System for verifying and re-training object detection ai and method for verifying and re-training object detection ai

Abstract

Provided is a system for verifying and re-training object detection AI capable of efficiently improving training accuracy in verification / re-training of the object detection AI. The present invention is characterized by comprising: an AI inference processing unit that infers an object to be detected from a verification image and obtains, as the inference result, a plurality of coordinate candidates on the verification image of the object to be detected; a coordinate variation calculation unit that obtains variations in the plurality of coordinate candidates of the object to be detected; a re-training necessity determination unit that determines the necessity of re-training in accordance with the variations in the coordinate candidates; and an AI training processing unit that performs re-training with the verification image for which the re-training necessity determination unit has determined the necessity of re-training.

Description

Object detection AI verification and re-learning system, object detection AI verification and re-learning method

[0001] The present invention relates to the configuration of a verification and relearning system for object detection AI and its method, and in particular to a technology that is effective when applied to object detection systems for autonomous vehicles.

[0002] By adopting AI (Artificial Intelligence) in the object detection system of an autonomous vehicle, the vehicle's onboard cameras and sensors can recognize and process objects around the vehicle, enabling the vehicle to drive safely while avoiding pedestrians and obstacles. Generally, the AI ​​system responsible for autonomous driving is installed on the cloud rather than in the vehicle itself, and controls the vehicle by communicating data between the vehicle and the cloud.

[0003] In order to expand the scope of applicability of autonomous driving, continuous re-learning is necessary to improve the accuracy of object detection by AI. However, with the conventional method of adding only images that the AI ​​has misrecognized to the re-learning images, it is difficult to efficiently improve learning accuracy.

[0004] As background art in this technical field, there is, for example, a technology such as that described in Patent Document 1. Patent Document 1 discloses an "image processing device that clarifies the area of ​​an object to be recognized while enabling machine learning including the area surrounding the area where the object exists."

[0005] JP 2018-45359 A

[0006] As described above, in the verification and relearning process of conventional object detection AI, only images that were incorrectly recognized during verification are subject to relearning. Of the verification images, only images that were incorrectly recognized are fed back into the relearning process, and no consideration is given to images that are correct but have low inference reliability. Furthermore, while object detection AI calculates multiple inference candidates, only the candidate that is finally selected is output externally and is not subject to verification.

[0007] The above-mentioned Patent Document 1 is an idea regarding a training method for image recognition AI to detect detection targets such as letters and numbers from an image. Since training is difficult using only the detection target area, it is proposed to train using an image of an area slightly larger than the detection target.

[0008] However, Patent Document 1 does not address the issues with verification and relearning in object detection AI as described above, and there is room for improvement.

[0009] Therefore, an object of the present invention is to provide an object detection AI verification and relearning system and an object detection AI verification and relearning method that can efficiently improve learning accuracy in verifying and relearning object detection AI.

[0010] In order to solve the above problems, the present invention is characterized by comprising an AI inference processing unit that infers a detection target object from a verification image and obtains multiple coordinate candidates of the detection target object on the verification image as an inference result; a coordinate variation calculation unit that obtains the variation of the multiple coordinate candidates of the detection target object; a relearning necessity determination unit that determines whether relearning is necessary based on the variation of the coordinate candidates; and an AI learning processing unit that performs relearning on the verification image that has been determined to require relearning by the relearning necessity determination unit.

[0011] The present invention is also characterized by including the steps of: (a) inferring a detection target object from a verification image and determining a plurality of coordinate candidates of the detection target object on the verification image; (b) determining the variance of the plurality of coordinate candidates of the detection target object; (c) determining whether or not re-learning is necessary based on the variance of the coordinate candidates; and (d) performing re-learning on the verification image determined to require re-learning in step (c).

[0012] According to the present invention, it is possible to realize a verification and relearning system and a verification and relearning method for object detection AI that can efficiently improve the learning accuracy in the verification and relearning of object detection AI.

[0013] This will improve the recognition accuracy and robustness of AI, which, for example, will expand the environmental conditions that autonomous driving can handle and contribute to reducing the need to hand over driving to the driver.

[0014] Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments.

[0015] 6A is a diagram showing a schematic configuration of an object detection AI verification and relearning system according to a first embodiment of the present invention. FIG. 6B is a block diagram showing a schematic configuration of an object detection AI verification and relearning system according to a second embodiment of the present invention. FIG. 6C is a diagram showing a schematic configuration of an object detection AI verification and relearning system according to a third embodiment of the present invention. FIG. 6D is a diagram showing an example of the hardware configuration of an object detection AI verification and relearning system according to a fourth embodiment of the present invention. FIG. 6E is a diagram showing a modified example of FIG. 6A. FIG. 6F is a block diagram showing a schematic configuration of an object detection AI verification and relearning system according to a fifth embodiment of the present invention. FIG. 6G is a block diagram showing a schematic configuration of an object detection AI verification and relearning system according to a sixth embodiment of the present invention. FIG. 6H is a flowchart showing a method for verifying and relearning an object detection AI according to the first embodiment of the present invention.

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same components are designated by the same reference numerals, and detailed description of overlapping parts will be omitted.

[0017] A verification and re-learning system and a verification and re-learning method for object detection AI according to a first embodiment of the present invention will be described with reference to FIGS. 1, 2, and 9.

[0018] Fig. 1 is a diagram showing a schematic diagram of a method for obtaining an inference result by an object detection AI. Fig. 2 is a block diagram showing a schematic configuration of a verification and re-learning system for an object detection AI according to the present embodiment. Fig. 9 is a flowchart showing a verification and re-learning method for an object detection AI according to the present embodiment.

[0019] First, a method for obtaining an inference result 103 by the object detection AI in the present invention will be described with reference to FIG.

[0020] The detection target image 101 in the present invention is, for example, an image of the surroundings of the vehicle captured by an on-board camera.

[0021] 1 shows an example of a two-dimensional image taken by a monocular camera (mono-camera), but the present invention is not limited to this, and a three-dimensional image taken by a stereo camera consisting of multiple (usually two) cameras may also be used. Furthermore, the detection target image 101 is not limited to one taken by a camera, and may also be point cloud data such as Lidar (Light Detection and Ranging).

[0022] The detection target object 102 is a vehicle, pedestrian, sign, road, etc. in the detection target image 101. The detection target image 101 can be set arbitrarily by the designer of the object detection AI verification and re-learning system of the present invention. The location of the detection target object 102 in the detection target image 101 can be expressed by coordinates (xmin, ymin, xmax, ymax).

[0023] As shown in the right diagram of Figure 1, the object detection AI of the present invention outputs multiple inference results 103 for one detection target object 102. Of these multiple inference results 103, the one with the highest class probability is selected as the final output result.

[0024] Next, the configuration of the object detection AI verification and relearning system of this embodiment will be described with reference to FIG.

[0025] As shown in FIG. 2, the object detection AI verification and re-learning system of this embodiment mainly comprises an AI verification unit and an AI re-learning unit.

[0026] The AI ​​verification unit includes an AI inference processing unit 201, a coordinate variation calculation unit 202, and a re-learning necessity determination unit 203.

[0027] The AI ​​re-learning unit has an AI learning processing unit 204.

[0028] The AI ​​inference processing unit 201 receives a landscape image around the vehicle captured by a camera or the like as a detection target image (verification image) 101, and outputs a plurality of inference results 103 indicating the class (car, person, etc.) and coordinates for a plurality of detection target objects 102 in the image. As described above, the detection target image (verification image) 101 may be point cloud data acquired by Lidar or the like instead of a camera.

[0029] The coordinate variation calculation unit 202 outputs the variation of the multiple inference results 103 for the same object based on the inference results 103 received from the AI ​​inference processing unit 201. As an example, this variation indicates the sum of the variances of each coordinate among the multiple inference results 103. Furthermore, when calculating the variation, not only the coordinate information of the inference results 103 but also class probability information may be referenced. The coordinate variation calculation unit 202 determines a variation calculation target based on the class probability of each inference candidate among the multiple inference candidates.

[0030] The relearning necessity determination unit 203 determines whether relearning is necessary and outputs the result, such as indicating that relearning is necessary if the magnitude of the variation received from the coordinate variation calculation unit 202 is equal to or greater than a predetermined threshold. The threshold may be a fixed value, or may be a variable value that refers to parameters obtained from the surrounding situation, etc., obtained from the detection target image (verification image) 101, the object class, etc.

[0031] The AI ​​learning processing unit 204 performs re-learning on the detection target image (verification image) 101 that has been determined by the re-learning necessity determination unit 203 to require re-learning, and outputs the AI ​​model after re-learning (trained AI model).

[0032] Next, a method for verifying and relearning an object detection AI using the object detection AI verification and relearning system of Fig. 2 will be described with reference to Fig. 9. Fig. 9 shows a flow of generating a relearning image using the variations calculated by the coordinate variation calculation unit 202.

[0033] When the object detection AI verification and relearning system starts processing, first, in step S101, the AI ​​inference processing unit 201 acquires verification image data (detection target image 101) to be used for learning.

[0034] Next, in step S102, the AI ​​inference processing unit 201 acquires correct answer data indicating in which part of the input data (detection target image 101) acquired in step S101 the detection target object 102 is located.

[0035] Next, in step S103, the AI ​​inference processing unit 201 actually performs inference using AI based on the input data (detection target image 101) used for learning, and outputs multiple inference results 103.

[0036] Next, in step S104, the coordinate variation calculation unit 202 calculates the variation between the multiple inference results 103 indicating the same object. This is performed for all of the detection target objects 102.

[0037] Next, in step s105, the relearning necessity determination unit 203 determines whether the value of the variation calculated in step s104 is equal to or greater than a predetermined threshold, and determines whether relearning is necessary for each object in the detection target image 101. If it is determined that relearning is necessary (Yes), the process proceeds to step s106. On the other hand, if it is determined that relearning is not necessary (No), the process returns to step s103, and the processes from step s103 onwards are repeated.

[0038] Next, in step s106, the AI ​​re-learning unit generates an image for re-learning for the object that was determined to require re-learning in step s105.

[0039] Finally, in step S107, the AI ​​is trained using the re-learning image generated in step S106, and the process ends.

[0040] As described above, the object detection AI verification and relearning system of this embodiment includes an AI inference processing unit 201 that infers a detection target object 102 from a verification image 101 and obtains multiple coordinate candidates of the detection target object 102 on the verification image 101 as an inference result 103, a coordinate variation calculation unit 202 that obtains the variation in the multiple coordinate candidates of the detection target object 102, a relearning necessity determination unit 203 that determines whether relearning is necessary based on the variation in the coordinate candidates, and an AI learning processing unit 204 that performs relearning on the verification image 101 that has been determined to require relearning by the relearning necessity determination unit 203. In other words, the coordinate variation in the inference candidates is calculated, and verification objects with large variations are added to the relearning.

[0041] By relearning not only images that were incorrectly recognized in the verification image 101, but also images that are recognized as correct but have low reliability, the recognition accuracy and robustness of the AI ​​are improved, thereby expanding the environmental conditions that autonomous driving can handle and reducing the need to hand over driving to the driver.

[0042] Second Embodiment A verification and relearning system and a verification and relearning method for object detection AI according to a second embodiment of the present invention will be described with reference to FIGS. 3 and 4. FIG.

[0043] Fig. 3 is a block diagram showing the general configuration of the object detection AI verification and re-learning system of this embodiment, and shows a more specific configuration example of the AI ​​re-learning unit compared to Example 1 (Fig. 2). Fig. 4 is a diagram showing the generation of a mask image 305 by the re-learning image generation unit 303 of Fig. 3.

[0044] As shown in FIG. 3, the AI ​​re-learning unit of this embodiment further includes a re-learning image generation unit 303 in addition to the components of the AI ​​re-learning unit of the first embodiment (FIG. 2).

[0045] For an image determined to require relearning by the relearning necessity determination unit 203, the relearning image generation unit 303 generates a mask image 305 (the right diagram in FIG. 4 ) by extracting an area to be used in relearning from the detection target image (verification image) 101. Note that the method of extracting the area at this time is not limited to generating the mask image 305, but may also be to generate an image in which only the relevant portion is cut out. Furthermore, the area extracted at this time may be a fixed value, or may be a variable value that extracts a portion corresponding to the range of variation that is the output result of the coordinate variation calculation unit 202.

[0046] The AI ​​learning processing unit 304 performs re-learning using the mask image 305 generated by the re-learning image generation unit 303 on the detection target image (verification image) 101 that has been determined to require re-learning by the re-learning necessity determination unit 203, and outputs the AI ​​model after re-learning (trained AI model).

[0047] As described above, the object detection AI verification and relearning system of this embodiment further includes a relearning image generation unit 303 that extracts a relearning image area from the verification image 101 that has been determined to require relearning by the relearning necessity determination unit 203.

[0048] Furthermore, the re-learning image generating unit 303 determines a re-learning image region from the verification image 101 based on the coordinate information of the inference candidate used by the coordinate variation calculating unit 202 .

[0049] Third Embodiment With reference to FIG. 5, a system and a method for verifying and re-learning an object detection AI according to a third embodiment of the present invention will be described.

[0050] FIG. 5 is a block diagram showing the general configuration of the object detection AI verification and re-learning system of this embodiment, and shows a more specific configuration example of the AI ​​re-learning unit compared to embodiment 2 (FIG. 3).

[0051] 5, the AI ​​re-learning unit of this embodiment further includes a similar image generation unit 401 and a generated similar image selection unit 402 in addition to the components of the AI ​​re-learning unit of Example 2 (FIG. 3). Similarly to the AI ​​verification unit, the AI ​​re-learning unit also includes an AI inference processing unit 201, a coordinate variation calculation unit 202, and a relearning necessity determination unit 203.

[0052] The AI ​​inference processing unit 201, coordinate variation calculation unit 202, and relearning necessity determination unit 203 of the AI ​​relearning unit have the same functions as the AI ​​verification unit, and the corresponding processing in the AI ​​relearning unit may be performed in the AI ​​verification unit. For ease of explanation, these are shown as separate blocks in Figure 5.

[0053] The similar image generation unit 401 generates an image similar to the mask image 305 generated by the re-learning image generation unit 303. Examples of the similar image to be generated include a new image with changed weather or surrounding environment, or an image that has been inverted or has had its contrast changed.

[0054] The generated similar image selection unit 402 uses the similar images generated by the similar image generation unit 401 based on the results of the inference processing by the AI ​​inference processing unit 201, and selects images that have been determined to require re-learning by the re-learning necessity determination unit 203 based on the magnitude of variation calculated by the coordinate variation calculation unit 202, and selects the selected similar images as images to be learned by the AI ​​learning processing unit 304.

[0055] The selected image may be added to or replaced with the detection target image (verification image) 101 or mask image 305 originally used for learning.

[0056] As described above, the object detection AI verification and relearning system of this embodiment further includes a similar image generation unit 401 that generates similar images of the relearning image area extracted by the relearning image generation unit 303, and a generated similar image selection unit 402 that inputs the generated similar images generated by the similar image generation unit 401 into the AI ​​inference processing unit 201, performs coordinate variation calculation (coordinate variation calculation unit 202) and determines whether relearning is necessary (relearning necessity determination unit 203) on the results, and selects generated similar images that are determined to require relearning as images for relearning.

[0057] Referring to Figures 6A and 6B, an object detection AI verification and relearning system and object detection AI verification and relearning method according to Example 4 of the present invention will be described.

[0058] Fig. 6A is a diagram illustrating an example of the hardware configuration of the verification and relearning system for object detection AI according to the present embodiment. Fig. 6B is a diagram illustrating a modified example of Fig. 6A.

[0059] The learning device 501 shown in Figure 6A is a device for performing AI learning in the AI ​​verification unit and AI re-learning unit described in each of the above embodiments, and an example of such a device is a server installed on the cloud.

[0060] As shown in FIG. 6A, the learning device 501 includes, as its main components, a GPU (Graphics Processing Unit) 504, a CPU (Central Processing Unit) 505, a RAM (Random Access Memory) 506, and a storage unit 507.

[0061] The storage unit 507 includes a learning data storage unit 508 and a threshold value storage unit 509 .

[0062] On the other hand, an on-board ECU 502 mounted on a vehicle that performs data communication with the learning device 501 includes a driving data storage unit 503 .

[0063] The learning device 501 receives vehicle driving data from the in-vehicle ECU 502 and performs AI learning. The learning device 501 also transmits the learned AI model to the in-vehicle ECU 502. The driving data storage unit 503 stores image data of the surroundings of the vehicle acquired from a camera or the like while the vehicle is driving.

[0064] In addition, when image data is sent from the on-board ECU 502 to the learning device 501 (server) in real time, the driving data storage unit 503 may not be provided, and the data may be stored directly from the on-board ECU 502 to the learning data storage unit 508 of the learning device 501 (server).

[0065] The GPU 504 is a device for performing a large amount of parallel calculations at high speed in order to perform AI calculations. Other devices that can be used include an AI accelerator and a CPU.

[0066] The CPU 505 is a device for calculating AI and controlling the overall operation of the learning device 501.

[0067] The RAM 506 is a device for temporarily storing calculation data.

[0068] The storage unit 507 is a device for storing data for a long period of time, and includes a learning data storage unit 508 and a threshold value storage unit 509 .

[0069] The learning data storage unit 508 stores group data used for learning the AI ​​model and transfers it to the CPU 505, GPU 504, and RAM 506, which are the main computing units, as needed.

[0070] A threshold value (threshold value) storage unit 509 transmits the threshold value (threshold value) to be used during learning to the CPU 505 and the GPU 504 .

[0071] Note that Figure 6A shows an example of a configuration in which data communication is performed between an on-board ECU 502 installed in a vehicle and a learning device 501 (server) installed on the cloud. However, as shown in the modified example of Figure 6B, by incorporating the learning device 501 into the on-board ECU 502, it is also possible to install an AI system responsible for autonomous driving in the vehicle itself and limit autonomous driving control to within the vehicle itself.

[0072] Referring to FIG. 7, a verification and re-learning system and a verification and re-learning method for object detection AI according to a fifth embodiment of the present invention will be described.

[0073] FIG. 7 is a block diagram showing the schematic configuration of the object detection AI verification and re-learning system of this embodiment, and shows an example of updating an AI model using a vehicle and a learning device.

[0074] 7, the learning device 604 of this embodiment mainly comprises an AI verification unit, an AI relearning unit, a peripheral information receiving unit 605, and a model transmitting unit 606. The configurations of the AI ​​verification unit and the AI ​​relearning unit are basically the same as those of the first embodiment (FIG. 2).

[0075] Furthermore, the in-vehicle ECU 601 of this embodiment has, as its main components, a peripheral information acquisition and storage unit 602, a peripheral information transmission unit 603, a model reception unit 607, and an AI inference processing unit 608.

[0076] The in-vehicle ECU 601 is a device that has the function of recognizing information about the surroundings of the vehicle and controlling the vehicle.

[0077] In the vehicle-mounted ECU 601 , a surrounding information acquisition and storage unit 602 acquires and stores surrounding information from a camera or the like, and the surrounding information is sent from a surrounding information transmission unit 603 to a surrounding information reception unit 605 in a learning device 604 .

[0078] The learning device 604 is a device for performing AI learning, and an example thereof is a server installed on the cloud.

[0079] The AI ​​inference processing unit 201 of the learning device 604 uses the information received from the surrounding information receiving unit 605 to output multiple inference results 103 indicating the class (car, person, etc.) and coordinates for multiple detection target objects 102 in the image.

[0080] The coordinate variation calculation unit 202 outputs the variation of the multiple inference results 103 for the same object based on the inference results 103 received from the AI ​​inference processing unit 201.

[0081] The relearning necessity determination unit 203 determines whether relearning is necessary, for example, if the magnitude of the variation received from the coordinate variation calculation unit 202 is equal to or greater than a predetermined threshold, and outputs the result.

[0082] The AI ​​learning processing unit 204 performs re-learning on the detection target image (verification image) 101 that has been determined by the re-learning necessity determination unit 203 to require re-learning, and outputs the AI ​​model after re-learning (trained AI model).

[0083] The model transmission unit 606 transmits the AI ​​model re-learned by the AI ​​learning processing unit 204 to the model reception unit 607 on the in-vehicle ECU 601 side.

[0084] The model receiving unit 607 transmits the received re-trained AI model to the AI ​​inference processing unit 608 on the in-vehicle ECU 601 side, and updates the AI ​​model.

[0085] The AI ​​inference processing unit 608 performs peripheral recognition and the like for vehicle control by the in-vehicle ECU 601.

[0086] The AI ​​inference processing unit 201 of the learning device 604 uses the same AI model as the AI ​​inference processing unit 608 on the in-vehicle ECU 601 side, and is able to reproduce the processing on the in-vehicle ECU 601 side and extract areas where recognition accuracy is insufficient.

[0087] As described above, the object detection AI verification and re-learning system of this embodiment is equipped with a model transmission unit 606 that transmits the AI ​​model re-learned by the AI ​​learning processing unit 204 to the on-board ECU 601, and the on-board ECU 601 updates the AI ​​model of the on-board ECU 601 with the re-learned AI model received from the model transmission unit 606.

[0088] In addition, the AI ​​inference processing unit 201, coordinate variation calculation unit 202, re-learning necessity determination unit 203, and AI learning processing unit 204 are arranged in a learning device 604 located at a position separated from the vehicle in which the on-board ECU 601 is installed.

[0089] With reference to FIG. 8, an object detection AI verification and relearning system and an object detection AI verification and relearning method according to a sixth embodiment of the present invention will be described.

[0090] 8 is a block diagram showing the schematic configuration of the object detection AI verification and relearning system of this embodiment, which corresponds to a modified example of the fifth embodiment (FIG. 7). In this embodiment, a configuration example for reducing the amount of communication between the learning device 604 and the on-board ECU 601 will be described.

[0091] 8, the learning device 604 of this embodiment mainly comprises an AI re-learning unit, a peripheral information receiving unit 605, and a model transmitting unit 606. The configuration of the AI ​​re-learning unit is basically the same as the configuration of the first embodiment (FIG. 2).

[0092] In addition, the in-vehicle ECU 601 of this embodiment has, as its main components, a peripheral information acquisition and storage unit 602, a peripheral information transmission unit 603, a model reception unit 607, an AI inference processing unit 608, a coordinate variation calculation unit 202, and a re-learning necessity determination unit 203.

[0093] In this embodiment (FIG. 8), compared to the configuration of embodiment 5 (FIG. 7), the coordinate variation calculation unit 202 and the relearning necessity determination unit 203 are moved to the in-vehicle ECU 601 side, and the function of the AI ​​inference processing unit 201 is also performed by the AI ​​inference processing unit 608. With this configuration, only images determined to require relearning are sent from the in-vehicle ECU 601 side to the learning device 604 side and used for relearning. Sending all images from the in-vehicle ECU 601 side to the learning device 604 side would require a huge amount of data communication, so even the selection of images that require relearning is performed on the in-vehicle ECU 601 side.

[0094] The surrounding information acquisition and storage unit 602 acquires and stores surrounding information from a camera or the like, and based on that surrounding information, the AI ​​inference processing unit 608 recognizes the surroundings, and from the results, the coordinate variation calculation unit 202 outputs the variation of multiple inference results 103 for the same object.

[0095] The re-learning necessity determination unit 203 determines whether re-learning is necessary, such as determining that re-learning is necessary if the magnitude of the variation received from the coordinate variation calculation unit 202 is equal to or greater than a predetermined threshold, and stores images that are determined to require re-learning in the peripheral information acquisition and storage unit 602.

[0096] As described above, in the object detection AI verification and relearning system of this embodiment, the AI ​​inference processing unit 608, coordinate variation calculation unit 202, and relearning necessity determination unit 203 are arranged in the on-board ECU 601, and the AI ​​learning processing unit 204 is arranged in a learning device 604 located at a position separated from the vehicle in which the on-board ECU 601 is installed.

[0097] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0098] 101...detection target image (verification image), 102...detection target object, 103...inference result, 201, 608...AI inference processing unit, 202...coordinate variation calculation unit, 203...relearning necessity determination unit, 204, 304...AI learning processing unit, 303...relearning image generation unit, 305...mask image, 401...similar image generation unit, 402...generated similar image selection unit, 501, 604...learning device, 502, 601...on-board ECU, 503...driving data storage unit, 504...GPU, 505...CPU, 506...RAM, 507...memory unit, 508...learning data storage unit, 509...threshold (threshold value) storage unit, 602...peripheral information acquisition and storage unit, 603...peripheral information transmission unit, 605...peripheral information reception unit, 606...model transmission unit, 607...model reception unit.

Claims

1. An object detection AI verification and re-learning system comprising: an AI inference processing unit that performs inference on a verification image to determine a detection target object and obtains multiple coordinate candidates of the detection target object on the verification image as an inference result; a coordinate variation calculation unit that determines the variation of the multiple coordinate candidates of the detection target object; a re-learning necessity determination unit that determines whether re-learning is necessary based on the variation of the coordinate candidates; and an AI learning processing unit that performs re-learning on the verification image determined to require re-learning by the re-learning necessity determination unit.

2. A verification and re-learning system for object detection AI as described in claim 1, comprising a re-learning image generation unit that extracts a re-learning image area from the verification image that has been determined to require re-learning by the re-learning necessity determination unit.

3. A verification and re-learning system for object detection AI as described in claim 1, wherein the coordinate variation calculation unit determines the target for variation calculation based on the class probability of each inference candidate among multiple inference candidates.

4. A verification and re-learning system for object detection AI as described in claim 2, wherein the re-learning image generation unit determines the re-learning image area from the verification image based on coordinate information of the inference candidate used in the coordinate variation calculation unit.

5. A verification and re-learning system for object detection AI as described in claim 2, comprising: a similar image generation unit that generates similar images of the re-learning image area extracted by the re-learning image generation unit; and a generated similar image selection unit that inputs the generated similar images generated by the similar image generation unit into the AI ​​inference processing unit, calculates coordinate variation and determines whether re-learning is necessary for the results, and selects generated similar images that are determined to require re-learning as images for re-learning.

6. An object detection AI verification and re-learning system as described in claim 1, comprising a model transmission unit that transmits the AI ​​model re-learned by the AI ​​learning processing unit to an on-board ECU, and the on-board ECU updates the AI ​​model of the on-board ECU with the re-learned AI model received from the model transmission unit.

7. A verification and re-learning system for object detection AI as described in claim 6, wherein the AI ​​inference processing unit, the coordinate variation calculation unit, the re-learning necessity determination unit, and the AI ​​learning processing unit are arranged in a learning device located at a position separated from the vehicle in which the on-board ECU is mounted.

8. A verification and relearning system for object detection AI as described in claim 6, wherein the AI ​​inference processing unit, the coordinate variation calculation unit, and the relearning necessity determination unit are arranged in the on-board ECU, and the AI ​​learning processing unit is arranged in a learning device located at a position separated from the vehicle in which the on-board ECU is mounted.

9. A verification and re-learning system for object detection AI as described in claim 1, wherein the verification image is one of a two-dimensional image taken by a monocular camera, a three-dimensional image taken by a stereo camera, and point cloud data taken by Lidar.

10. A method for verifying and re-learning an object detection AI, comprising the following steps: (a) a step of inferring a detection target object from a verification image and determining a plurality of coordinate candidates of the detection target object on the verification image; (b) a step of determining the variance of the plurality of coordinate candidates of the detection target object; (c) a step of determining whether or not re-learning is necessary based on the variance of the coordinate candidates; and (d) a step of re-learning the verification image determined to require re-learning in step (c).

Images