Information processing program, information processing method, and information processing device

Abstract

To provide an information processing program, an information processing method, and an information processing device which suppress deterioration of detection accuracy of an object detection model.SOLUTION: An information processing device 10 comprises: an inference processing unit 141 for acquiring a detection result for a first image of a first object detection model for detecting an object; an annotation unit 142 for accepting specification from a user to a partial region of the first image according to the detection result; a mask processing unit 143 for generating a second image in which the object included in the partial region is masked from the first image; and a learning processing unit 140 for executing mechanical learning using the second image, and generating a second object detection model for detecting the object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing program, an information processing method, and an information processing device. [Background technology]

[0002] In the field of machine learning, there is an object detection technology that uses a machine learning model to predict the area of ​​an object present in an image and the name of the object. A machine learning model for object detection is called an object detection model. The object detection technology is a supervised learning technology, and machine learning is performed using annotation data to which area information indicating the area of ​​an object and a label indicating the name of the object are attached for each object present in an image.

[0003] In the prior art, a technique has been disclosed in which an image is input to an existing object detection model, and the obtained detection result is used as annotation data for the input image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2023 / 286847 [Patent Document 2] Patent Publication No. 2021-089491 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if a detection result including a false detection is used as annotation data as is, the annotation data will contain erroneous information, which may degrade the detection accuracy of an object detection model generated using the annotation data.

[0006] In one aspect, the present invention aims to provide an information processing program, an information processing method, and an information processing device that can suppress deterioration of the detection accuracy of an object detection model. [Means for solving the problem]

[0007] In one aspect, an information processing program is provided that causes a computer to execute a process of obtaining a detection result for a first image of a first object detection model that detects an object, receiving a designation for a partial area of ​​the first image in accordance with the detection result, generating a second image from the first image in which an object included in the partial area is masked, performing machine learning using the second image, and generating a second object detection model that detects the object.

[0008] In one aspect, there is provided an information processing method in which a computer executes a process similar to the process based on the above-mentioned information processing program.

[0009] In one aspect, an information processing device that executes a process similar to the process based on the above information processing program is provided. Effect of the Invention

[0010] According to one aspect, it is possible to suppress deterioration in the detection accuracy of an object detection model. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a functional block diagram illustrating an example of a functional configuration of an information processing device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of relearning according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a screen on which a detection result of object detection according to the embodiment is displayed. [Figure 4] FIG. 4 is a diagram illustrating the first phase of the mask processing according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating the second phase of the mask processing according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating a process flow according to the embodiment. [Figure 7]FIG. 7 is a diagram illustrating an example of a second image according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a hardware configuration of an information processing device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an information processing program, an information processing method, and an information processing device according to the present invention will be described in detail with reference to the drawings. Note that these examples are merely examples for implementing the present invention, and do not limit the present invention. In addition, in this embodiment, an embodiment for object detection that detects vehicles from an image of a road is described, but the image target and the detection target are not limited to the contents of this embodiment. EXAMPLES

[0013] The information processing device 10 according to the present embodiment is a device that performs re-learning of object detection. Re-learning in the present embodiment is a learning method that performs machine learning by adding a new image to an image used in machine learning of an existing object detection model. The information processing device 10 inputs a first image to be added in re-learning to a first object detection model and performs inference processing. The information processing device 10 acquires a detection result for the first image by the inference processing. The information processing device 10 displays an image in which the detection result is superimposed on the first image on a display or the like. The user checks the displayed image, specifies a mask area that is a part of the first image, and inputs it to the information processing device 10 via a mouse or the like. The information processing device 10 accepts a designation from the user for the mask area of ​​the first image. The information processing device 10 generates a second image in which an object included in the mask area is masked. The masking process is a process of covering and concealing an object included in the mask area. The information processing device 10 performs re-learning using the generated second image to generate a second object detection model.

[0014] [Function configuration] Fig. 1 is a functional block diagram showing an example of the functional configuration of an information processing device according to an embodiment. The functional configuration of an information processing device 10 according to an embodiment will be described with reference to Fig. 1. The information processing device 10 has a communication unit 11, an input unit 12, an output unit 13, a control unit 14, and a storage unit 15.

[0015] The communication unit 11 is a processing unit that controls communication with other devices, and is realized by a communication interface such as a network interface card (NIC). Through the communication unit 11, for example, first training data, a first object detection model, a first image, a detection result, first annotation data, second annotation data, a second image, second training data, and a second object detection model are input and output.

[0016] The input unit 12 is a processing unit that controls input from a user to the information processing device 10, and is realized by, for example, a mouse, a pen tablet, etc. For example, the annotation unit 142 accepts designation for a mask region from the user via the input unit 12.

[0017] The output unit 13 is a processing unit that controls the output from the information processing device 10 to the user, and is realized by, for example, a display. For example, the annotation unit 142 displays to the user, via the output unit 13, an image in which the detection result is superimposed on the first image.

[0018] The storage unit 15 stores various data or various programs executed by the control unit 14. The storage unit 15 is realized by, for example, a main storage device such as a Random Access Memory (RAM) or an auxiliary storage device such as a Hard Disk Drive (HDD). The storage unit 15 stores, for example, a first training data DB150, a first object detection model DB151, a first image DB152, a detection result DB153, a first annotation data DB154, a second annotation data DB155, a second image DB156, a second training data DB157, and a second object detection model DB158.

[0019] The first training data DB150 is a database that stores the first training data. The first training data is data used for machine learning of the first object detection model. The first training data is training data that includes an image and annotation data for the image. As an example, when there are multiple images, the annotation data for the image is stored in an individual file for each image.

[0020] The first object detection model DB151 is a database that stores a first object detection model. The first object detection model is an object detection model generated by machine learning using first training data. The first object detection model is a neural network, and examples of the first object detection model include Region Based Convolutional Neural Networks (R-CNN), Single Shot multibox Detector (SSD), and You Only Look Once (YOLO) based on a Convolutional Neural Network (CNN). The first object detection model is data including various parameters such as feature amounts obtained by machine learning.

[0021] The first image DB 152 is a database that stores a first image. The first image is an image including an object that will be erroneously detected as a detection result by the first object detection model. As an example, the inference processing unit 141 collects images including an object that will be erroneously detected by performing an inference process using the first object detection model for a certain period of time, and sets the collected images as the first images.

[0022] The detection result DB 153 is a database that stores the detection results. The detection results are inference results for the first image of the first object detection model. The detection results are data including area information, labels, and confidence levels for each object included in the first image. The confidence level is a value indicating how certain the detection result for each object is. The area information and labels are output in the same format as the area information and labels of the annotation data. As an example, when there are multiple first images, the inference processing unit 141 generates the detection results in separate files for each image.

[0023] The first annotation data DB154 is a database that stores the first annotation data. The first annotation data is annotation data for the first image, and is annotation data including region information and a label for the mask region. As an example, the first annotation data is annotation data including corrected region information and a label for an object that is erroneously detected in the detection result. As an example, the first annotation data is annotation data including region information and a label for an object that is correctly detected in the detection result. As an example, the region information is information based on the x coordinate and y coordinate of the image. As an example, the region information is information of a rectangular region that is defined by the x coordinate and y coordinate of the center of the region, the width of the region, and the height of the region, with the upper left corner of the image as the starting point (x coordinate, y coordinate) = (0, 0). As an example, the label is a label stored in the annotation data as an arbitrary natural number corresponding to the name, such as "1: car, 2: truck, 3: masking". "car" is an example of a label indicating a car, "truck" is an example of a label indicating a truck, and "masking" is an example of a label indicating a masking region. As an example, when there are multiple first images, the annotation unit 142 generates the first annotation data in a separate file for each image.

[0024] The second annotation data DB 155 is a database that stores the second annotation data. The second annotation data is annotation data used for training a second object detection model. The second annotation data is annotation data for a first image, and is annotation data obtained by deleting area information and labels for mask areas from the first annotation data. As an example, when there are multiple first images, the annotation unit 142 generates the second annotation data in an individual file for each image.

[0025] The second image DB156 is a database that stores a second image. The second image is an image in which an object included in a mask area designated by a user among areas included in the first image is masked. The second image is an image included in second training data that is added to the first training data in re-learning, and is used for machine learning of a second object detection model.

[0026] The second training data DB 157 is a database that stores second training data. The second training data is added to the first training data in re-learning and is used for machine learning of the second object detection model. The second training data is training data that includes a second image and second annotation data.

[0027] The second object detection model DB158 is a database that stores a second object detection model. The second object detection model is an object detection model generated by machine learning using a second image. The second object detection model is an object detection model generated by re-learning. The second object detection model is an object detection model generated by machine learning using the first training data and the second training data. The second object detection model is data including various parameters such as features obtained by machine learning. The second object detection model is a neural network, and examples thereof include R-CNN, SSD, and YOLO based on CNN.

[0028] Next, the control unit 14 will be described. The control unit 14 controls the overall processing of the information processing device 10. The control unit 14 is realized by a processor such as a Central Processing Unit (CPU), a Graphical Processing Unit (GPU), or a Digital Signal Processor (DSP) reading a program stored in a storage device, and expanding and executing the program in a main storage device such as a RAM. The control unit 14 may be realized by including an integrated circuit such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA). The control unit 14 has a learning processing unit 140, an inference processing unit 141, an annotation unit 142, and a mask processing unit 143.

[0029] The learning processing unit 140 generates a first object detection model by machine learning using the first training data. The learning processing unit 140 stores the generated first object detection model in the storage unit 15. The learning processing unit 140 generates a second object detection model by machine learning using the second image. The learning processing unit 140 generates a second object detection model by machine learning using the first training data and the second training data. The learning processing unit 140 stores the generated second object detection model in the storage unit 15.

[0030] The inference processing unit 141 acquires a detection result for the first image of the first object detection model. Specifically, the inference processing unit 141 inputs the first image to the first object detection model, and executes an inference process of object detection that predicts the area and name of an object present in the first image. The inference processing unit 141 acquires a detection result by the inference process. The inference processing unit 141 stores the detection result in the storage unit 15.

[0031] The annotation unit 142 accepts a designation for the mask region from the user in accordance with the detection result. As an example, the annotation unit 142 displays an image in which the detection result is superimposed on the first image on a screen such as a display. The user checks the displayed image and performs annotation to designate region information and a label for the mask region. The annotation unit 142 accepts a designation for the mask region from the user and generates first annotation data reflecting the designation of the mask region. As an example, the user checks the displayed image and performs annotation to correct the detection result. As an example, the annotation unit 142 accepts a correction of the detection result from the user and generates first annotation data reflecting the designation of the mask region and the correction of the detection result. The annotation unit 142 stores the generated first annotation data in the storage unit 15. In this way, the task of designating region information or a label or the task of correcting is called annotation.

[0032] The annotation unit 142 deletes the area information and the label for the mask area from the first annotation data, and generates the second annotation data. As an example, the annotation unit 142 deletes the label of "masking" and the area information corresponding to the label of "masking" from the first annotation data, and generates the second annotation data. The annotation unit 142 stores the second annotation data in the storage unit 15.

[0033] The mask processing unit 143 generates a second image in which an object included in a mask area is masked from the first image. Specifically, the mask processing unit 143 refers to the first annotation data and acquires area information of the mask area. The mask processing unit 143 masks the mask area based on the acquired area information, and generates the second image.

[0034] [About re-learning process] 2 is a diagram illustrating an example of the re-learning process according to the embodiment. The re-learning process includes a learning phase 20 and a re-learning phase 21.

[0035] First, the learning phase 20 will be described. In the learning phase 20, a first object detection model is generated by a learning process using the first training data. Specifically, in the learning phase 20, the learning processing unit 140 acquires the first training data from the first training data DB 150, executes a learning process for executing machine learning, and generates the first object detection model.

[0036] Next, the re-learning phase 21 will be described. In the re-learning phase 21, a second object detection model is generated by a learning process using the first training data and the second training data. Specifically, in the re-learning phase 21, the learning processing unit 140 acquires the first training data from the first training data DB 150, acquires the second training data from the second training data DB 157, executes a learning process for executing machine learning, and generates a second object detection model.

[0037] In this way, the re-learning according to the present embodiment is a technique in which the second training data, which is new training data, is added to the first training data used in the learning process of the first object detection model, and the learning process is executed. The re-learning is executed, for example, when the first object detection model is verified or operated, and it is found that the required detection accuracy cannot be obtained.

[0038] [Mask processing] The masking process in the information processing device 10 will be described with reference to FIGS.

[0039] Fig. 3 is a diagram illustrating an example of a screen on which a detection result of object detection according to the embodiment is displayed. First, with reference to Fig. 3, object detection based on an object detection model generated by the information processing device 10 will be described. Note that this embodiment will be described with an example of object detection for detecting vehicles such as cars and trucks from an image of a road.

[0040] The inference processing unit 141 executes an inference process that predicts the area and name of an object to be detected that exists in the first image using the first object detection model, and obtains a detection result. The annotation unit 142 displays an image in which the first image and the detection result for the first image are superimposed on a screen such as a display. As an example, the annotation unit 142 displays the detection result by showing the area of ​​the object as a rectangle as in box 35, and showing the label of the object surrounded by box 35 at the top of box 35. For example, the annotation unit 142 displays a detection result indicating that the label of the object in the area of ​​box 32 is "truck" and the label of the object in the area of ​​box 35 is "car" as in FIG. 3.

[0041] Here, if the detection result includes a false positive, the annotation unit 142 accepts corrections of the area information and the label for the false positive object from the user. This is because if the detection result including the false positive is used as annotation data as it is, the annotation data may include erroneous information, and the detection accuracy of the second object detection model generated by re-learning may deteriorate. Examples of false positives include overdetection in which an object that has not been trained as a detection target is detected, such as the signboard 30 in FIG. 3, a case where an object is detected but has an incorrect label, such as the truck 34, and undetection in which an object that has been trained as a detection target is not detected, such as the car 33. As an example, if there is an overdetection, the annotation unit 142 accepts annotations from the user that delete the area information and the label. As an example, if the label is incorrect, the annotation unit 142 accepts annotations from the user that correct the label. As an example, if there is an undetection, the annotation unit 142 accepts annotations from the user that attach area information and a label to the object.

[0042] However, the more correction targets there are in the detection results, the greater the annotation workload on the user becomes. Also, compared to a large-sized object such as a car 33, a small-sized object such as the car surrounded by the dotted line 31 or an object that overlaps with other objects requires detailed annotation work, which increases the annotation workload.

[0043] Furthermore, when the detection target to be improved in accuracy by re-learning is a car 33, small objects such as the car surrounded by the dotted line 31 or objects that overlap each other may have a low priority for re-learning. In such a case, it may be possible to choose not to annotate objects with a low priority for re-learning in order to reduce the annotation workload. However, if an unannotated object exists in the image used for re-learning, the detection accuracy of the second object detection model generated by re-learning may deteriorate.

[0044] 4 is a diagram for explaining the first phase of the mask processing according to the embodiment. The first phase of the mask processing in the information processing device 10 will be explained with reference to FIG.

[0045] The annotation unit 142 accepts a designation for a mask region from a user. The mask region is a region that includes an object that is erroneously detected by the first object detection model. As an example, the mask region is a region that includes an object that is erroneously detected by the first object detection model and has a low priority for re-learning. As an example, an object that has a low priority for re-learning is an object that requires costs such as manpower and time for annotation, such as a small-sized object or an object that overlaps with another object. When a user wants to set an area that includes an object surrounded by a dotted line 31 as a mask region, the annotation unit 142 accepts a designation for a mask region from a user, attaches a box 40 to the area that includes the object surrounded by the dotted line 31, and displays the box 40 with a label of "masking". As an example, the annotation unit 142 accepts an annotation that corrects the detection result. In the image shown in Figure 4, annotations that correct the detection results have been used to delete the box and label of the signboard 30 that was overdetected in Figure 3, the label of the mislabeled truck 34 has been corrected to "truck," and a box and label have been added to the undetected car 33.

[0046] In this way, the annotation unit 142 receives annotations for the mask region and annotations for correcting the detection result. The annotation unit 142 generates first annotation data based on the region information and label for the mask region and the corrected region information and label.

[0047] 5 is a diagram for explaining the second phase of the mask processing according to the embodiment. The mask processing in the information processing device 10 will be explained with reference to FIG.

[0048] The mask processing unit 143 acquires first annotation data. The mask processing unit 143 acquires region information of a mask region labeled with "masking" from the first annotation data. In other words, the mask processing unit 143 acquires region information of the box 40 in Fig. 4 from the first annotation data. The mask processing unit 143 generates a second image in which an object included in the mask region is masked based on the acquired region information.

[0049] As an example, the mask processing unit 143 masks an object included in the mask area by filling the mask area with an arbitrary color. The color may be a single color. FIG. 5 is an example of the second image when the mask area is filled with an arbitrary single color. The process of filling the mask area with an arbitrary color is an example of a masking process.

[0050] The information processing device 10 executes re-learning using the generated second image. In this way, the information processing device 10 can execute re-learning using the second image in which objects that are erroneously detected by the first object detection model or objects with low priority for re-learning are excluded from the first image added by re-learning. This allows the information processing device 10 to suppress deterioration of the detection accuracy of the second object detection model generated by re-learning. The information processing device 10 can reduce the workload of annotation during re-learning.

[0051] [Processing flow] Fig. 6 is a flowchart for explaining the flow of processing according to the embodiment. With reference to Fig. 6, the flow of processing in which the information processing device 10 accepts a mask region designation from a user, generates a second image, and performs re-learning using the second image will be explained.

[0052] First, the learning processing unit 140 performs a learning process using first training data to generate a first object detection model (S100).

[0053] Next, the inference processing unit 141 inputs the first image to the first object detection model and executes the inference processing (S101). The inference processing unit 141 causes the storage unit 15 to store the obtained detection result.

[0054] Next, the annotation unit 142 displays an image in which the detection result is superimposed on the first image on a display or the like (S102). The user checks the displayed image and specifies a box and a label of "masking" for the mask region. The user may check the displayed image and perform annotation to correct region information and labels of erroneously detected objects. The annotation unit 142 generates first annotation data including region information and labels for the mask region and corrected region information and labels, and stores the first annotation data in the storage unit 15. The annotation unit 142 generates second annotation data in which the label of "masking" and region information corresponding to the label of "masking" are deleted from the first annotation data, and stores the second annotation data in the storage unit 15.

[0055] The mask processing unit 143 generates a second image based on the first annotation data (S103). The mask processing unit 143 specifies a mask area from area information of an object labeled with "masking" included in the first annotation data. The mask processing unit 143 fills the specified mask area with an arbitrary color to generate a second image in which the object included in the mask area is masked. The mask processing unit 143 stores the generated second image in the storage unit 15.

[0056] The learning processing unit 140 generates second training data including the second annotation data and the second image. The learning processing unit 140 executes a re-learning process using the first training data and the second training data to generate a second object detection model (S104). The learning processing unit 140 stores the generated second object recognition model in the storage unit 15.

[0057] [Variations] Although the embodiments of the present invention have been described above, the present invention may be embodied in various different forms other than the above-described embodiments.

[0058] As an example, the mask processing unit 143 generates the second image by replacing the mask area with a third image. The process of replacing the mask area with the third image is an example of a masking process. As an example, the third image is an image of an arbitrary color. As an example, the color is a single color. As an example, the third image is an image of an arbitrary pattern. As an example, the third image is an image obtained by capturing an image of the mask area in a state where there is no object. FIG. 7 is an example of a second image masked with an image obtained by capturing an image of the mask area in a state where there is no object as the third image.

[0059] As an example, the mask processing unit 143 has a check function for checking whether or not an object detected by the first object detection model is present in the third image. An example of the check function will be described below. The inference processing unit 141 inputs the third image to the first object detection model and obtains a detection result for the third image of the first object detection model. The mask processing unit 143 refers to the detection result, and if the certainty of the object included in the detection result is lower than a threshold value, or if an object is not detected from the third image, determines that there is no problem in using the third image for masking, and generates a second image by replacing the mask area with the third image. If the certainty of the detected object is equal to or higher than a threshold value, the mask processing unit 143 determines that the third image is not to be used for masking. This allows the information processing device 10 to prevent the object detected by the first object detection model from being present in the third image, and suppresses deterioration of the detection accuracy of the second object detection model generated by re-learning.

[0060] The mask area may have a shape other than a rectangle. As an example, the mask area is a figure such as a circle or a triangle. As an example, the mask area is an arbitrary shape along the trajectory of the input from the user to the information processing device 10. The second image does not have to be generated by the mask processing unit 143 based on the first annotation data. As an example, the annotation unit 142 generates the second image by filling in an object included in the mask area along the trajectory of the input from the user using a mouse, a pen tablet, or the like. The process of filling in an object present in the mask area along the trajectory of the input from the user using a mouse, a pen tablet, or the like is an example of a masking process. As an example, the annotation unit 142 has an annotation mode in which the user performs annotation to correct the detection result, and a masking mode in which the user masks the mask area. As an example, in the masking mode, when the mask area is specified by the user, the annotation unit 142 masks an area corresponding to the mask area of ​​the first image and generates a second image. As an example, in the masking mode, the annotation unit 142 fills in an object included in a mask area along a trajectory of an input from a user to the information processing device 10 using a mouse, a pen tablet, or the like, to generate a second image.

[0061] As an example, the learning processing unit 140 generates a second object detection model using the second training data without using the first training data. This allows the information processing device 140 to reduce the annotation workload even when generating an object detection model without using the first training data.

[0062] As an example, the information processing device 10 accepts a designation of a mask region from a user for an image used in the learning phase, generates an image in which an object included in the mask region is masked, and generates a first object detection model by machine learning using the masked image. An example of a process of generating a first object detection model by machine learning using a masked image will be described below. The annotation unit 142 displays an image used in the learning phase on a screen such as a display. The user checks the displayed image and performs annotation to specify region information and a label for an object included in the image. The user checks the displayed image and performs annotation to specify region information and a label for the mask region. The annotation unit 142 accepts designation of an object and a mask region from a user, and generates annotation data reflecting the designation of the object and the mask region. The mask processing unit 143 acquires region information of the mask region from the generated annotation data, and generates an image in which an object included in the mask region is masked. The annotation unit 142 deletes region information and a label for the mask region from the generated annotation data, and generates annotation data for the object. The learning processing unit 140 generates a first object detection model by using first training data including a masked image and annotation data for the object. This allows the information processing device 10 to reduce the annotation workload even in the annotation during the learning phase.

[0063] As an example, the information processing device 10 may acquire all or a part of the first training data, the first object detection model, the detection result, the first annotation data, the second annotation data, the second image, and the second training data generated by an external device other than the information processing device 10 via the communication unit 11, and store them in the memory unit 15.

[0064] [effect] As described above, the information processing device 10 inputs the first image to be added in the re-learning to the trained first object detection model, executes inference processing, and acquires a detection result. The information processing device 10 displays the first image and the acquired detection result on a display or the like, and accepts a designation of a mask region from a user. The information processing device 10 generates a second image in which an object included in the mask region is masked. The information processing device 10 executes re-learning using the generated second image. In this way, the information processing device 10 can execute re-learning using an image in which an object that is erroneously detected by the first object detection model or an object with a low priority for re-learning is excluded. This allows the information processing device 10 to suppress deterioration of the detection accuracy of the second object detection model generated by re-learning. The information processing device 10 can reduce the workload of annotation for objects included in the mask region in the re-learning.

[0065] The information processing device 10 may accept annotations for correcting the detection results together with the designation of the mask region. This enables the information processing device 10 to increase the number of training targets for re-learning, and suppress deterioration of the detection accuracy of the second object detection model.

[0066] The information processing device 10 may generate the second image by filling the mask region with an arbitrary single color, which allows the information processing device 10 to efficiently generate the second image by general image processing.

[0067] The information processing device 10 may generate the second image by replacing the mask area with a third image obtained by capturing the mask area in a state where there is no object. This allows the information processing device 10 to perform re-learning using an image that is close to the captured object.

[0068] The information processing device 10 may have a function of checking whether or not an object detected by the first object detection model is present in the third image. This enables the information processing device 10 to prevent an object detected by the first object detection model from being present in the third image, and suppress deterioration of the detection accuracy of the second object detection model.

[0069] When object detection is performed using an outdoor image such as a road as in this embodiment, false detection may occur due to changes in the shooting environment due to seasons and time periods or changes in the situation at the site, and re-learning may be performed frequently. In addition, when the first object detection model begins to be used at the site, re-learning may be performed by a user at the site, not a machine learning expert such as a vendor. Even in such a case, the user can generate a second image with a simple operation using the information processing device 10, and a second object detection model in which deterioration of detection accuracy is suppressed can be generated. The information processing device 10 can reduce the annotation workload of the user at the site.

[0070] [Hardware] Fig. 8 is a diagram illustrating an example of a hardware configuration of an information processing device according to an embodiment. An example of a hardware configuration of the information processing device 10 will be described with reference to Fig. 8. As illustrated in Fig. 8, the information processing device 10 includes, as an example, a CPU 100, a RAM 101, an input / output interface 102, a communication interface 103, and a HDD 104 as components, and these components are connected via a bus 105.

[0071] The CPU 100 is a processor that operates a process that executes each function of the control unit 14. Specifically, the CPU 100 reads out a program that executes the same functions as the learning processing unit 140, the inference processing unit 141, the annotation unit 142, and the mask processing unit 143 from the HDD 104 or the like, develops it in the RAM 101, and executes a process that executes the functions of the learning processing unit 140, the inference processing unit 141, the annotation unit 142, and the mask processing unit 143. The CPU 100 may obtain the above programs and data used when executing the above programs from a medium reading device, or may obtain them via the communication interface 103. The CPU 100 may have one or more processor cores. The information processing device 10 may be provided with a processor other than a CPU, or may be provided with multiple types of processors.

[0072] The RAM 101 operates as a main storage device of the information processing device 10, and stores programs read from an auxiliary storage device such as the HDD 104, data used to execute the programs, etc. The information processing device 10 may include a memory other than the RAM, or may include multiple memories.

[0073] The input / output interface 102 is an interface that inputs signals to the information processing device 10 and outputs signals from the information processing device 10. As an example, the input / output interface 102 receives signals from input devices such as a keyboard or a mouse connected to the information processing device 10, and transmits signals such as images to output devices such as a display connected to the information processing device 10. A plurality of input devices and output devices may be connected to the information processing device 10 via the input / output interface 102.

[0074] The communication interface 103 is an interface for connecting the information processing device 10 to a network. As an example, the standard of the communication interface 103 may be a communication standard for a wired LAN such as Ethernet (registered trademark), a communication standard for a wireless LAN such as WiFi (registered trademark), or a wireless standard for mobile communication such as local 5G.

[0075] The HDD 104 operates as an auxiliary storage device for storing programs for executing the functions of the operating system (OS) and the control unit 14 of the information processing device 10 and data used by the programs. The information processing device 10 may be provided with an auxiliary storage device other than an HDD, such as a solid state drive (SSD), or may be provided with multiple auxiliary storage devices.

[0076] Although the embodiments of the present invention have been described above, the embodiments of the present invention are not limited to those described above. The present invention may be embodied in various different forms other than the above-described embodiments.

[0077] The processing procedures, processing methods, names of parts, various data, etc. shown in the embodiments and drawings are merely examples and may be changed arbitrarily unless otherwise specified.

[0078] The functional configurations and hardware configurations shown in the embodiments and drawings are merely examples, and therefore may not necessarily be configured or arranged as shown in the drawings, and may be changed as desired unless otherwise specified. For example, the components of the functional configurations and hardware configurations may be distributed or integrated in any unit, such as a function.

[0079] The information processing program according to the embodiment is not limited to being executed by the information processing device 10. For example, the information processing program according to the embodiment may be executed across a plurality of devices.

[0080] The information processing program according to the embodiment may be distributed via a network such as the Internet. The information processing program according to the embodiment may be recorded on a computer-readable recording medium and sold. Examples of the computer-readable recording medium include a Compact Disc Read Only Memory (CD-ROM), a Digital Versatile Disc (DVD), a Universal Serial Bus Memory (USB memory), a floppy disk, and a Magneto-Optical Disk (MO). The information processing program according to the embodiment may be read from the computer-readable recording medium by a computer and executed by the computer. [Explanation of symbols]

[0081] 10. Information processing device 11 Communications Department 12 Input section 13 Output section 14 Control section 15 Storage section 20 Learning Phase 21 Relearning Phase 140 Learning processing unit 141 Inference processing unit 142 Annotation Section 143 Mask processing section 150 First training data DB 151 First Object Detection Model DB 152 First Image DB 153 Detection result DB 154 The first annotation data DB 155 Second Annotation Data DB 156 Second Image DB 157 Second training data DB 158 Second Object Detection Model DB

Claims

1. Obtaining a detection result for a first image of a first object detection model that detects an object; when a designation for a portion of an area included in the first image is accepted according to the detection result, a second image is generated from the first image in which the object included in the portion of the area is masked; A second object detection model for detecting the object is generated by machine learning using the second image. An information processing program that causes a computer to carry out a process.

2. Accepting a correction of the detection result for a part of the objects included in the first image in accordance with the detection result; The process of generating the second object detection model is a process of generating the second object detection model by machine learning using the second image and data generated based on the correction.

2. The information processing program according to claim 1,

3. The process of generating the second image is a process of generating the second image by filling the partial area with a single color.

2. The information processing program according to claim 1,

4. The process of generating the second image is a process of generating the second image by replacing the partial area with a third image obtained by capturing an image of the partial area in a state in which the object is not present.

2. The information processing program according to claim 1,

5. Obtaining a detection result for a third image of the first object detection model; The process of generating the second image is a process of generating the second image by replacing the partial area with the third image when a confidence level included in the detection result for the third image is lower than a threshold value or when the object is not detected from the third image.

2. The information processing program according to claim 1,

6. Obtaining a detection result for a first image of a first object detection model that detects an object; when a designation for a portion of an area included in the first image is accepted according to the detection result, a second image is generated from the first image in which the object included in the portion of the area is masked; A second object detection model for detecting the object is generated by machine learning using the second image. An information processing method, characterized in that the processing is carried out by a computer.

7. Obtaining a detection result for a first image of a first object detection model that detects an object; when a designation for a portion of an area included in the first image is accepted according to the detection result, a second image is generated from the first image in which the object included in the portion of the area is masked; A second object detection model for detecting the object is generated by machine learning using the second image. An information processing device comprising a control unit.

Images