Information processing device, information processing method, and recording medium

Abstract

An information processing device according to the present disclosure comprises an image analysis unit, an extraction unit, a generation unit, and a learning unit. The image analysis unit uses a machine learning model to detect a designated object from processed images that are obtained by imaging a predetermined place. The extraction unit extracts, on the basis of a detection result based on the processed images, a non-detection image that is a processed image including objects other than the designated object. The generation unit generates teacher data on the basis of the non-detection image. The learning unit retrains the machine learning model using the teacher data.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] A technology related to this disclosure is disclosed in Patent Document 1. The technology disclosed in Patent Document 1 extracts face images whose similarity to labeled face images is equal to or greater than a threshold from a large number of unlabeled face images, and generates training data by assigning new labels to the extracted face images.

[0003] Japanese Patent Application Laid-Open No. 2022-180959

[0004] A technology for detecting a specified object using a machine learning model generated by machine learning is widely used. To improve the accuracy of detecting a specified object in this technology, it is preferable to learn a larger amount of training data. However, generating a large amount of training data requires a lot of time and effort. In the case of the technology disclosed in Patent Literature 1, a large amount of time and effort is required to prepare a large number of unlabeled face images.

[0005] One example of the purpose of this disclosure is to provide a new technology for generating training data for learning in a machine learning model.

[0006] According to one aspect of this disclosure, there is provided an information processing device having: an image analysis means for detecting a specified object from a processed image taken of a predetermined location using a machine learning model; an extraction means for extracting a non-detection image, which is the processed image in which an object is captured but the specified object is not detected, based on a detection result based on the processed image; a generation means for generating training data based on the non-detection image; and a learning means for re-training the machine learning model using the training data.

[0007] Furthermore, according to one aspect of this disclosure, an information processing method is provided in which one or more computers use a machine learning model to detect a specified object from a processed image taken of a predetermined location, extract non-detection images, which are processed images in which an object is captured but the specified object is not detected, based on the detection results based on the processed images, generate training data based on the non-detection images, and re-train the machine learning model using the training data.

[0008] Furthermore, according to one aspect of this disclosure, there is provided a program that causes a computer to function as: an image analysis means that uses a machine learning model to detect a specified object from a processed image taken of a predetermined location; an extraction means that extracts a non-detection image, which is the processed image in which an object is captured but the specified object is not detected, based on a detection result based on the processed image; a generation means that generates training data based on the non-detection image; and a learning means that re-trains the machine learning model using the training data.

[0009] According to one example of this disclosure, a new technology is realized for generating training data for learning in a machine learning model.

[0010] FIG. 1 is a diagram illustrating an example of a functional block diagram of an information processing device. FIG. 2 is a flowchart illustrating an example of a processing flow of the information processing device. FIG. 3 is a diagram illustrating an example of a hardware configuration of the information processing device. FIG. 4 is a diagram illustrating an example of a processing executed by the information processing device. FIG. 5 is a flowchart illustrating another example of a processing flow of the information processing device. FIG. 6 is a diagram illustrating another example of a processing executed by the information processing device. FIG. 7 is a flowchart illustrating another example of a processing flow of the information processing device. FIG. 8 is a diagram illustrating another example of a functional block diagram of an information processing device. FIG. 9 is a flowchart illustrating another example of a processing flow of the information processing device. FIG. 10 is a diagram illustrating another example of a functional block diagram of an information processing device. FIG. 11 is a flowchart illustrating another example of a processing flow of the information processing device. FIG. 12 is a diagram illustrating another example of a functional block diagram of an information processing device. FIG. 13 is a flowchart illustrating another example of a processing flow of the information processing device.

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are designated by similar reference numerals, and descriptions thereof will be omitted as appropriate.

[0012] First Embodiment Fig. 1 is a functional block diagram showing an overview of an information processing device 10. Fig. 2 is a flowchart showing an example of the flow of processing executed by the information processing device 10.

[0013] 1, the information processing device 10 includes an image analysis unit 11, an extraction unit 12, a generation unit 13, and a learning unit 14. These functional units execute the processing of the flowchart in FIG.

[0014] In S10, the image analysis unit 11 uses a machine learning model to detect a specified object from a processed image captured of a predetermined location. In S11, the extraction unit 12 extracts non-detection images, which are processed images in which an object is captured but the specified object is not detected, based on the detection results based on the processed image in S10. In S12, the generation unit 13 generates training data based on the non-detection images extracted in S11. In S13, the learning unit 14 re-trains the machine learning model used in S10 using the training data generated in S12.

[0015] In this way, the information processing device 10 can extract non-detection images from "processed images that have actually been processed to detect a specified object using a machine learning model" and generate training data based on the extracted non-detection images. The machine learning model processes a large number of images during actual operation (hereinafter sometimes simply referred to as "actual operation") to detect a specified object from images taken at a predetermined location, or during a test stage before actual operation. The information processing device 10 can generate training data based on non-detection images extracted from a large number of images that have actually been processed by such a machine learning model. As a result, training data can be generated efficiently.

[0016] In one example, the information processing device 10 can generate training data based on images (processed images) actually processed in the actual operation and the detection results thereof while performing actual operation using the machine learning model. The generated training data can then be used to retrain the machine learning model. Such an information processing device 10 can efficiently improve the detection accuracy of the machine learning model for a specified object while performing actual operation using the machine learning model. With this configuration, the more the machine learning model is used in a certain location, the more the detection accuracy at that location improves.

[0017] However, when images for training a machine learning model are collected by any means, there is a possibility that images unsuitable for training the machine learning model will be collected. Images unsuitable for training the machine learning model are, for example, images captured under different conditions (camera orientation (elevation and depression angles, azimuth angles), distance between the camera and the subject, various camera settings, lighting conditions, etc.) than images processed during actual operation using the machine learning model. Furthermore, images unsuitable for training the machine learning model are, for example, images that contain a specified object that will not appear in images processed during actual operation using the machine learning model. Training based on such images unsuitable for training the machine learning model not only wastes time and effort, but may also affect the detection accuracy of the specified object.

[0018] In contrast, the above-mentioned inconvenience can be alleviated by using the information processing device 10, which generates new training data based on images actually processed during actual operation using a machine learning model, as in the above example. From this perspective, it can be said that with the information processing device 10, the more a machine learning model is used in a certain location, the more the detection accuracy in that location improves.

[0019] Furthermore, the information processing device 10 can perform processing to detect a designated object from a processed image using a machine learning model and generate training data based on the results. Specifically, the information processing device 10 extracts "processed images in which a designated object is not detected by the machine learning model" based on the processing results, and generates training data based on the extracted processed images. The information processing device 10 then retrains the machine learning model using the generated training data.

[0020] According to this information processing device 10, it is possible to generate training data based on processed images that show a specified object that cannot be detected by the machine learning model, or processed images that show a specified object in a manner (orientation, size, etc.) that cannot be detected by the machine learning model. The machine learning model can then be retrained using the training data generated in this manner. According to the information processing device 10, rather than performing a large amount of training indiscriminately, retraining can be performed to compensate for parts that cannot be detected by the machine learning model. As a result, it is possible to efficiently improve the detection accuracy of the specified object by the machine learning model.

[0021] Furthermore, the information processing device 10 can generate training data based on "processed images in which an object is captured but the designated object is not detected (non-detection images)" among "processed images in which the designated object is not detected by the trained machine learning model." Processed images in which the designated object is not detected by the trained machine learning model also include processed images in which the designated object is not captured in the first place. Processed images in which the designated object is not captured are not suitable as training data.

[0022] The above-mentioned inconvenience can be alleviated by narrowing down the images for generating training data to "processed images (non-detection images) that contain an object but in which the specified object was not detected," as in the case of the information processing device 10. Note that the presence of an object in a processed image can be realized by using techniques such as object detection by image analysis or linking with the results of object detection using a sensor.

[0023] In this way, the information processing device 10 realizes a new technology for generating training data for learning by a machine learning model.

[0024] <<Second Embodiment>> <Overview> An information processing device 10 according to a second embodiment uses a distinctive technique to extract, from processed images, "processed images in which an object is captured but a specified object is not detected (non-detection images)" that are used to generate training data. Specifically, the information processing device 10 extracts the non-detection images based on "the results of a process for detecting a specified object from processed images captured of a specified location using a machine learning model" and "the results of a process for detecting an object present in the specified location using a sensor." This will be described in detail below.

[0025] <Hardware Configuration> First, an example of the hardware configuration of the information processing device 10 will be described. Each functional unit of the information processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. The software includes programs that are pre-stored in the device before shipping, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0026] FIG. 3 is a block diagram illustrating an example of the hardware configuration of an information processing device 10. As shown in FIG. 3, the information processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The information processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the information processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices may have the above hardware configuration.

[0027] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a central processing unit (CPU) or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, and touch panel. Examples of output devices include a display, projection device, speaker, printer, and mailer. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0028] <Functional Configuration> Next, a detailed description will be given of the functional configuration of the information processing device 10. Fig. 1 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an image analysis unit 11, an extraction unit 12, a generation unit 13, and a learning unit 14.

[0029] The image analysis unit 11 uses a machine learning model to detect a specified object from a processed image of a predetermined location.

[0030] A "machine learning model" is a model generated by machine learning, which detects a specified object from an input image. The machine learning model is composed of, for example, a classifier.

[0031] A "specified object" is an object detected by a machine learning model. The machine learning model is trained to detect the specified object from an input image. For example, the specified object may be, but is not limited to, a person, a non-human organism, an artificial object (vehicle, clothing, belongings, etc.), a natural object, etc. The specified object may be more specific. For example, if the specified object is a person, the specified object may be more specific using gender, age, nationality, hairstyle, body type, etc. (e.g., a man in his 40s). Furthermore, if the specified object is an object, the specified object may be more specific using color, shape, material, etc. (e.g., a black handbag).

[0032] The machine learning model may be trained to detect one type of specified object (e.g., a person), or may be trained to detect multiple types of specified objects (e.g., a person, a motorcycle, a car, a bus, a truck).

[0033] A "predetermined location" is a location where a camera is installed to take an image. Various locations can be the predetermined location. The predetermined location may be a public location or a private location. The predetermined location may also be outdoors or indoors. Furthermore, the predetermined location does not have to be fixed to a specific location, but may be attached to a mobile object or the like, and the camera may move as the mobile object moves. In this case, the location photographed by the camera attached to the mobile object is the predetermined location. Note that a mobile object may be, for example, a car, a ship, a drone, etc.

[0034] The predetermined location may be a location where a camera is installed to capture images when the machine learning model is actually used to detect a specified object (hereinafter referred to as an "operation location"). For example, the operation location may be a road, an entrance / exit to a facility (such as an entrance gate or an exit gate), or a location requiring high security.

[0035] Alternatively, the predetermined location may be a location different from the operational location. For example, the predetermined location may be a location different from the operational location that is selected for retraining the machine learning model.

[0036] A "processed image" is an image that the image analysis unit 11 processes using a machine learning model. The processed image is an image generated by a camera installed to capture a predetermined location. In one example, the processed image is an image generated by a camera installed to capture an image of an operation location. The processed image may be an image captured at the operation location during a test phase before actual operation using the machine learning model begins. Alternatively, the processed image may be an image captured at the operation location during actual operation using the machine learning model.

[0037] In another example, the processed image is an image generated by a camera installed to capture an image of a location other than the operation location. In this case, the processed image is preferably an image captured under the same shooting conditions (camera orientation (elevation and depression angles, azimuth angles), distance between the camera and the subject, various camera settings, lighting conditions, etc.) as the image generated by the camera capturing the operation location in actual operation. In other words, the camera capturing an image of a location other than the operation location is preferably installed, configured, etc. to capture an image under the same shooting conditions (camera orientation (elevation and depression angles, azimuth angles), distance between the camera and the subject, various camera settings, lighting conditions, etc.) as the camera capturing the operation location in actual operation.

[0038] The processed image may be a still image or a moving image, or may be an image generated by detecting visible light or other electromagnetic waves such as infrared light.

[0039] When the image analysis unit 11 acquires the processed image, it inputs the acquired processed image into a machine learning model to detect a specified object from within the processed image.

[0040] "Acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition); and a device inputting data or information output from another device (passive acquisition). Examples of active acquisition include making a request to another device and receiving a response, and accessing and reading information from another device or storage medium. An example of passive acquisition is receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, acquisition may be selecting and acquiring from received data or information, or selecting and receiving distributed data or information.

[0041] The extraction unit 12 extracts non-detected images based on the detection results of the processed images by the image analysis unit 11 and the detection results by the detection means.

[0042] A "non-detected image" is a part or all of a processed image, which is a "processed image in which an object is captured but in which the specified object was not detected in processing using a machine learning model."

[0043] The "detection result of the processed image by the image analysis unit 11" indicates whether or not a specified object has been detected by processing using a machine learning model from each of a plurality of processed images generated by photographing a predetermined location at each of a plurality of timings. The detection result may be accompanied by information on the photographing date and time of each processed image.

[0044] The "detection means" detects an object present at the predetermined location using a process different from the detection of a specified object based on a machine learning model. In one example, the detection means can detect an object present at the predetermined location using an object detection sensor. Examples of the object detection sensor include, but are not limited to, a photoelectric sensor, a laser sensor, a proximity sensor, an ultrasonic sensor, a weight sensor, and a vibration sensor.

[0045] The "detection result by the detection means" indicates whether or not an object was detected at a predetermined location by the detection means at each of a plurality of timings. The detection result may be accompanied by date and time information for each of the plurality of timings.

[0046] The extraction unit 12 extracts non-detection images based on the detection results of the processed images by the image analysis unit 11 and the detection results by the detection means. Specifically, the extraction unit 12 extracts, as non-detection images, processed images that were taken at the time an object was detected by the detection means and in which the specified object was not detected by the processing using the machine learning model. Note that the extraction unit 12 can time-synchronize the two detection results based on the date and time information assigned to each of the above-mentioned detection results.

[0047] The extraction process by the extraction unit 12 will now be described with reference to Fig. 4. In Fig. 4, a plurality of processed images F are generated by photographing a predetermined location at each of a plurality of timings. １ ~F １５ Then, by processing using the machine learning model, each processed image F １ ~F １５ The processed image F indicates whether or not the designated object has been detected (the detection result of the processed image by the image analysis unit 11). １ ~F ３ It is shown that the specified object has been detected. １２ ~F １５ indicates that the specified object was not detected.

[0048] 4 also shows whether or not an object has been detected at a predetermined location by the detection means at each of a plurality of timings (detection results by the detection means). １ ~F ３、 F １２ It is shown that an object was detected at the time when the processed image F １３ ~F １５ This shows that no object was detected at the time the image was taken.

[0049] In this example, the extraction unit 12 extracts a processed image F captured at the time when the object is detected by the detection unit. １ ~F ３、 F １２ In the processed image F, the specified object was not detected by the machine learning model. １２ is extracted as a non-detection image.

[0050] 1, the generation unit 13 generates training data based on the non-detection images extracted by the extraction unit 12. The generation unit 13 can generate training data by attaching a label to the non-detection images.

[0051] Additionally, the generation unit 13 may process the non-detection image to generate a processed image, and generate training data by attaching a label to the processed image. In one example, the generation unit 13 may process the non-detection image using a generative AI (artificial intelligence) technique. For example, the generation unit 13 may generate a plurality of processed images by changing the color, size, angle, material, etc. of an object appearing in the non-detection image, and may attach a label to each of the plurality of processed images to generate training data.

[0052] In addition, the generation unit 13 may perform object detection on the non-detection image, cut out the part where the detected object is located, enlarge the cut-out part, or make the part other than the part where the detected object is located a single color such as white or black to generate a processed image.

[0053] The generation unit 13 can employ various methods for assigning labels. For example, the generation unit 13 may output a non-detected image or an edited image to a user and receive input from the user specifying a label. Alternatively, the generation unit 13 may assign labels using generative AI technology. For example, the generation unit 13 may input the non-detected image or the edited image and a pre-prepared prompt, such as "Please tell me the name of the object in the image," into an LLM (Large Language Model) equipped with image recognition functionality, and use the name of the object included in the output result as the label. Alternatively, the LLM may acquire characteristic words included in the prompt entered by the user and generate a label based on the word. The generated label may then be associated with the non-detected image or the edited image.

[0054] The learning unit 14 retrains the machine learning model using the training data generated by the generation unit 13. The machine learning model to be retrained is the machine learning model used by the image analysis unit 11 to detect a specified object from within a processed image. Retraining of the machine learning model is achieved using widely known techniques.

[0055] Next, an example of the process flow of the information processing device 10 will be described using the flowcharts of Figures 2 and 5. Note that the purpose here is to explain the process flow. Details of each process have been described above, so a description thereof will be omitted here.

[0056] First, the information processing device 10 detects a specified object from a processed image captured at a predetermined location using a machine learning model (S10). Next, the information processing device 10 extracts non-detection images, which are processed images in which an object is captured but the specified object is not detected, based on the detection results based on the processed image in S10 (S11).

[0057] Next, the information processing device 10 generates training data based on the non-detection images extracted in S11 (S12).Then, the information processing device 10 retrains the machine learning model used in S10 using the training data generated in S12 (S13).

[0058] Next, the process flow of S11 will be described with reference to the flowchart of FIG.

[0059] First, the information processing device 10 identifies the timing at which the detection device detected an object at a predetermined location based on the detection result of the detection device (S20). The information processing device 10 can acquire the detection result of the detection device by any means (communication with the detection device, input by a user, etc.). The information processing device 10 may acquire the detection result of the detection device by real-time processing or by batch processing.

[0060] Next, the information processing device 10 extracts, as a non-detection image, a processed image that was captured at a predetermined location at the timing identified in S20 and in which the designated object was not detected in S10 (S21).

[0061] The information processing device 10 may perform real-time processing of the detection result obtained in S10 and execute S11 to S13. Alternatively, the information processing device 10 may perform batch processing of the detection result obtained in S10 and execute S11 to S13.

[0062] Other configurations of the information processing device 10 can be the same as those in the first embodiment.

[0063] <Effects> The information processing device 10 of the second embodiment can achieve the same effects as the information processing device 10 of the first embodiment. Furthermore, the information processing device 10 can extract a "processed image (non-detection image) in which an object is captured but the specified object is not detected" based on the result of detecting an object present in a predetermined location using a process different from the detection of the specified object based on a machine learning model. For example, the information processing device 10 can extract a non-detection image based on the result of detecting an object present in a predetermined location using a sensor.

[0064] According to the information processing device 10, it is possible to extract with high accuracy "processed images in which an object is photographed but the specified object is not detected (non-detection images)."

[0065] <<Third Embodiment>> <Overview> An information processing device 10 of a third embodiment uses a distinctive method different from that of the second embodiment to extract, from processed images, "processed images in which an object is captured but a specified object is not detected (non-detection images)" that are used to generate training data. Specifically, the information processing device 10 extracts non-detection images based on "the results of processing to detect a specified object from processed images using a machine learning model" and "the results of processing to detect an object from processed images using object detection technology." This will be described in detail below.

[0066] <Hardware Configuration> The hardware configuration of the information processing apparatus 10 of the third embodiment can be the same as the hardware configuration described in the second embodiment.

[0067] <Functional Configuration> Next, a detailed description will be given of the functional configuration of the information processing device 10. Fig. 1 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an image analysis unit 11, an extraction unit 12, a generation unit 13, and a learning unit 14.

[0068] The image analysis unit 11 detects an object in the processed image using an object detection technique. The image analysis unit 11 can use various object detection techniques. For example, the image analysis unit 11 may detect an object included in the processed image (an object that does not exist in the background image) by detecting a difference between a background image prepared in advance and the processed image.

[0069] The image analysis unit 11 may divide the background image and the processed image into a plurality of areas and detect the difference for each area, thereby further specifying the position of the detected object within the image.

[0070] The division rule (16 divisions, 36 divisions, etc.) is defined in advance and stored in the information processing device 10. The image analysis unit 11 divides the background image and the processed image into multiple areas according to the rule. In one example, the division rule (16 divisions, 36 divisions, etc.) is defined in advance so that the entirety or most of the designated object fits into one area. If the camera capturing the specified location is a fixed camera, the range of sizes of the designated object captured in the image generated by the camera can fit within a certain range. The division rule can be defined in advance based on that range.

[0071] After detecting the object in the processed image, the image analysis unit 11 processes the processed image in which the object has been detected using a machine learning model to detect the specified object in the processed image. The process of detecting the specified object in the processed image using the machine learning model is as described in the second embodiment.

[0072] In addition, when the background image and the processed image are divided into multiple areas, the image analysis unit 11 may detect the specified object based on the image within the area where the object is detected. For example, the image analysis unit 11 may perform a predetermined process (e.g., converting the image to a single color such as white or black) on the areas other than the area where the object is detected, and then process the processed image using a machine learning model to detect the specified object. Alternatively, the image analysis unit 11 may cut out the area where the object is detected, perform a process such as enlarging the cut-out image, and then process the processed image using a machine learning model to detect the specified object.

[0073] The extraction unit 12 extracts, as a non-detection image, a processed image in which an object is detected but in which the specified object is not detected.

[0074] 6, the information processing device 10 performs object detection on a group of processed images and extracts a group of processed images in which an object has been detected from the group of processed images. Then, the information processing device 10 performs designated object detection on the group of processed images in which an object has been detected, and extracts a group of processed images in which the designated object has not been detected from the group of processed images in which an object has been detected as non-detected images.

[0075] Next, an example of the flow of processing by the information processing device 10 will be described using the flowchart in Figure 7. Note that the purpose here is to explain the flow of processing. Details of each process have been described above, so description here will be omitted.

[0076] First, the information processing device 10 uses an object detection technique to detect an object from a processed image of a predetermined location (S30). Next, the information processing device 10 uses a machine learning model to detect a specified object from the processed image in which the object has been detected (S31).

[0077] Next, the information processing device 10 extracts, as non-detection images, processed images in which an object was detected but the specified object was not detected (S32). Next, the information processing device 10 generates training data based on the non-detection images extracted in S32 (S33). Then, the information processing device 10 retrains the machine learning model used in S31 using the training data generated in S33 (S34).

[0078] The information processing device 10 may perform real-time processing of the detection results obtained in S30 and S31 and execute S32 to S34. Alternatively, the information processing device 10 may perform batch processing of the detection results obtained in S30 and S31 and execute S32 to S34.

[0079] Other configurations of the information processing device 10 can be the same as those in the first and second embodiments.

[0080] <Effects> According to the information processing device 10 of the third embodiment, it is possible to achieve the same effects as the information processing device 10 of the first and second embodiments. Furthermore, the information processing device 10 can extract "processed images in which an object is photographed but the specified object was not detected (non-detected images)" based on the result of detecting an object from a processed image using object detection technology.

[0081] According to the information processing device 10, it is possible to extract with high accuracy "processed images in which an object is photographed but the specified object is not detected (non-detection images)."

[0082] <<Fourth Embodiment>> <Overview> An information processing device 10 according to a fourth embodiment collects “information related to a specified object not detected by a machine learning model” based on an extracted non-detected image, and generates training data based on the collected information. This will be described in detail below.

[0083] <Hardware Configuration> The hardware configuration of the information processing apparatus 10 of the fourth embodiment can be the same as the hardware configuration described in the second embodiment.

[0084] <Functional Configuration> Next, a detailed description will be given of the functional configuration of the information processing device 10. Fig. 8 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an image analysis unit 11, an extraction unit 12, a generation unit 13, a learning unit 14, and a search unit 15.

[0085] The search unit 15 searches the open data for relevant information about the undetected object.

[0086] An "undetected object" is a specified object that was not detected in a processed image by processing using a machine learning model. More specifically, an undetected object is an object that appears in an undetected image.

[0087] "Open data" is data that is made publicly available for free use. Open data may be data that is made public on the Internet, data that is made public within a local network, or other such data.

[0088] The search unit 15 can search for related information about non-detected objects from such open data. Various search algorithms are available, and widely known technologies can be used. For example, the search unit 15 may perform a search using a non-detected image as a search query. Alternatively, the search unit 15 may generate keywords from the non-detected image and perform a search using the generated keywords as a search query. There are various means for generating keywords from a non-detected image. For example, the search unit 15 may input the non-detected image and a pre-prepared prompt such as "Please tell me the keywords to search for objects in the image" to an LLM equipped with an image recognition function, and obtain keywords included in the output results.

[0089] "Related information" includes at least one of images and text data. Images (related information) included in the search results are expected to be images including undetected objects, images including objects similar to undetected objects, images taken under similar shooting conditions to the undetected images, etc. Text data (related information) included in the search results is text described in pages or files found in a search using the search query. The text data (related information) included in the search results is expected to include, for example, characteristics of the undetected objects (color, size, material, brand, etc.). The search unit 15 can extract various characteristics of the undetected objects from the text data (related information) included in the search results and create a database.

[0090] The generating unit 13 can generate training data using the related information searched by the searching unit 15 .

[0091] In one example, the generation unit 13 can generate training data by attaching labels to images (related information) included in the search results. Alternatively, the generation unit 13 can process images (related information) included in the search results to generate processed images, and attach labels to the processed images to generate training data. The generation of processed images and the assignment of labels can be achieved by various means, and for example, the method described in the second embodiment can be adopted.

[0092] In another example, the generation unit 13 may generate a drawn image of a specified object based on at least one of a non-detected image, an image (related information) included in the search results, and text data (related information) included in the search results. The generation unit 13 may generate a two-dimensional image or a three-dimensional image. The generation of the drawn image can be achieved by various means. For example, the generation unit 13 may generate the drawn image by inputting the above-described information into an image generation tool generated using a generative AI technology.

[0093] The generation unit 13 can then generate training data based on the generated drawing image. For example, the generation unit 13 can generate training data by attaching a label to a two-dimensional drawing image. Alternatively, the generation unit 13 can process the two-dimensional drawing image to generate a processed image, and attach a label to the processed image to generate training data. The generation of the processed image and the attachment of a label can be achieved by various means, and for example, the method described in the second embodiment can be adopted.

[0094] Alternatively, the generation unit 13 may generate a two-dimensional drawing image showing a state in which a specified object is observed under predetermined conditions from the three-dimensional drawing image, and generate training data by labeling the generated two-dimensional drawing image. Furthermore, the generation unit 13 may process the two-dimensional drawing image generated from the three-dimensional drawing image to generate a processed image, and generate training data by labeling the processed image. The generation of the processed image and the labeling can be realized by various means, and for example, the method described in the second embodiment can be adopted.

[0095] The "specified conditions" in "observation under specified conditions" are the same as the shooting conditions (camera orientation (elevation and depression angles, azimuth angles), distance between the camera and the subject, various camera settings, lighting conditions, etc.) of the images processed during actual operation using the machine learning model. Lighting conditions indicate the type of light source (sunlight, indoor lighting, indirect lighting, etc.), light intensity, and the relative positions of the light source, subject, and camera, etc.

[0096] The information processing device 10 stores in advance the shooting conditions for an image to be processed during actual operation using the machine learning model. The generation unit 13 places a subject (a specified object shown in a three-dimensional rendered image) in a virtual space, captures the subject with a virtual camera in the virtual space, and generates a two-dimensional rendered image. The virtual camera preferably captures the subject in the virtual space under shooting conditions that are the same as or equivalent to the shooting conditions for the image to be processed during actual operation using the machine learning model. For example, the virtual camera has the same performance as a camera that captures the operation location during actual operation using the machine learning model, and can capture images under the same shooting conditions (camera orientation (elevation and depression angles, azimuth angles), distance between the camera and the subject, various camera settings, lighting conditions, etc.) as the camera.

[0097] Note that additional conditions to be imposed on the subject may be determined in advance and stored in the information processing device 10. Then, the generation unit 13 may place the subject (a designated object shown in a three-dimensional rendered image) in the virtual space so as to satisfy the additional conditions, and may capture an image of the subject in that state with a virtual camera to generate a two-dimensional rendered image.

[0098] The additional conditions can include at least one of the following: Position of the subject Posture of the subject Orientation of the subject relative to the virtual camera State of the subject (placed on the ground, placed on a desk, held in a person's hand, carried on a person's back, worn by a person, etc.) Characteristics of the object on which the subject is placed (type, color, shape, size, design, etc. of the desk) Attributes of the person holding the subject (held in a hand, carried on a person's back, worn, etc.) (gender, age, nationality, hairstyle, body type, etc.)

[0099] If the additional condition indicates a relationship between the subject and another object, the generation unit 13 further places the other object in the virtual space so that the relationship between the subject and the other object satisfies the additional condition. For example, the generation unit 13 places a person with attributes indicated by the additional condition in the virtual space and sets the state of the person and the subject to the state indicated by the additional condition (e.g., holding in hand). The generation unit 13 can then photograph the subject in that state with a virtual camera in the virtual space to generate a two-dimensional rendered image.

[0100] The background of the drawn image may have various contents. For example, the background of the drawn image may be a single color such as black or white, or may have a predetermined pattern. The environment of the operation location may also be reproduced in a virtual space. Then, a subject placed in that environment may be photographed with a virtual camera to generate the drawn image. In this case, objects present in the operation location (buildings, natural objects, roads, etc.) appear in the background of the drawn image.

[0101] As a modification of the fourth embodiment, the information processing device 10 can repeat the above-described process. That is, the search unit 15 can newly search for related information about the undetected object from the open data based on the drawn image generated by the generation unit 13. The generation unit 13 can generate a new drawn image based on the newly searched related information. Note that the generation unit 13 may generate a new drawn image using both the related information previously searched and the newly searched related information. Then, the generation unit 13 can generate training data based on the newly generated drawn image.

[0102] Furthermore, the generation unit 13 can generate a database by linking the generated drawn image with the characteristics (color, size, material, brand, etc.) of the undetectable object appearing in the drawn image. The characteristics of the undetectable object may be information included in the related information. Alternatively, the generation unit 13 may identify the characteristics of the undetectable object appearing in the drawn image by using an LLM with an image recognition function.

[0103] Next, an example of the flow of processing by the information processing device 10 will be described using the flowchart in Fig. 9. Note that the purpose here is to explain the flow of processing. Details of each process have been described above, so description thereof will be omitted here.

[0104] First, the information processing device 10 uses a machine learning model to detect a specified object from processed images captured of a predetermined location (S40). Next, the information processing device 10 extracts processed images in which an object is detected but in which the specified object is not detected as non-detection images (S41). The information processing device 10 can extract non-detection images using, for example, the method described in the second embodiment or the method described in the third embodiment.

[0105] Next, the information processing device 10 searches open data for related information about the undetected object based on the undetected image extracted in S41 (S42). Next, the information processing device 10 generates a drawn image of the specified object based on the related information searched in S42 (S43). Next, the information processing device 10 generates training data based on the drawn image generated in S43 (S44). Then, the information processing device 10 retrains the machine learning model used in S40 using the training data generated in S44 (S45).

[0106] The information processing device 10 may perform real-time processing of the detection result obtained in S40 and execute S41 to S45. Alternatively, the information processing device 10 may perform batch processing of the detection result obtained in S40 and execute S41 to S45.

[0107] Other configurations of the information processing device 10 can be the same as those of the first to third embodiments.

[0108] <Effects> According to the information processing device 10 of the fourth embodiment, it is possible to achieve the same effects as the information processing device 10 of the first to third embodiments. Furthermore, the information processing device 10 can collect related information of the non-detected object from open data based on the non-detected image, and generate training data based on the collected related information.

[0109] For example, the information processing device 10 can generate training data by assigning labels to images included in the collected related information.

[0110] Furthermore, the information processing device 10 can generate a drawn image of an undetectable object (designated object) based on the collected related information and generate training data based on the generated drawn image. According to the information processing device 10 that generates training data by drawing an image, an image of an undetectable object captured in a desired state can be generated and the desired image can be used as training data. For example, an image of an undetectable object captured under specified shooting conditions, an image of an undetectable object in a specified posture, or an image of an undetectable object facing a specified direction relative to the camera can be generated and used as training data. Additionally, an image of an undetectable object in a specified state can be generated and used as training data. Examples of undetectable objects in a specified state include an undetectable object placed on a desk with specified characteristics, an undetectable object held in the hand of a person with specified attributes, an undetectable object carried on the back of a person with specified attributes, an undetectable object worn by a person with specified attributes, etc.

[0111] Such information processing device 10 can easily generate a wide variety of training data. Furthermore, the information processing device 10 that generates training data based on a drawing image can easily acquire an image (drawing image) of an undetected object captured in a desired state, and can train a machine learning model based on the image.

[0112] <<Fifth Embodiment>> <Overview> The information processing device 10 of the fourth embodiment generates a search query based on a non-detected image and collects information related to a specified object based on the search query. The information processing device 10 of the fifth embodiment collects information related to a specified object based on a search query specified by a user. This will be described in detail below.

[0113] <Hardware Configuration> The hardware configuration of the information processing apparatus 10 of the fifth embodiment can be the same as the hardware configuration described in the second embodiment.

[0114] <Functional Configuration> Next, the functional configuration of the information processing device 10 will be described in detail. Fig. 10 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has a generation unit 13, a learning unit 14, and a search unit 15. Note that the information processing device 10 may further have an image analysis unit 11 and an extraction unit 12.

[0115] The search unit 15 searches open data for information related to the specified object based on a search query specified by the user.

[0116] The "user-specified search query" includes at least one of a keyword and an image. The user inputs a search query related to a specified object for which the user wants to improve the detection accuracy. For example, the user can input the type (bag, shoes, etc.), name (item name, etc.), characteristics (color, size, material, brand, etc.), image, etc. of the specified object as the search query.

[0117] The related information and open data are as described in the fourth embodiment.

[0118] The generating unit 13 can generate teacher data using the related information searched by the searching unit 15. The generating unit 13 can generate teacher data by the method described in the fourth embodiment.

[0119] As a modification of the fifth embodiment, the information processing device 10 can perform the following processing in addition to the above-described processing.

[0120] The search unit 15 can process a search query specified by a user and search for related information using the processed search query. Processing of a search query can be achieved by various means. For example, the search unit 15 may input the search query specified by the user and a pre-prepared prompt, such as "Please tell me a search query to search for variations of an object searched for with this search query," to the LLM, and obtain the search query included in the output results. In addition to the prompt, pre-prepared information such as "color variations, material variations, shape variations, etc." may also be input to the LLM.

[0121] Furthermore, the generation unit 13 can generate a database by linking the generated drawn image with the characteristics (color, size, material, brand, etc.) of the specified object depicted in the drawn image. The characteristics of the specified object may be information included in the related information. Alternatively, the generation unit 13 may identify the characteristics of the specified object depicted in the drawn image by using an LLM with an image recognition function.

[0122] Next, an example of the flow of processing by the information processing device 10 will be described using the flowchart in Fig. 11. Note that the purpose here is to explain the flow of processing. Details of each process have been described above, so description thereof will be omitted here.

[0123] First, the information processing device 10 searches open data for related information about the specified object based on a search query specified by the user (S50). Next, the information processing device 10 generates a drawn image of the specified object based on the searched related information (S51). Then, the information processing device 10 generates training data based on the drawn image (S52).

[0124] Other configurations of the information processing device 10 can be the same as those of the first to fourth embodiments.

[0125] <Effects> According to the information processing device 10 of the fifth embodiment, it is possible to achieve the same effects as the information processing device 10 of the first to fourth embodiments. In addition, the information processing device 10 can collect related information of a specified object from open data based on a search query specified by a user, and generate training data based on the collected related information.

[0126] For example, the information processing device 10 can generate training data by assigning labels to images included in the collected related information.

[0127] Furthermore, the information processing device 10 can generate a drawn image of a designated object based on the collected related information and generate training data based on the generated drawn image. According to the information processing device 10 that generates training data by drawing an image, an image of a designated object captured in a desired state can be generated and the desired image can be used as training data. For example, an image of a designated object captured under predetermined shooting conditions, an image of a designated object in a predetermined posture, or an image of a designated object facing a predetermined direction relative to the camera can be generated and used as training data. Additionally, an image of a designated object in a predetermined state can be generated and used as training data. Examples of a designated object in a predetermined state include a designated object placed on a desk with predetermined characteristics, a designated object held in the hand of a person with predetermined attributes, a designated object carried on the back of a person with predetermined attributes, and a designated object worn by a person with predetermined attributes.

[0128] Such information processing device 10 can easily generate a wide variety of training data. Furthermore, the information processing device 10 that generates training data based on a drawing image can easily obtain an image (drawing image) of a specified object captured in a desired state, and can train a machine learning model based on the image.

[0129] <<Sixth Embodiment>> <Overview> An information processing apparatus 10 according to a sixth embodiment has a function of determining the detection accuracy of a machine learning model trained by the methods according to the first to fifth embodiments. This will be described in detail below.

[0130] <Hardware Configuration> The hardware configuration of the information processing apparatus 10 of the sixth embodiment can be the same as the hardware configuration described in the second embodiment.

[0131] <Functional Configuration> Next, the functional configuration of the information processing device 10 will be described in detail. Fig. 12 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an image analysis unit 11, an extraction unit 12, a generation unit 13, a learning unit 14, and a determination unit 16. Note that the information processing device 10 may further have a search unit 15. Alternatively, the information processing device 10 may have the search unit 15 but not have the image analysis unit 11 and the extraction unit 12.

[0132] The determination unit 16 causes the image analysis unit 11 to execute "processing of detecting a designated object from a virtual image captured by placing the designated object in a virtual space that reproduces the operation location, using a machine learning model." Then, the determination unit 16 determines the detection accuracy of the machine learning model based on the result of this processing.

[0133] In the "virtual space recreating the operation location," objects (buildings, natural objects, roads, etc.) that exist in the operation location exist in the same positional relationship as the operation location. The determination unit 16 can generate such a virtual space by using so-called digital twin technology.

[0134] A "virtual image" is an image captured by a virtual camera of a specified object placed in a virtual space that recreates the operating location. The virtual camera captures the specified object in the virtual space under the same or equivalent shooting conditions as the shooting conditions for the image to be processed during actual operation using the machine learning model. For example, the virtual camera has the same performance as the camera that captures the operating location during actual operation using the machine learning model, and captures the image under the same shooting conditions as the camera (camera orientation (elevation and depression angles, azimuth angles), distance between the camera and the subject, various camera settings, lighting conditions, etc.).

[0135] The determination unit 16 places the designated object in the virtual space using a prepared 3D model of the designated object. A 3D model of one type of designated object may be generated in advance, or 3D models of multiple types of designated objects may be generated in advance. In one example, a user may generate a 3D model of the designated object and input it into the information processing device 10. For example, the user may generate the 3D model of the designated object using a tool such as 3D CAD (computer-aided design) or an image generation tool generated using generative AI technology. Alternatively, the determination unit 16 may use the "3D rendering image of the designated object generated from the related information of the designated object" described in the fourth and fifth embodiments as the 3D model of the designated object.

[0136] Various placement conditions may be imposed on the designated object to be placed in the virtual space, and the determination unit 16 may place the designated object in the virtual space so as to satisfy the placement conditions.

[0137] The placement conditions can include at least one of the following: - Position of the specified object - Attitude of the specified object - Orientation of the specified object relative to the virtual camera - State of the specified object (placed on the ground, placed on a desk, held in a person's hand, carried on a person's back, worn by a person, etc.) - Characteristics of the object on which the specified object is placed (type of desk, color, shape, size, design, etc.) - Attributes of the person holding (held in a person's hand, carried on a person's back, worn, etc.) the specified object (gender, age, nationality, hairstyle, body type, etc.) - Light intensity of the light source present in the virtual space

[0138] For example, an area in the virtual space where a designated object should be placed may be defined in advance and stored in the information processing device 10. The determination unit 16 can place the designated object at any position within that area.

[0139] Furthermore, if the placement condition indicates a relationship between the designated object and another object, the determination unit 16 further places the other object in the virtual space so that the relationship between the designated object and the other object satisfies the placement condition. For example, the determination unit 16 places a person with attributes indicated by the placement condition in the virtual space, and sets the state of the person and the designated object to the state indicated by the placement condition (e.g., held in hand).

[0140] Note that each of the multiple types of placement conditions described above may have multiple options. For example, the location where the designated object indicated by the placement condition should be placed may not be a single pinpoint location, but may include multiple locations. Furthermore, the orientation of the designated object indicated by the placement condition may be multiple, rather than just one. The determination unit 16 may then randomly or according to a predetermined rule select one of the multiple options and place the designated object in the virtual space so as to satisfy the selected orientation. In this manner, the manner in which the designated object is placed in the virtual space is not limited to one, but may include multiple various orientations. As a result, the designated object in the virtual space in multiple various orientations can be photographed with a virtual camera to generate multiple virtual images in various orientations. By assessing the detection accuracy of the machine learning model using such multiple virtual images in various orientations, the accuracy of the determination is improved.

[0141] The image analysis unit 11 executes, using each of the multiple virtual images, a process of detecting a designated object from a virtual image captured by placing the designated object in a virtual space that recreates the operating location, using a machine learning model. The image analysis unit 11 then inputs the results of this process to the determination unit 16.

[0142] The determination unit 16 determines the detection accuracy of the machine learning model based on the results of the above processing. Note that there are various ways to obtain the detection accuracy.

[0143] In one example, the determination unit 16 can calculate the ratio of the "number of virtual images in which the specified object is detected" to the "number of virtual images processed by the machine learning model" as the detection accuracy.

[0144] As a modified example, the determination unit 16 may generate a "virtual image captured without placing a designated object in the virtual space that recreates the operation location" in addition to a "virtual image captured with a designated object placed in the virtual space that recreates the operation location." The "virtual image captured without placing a designated object in the virtual space that recreates the operation location" may be, for example, a "virtual image captured with an object other than the designated object placed in the virtual space that recreates the operation location" or a "virtual image captured without placing any object in the virtual space that recreates the operation location." The determination unit 16 may then cause the image analysis unit 11 to analyze test images including a "virtual image captured with a designated object placed in the virtual space that recreates the operation location" and a "virtual image captured without placing a designated object in the virtual space that recreates the operation location." The image analysis unit 11 uses a machine learning model to detect the designated object from each test image.

[0145] In this case, the determination unit 16 can calculate the ratio of "the number of test images in which the detection result of the designated object is correct" to "the number of test images processed by the machine learning model" as the detection accuracy. In "virtual images taken with a designated object placed in a virtual space that recreates the operation location," if the detection result is "designated object detected," it is determined to be "correct," and if the detection result is "designated object not detected," it is determined to be "incorrect." Also, in "virtual images taken without a designated object placed in a virtual space that recreates the operation location," if the detection result is "designated object not detected," it is determined to be "correct," and if the detection result is "designated object detected," it is determined to be "incorrect."

[0146] The determination unit 16 can execute processing according to the detection accuracy. For example, if the detection accuracy is equal to or greater than a predetermined threshold, the determination unit 16 may notify the user that the machine learning model is usable in actual operation. The notification may be performed via an output device (such as a display, a projection device, a speaker, or a lamp) included in the information processing device 10. Alternatively, the notification may be realized by transmitting information to a predetermined user device.

[0147] In addition, if the detection accuracy is equal to or greater than a predetermined threshold, the determination unit 16 may automatically or semi-automatically replace the machine learning model used in actual operation with a machine learning model whose detection accuracy is equal to or greater than a predetermined threshold.

[0148] In one example, the information processing device 10 can retrain a machine learning model while performing actual operation using the machine learning model. For example, the information processing device 10 can retrain a copy of the machine learning model, rather than retraining the machine learning model itself that is actually used in actual operation using the machine learning model. Then, when the detection accuracy of the machine learning model exceeds a threshold value due to the retraining, the information processing device 10 can replace the machine learning model described above. This configuration can prevent problems that occur in actual operation due to unexpected inconveniences (deterioration in detection accuracy) caused by retraining the machine learning model.

[0149] Furthermore, if a deterioration in detection accuracy is detected, the determination unit 16 may perform a warning process. If a deterioration in detection accuracy is detected, the user needs to reconsider the re-learning method.

[0150] For example, the determination unit 16 may determine the detection accuracy of one machine learning model multiple times. The determination unit 16 may determine the detection accuracy at predetermined intervals (e.g., every day, every week, every 10 days, or every month). Alternatively, the determination unit 16 may determine the detection accuracy every time the machine learning model is re-trained a predetermined number of times.

[0151] The determination unit 16 can then accumulate a history of past detection accuracy. The determination unit 16 can recognize a change in detection accuracy based on the detection accuracy measured multiple times in the past. The determination unit 16 can then perform a warning process if the change satisfies a predetermined warning condition. Examples of the warning condition include, but are not limited to, "worsening from the immediately preceding detection accuracy" and "worsening from the immediately preceding detection accuracy a predetermined number of times in a row."

[0152] Next, an example of the flow of processing by the information processing device 10 will be described using the flowchart in Fig. 13. Note that the purpose here is to explain the flow of processing. Details of each process have been described above, so description thereof will be omitted here.

[0153] First, the information processing device 10 uses a machine learning model to detect a designated object from a virtual image captured by placing the designated object in a virtual space that recreates the operating location (S60). Next, the information processing device 10 determines the detection accuracy of the machine learning model based on the results of the process of detecting the designated object from the virtual image (S61). Then, the information processing device 10 executes a process according to the detection accuracy (S62).

[0154] Other configurations of the information processing device 10 can be the same as those of the first to fifth embodiments.

[0155] <Effects> The information processing device 10 of the sixth embodiment can achieve the same effects as the information processing devices 10 of the first to fifth embodiments. Furthermore, the information processing device 10 can determine the detection accuracy of a machine learning model trained using the techniques of the first to fifth embodiments. As a result, a user can recognize the learning effect of the machine learning model. Furthermore, based on the detection accuracy, a user can determine whether to replace a machine learning model currently being used in actual operation with a newly retrained machine learning model.

[0156] <<Modifications>> Modifications applicable to the first to sixth embodiments will be described below. These modifications can also achieve the same effects as the first to sixth embodiments.

[0157] <Variation 1> Areas in the processed image where various designated objects are detected are specified in advance. For example, an area in the processed image where a sidewalk appears is specified as an area where a person is detected. If the camera capturing the predetermined location is a fixed camera, the state of the background (sidewalk, etc.) captured in the image generated by that camera is fixed.

[0158] The image analysis unit 11 can execute a process for detecting a designated object in a pre-specified "area where a designated object is detected" in the processed image. For example, the image analysis unit 11 may perform a predetermined process (e.g., converting the area other than the "area where a designated object is detected" to a single color such as white or black) on the other areas, and then process the processed image with a machine learning model to detect the designated object. Alternatively, the image analysis unit 11 may cut out the "area where a designated object is detected," perform a process such as enlarging the cut-out image, and then process the processed image with a machine learning model to detect the designated object.

[0159] <Variation 2> In Variation 2, a machine learning model is prepared for each "operation location" where a camera is installed and images are captured when a machine learning model is actually used to detect a designated object. The information processing device 10 then retrains and determines the accuracy of the machine learning model for each operation location using the techniques described in the first to sixth embodiments. By retraining the machine learning model for each operation location using training data appropriate for each operation location, the detection accuracy of each machine learning model can be efficiently improved.

[0160] Although this disclosure has been described above with reference to the embodiments, this disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of this disclosure within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0161] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of the content.

[0162] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. 1. An information processing device comprising: image analysis means that detects a designated object from a processed image captured of a predetermined location using a machine learning model; extraction means that extracts non-detection images, which are processed images in which an object is captured but the designated object is not detected, based on a detection result based on the processed image; generation means that generate training data based on the non-detection images; and learning means that re-train the machine learning model using the training data. 2. The information processing device described in 1, wherein the extraction means extracts the non-detection images based on a detection result of a detection means that detects an object present at the predetermined location by a process different from the detection of the designated object based on the machine learning model, and on the detection result of the processed images. 3. The information processing device described in 2, wherein the extraction means extracts, as the non-detection images, the processed images captured at the time an object is detected by the detection means and in which the designated object is not detected. 4. 5. The information processing device according to any one of 1 to 4, further comprising a search means for searching open data for related information related to the designated object, and wherein the generation means generates a drawn image of the designated object based on the related information, and generates the training data based on the drawn image. 6. The information processing device according to 5, wherein the generation means generates a three-dimensional image of the designated object based on the related information, and generates from the three-dimensional image the two-dimensional drawn image showing the designated object as observed under predetermined conditions. 7. The information processing device according to 6, wherein the predetermined conditions are the same as the shooting conditions of images to be processed during actual operation using the machine learning model.8. The information processing device according to any one of 1 to 7, further comprising a determination means for determining the detection accuracy of the machine learning model based on the results of processing using the machine learning model to detect the designated object from a virtual image in which the designated object is placed in a virtual space that reproduces a location where actual operation using the machine learning model will be performed. 9. An information processing method in which one or more computers: use a machine learning model to detect a designated object from processed images taken of a predetermined location; extract non-detection images, which are the processed images in which an object is shown but the designated object is not detected, based on the detection results based on the processed images; generate teacher data based on the non-detection images; and re-train the machine learning model using the teacher data. 10. A program that causes a computer to function as: image analysis means for detecting a designated object from processed images taken of a predetermined location using a machine learning model; extraction means for extracting non-detection images, which are the processed images in which an object is shown but the designated object is not detected, based on the detection results based on the processed images; generation means for generating teacher data based on the non-detection images; and learning means for re-training the machine learning model using the teacher data.

[0163] Some or all of Supplements 2 to 8 that are dependent on the information processing device of Supplement 1 described above may also be dependent on the information processing method of Supplement 9 and the program of Supplement 10 in the same dependent relationship as Supplement 1 and Supplements 2 to 8. Furthermore, within the scope of each of the above-mentioned embodiments, some or all of the configurations described as Supplements can be realized in various hardware, software, various recording means for recording software, or systems.

[0164] This application claims priority based on Japanese Patent Application No. 2024-110831, filed July 10, 2024, the disclosure of which is incorporated herein by reference in its entirety.

[0165] REFERENCE SIGNS LIST 10 Information processing device 11 Image analysis unit 12 Extraction unit 13 Generation unit 14 Learning unit 15 Search unit 16 Determination unit 1A Processor 2A Memory 3A Input / output I / F 4A Peripheral circuit 5A Bus

Claims

1. An information processing device having: an image analysis means for detecting a specified object from a processed image taken of a predetermined location using a machine learning model; an extraction means for extracting non-detection images, which are processed images in which an object is captured but the specified object is not detected, based on a detection result based on the processed image; a generation means for generating training data based on the non-detection images; and a learning means for re-training the machine learning model using the training data.

2. The information processing device described in claim 1, wherein the extraction means extracts the non-detected image based on the detection result of a detection means that detects an object present at the specified location using a process different from the detection of the specified object based on the machine learning model and the detection result of the processed image.

3. The information processing device according to claim 2, wherein the extraction means extracts as the non-detection image the processed image captured at the timing when the object is detected by the detection means, in which the specified object is not detected.

4. An information processing device according to any one of claims 1 to 3, wherein the image analysis means detects an object from within the processed image, and executes a process for detecting the specified object for the processed image in which the object is detected, and the extraction means extracts the processed image in which the object is detected but the specified object is not detected as the non-detection image.

5. An information processing device according to any one of claims 1 to 4, further comprising a search means for searching open data for related information relating to the specified object, wherein the generation means generates a drawing image of the specified object based on the related information, and generates the training data based on the drawing image.

6. An information processing device as described in claim 5, wherein the generation means generates a three-dimensional image of the specified object based on the related information, and generates a two-dimensional drawing image from the three-dimensional image that shows the specified object as observed under predetermined conditions.

7. The information processing device according to claim 6, wherein the predetermined conditions are the same as the shooting conditions of the image to be processed during actual operation using the machine learning model.

8. An information processing device according to any one of claims 1 to 7, further comprising a determination means for determining the detection accuracy of the machine learning model based on the results of a process for detecting the specified object from a virtual image in which the specified object is placed in a virtual space that reproduces a location where actual operation using the machine learning model is performed.

9. An information processing method in which one or more computers use a machine learning model to detect a specified object from processed images taken of a specified location, extract non-detection images, which are processed images in which an object is captured but the specified object is not detected, based on the detection results based on the processed images, generate training data based on the non-detection images, and re-train the machine learning model using the training data.

10. The information processing method described in claim 9, wherein the one or more computers extract the non-detected image based on the detection result of a detection means that detects an object present at the specified location using a process different from the detection of the specified object based on the machine learning model and the detection result of the processed image.

11. The information processing method according to claim 10, wherein the one or more computers extract as the non-detection image the processed image that was taken at the time when the object was detected by the detection means and in which the specified object was not detected.

12. An information processing method according to any one of claims 9 to 11, wherein the one or more computers detect an object from within the processed image, perform a process to detect the specified object for the processed image in which the object is detected, and extract the processed image in which the object is detected but the specified object is not detected as the non-detection image.

13. An information processing method according to any one of claims 9 to 12, wherein the one or more computers search for related information about the specified object from open data, generate a drawing image of the specified object based on the related information, and generate the training data based on the drawing image.

14. An information processing method as described in claim 13, wherein the one or more computers generate a three-dimensional image of the specified object based on the related information, and generate a two-dimensional drawing image from the three-dimensional image that shows the specified object as observed under predetermined conditions.

15. A recording medium having recorded thereon a program that causes a computer to function as: an image analysis means that uses a machine learning model to detect a specified object from a processed image taken of a specified location; an extraction means that extracts a non-detection image, which is a processed image that contains an object but in which the specified object was not detected, based on a detection result based on the processed image; a generation means that generates training data based on the non-detection image; and a learning means that retrains the machine learning model using the training data.

16. The recording medium described in claim 15, wherein the extraction means extracts the non-detected image based on the detection result of a detection means that detects an object present at the specified location using a process different from the detection of the specified object based on the machine learning model, and the detection result of the processed image.

17. The recording medium according to claim 16, wherein the extraction means extracts as the non-detection image the processed image that was taken at the time when the object was detected by the detection means and in which the specified object was not detected.

18. A recording medium according to any one of claims 15 to 17, wherein the image analysis means detects an object from within the processed image, and executes a process for detecting the specified object for the processed image in which the object is detected, and the extraction means extracts the processed image in which the object is detected but the specified object is not detected as the non-detection image.

19. A recording medium according to any one of claims 15 to 18, further causing the computer to function as a search means for searching open data for related information relating to the specified object, and wherein the generation means generates a drawing image of the specified object based on the related information, and generates the training data based on the drawing image.

20. A recording medium as described in claim 19, wherein the generating means generates a three-dimensional image of the specified object based on the related information, and generates a two-dimensional drawing image from the three-dimensional image that shows the specified object as observed under predetermined conditions.

Images