Information processing device, detection device, information processing method, and program

Abstract

To provide an information processing device, a detection device, an information processing method, and a program that suppress at least one of erroneous detection and oversight of a target object.SOLUTION: In a detection system 100, an information processing device 10 includes: a first image acquisition unit that acquires image data of a first image where an object area AR, which is a part of a detection area AR0 to be monitored within a facility W, is photographed and where a target object A is not photographed; a detection unit that detects, on the basis of the image data of the first image, from the detection area, at least one of an erroneous detection area where an erroneous detection of the target object occurs and an oversight area where an oversight of the target object occurs; a second image acquisition unit that acquires a second image that is at least one of an image where the target object is superimposed on an image showing the erroneous detection area and an image where the target object is superimposed on an image showing the oversight area; and a learning unit that uses the image data of the second image as training data to train a machine learning model to distinguish the presence or absence of the target object in the image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a detection apparatus, an information processing method, and a program.

Background Art

[0002] There is an apparatus that analyzes an image acquired by a camera or the like and detects an object included in the image. For example, Patent Document 1 describes that a discriminator that corrects a partial image including an inspection object and detects the inspection object using a composite image obtained by combining the corrected partial image and another image is learned.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when detecting an object using a discriminator trained by machine learning, there is a possibility that a non-object is detected as an object (false detection), or an existing object is overlooked and not detected. Therefore, it is required to suppress at least one of false detection and overlooking of an object.

[0005] The present disclosure solves the above-described problems, and an object thereof is to provide an information processing apparatus, a detection apparatus, an information processing method, and a program capable of suppressing at least one of false detection and overlooking of an object.

Means for Solving the Problems

[0006] The information processing device according to this disclosure includes: a first image acquisition unit that acquires image data of a first image in which a target area, which is a part of the detection area to be monitored, is shown, but the target object is not shown; a detection unit that, based on the image data of the first image, detects from the detection area at least one of a false detection area in which the target object is falsely detected and a missed area in which the target object is missed; a second image acquisition unit that acquires a second image which is at least one of an image in which the target object is superimposed on an image in which the false detection area is shown and an image in which the target object is superimposed on an image in which the missed area is shown; and a learning unit that uses the image data of the second image as training data to train an identification model to determine the presence or absence of the target object in the image.

[0007] The detection device relating to this disclosure detects the presence or absence of an object in the detection area using the identification model that has been trained by the learning unit of the information processing device.

[0008] The information processing method relating to this disclosure includes the steps of: acquiring image data of a first image in which a target area, which is a part of the detection area to be monitored, is shown, but the target object is not shown; detecting at least one of a false detection area where the target object is falsely detected and a missed area where the target object is missed from the detection area based on the image data of the first image; acquiring image data of a second image, which is at least one of an image in which the target object is superimposed on an image in which the false detection area is shown and an image in which the target object is superimposed on an image in which the missed area is shown; and using the image data of the second image as training data to train an identification model to determine the presence or absence of the target object in the image.

[0009] The program relating to this disclosure causes a computer to perform the following steps: acquire image data of a first image in which a target area, which is a part of the detection area to be monitored, is shown, but the target object is not shown; detect at least one of a false detection area and a missed area from the detection area based on the image data of the first image; acquire a second image which is at least one of an image in which the target object is superimposed on an image showing the false detection area and an image in which the target object is superimposed on an image showing the missed area; and use the image data of the second image as training data to train an identification model to determine the presence or absence of the target object in the image. [Effects of the Invention]

[0010] According to this disclosure, it is possible to suppress at least one of false detections and missed detections of objects. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a schematic diagram of the detection system according to this embodiment. [Figure 2] Figure 2 is a schematic block diagram of the control device for the detection device. [Figure 3] Figure 3 is a schematic block diagram of an information processing device. [Figure 4] Figure 4 is a schematic diagram illustrating the first training image. [Figure 5] Figure 5 is a schematic diagram showing an example of the difficulty score calculation results. [Figure 6] Figure 6 is a schematic diagram illustrating the second training image. [Figure 7] Figure 7 is a schematic diagram illustrating the second training image. [Figure 8] Figure 8 is a schematic diagram illustrating the detection of missed areas and false positive areas. [Figure 9] Figure 9 is a schematic diagram illustrating the detection of missed areas and false positive areas. [Figure 10]Figure 10 is a schematic diagram showing an example of the second image. [Figure 11] Figure 11 is a schematic diagram showing another example of the second image. [Figure 12] Figure 12 is a flowchart illustrating the processing flow of an information processing device. [Modes for carrying out the invention]

[0012] Preferred embodiments of the present disclosure will be described in detail below with reference to the attached drawings. However, this disclosure is not limited to these embodiments, and if there are multiple embodiments, they may be combinations of these embodiments.

[0013] (Detection system) Figure 1 is a schematic diagram of the detection system according to this embodiment. The detection system 100 according to this embodiment is a system for detecting object A. The detection system 100 has a detection device 1 and an information processing device 10. The detection device 1 is a device deployed within the facility W that detects object A. The detection device 1 detects whether object A is present in the detection area AR0, which is the area to be monitored within the facility W. The information processing device 10 sets an identification model M, which is an identifier for the detection device 1 to detect object A. The identification model M is an AI (Artificial Intelligence) model that executes processing for detecting object A, that is, a program that executes processing for detecting object A. The information processing device 10 and the identification model M will be described later. Note that the facility W is an indoor facility such as a factory or warehouse, but is not limited to that and may be any facility, for example, an outdoor facility. In other words, the facility W may refer to any area. Also, object A is a person in this embodiment, but is not limited to that and may be any type of object to be detected, such as a vehicle.

[0014] (Detection device) The detection device 1 includes an imaging device (camera) C and a control device 2. The detection device 1 captures an image of the target area AR with the imaging device C, and analyzes the image of the target area AR captured by the control device 2 to detect whether the target object A exists in the target area AR. The target area AR is at least a part of the detection area AR0, and can be said to be an area within the imaging range of the image captured by the imaging device C among the detection areas AR0. In this embodiment, the detection device 1 detects whether the target object A exists in each of a plurality of target areas AR located at different positions within the facility W. In this embodiment, the detection device 1 is a moving body that moves automatically. Since the detection device 1 detects the target object A while moving, it can be said that it detects whether the target object A exists in each of the target areas AR whose positions change due to movement. However, the detection device 1 is not limited to being a moving body, and may be a device whose position is fixed within the facility W. In this case, for example, a plurality of imaging devices C are provided at different positions in the facility W, and each imaging device C captures an image of the target area AR. The detection device 1 analyzes the captured images of each target area AR by each imaging device C to detect whether the target object A exists in each target area AR. However, the detection device 1 is not limited to detecting the target object A in a plurality of target areas AR, and may be one that detects the target object A in one target area AR with a fixed position.

[0015] Figure 2 is a schematic block diagram of the control device of the detection device. The control device 2 is, for example, a computer and has a communication unit 3, a storage unit 4, and a control unit 6 as shown in Figure 2. The communication unit 3 is a communication module that communicates with an external device and may be, for example, an antenna or the like. The detection device 1 communicates with an external device by wireless communication, but may also use wired communication, and the communication method may be arbitrary. The storage unit 4 is a memory that stores the calculation content of the control unit 6, program information, the identification model M, etc., and includes, for example, at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), and an external storage device such as a HDD (Hard Disk Drive). The program for the control unit 6 and the identification model M stored in the storage unit 4 may be stored in a recording medium readable by the detection device 1.

[0016] The control unit 6 is an arithmetic device and includes an arithmetic circuit such as a CPU (Central Processing Unit). The control unit 6 includes an image acquisition unit 7 and an object detection unit 8. The control unit 6 reads and executes a program (software) from the storage unit 4 to realize the image acquisition unit 7 and the object detection unit 8 and execute their processes. Note that the control unit 6 may execute these processes by one CPU, or may include a plurality of CPUs and execute the processes with these plurality of CPUs. Also, at least a part of the processes of the image acquisition unit 7 and the object detection unit 8 may be realized by a hardware circuit.

[0017] The image acquisition unit 7 acquires a captured image (an image including the target area AR) in which the target area AR is captured by the imaging device C. The image acquisition unit 7 controls the imaging device C to capture the target area AR and acquires a captured image in which the target area AR is captured.

[0018] The object detection unit 8 reads the machine learning-trained identification model M from the storage unit 4 and inputs the image data of the captured image acquired by the image acquisition unit 7 to the read-out identification model M, thereby obtaining output data from the identification model M indicating whether object A is present in the target area AR. Based on the output data indicating whether object A is present in the target area AR, the object detection unit 8 detects object A. That is, for example, if the output data indicates that object A is present in the target area AR, the object detection unit 8 determines that object A is present, and if the output data indicates that object A is not present in the target area AR, the object detection unit 8 determines that object A is not present. If the detection device 1 is a moving object, for example, if it is determined that object A is present, the detection device 1 may stop.

[0019] The identification model M read out by the object detection unit 8 is a model that has been trained by the information processing device 10 described later, and can be said to be a model that has been trained to determine the presence or absence of object A in an image including the target region AR. In other words, the identification model M read out by the object detection unit 8 can be said to be a model that has been trained on the feature quantities of an image when object A is present in the target region AR. Therefore, when an image including the target region AR is input to the object detection unit 8, the identification model M read out by the object detection unit 8 can perform calculations and output a result of determining whether object A is present in the target region AR.

[0020] (Information processing device) The configuration of the information processing device 10 for training the identification model M will be described below. In this embodiment, the information processing device 10 for training the identification model M and the detection device 1 (control device 2) for detecting the object A using the trained identification model M are separate hardware. However, this is not limited to the latter; the information processing device 10 and the control device 2 may be the same hardware. In other words, the information processing device 10 (or the control device 2) may perform both the machine learning of the identification model M and the detection of the object A using the trained identification model M.

[0021] Figure 3 is a schematic block diagram of an information processing device. The information processing device 10 is, for example, a computer and, as shown in Figure 3, has a communication unit 12, a storage unit 14, and a control unit 16. The information processing device 10 may also have an input unit for receiving information and an output unit (such as a display unit) for outputting information. In the example of Figure 1, the information processing device 10 is located outside the equipment W, but the location where the information processing device 10 is installed can be arbitrary. Furthermore, the information processing device 10 may be configured as a standalone device, as an integral part of other devices, or as a system combining various devices such as an arithmetic unit and a data server, and is not particularly limited.

[0022] The communication unit 12 is a communication module that communicates with an external device, and may be, for example, an antenna. The information processing device 10 communicates with an external device via wireless communication, but it may also use wired communication, and the communication method may be arbitrary.

[0023] The memory unit 14 is a memory that stores information such as the calculation contents and program of the control unit 16, the identification model M, the missed identification model MA (described later), and the false detection identification model MB (described later), and for example, it includes at least one of RAM, ROM, and an external storage device such as an HDD. The program for the control unit 16, the identification model M, the missed identification model MA, and the false detection identification model MB stored in the memory unit 14 may be stored on a recording medium that the information processing device 10 can read.

[0024] The control unit 16 is an arithmetic unit and includes arithmetic circuits such as a CPU. The control unit 16 includes a first image acquisition unit 20, a missed detection learning unit 22, a false detection learning unit 24, a detection unit 26, a second image acquisition unit 28, and a learning unit 30. The control unit 16 reads a program (software) from the storage unit 14 and executes it to realize the first image acquisition unit 20, the missed detection learning unit 22, the false detection learning unit 24, the detection unit 26, the second image acquisition unit 28, and the learning unit 30, and executes their processing. The control unit 16 may execute these processes with a single CPU, or it may have multiple CPUs and execute the processing with those multiple CPUs. In addition, at least a part of the processing of the first image acquisition unit 20, the missed detection learning unit 22, the false detection learning unit 24, the detection unit 26, the second image acquisition unit 28, and the learning unit 30 may be realized with hardware circuits.

[0025] The first image acquisition unit 20 acquires the first image P1, the missed detection learning unit 22 trains the missed detection identification model MA, and the false detection learning unit 24 trains the false detection identification model MB. The detection unit 26 detects missed detection areas and false detection areas from the detection area AR0 based on the first image P1. The second image acquisition unit 28 acquires the second image P2, in which the image of object A is superimposed on the images of the missed detection areas and false detection areas. The learning unit 30 trains the identification model M based on the second image P2. The specific processing details of each of these units will be described later.

[0026] (Processing by information processing equipment) As described above, the identification model M is a classifier capable of detecting object A. However, when detecting object A using a machine learning classifier, there is a risk of false detection of object A or missing object A. False detection refers to detecting an object that is not object A as object A, and missing object A refers to failing to detect an existing object A. In response to this, the information processing device 10 trains the identification model M to suppress both missing objects and false detections. In this embodiment, the information processing device 10 trains the identification model M to suppress both missing objects and false detections, but is not limited to this, and may train the identification model M to suppress only one of them.

[0027] The information processing device 10 uses the first image P1, which shows the target area AR, to detect missed areas and false detection areas using the machine learning-trained missed detection model MA and false detection model MB. Then, the information processing device 10 uses the data from the second image P2, which includes the missed areas and false detection areas, as training data to train the identification model M. The method for training the identification model M will be described in detail below.

[0028] (Machine learning for missed detection models) The information processing device 10 uses the missed detection learning unit 22 to train the missed detection identification model MA, an AI model for detecting missed areas, to learn the correspondence between images and their difficulty scores. A missed area refers to an area within the detection area AR0 where there is a high probability of missing an image. This allows the missed detection identification model MA to calculate a difficulty score for an image once image data is input. The difficulty score is an index indicating the probability of missing an object A. Any AI model can be applied as the missed detection identification model MA, but for example, a CNN (Convolutional Neural Network) model may be used.

[0029] The missed image detection learning unit 22 may train the missed image recognition model MA using any method, but an example of the training method in this embodiment will be described. Figure 4 is a schematic diagram illustrating the first training image.

[0030] (Acquisition of the first training image) The missed detection learning unit 22 acquires a first training image (training image) PA, which will serve as training data for the missed detection recognition model MA. The first training image PA is an image in which a background image PA1 and an object image PA2 are superimposed. The object image PA2 is an image of object A, as shown in the example in Figure 4, or in other words, an image of the same type of object (a person in this embodiment) as object A that is to be detected. The background image PA1 is the image that forms the background of object image PA2, and as shown in the example in Figure 4, it can be said to be an image that does not contain object A (the same type of object as object A that is to be detected). The squares and triangles shown in the background image PA1 in the example in Figure 4 exemplify objects that are not object A that are shown in the background image PA1. Since the first training image PA is an image in which such a background image PA1 and object image PA2 are superimposed, as shown in the example in Figure 4, it can also be said to be an image in which the region RA in which object A is shown is superimposed on a background that does not contain object A, as shown in the background image PA1.

[0031] The missed detection learning unit 22 may acquire any image that does not contain object A as background image PA1. For example, it may acquire an image that does not contain the target area AR (an image in which the target area AR is not visible) as background image PA1. It is preferable for the missed detection learning unit 22 to acquire multiple types of background image PA1. The missed detection learning unit 22 may acquire background image PA1 by any method. For example, the missed detection learning unit 22 may acquire an existing image that does not contain object A or the target area AR from an external device (server) via the communication unit 12 as background image PA1.

[0032] The missed detection learning unit 22 may acquire any image containing object A (an object of the same type as object A to be detected) as object image PA2. It is preferable for the missed detection learning unit 22 to acquire multiple types of object image PA2. The missed detection learning unit 22 may acquire object image PA2 by any method. The missed detection learning unit 22 may acquire existing images containing object A from an external device (server) or the like via the communication unit 12 as object image PA2. Alternatively, for example, the missed detection learning unit 22 may acquire image data (3D image data) of object A and generate multiple object image PA2 by varying the size and orientation of object A based on the image data of object A.

[0033] The missed image learning unit 22 generates a first learning image PA by superimposing the acquired background image PA1 and object image PA2. The missed image learning unit 22 may generate the first learning image PA by superimposing the background image PA1 and object image PA2 in any way. For example, as shown in Figure 4, the missed image learning unit 22 may generate a first learning image PA for each background image PA1 and object image PA2 by superimposing a background image PA1 selected from each background image PA1 and an object image PA2 extracted from each object image PA2. Alternatively, for example, the missed image learning unit 22 may generate multiple first learning image PAs using the same background image PA1 and object image PA2 by superimposing the same object image PA2 with different sizes and positions onto the extracted background image PA1.

[0034] Thus, in this embodiment, the missed image learning unit 22 generated the first learning image PA by superimposing the background image PA1 and the object image PA2. However, the method for acquiring the first learning image PA is not limited to this and may be arbitrary. For example, the missed image learning unit 22 may acquire an existing image including the object A and the background as the first learning image PA from an external device (server) or the like.

[0035] (Machine learning for missed detection models) The missed image detection learning unit 22 sets a dataset containing the image data of the first training image PA and the designation information indicating that the first training image PA contains object A, as training data for the missed image recognition model MA. The missed image detection learning unit 22 prepares multiple training datasets by preparing a dataset (training data) containing the first training image PA and the designation information for each first training image PA. The missed image detection learning unit 22 inputs the prepared training data into the untrained missed image recognition model MA, thereby allowing the missed image recognition model MA to learn the correspondence between images and confidence scores. The confidence score is a score correlated with the difficulty score mentioned above, and is an indicator that shows the judgment result of whether or not object A is present in the image, and the probability that the judgment result is correct. That is, for example, if the confidence score indicates a judgment result that object A is present, the higher the confidence score, the higher the probability that object A is present.

[0036] More specifically, in this embodiment, it is preferable that the missed image learning unit 22 trains the missed image recognition model MA to perform machine learning so that it can calculate a confidence score for each position in the image. In this case, for example, the missed image learning unit 22 sets the image data of the first learning image PA and the specified information indicating the position of object A in the first learning image PA as training data. The missed image learning unit 22 inputs the prepared training data into the untrained missed image recognition model MA, thereby training the missed image recognition model MA to learn the correspondence between images and the confidence score at their positions. That is, in this case, the missed image learning unit 22 inputs as specified information that object A exists at the position indicated by the specified information (the probability of object A being present is 100%) into the missed image recognition model MA. Furthermore, the missed image learning unit 22 prepares multiple first learning images PA in which object A (region RA shown in Figure 4) is placed at different positions in the object image PA1 for the same background image PA2, and trains each of the first learning images PA as training data as described above. This allows the machine learning to be performed on each position of the first training image PA, so that the missed detection model MA learns the correspondence between the image and the confidence score for each position. By performing machine learning in this way, the missed detection model MA can output a confidence score for each position of the image when an image is input. In other words, the missed detection model MA can output an indicator showing the judgment result of whether object A is present or not, and the probability that the judgment result is correct, for each position in the image.

[0037] In this embodiment, the missed detection learning unit 22 configures the missed detection model MA, which has learned the correspondence between images and confidence scores, so that it can calculate a difficulty score. That is, the missed detection learning unit 22 configures the missed detection model MA so that it calculates a difficulty score based on the calculated confidence score. As a result, when an image is input, the missed detection model MA can output a difficulty score for each position in the image. The difficulty score, as described above, is an index that indicates the probability of missing object A and correlates with the confidence score. For example, if the confidence score indicates that object A exists, the lower the probability that the confidence score is correct, the higher the difficulty score will be (i.e., it will be easier to miss). Also, for example, if the confidence score indicates that object A does not exist, the lower the probability that the confidence score is correct, the higher the difficulty score will be (i.e., it will be easier to miss). The missed detection learning unit 22 may configure the missed detection model MA to calculate the difficulty score based on the calculated confidence score, based on the correspondence between the confidence score and the difficulty score, which is set in advance. The correspondence between the confidence score and the difficulty score can be set arbitrarily, but for example, if the confidence score is p and the difficulty score is q, the correspondence may be set as shown in equation (1) below.

[0038] q = 1 - p ... (1)

[0039] Figure 5 is a schematic diagram showing an example of the difficulty score calculation result. As described above, in this embodiment, when an image is input, the missed image recognition model MA is trained to calculate the difficulty score for each position in the image. For example, the missed image recognition model MA may be constructed so as shown in Figure 5, that it outputs an image H showing the difficulty score for each position in the image. Image H may be, for example, an image (heatmap) that shows the difficulty score for each position in the image by color, by dividing the possible numerical ranges of the difficulty score and setting a color for each divided numerical range.

[0040] As described above, in this embodiment, the information processing device 10 uses the missed detection learning unit 22 to train the missed detection identification model MA, but it is not limited to this, and for example, the missed detection identification model MA that has been trained by another device may be acquired.

[0041] (Machine learning for false positive / false positive identification models) The information processing device 10 uses a false detection learning unit 24 to train a false detection identification model MB, which is an AI model for detecting false detection regions, to learn about the presence or absence of false detection object B in an image. A false detection region refers to a region within the detection region AR0 that is highly likely to occur, and false detection object B refers to an object that is falsely identified as target object A. Specifically, the false detection learning unit 24 trains the false detection identification model MB to learn the correspondence between an image and the presence or absence of false detection object B. As a result, when image data is input to the false detection identification model MB, it can output a result of determining the presence or absence of false detection object B in that image. The following describes in more detail how to train the false detection identification model MB. Any AI model can be applied as the false detection identification model MB, but for example, a CNN model may be used.

[0042] The false detection learning unit 24 may train the false detection identification model MB using any method, but an example of the training method in this embodiment will be described. Figures 6 and 7 are schematic diagrams illustrating the second training image.

[0043] (Acquisition of the second training image) The false detection learning unit 24 acquires a second training image PB, which will serve as training data for the false detection identification model MB. The second training image PB is an image that does not contain object A (an image in which object A is not visible). In this embodiment, the false detection learning unit 24 acquires an image that does not contain object A and does not contain the detection region AR0 (an image in which the detection region AR0 is not visible) as the second training image PB. However, it is not limited to this, and the false detection learning unit 24 may acquire any image that does not contain object A as the second training image PB. For example, it may acquire an image showing a location close to the environment of equipment W as the second training image PB. A location close to the environment of equipment W may refer to equipment of the same type as equipment W, that is, if equipment W is a warehouse, an image of a warehouse other than equipment W may be used as the second training image PB. The rectangles shown in the second training image PB in the example in Figure 6 exemplify objects that are not object A and are shown in the second training image PB. However, the second training image PB is not limited to being an image that does not contain object A, but may also be an image that contains object A. The false detection learning unit 24 may acquire the second training image PB by any method, for example, by obtaining an existing image that does not contain object A from an external device (server) via the communication unit 12 as the second training image PB.

[0044] The false detection learning unit 24 may use the acquired second learning image PB to generate multiple second learning images PBa in which at least a portion of the imaging area (the area within the field of view of the image) overlaps with each other, and the image data is different from each other, and use the second learning images PBa for training the false detection discrimination model MB. In other words, each second learning image PBa is an image taken of the same location, but it can be said that the imaging conditions, such as the imaging area and the surrounding environment (e.g., brightness), are different. Figure 6 shows an example in which the false detection learning unit 24 generates second learning images PBa1, PBa2, and PBa3 as second learning images PBa.

[0045] Note that overlapping imaging areas means that each of the second training images PBa has a common imaging area. In the example in Figure 6, the second training images PBa1, PBa2, and PBa3 have overlapping imaging areas and can be said to correspond to images taken of the same location with the same imaging range. However, the second training images PBa may have different imaging ranges in some parts. In this case, for example, the false detection learning unit 24 may generate multiple second training images PBa with different imaging ranges in some parts by changing the image cropping range of the acquired second training image PB. The image cropping range is preferably set based on the changes in the mounting position and angle of the imaging device C in the equipment W. For example, when the imaging device C is attached to a moving object, the difference between the passage of the moving object and the vehicle width becomes the change in the mounting position of the imaging device C. The change in the angle of the moving object caused by the orientation of the moving object and the slope of the ground that can be taken with respect to the passage width becomes the change in the camera angle. In this case, for example, in equipment W, an imaging device C may be installed within the estimated range of change, markers may be placed at the expected positions, images may be comprehensively captured, and the change in the position of the markers within the acquired images may be used as the range of the cropped position. The size and aspect ratio of the cropped image may be set based on the image obtained by the imaging device C used for detection.

[0046] Furthermore, "at least a portion of the image data is different" means, for example, that at least a portion of the pixel data (grayscale value) of each pixel in each second training image PBa is different. In this embodiment, this means that at least one of the brightness value, contrast, and hue of at least some of the pixels is different. For example, the false detection learning unit 24 may generate multiple second training images PBa in which at least a portion of the image data is different by adjusting at least one of the brightness value, contrast, and hue of the acquired second training image PB. The amount of adjustment for at least one of the brightness value, contrast, and hue may be set based on the amount of change in brightness value, contrast, and hue of images acquired comprehensively within the operating time by installing a fixed-point camera in the facility W. In the example in Figure 6, a second training image PBa1 with the same brightness value as the acquired second training image PB, a second training image PBa2 with a lower brightness value than the second training image PBa1, and a second training image PBa3 with a lower brightness value than the second training image PBa2 are generated.

[0047] The false detection learning unit 24 acquires multiple types of second learning images PB with different imaging areas, and prepares multiple second learning images PBa for each acquired second learning image PB, as described above. In other words, although Figure 6 shows an example in which multiple second learning images PBa (second learning images PBa1, PBa2, PBa3) are prepared from one second learning image PB, it can be said that the false detection learning unit 24 prepares multiple second learning images PBa for each acquired second learning image PB.

[0048] The method for generating the second training image PBa is not limited to the method described above and may be arbitrary. For example, the acquired second training image PB may be used as the second training image PBa as is. Also, the number of second training image PBa generated from one second training image PB is not limited to three and may be arbitrary. Furthermore, the false detection learning unit 24 generates multiple second training image PBa based on the acquired second training image PB, but it is not limited to generating them itself; it may also acquire multiple pre-generated second training image PBa, for example, via the communication unit 12.

[0049] (Extraction of the second training image containing false positives) The false detection learning unit 24 selects a second training image PB containing the falsely detected object B from among the acquired second training images PB. The false detection learning unit 24 may select a second training image PB containing the falsely detected object B by any method, but in this embodiment, a provisional identification model that has been trained to detect the presence or absence of object A is used to extract the second training image PB containing the falsely detected object B. Specifically, the false detection learning unit 24 reads the provisional identification model from the storage unit 14 and inputs the data of the second training image PBa into the provisional identification model to obtain a detection result of whether object A is contained in the second training image PBa. The provisional identification model is a model that has been trained to detect the presence or absence of object A in an image. Since object A is not actually contained in the second training image PBa, the detection result by the provisional identification model of whether object A is contained in the second training image PBa can be rephrased as the detection result of whether falsely detected object B is contained in the second training image PBa. In other words, object A detected by the provisional identification model in the second training image PBa is a falsely detected object (i.e., falsely detected object B). Any AI model can be used as the provisional classification model, but for example, a CNN (Conventional Neural Network) model may be used. For example, the provisional classification model may be a pre-trained model that has been trained on a publicly available dataset.

[0050] The false detection learning unit 24 inputs the image data of the second training image PBa into the provisional identification model, causing the provisional identification model to output a judgment result on whether the second training image PBa contains object A (falsely detected object B), and obtains that judgment result. Note that the second training image PBa is an image that does not contain object A, so the judgment result that the second training image PBa contains object A can be said to be a judgment result that something that is not object A was mistakenly detected as object A, that is, a judgment result that falsely detected object B is included.

[0051] More specifically, it is preferable that the provisional identification model is trained to output, as a result of determining whether an image contains object A, the position information of object A within the image, i.e., the position information of the region occupied by object A within the image. Therefore, it is preferable that the false detection learning unit 24 inputs the image data of the second learning image PBa into the provisionally trained second learning image PBa, thereby obtaining the position information of object A within the second learning image PBa (the position information of the region occupied by object A) from the second learning image PBa. The position information of object A obtained from the provisional identification model can be said to be the position information of the falsely detected object A (i.e., the falsely detected object B).

[0052] The false detection learning unit 24 inputs the image data of each second learning image PBa into the provisional identification model, thereby obtaining a determination result of whether object A is included and the location information of object A for each second learning image PBa. As described above, in this embodiment, multiple second learning images PBa are prepared for each of the multiple types of second learning images PB with different imaging areas. For each of the second learning images PBa set for each of these second learning images PB, the false detection learning unit 24 obtains a determination result of whether object A is included and the location information of object A using the provisional identification model.

[0053] (Setting up training data for the false positive / false positive model) The false detection learning unit 24 sets the second training image PB, which contains the falsely detected object B selected as described above, as training data for the false detection identification model MB. In this embodiment, the false detection learning unit 24 sets a dataset including the second training image PB corresponding to the second training image PBa in which object A is determined to be included, and specification information about that second training image PB, as training data for the untrained false detection identification model MB. The specification information is information indicating whether or not the falsely detected object B is included in that second training image PB. That is, the false detection learning unit 24 uses the data of the second training image PB in which object A is determined to be included, and the information indicating that the falsely detected object B is included in that second training image PB, as training data for the false detection identification model MB. The false detection learning unit 24 does not need to use the second training image PB corresponding to the second training image PBa in which object A is determined not to be included as training data.

[0054] More specifically, the false detection learning unit 24 sets the position of falsely detected object B in the second learning image PB corresponding to the second learning image PBa, based on the position of object A in the second learning image PBa indicated by the provisional identification model. The false detection learning unit 24 then uses the position information of falsely detected object B in the second learning image PB as specified information. In other words, the false detection learning unit 24 uses the data of the second learning image PB corresponding to the second learning image PBa in which object A is determined to be included, and the position information of falsely detected object B in the second learning image PB, as training data for the false detection identification model MB.

[0055] In this embodiment, as described above, multiple second learning images PBa with different image data are set for the same second learning image PB, and position information of object A is acquired for each second learning image PBa. Therefore, the false detection learning unit 24 sets the position of the falsely detected object B in the second learning image PB based on the position of the object for each second learning image PBa corresponding to the same second learning image PB, and uses this as specified information. For example, if the positions of object A in each second learning image PBa overlap, the false detection learning unit 24 sets the region including the positions of those object A as the position of the falsely detected object in the second learning image PB. Alternatively, if the positions of object A in each second learning image PBa do not overlap, the false detection learning unit 24 sets the position of each object A as the position of the falsely detected object B in the second learning image PB.

[0056] Furthermore, in this embodiment, multiple second training images PBa are prepared for each of the multiple types of second training images PB with different imaging areas. The false detection learning unit 24 performs the above processing for each of these second training images PB, and for each second training image PB that is determined to contain object A, it prepares a dataset including the data of the second training image PB and the specified information, which is used as training data for the false detection identification model MB. Figure 7 shows an example of training data consisting of a second training image PB and specified information indicating the location (region RB) of a falsely detected object within the second training image PB.

[0057] (Machine learning for false positive / false positive identification models) The false detection learning unit 24 inputs the training data (the second training image PB, which is determined to contain object A, and the specified information) set as described above into the untrained false detection identification model MB, thereby training the false detection identification model MB to learn whether or not falsely detected object B is present in the image. As a result, when unknown image data is input, the false detection identification model MB can output a result of determining whether or not falsely detected object B is present in that image. More specifically, it is preferable that the false detection learning unit 24 uses the data of the second training image PB, which is determined to contain object A, and the specified information indicating the position of falsely detected object B in the second training image PB as training data, and trains the false detection identification model MB to learn the position of falsely detected object B in the image. As a result, when unknown image data is input, the false detection identification model MB can output a result of determining whether or not falsely detected object B is present, and the position information of falsely detected object B in that image. In other words, the trained false positive / false detection model MB is a model that has been trained to recognize the presence or absence of false positive object B in an image and the location information of false positive object B. It can also be said that the model has been trained to recognize the features of images where false positive object B is present. Any AI model can be applied as the false positive / false detection model MB, but a CNN model, for example, can be used.

[0058] Furthermore, the false detection learning unit 24 may train the false detection identification model MB to perform machine learning so that, in addition to determining the presence or absence of false detection object B and the location of false detection object B, it can also calculate a probability score indicating the probability that the determination that false detection object B exists is correct. The false detection learning unit 24 inputs the prepared training data into the untrained false detection identification model MB, thereby training the false detection identification model MB to learn the correspondence between images and the presence or absence of false detection object B, the location of false detection object B, and the probability score. In this case, the false detection learning unit 24 inputs the information that false detection object B exists at the location indicated by the specified information into the false detection identification model MB as specified information. By performing machine learning in this way, the false detection identification model MB can output the presence or absence of false detection object B, the location of false detection object B, and the probability score by performing calculations when an image is input.

[0059] As described above, in this embodiment, the information processing device 10 uses the false detection learning unit 24 to train the false detection identification model MB, but it is not limited to this, and for example, the false detection identification model MB that has been trained by another device may be acquired.

[0060] (Acquisition of the first image) The first image acquisition unit 20 acquires data for a first image P1 in which the target area AR (a part of the detection area AR0) is captured, but the object A is not captured. In other words, the first image P1 can be said to be an image of a part of the detection area AR0 in a state where the object A is not present, as shown in Figure 1, for example. In this embodiment, the first image acquisition unit 20 acquires the first image P1 for each of the multiple target areas AR. In other words, for example, if the detection device 1 is a moving object, the first image P1 can be said to be acquired for each of the target areas AR whose position changes as the detection device 1 moves. The first image acquisition unit 20 may acquire the first image P1 by any method, but for example, an image of the target area AR within the facility W captured by the detection device 1 or another device may be acquired as the first image P1 via the communication unit 12. Note that the first image P1 is not limited to an image that does not include the object A, and may be an image of the target area AR that includes the object A.

[0061] (Detection of missed areas) Figures 8 and 9 are schematic diagrams illustrating the detection of missed areas and false detection areas. The detection unit 26 detects missed areas by inputting image data of the first image P1 into the machine-learned missed detection model MA. As described above, the missed detection model MA has been machine-learned to recognize the correspondence between images and difficulty scores. Therefore, the detection unit 26 obtains the difficulty score of the first image P1 by inputting the image data of the first image P1 into the missed detection model MA. More specifically, the detection unit 26 obtains the difficulty score for each position of the first image P1 by inputting the image data of the first image P1 into the missed detection model MA.

[0062] The detection unit 26 determines whether the target area AR captured in the first image P1 contains a missed area, based on the difficulty score of the first image P1. The detection unit 26 may determine whether a missed area is included using any method based on the difficulty score. For example, the detection unit 26 determines that a missed area is included if the value based on the difficulty score for each location is greater than or equal to a predetermined threshold, and determines that no missed area is included if the value based on the difficulty score for each location is less than the threshold. The value based on the difficulty score for each location may be, for example, the sum, mean, or variance of the difficulty scores for each location.

[0063] The detection unit 26 performs the above processing for each first image P1 to determine whether the target area AR captured in the first image P1 contains any missed areas. Line L1 in Figure 8 shows an example of a value based on the difficulty score for each first image P1. In the example in Figure 8, the detection unit 26 determines that the target area AR of the first image P1 in which the value based on the difficulty score is equal to or greater than the threshold contains any missed areas.

[0064] Furthermore, the detection unit 26 may detect the location of missed areas in the target area AR captured in the first image P1 based on the difficulty score for each position in the first image P1. In this case, for example, the detection unit 26 may detect the locations (areas) in the first image P1 where the difficulty score is above a threshold as the locations of missed areas.

[0065] The detection unit 26 acquires positional information of the target area AR that has been determined to contain a missed area in the coordinate system of the detection area AR0 (equipment W). The detection unit 26 detects the position (area) of the target area AR that has been determined to contain a missed area within the entire area of ​​the detection area AR0 as the missed area in the coordinate system of the detection area AR0. In the example in Figure 9, the difficulty score of the first image P1 in which the target area ARa is captured is above the threshold, so it is determined that the target area ARa contains a missed area. In this case, the detection unit 26 detects the area that overlaps with the target area ARa, at least partially, in the coordinate system of the detection area AR0 as the missed area. The method for acquiring the positional information of the target area AR in the coordinate system of the detection area AR0 is arbitrary; for example, the positional information of the target area AR may be calculated based on the position and orientation in which the first image P1 in which the target area AR is captured was taken in the coordinate system of the detection area AR0.

[0066] In this way, the detection unit 26 inputs the image data of the first image P1 into the machine-learned missed detection model MA to determine whether the target area AR in the first image P1 contains a missed area, and detects the missed area. However, it is not limited to this, and the detection unit 26 may determine whether the target area AR in the first image P1 contains a missed area by any method based on the image data of the first image P1, and detect the missed area.

[0067] (Detection of false positive areas) The detection unit 26 detects the false detection area by inputting the image data of the first image P1 into the false detection identification model MB. As described above, the false detection identification model MB has been trained to determine the presence or absence of the false detection object B. Therefore, the detection unit 26 inputs the image data of the first image P1 into the false detection identification model MB to obtain the result of determining the presence or absence of the false detection object B in the target area AR shown in the first image P1.

[0068] The detection unit 26 determines whether the target area AR in the first image P1 contains the falsely detected object B, based on the determination result of whether the falsely detected object B is present or not. The detection unit 26 may determine whether the missed area is included by any method based on the determination result of whether the falsely detected object B is present or not. For example, if the detection unit 26 obtains a determination result that the falsely detected object B is present, it may determine that the target area AR in the first image P1 contains the falsely detected object B, and if it obtains a determination result that the falsely detected object B is not present, it may determine that the target area AR does not contain the falsely detected object B. Alternatively, for example, the detection unit 26 may obtain a probability score from the false detection identification model MB that indicates the probability that the determination result of whether the falsely detected object B is present or not is correct. The detection unit 26 may then determine that the falsely detected object B is present if the probability score is above a predetermined threshold, and that the falsely detected object B is not present if the probability score is below a predetermined threshold.

[0069] The detection unit 26 performs the above processing for each first image P1 to determine whether the target area AR in the first image P1 contains the falsely detected object B. Line L2 in Figure 8 shows an example of a value based on the probability score for each first image P1. In the example in Figure 8, the detection unit 26 determines that the target area AR of the first image P1 in which the probability score is equal to or greater than the threshold contains the falsely detected object B.

[0070] The detection unit 26 acquires the position information of the falsely detected object B in the coordinate system of the detection area AR0 (equipment W). The detection unit 26 detects the position (area) that includes the falsely detected object B in the coordinate system of the detection area AR0 within the entire area of ​​the detection area AR0 as the falsely detected area. In the example in Figure 9, since area ARb was determined to be the falsely detected object B, the detection unit 26 detects the area that overlaps with area ARb at least partially in the coordinate system of the detection area AR0 as the falsely detected area. Note that the method for acquiring the position information of the falsely detected object B in the coordinate system of the detection area AR0 is arbitrary. For example, the detection unit 26 acquires the position information of the falsely detected object B in the first image P1 (the position of the falsely detected object B in the coordinate system of the first image P1) from the false detection identification model MB. The detection unit 26 may then calculate the position information of the falsely detected object B in the coordinate system of the detection region AR0 based on the position of the falsely detected object B in the first image P1 and the position and orientation in the coordinate system of the detection region AR0 from which the first image P1, which shows the target region AR, was captured.

[0071] In this way, the detection unit 26 inputs the image data of the first image P1 into the machine-learned false detection identification model MB to determine whether the target area AR in the first image P1 contains the falsely detected object B, and detects the falsely detected area. However, it is not limited to this, and the detection unit 26 may determine whether the target area AR in the first image P1 contains a false detection by any method based on the image data of the first image P1, and detect the falsely detected area.

[0072] As described above, in this embodiment, the detection unit 26 detects both missed areas and falsely detected areas, but is not limited to this, and may detect only one of the missed areas or falsely detected areas based on the image data of the first image P1.

[0073] (Acquiring the second image) Figure 10 is a schematic diagram showing an example of a second image. The second image acquisition unit 28 acquires a second image P2 for training the identification model M based on the detection result of at least one of the missed area and the false detection area by the detection unit 26. In this embodiment, the second image acquisition unit 28 acquires an image in which the object A is superimposed on an image in which the false detection area is shown, and an image in which the object is superimposed on an image in which the missed area is shown, as the second image P2. However, it is not limited to this, and the second image acquisition unit 28 may acquire at least one of an image in which the object A is superimposed on an image in which the false detection area is shown, and an image in which the object is superimposed on an image in which the missed area is shown, as the second image P2.

[0074] The second image acquisition unit 28 preferably prepares a second image P2 for each false detection region and missed detection region. Hereafter, in the second image P2, the image superimposed with the image of object A, that is, the image showing the missed detection region or false detection region, will be referred to as the background image P2A. The image of object A superimposed with the background image P2A will be referred to as the object image P2B. In other words, as shown in Figure 10, the second image P2 can be said to be an image in which the object image P2B is superimposed on the background image P2A. Note that the rectangles shown in the second image P2 in Figure 10 are examples of objects other than object A.

[0075] (When the captured image is used as the second image) The method by which the second image acquisition unit 28 acquires the second image P2 is arbitrary. For example, the second image acquisition unit 28 may acquire an image in which the object A and the missed area or false detection area are captured (an image in which the object A and the missed area or false detection area are included within the imaging range) as the second image P2. In this case, for example, the object A is actually positioned in the missed area or false detection area, and imaging is performed so that the object A and the missed area or false detection area are included within the imaging range. The second image acquisition unit 28 acquires the image thus captured as the second image P2. When imaging with the object A positioned in the missed area, the object A may be positioned at any position within the missed area, but it is preferable to position the object A at the position within the missed area that has the highest difficulty score. Furthermore, when imaging with object A positioned in the false detection area, object A may be positioned at any position within the false detection area, but it is preferable to position object A at a position within that false detection area where the distance to the falsely detected object B is less than a predetermined value.

[0076] The state of object A when capturing an image can be arbitrary. The state of object A includes its type, size, orientation, and posture, and the likelihood of missed detection or false detection changes depending on the state of object A. The second image acquisition unit 28 may acquire multiple images captured with object A in different states as the second image P2. Alternatively, for example, a correspondence between the state of object A and the likelihood of missed detection or false detection may be set in advance, and an image captured with object A positioned such that the likelihood of missed detection or false detection exceeds a predetermined threshold may be acquired as the second image P2. The correspondence between the state of object A and the likelihood of missed detection or false detection can be set arbitrarily.

[0077] (When generating a second image) The second image acquisition unit 28 may generate a second image P2 by superimposing a background image P2A and an object image P2B. In this case, the second image acquisition unit 28 acquires a background image P2A that shows the missed area or the falsely detected area. The second image acquisition unit 28 may acquire the background image P2A by any method, for example, by acquiring an image captured of the missed area or falsely detected area where the object A is not located as the background image P2A. The second image acquisition unit 28 may also acquire an object image P2B by any method. The second image acquisition unit 28 may acquire an existing image showing the object A from an external device (server) or the like via the communication unit 12 as the object image P2B. Alternatively, for example, the second image acquisition unit 28 may acquire image data (3D image data) of the object A and generate the object image P2B based on the image data of the object A.

[0078] The second image acquisition unit 28 may superimpose the background image P2A and the object image P2B so that the object image P2B is positioned at an arbitrary position on the background image P2A. For example, when superimposing on the background image P2A of a missed area, it is preferable to position the object image P2B at the position in that missed area where the difficulty score is maximum. Also, when superimposing on the background image P2A of a false detection area, it is preferable to position the object image P2B at a position in that false detection area where the distance to the falsely detected object B is less than a predetermined value.

[0079] The state of object A in object image P2B (type, size, orientation, posture, etc.) can be arbitrary. The second image acquisition unit 28 may generate object images P2B with different states of object A and acquire multiple second images P2 with different states of object A. Alternatively, for example, a correspondence between the state of object A and the likelihood of missed detection or false detection may be set in advance, and an object image P2B in which the likelihood of missed detection or false detection is above a predetermined threshold may be superimposed on the background image P2A to generate a second image P2.

[0080] When generating a second image P2 by superimposing a background image P2A and an object image P2B, the second image acquisition unit 28 may superimpose the background image P2A and the object image P2B based on depth information indicating the depth at each position in the background image P2A. Depth refers to the position in the depth direction in the image. The second image acquisition unit 28 may acquire the depth information of the background image P2A by any method, but for example, it may calculate the depth of each position in the background image P2A from captured images taken from a common area captured by multiple cameras. Alternatively, for example, it may calculate the depth of each position in the background image P2A by performing depth estimation using deep learning in a known way based on the background image P2A.

[0081] Figure 11 is a schematic diagram showing another example of the second image. After acquiring depth information of the background image P2A, the second image acquisition unit 28 sets the depth of the object image P2B. The second image acquisition unit 28 generates the second image P2 by superimposing the background image P2A and the object image P2B so that they are located at the set depth. For example, in Figure 11, the depth is set so that the object image P2B is behind the object P2A1 included in the background image P2A. In this case, as shown in Figure 11, the second image acquisition unit 28 generates the second image P2 so that the object image P2B is behind the object P2A1.

[0082] In the above explanation, the second image P2 containing the false detection region was an image in which the object image P2B is superimposed on the background image P2A of the false detection region. However, it is not limited to this, and an image in which the object image P2B is superimposed on the background image P2A of the false detection region, and an image that does not include the object image P2B but includes the background image P2A of the false detection region (an image in which the object A is not shown but the false detection region is shown) may also be obtained as the second image P2 containing the false detection region.

[0083] (Machine learning of discriminative models) Once the second image P2 is acquired, the learning unit 30 uses the image data of the second image P2 as training data to train the discrimination model M to determine the presence or absence of object A in the image. More specifically, the learning unit 30 sets a dataset containing the image data of the second image P2 and specification information indicating the presence or absence of object A in the second image P2 as training data for the discrimination model M. The learning unit 30 prepares multiple training datasets by preparing a dataset (training data) containing the second image P2 and the specification information for each second image P2. Since the second image P2 includes the object image P2B, the specification information will be information indicating that object A is included in the second image P2. However, as mentioned above, as a second image P2 that includes a false detection area, an image is also prepared that does not include the object image P2B but includes the background image P2A of the false detection area. Since such a second image P2 does not include the object image P2B, the specification information will be information indicating that object A is not included in the second image P2.

[0084] The learning unit 30 inputs the respective training data (image data and specified information of the second image P2) into the untrained discriminative model M, thereby training the discriminative model M to learn the correspondence between images and the presence or absence of object A in those images. Through this machine learning process, the discriminative model M can perform calculations when an image is input and output a result determining whether or not object A is present in that image. Any AI model can be applied as the discriminative model M, but for example, a CNN model may be used.

[0085] The learning unit 30 may use the image data of the second image P2 and the specified information indicating the position of object A in the second image P2 as training data to train the discrimination model M to learn the position of object A within an image. This allows the discrimination model M to output a result of determining whether or not the discrimination model M exists, and the position information of the discrimination model M within that image, when unknown image data is input.

[0086] The information processing device 10 transmits the identification model M, which has been trained using the second image P2 as training data, to the detection device 1, for example, via the communication unit 12. The detection device 1 uses the trained identification model M, as described above, to detect the object A.

[0087] (Processing flow) Next, the processing flow of the information processing device 10 described above will be explained. Figure 12 is a flowchart illustrating the processing flow of the information processing device. As shown in Figure 12, the information processing device 10 acquires a first image P1 in which the target area AR is visible and the target object A is not visible using the first image acquisition unit 20 (step S10). Then, the information processing device 10 detects the missed area based on the first image P1 using the detection unit 26 (step S12), and acquires a second image P2 in which the missed area (background image P2A) and the target object A (target object image P2B) are superimposed using the second image acquisition unit 28 (step S14). In this embodiment, the detection unit 26 inputs the image data of the first image P1 into a machine learning-trained missed detection model MA to obtain a difficulty score for the first image P1, and detects the missed area based on the difficulty score. Furthermore, the information processing device 10 detects a false detection region based on the first image P1 using the detection unit 26 (step S16), and acquires a second image P2 in which the false detection region (background image P2A) and object A (object image P2B) are superimposed using the second image acquisition unit 28 (step S18). In this embodiment, the detection unit 26 inputs the image data of the first image P1 into a machine learning-trained false detection identification model MB to obtain a determination result on whether the first image P1 contains the false detection object B, and detects the false detection region based on that determination result.

[0088] Once the second image P2 is acquired, the information processing device 10 uses the second image P2 as training data to train the discrimination model M (step S20).

[0089] As explained above, in this embodiment, based on the first image P1 in which object A is not visible, false detection areas and missed detection areas are detected from the detection area AR0. Then, using the second image P2 in which the false detection areas and missed detection areas are superimposed with object A, the identification model M is trained to learn the presence or absence of object A. In this embodiment, false detection areas prone to false detection and missed detection areas prone to missed detection are detected, and the second image P2, which appropriately reproduces the situation in which false detection and missed detection are likely to occur, is used as training data. This allows the machine learning to focus on areas prone to false detection and missed detection. Therefore, by using the identification model M trained using the method of this embodiment, false detection and missed detection of object A can be appropriately suppressed.

[0090] (effect) As described above, the information processing device 10 according to the first aspect of this disclosure includes a first image acquisition unit 20, a detection unit 26, a second image acquisition unit 28, and a learning unit 30. The first image acquisition unit 20 acquires image data of a first image P1 in which a target area AR, which is a part of the detection area AR0 to be monitored, is captured, and object A is not captured. Based on the image data of the first image P1, the detection unit 26 detects at least one of a false detection area where object A is falsely detected and a missed area where object A is missed from within the detection area AR0. The second image acquisition unit 28 acquires a second image P2, which is at least one of an image in which object A is superimposed on an image in which the false detection area is captured, and an image in which object A is superimposed on an image in which the missed area is captured. The learning unit 30 uses the image data of the second image P2 as training data to train an identification model M to determine the presence or absence of object A in the image. According to this disclosure, by detecting areas prone to false detection and areas prone to missed detection, and using images containing these areas as training data, machine learning can be performed intensively using areas prone to false detection and missed detection. Therefore, according to this disclosure, at least one of false detection and missed detection of object A can be appropriately suppressed.

[0091] The information processing device 10 according to the second aspect of this disclosure is the same as the information processing device 10 according to the first aspect, wherein the detection unit 26 detects missed areas by inputting image data of the first image P1 to a missed area identification model MA that has been trained on the correspondence between an image and a difficulty score indicating the probability of missing an object A. According to this disclosure, by using a missed area identification model MA that has been trained on the difficulty score, missed areas can be appropriately detected and missed areas can be appropriately suppressed.

[0092] The information processing device 10 according to the third aspect of this disclosure is the information processing device 10 according to the second aspect, further including a missed detection learning unit 22 that trains a missed detection identification model MA. The missed detection learning unit 22 trains the missed detection identification model MA using a first learning image (learning image) PA in which an object A is superimposed on the background and designation information indicating that an object A exists in the first learning image PA as training data. The detection unit 26 inputs the image data of the first image P1 to the trained missed detection identification model MA to obtain a difficulty score for the first image P1, and determines whether the target area AR contains a missed area based on the difficulty score. According to this disclosure, by using a missed detection identification model MA that has been trained on difficulty scores, missed areas can be appropriately detected and missed detections can be appropriately suppressed.

[0093] The information processing device 10 according to the fourth aspect of this disclosure is the same as the information processing device 10 according to the third aspect, wherein the missed learning unit 22 is trained to learn the difficulty score for each position of the first learning image PA, and the detection unit 26 inputs the image data of the first image P1 to the missed identification model MA to obtain the difficulty score for each position of the first image P1. According to this disclosure, by using the missed identification model MA which has learned the difficulty score for each position, missed areas can be appropriately detected and missed areas can be appropriately suppressed.

[0094] The information processing device 10 according to the fifth aspect of this disclosure is an information processing device 10 according to any of the first to fourth aspects, wherein the detection unit 26 detects a false detection region by inputting image data of the first image P1 to a false detection identification model MB that has been trained to determine the presence or absence of a false detection object B that is falsely detected as object A. According to this disclosure, by using a false detection identification model MB that has been trained to determine the presence or absence of a false detection object B, the false detection region can be appropriately detected and false detections can be appropriately suppressed.

[0095] The information processing device 10 according to the sixth aspect of this disclosure is the information processing device 10 according to the fifth aspect, further including a false detection learning unit 24 that trains a false detection identification model MB. The false detection learning unit 24 trains the false detection identification model MB using a second learning image PB that does not contain the target object A and designation information indicating that the second learning image PB contains a false detection object B as training data. The detection unit 26 inputs image data of the first image P1 to the false detection identification model MB to determine whether the target region AR contains a false detection object B. According to this disclosure, by using a false detection identification model MB that has learned the presence or absence of a false detection object B through machine learning, the false detection region can be appropriately detected and false detections can be appropriately suppressed.

[0096] The information processing device 10 according to the seventh aspect of this disclosure is an information processing device 10 according to any of the first to sixth aspects, wherein the second image acquisition unit 28 acquires an image in which the false detection area or missed area and the object A are captured as a second image P2. By performing machine learning using such a second image P2, at least one of the false detection and missed detection of object A can be appropriately suppressed.

[0097] The information processing device 10 according to the eighth aspect of this disclosure is an information processing device 10 according to any of the first to seventh aspects, wherein the second image acquisition unit 28 generates a second image P2 by superimposing an object image P2B, which is an image of object A, onto a background image P2A in which a false detection area or a missed area is captured. By performing machine learning using such a second image P2, at least one of false detection and missed detection of object A can be appropriately suppressed.

[0098] The information processing device 10 according to the ninth aspect of this disclosure is the same as the information processing device 10 according to the eighth aspect, wherein the second image acquisition unit 28 superimposes the background image P2A and the object image P2B based on depth information indicating the depth at each position of the background image P2A. By superimposing the images using depth information as well, it becomes possible to generate a second image P2 that appropriately reproduces situations in which false detection or oversight is likely to occur, and at least one of false detection or oversight of object A can be appropriately suppressed.

[0099] The detection device 1 according to the tenth aspect of this disclosure detects the presence or absence of object A in the detection area AR0 using an identification model M that has been trained by machine learning using the learning unit 30 of the information processing device 10 according to any of the first to ninth aspects. According to this disclosure, at least one of false detections and missed detections of object A can be appropriately suppressed.

[0100] An information processing method according to the eleventh aspect of this disclosure includes the steps of: acquiring image data of a first image P1 in which a target area AR, which is a part of the detection area AR0 to be monitored, is shown, but object A is not shown; detecting at least one of a false detection area where object A is falsely detected and a missed area where object A is missed from the detection area AR0 based on the image data of the first image P1; acquiring a second image P2 which is at least one of an image in which object A is superimposed on an image showing a false detection area and an image in which object A is superimposed on an image showing a missed area; and using the image data of the second image P2 as training data to train an identification model M to determine the presence or absence of object A in the image. According to this disclosure, at least one of false detection and missed detection of object A can be appropriately suppressed.

[0101] A program according to the twelfth aspect of this disclosure causes a computer to perform the following steps: acquire image data of a first image P1 in which a target area AR, which is a part of the detection area AR0 to be monitored, is shown, but object A is not shown; detect at least one of a false detection area and a missed area from the detection area AR0 based on the image data of the first image P1, in which object A is falsely detected; acquire a second image P2 which is at least one of an image in which object A is superimposed on an image showing a false detection area and an image in which object A is superimposed on an image showing a missed area; and use the image data of the second image P2 as training data to train an identification model M to determine the presence or absence of object A in an image. According to this disclosure, at least one of false detection and missed detection of object A can be appropriately suppressed.

[0102] Although embodiments of the present disclosure have been described above, the embodiments are not limited to those described herein. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the embodiments described above. [Explanation of symbols]

[0103] 10 Information Processing Devices 20 First Image Acquisition Unit 22 Missed Learning Department 24 False Detection Learning Unit 26 Detection unit 28 Second Image Acquisition Unit 30 Learning Department A Object AR target area AR0 detection area B False detection M-Discrimination Model MA Missed Detector Model MB False Positive Recognition Model P1 First image P2 Second image

Claims

1. A first image acquisition unit acquires image data of a first image that shows a target area, which is a part of the detection area to be monitored, but does not show the target object. A detection unit that, based on the image data of the first image, detects at least one of the false detection region where the object is falsely detected and the missed region where the object is missed from within the detection region, A second image acquisition unit acquires a second image which is at least one of an image in which the object is superimposed on an image showing the false detection area, and an image in which the object is superimposed on an image showing the missed area. A learning unit that uses the image data of the second image as training data to perform machine learning on an identification model to determine the presence or absence of the object in the image, including, Information processing device.

2. The information processing apparatus according to claim 1, wherein the detection unit detects the missed area by inputting the image data of the first image into a missed object identification model that has been machine-learned to determine the correspondence between an image and a difficulty score indicating the probability of missing an object.

3. The aforementioned missed detection model is further equipped with a machine learning unit for learning missed detections, The aforementioned missed learning section is, Using a training image in which the object is superimposed on the background, and designation information indicating that the object is present in the training image, the missed detection model is subjected to machine learning. The detection unit is By inputting the image data of the first image into the machine-learned missed detection model, the difficulty score for the first image is obtained. The information processing apparatus according to claim 2, which determines whether the target area includes the overlooked area based on the difficulty score.

4. The aforementioned missed learning unit uses machine learning to determine the difficulty score for each position in the learning image. The information processing apparatus according to claim 3, wherein the detection unit inputs image data of the first image to the missed detection model to obtain the difficulty score for each position of the first image.

5. The information processing apparatus according to claim 1 or 2, wherein the detection unit detects the false detection region by inputting image data of the first image into a false detection identification model that has been trained to determine the presence or absence of false detection objects that are falsely detected as the target object.

6. The system further includes a false detection learning unit that performs machine learning on the aforementioned false detection identification model, The false detection learning unit uses an image that does not contain the target object and designation information indicating that the image contains the falsely detected object as training data to train the false detection identification model. The information processing apparatus according to claim 5, wherein the detection unit determines whether the target area contains the falsely detected object by inputting the image data of the first image to the false detection identification model.

7. The information processing apparatus according to claim 1 or 2, wherein the second image acquisition unit acquires an image in which the false detection area or the missed area and the target object are captured as the second image.

8. The information processing apparatus according to claim 1 or 2, wherein the second image acquisition unit generates an image as the second image by superimposing the image of the object onto a background image in which the falsely detected area or the missed area is captured.

9. The information processing apparatus according to claim 8, wherein the second image acquisition unit superimposes the background image and the image of the object based on depth information indicating the depth at each position of the background image.

10. A detection device that detects the presence or absence of an object in the detection area using the identification model that has been trained by the learning unit of the information processing device according to claim 1 or claim 2.

11. An information processing method performed by an information processing device, The steps include: acquiring image data of a first image that shows a target area, which is a part of the detection area to be monitored, but does not show the target object; A step of detecting, based on the image data of the first image, at least one of the detection areas in which the object is falsely detected and the missed area in which the object is missed, The step of acquiring image data of a second image which is at least one of an image in which the object is superimposed on an image in which the false detection area is captured, and an image in which the object is superimposed on an image in which the missed area is captured. The steps include: using the image data of the second image as training data to train a recognition model to determine the presence or absence of the object in the image; including, Information processing methods.

12. The steps include: acquiring image data of a first image that shows a target area, which is a part of the detection area to be monitored, but does not show the target object; A step of detecting, based on the image data of the first image, at least one of the detection areas in which the object is falsely detected and the missed area in which the object is missed, A step of acquiring a second image which is at least one of an image in which the object is superimposed on an image showing the false detection area and an image in which the object is superimposed on an image showing the missed area. The steps include: using the image data of the second image as training data to train a recognition model to determine the presence or absence of the object in the image; Make the computer execute it. program.

Images