Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device which can generate a learned model with an easy labeling operation.SOLUTION: An information processing device 1 comprises: an acquisition unit 20; a definition unit 30: and a learning unit 40. The acquisition unit 20 acquires an image for learning including an object region being a region of each of a plurality of types of detection objects and a mixed region being a region in which the two or more types of detection objects are mixed. The definition unit 30 generates label data of distinguishing the plurality of regions including the object region and the mixed region in the image for learning on the basis of a labeling operation by a user to the image for learning. The learning unit 40 generates a learned model for detecting the object region and the mixed region from an input image on the basis of the label data and the image for learning.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] A technique called semantic segmentation using deep learning has been known in the past. In this technique, for example, an image captured by an imaging device is used as an input image, and an object region is detected from the input image on a pixel-by-pixel basis.

[0003] In an image captured by an imaging device, there may be an area where multiple objects are intertwined. Patent Document 1 discloses a technique for dividing a training image into multiple small areas based on area class label data in which an area class label is assigned to each pixel during the learning process, and assigning a mixed area label to an area among the divided small areas where labels from multiple classes are mixed. This technique enables the identification of mixed areas during the inference process. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-117019 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology described in Patent Document 1, a model for detecting mixed regions is generated using label data in which each pixel of a training image is assigned a label of the corresponding object class, which poses a problem in that the labeling operation for an image in which multiple objects are mixed is difficult and time-consuming.

[0006] The present disclosure has been made in consideration of the above, and aims to provide an information processing device that can generate a trained model with a simple labeling operation. [Means for solving the problem]

[0007] In order to solve the above-described problems and achieve the object, the information processing device of the present disclosure includes an acquisition unit, a definition unit, and a learning unit. The acquisition unit acquires training images including target regions, which are regions for each of multiple types of detection targets, and mixed regions, which are regions where two or more types of detection targets are mixed. The definition unit generates label data for distinguishing between multiple regions in the training images, including the target regions and the mixed regions, based on a labeling operation performed by a user on the training images. The learning unit generates a trained model for detecting the target regions and the mixed regions from an input image, based on the label data and the training images. The learning unit generates a trained model by reducing the error value when an area in the learning image that is indicated as a mixed area is mistakenly detected as the target area of ​​one of two types of detection targets compared to the error value when the target area is mistakenly detected as a different area. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to generate a trained model with a simple labeling operation. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to a first embodiment. [Figure 2] FIG. 10 is a diagram illustrating an example of labeling processing by a definition unit of the information processing device according to the first embodiment; [Figure 3] FIG. 10 is a diagram showing another example of the labeling process performed by the definition unit of the information processing device according to the first embodiment; [Figure 4] FIG. 10 is a diagram illustrating an example of labeling processing in a case where the definition unit of the information processing device according to the first embodiment does not have a function of adding a vehicle-water boundary label to a boundary region between a vehicle and water. [Figure 5] FIG. 10 is a diagram showing yet another example of the labeling process by the definition unit of the information processing device according to the first embodiment; [Figure 6]FIG. 10 is a diagram illustrating an example of a detection result of an inference unit of the information processing device according to the first embodiment and an image displayed on a display unit by a display processing unit. [Figure 7] FIG. 10 is a diagram illustrating another example of the detection result of the inference unit of the information processing device according to the first embodiment and the image displayed on the display unit by the display processing unit. [Figure 8] 1 is a flowchart illustrating an example of processing by the information processing device according to the first embodiment. [Figure 9] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device according to a first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] An information processing device, an information processing method, and an information processing program according to embodiments will be described in detail below with reference to the accompanying drawings.

[0011] Embodiment 1 1 is a diagram illustrating an example of a configuration of an information processing system according to the first embodiment. As illustrated in FIG. 1, the information processing system 100 according to the first embodiment includes an information processing device 1, an imaging device 2, and a data server 3.

[0012] The information processing device 1, the imaging device 2, and the data server 3 are connected to a communication network 4 and are capable of transmitting and receiving information to and from each other. The communication network 4 is, for example, a WAN (Wide Area Network) such as the Internet, but may also be a LAN (Local Area Network).

[0013] The imaging device 2 includes an imaging section and a communication section (not shown), and transmits a plurality of captured images obtained by imaging using the imaging section to the data server 3 via the communication section and the communication network 4. The data server 3 receives the plurality of captured images transmitted from the imaging device 2 via the communication network 4, and stores the received plurality of captured images.

[0014] The information processing device 1 acquires each of the multiple captured images stored in the data server 3 as a learning image, and generates a trained model that detects multiple types of regions from an input image based on the acquired learning images. The information processing device 1 also acquires the captured images stored in the data server 3 as input images, and inputs the acquired input images into the trained model to detect multiple types of regions from the input image.

[0015] The imaging device 2 is mounted on, for example, a drone, a helicopter, an airplane, or a satellite, and can capture images of the ground. Note that the imaging device 2 may also be mounted on, for example, a smartphone, a tablet PC (Personal Computer), a notebook PC, or a monitoring device.

[0016] In the following, an example will be described in which the information processing system 100 is used to detect areas that are in a disaster state, such as flooded areas, using deep learning technology during a disaster caused by heavy rain, etc., but the information processing system 100 is not limited to this example and can detect various conditions.

[0017] The information processing device 1 includes a communication unit 10, an acquisition unit 20, a definition unit 30, a learning unit 40, an inference unit 50, and a display processing unit 60. The communication unit 10 is communicably connected to a communication network 4, and transmits and receives information to and from the imaging device 2 and the data server 3.

[0018] The acquisition unit 20 includes a learning image acquisition unit 21 that acquires learning images, and an input image acquisition unit 22 that acquires input images. The learning image acquisition unit 21 acquires, for example, each of the plurality of captured images as a learning image from the data server 3 via the communication network 4 and the communication unit 10. The learning image acquisition unit 21 can also acquire each of the plurality of captured images as a learning image from the imaging device 2 via the communication network 4 and the communication unit 10.

[0019] The input image acquisition unit 22 acquires, for example, a captured image as an input image from the data server 3 via the communication network 4 and the communication unit 10. The input image acquisition unit 22 can also acquire a captured image as an input image from the imaging device 2 via the communication network 4 and the communication unit 10.

[0020] The definition unit 30 acquires the learning images acquired by the learning image acquisition unit 21, and performs labeling processing based on an operation on the operation unit 70 by the user of the information processing device 1. In the following, the user of the information processing device 1 will be simply referred to as the user.

[0021] In the labeling process, the definition unit 30 performs labeling on a plurality of regions in the learning image, including a plurality of types of target regions, a plurality of types of mixed regions, and a plurality of types of boundary regions. The definition unit 30 assigns a different index label to each type of region, and for example, assigns an index label assigned to the type of region corresponding to each pixel.

[0022] The operation unit 70 is, for example, at least one of a mouse and a keyboard, but may also be a touch panel, etc. The display processing unit 60 displays the labeling status of the learning image on the display unit 80, and the user can label each region in the learning image by performing a labeling operation on the operation unit 70 while checking the labeling status of the learning image displayed on the display unit 80.

[0023] The target area is an area for each detection target, and examples of multiple types of detection targets include, but are not limited to, buildings, vehicles, and water. For example, the detection targets may be roads, earth and sand, banks, people, bridges, or tunnels. The mixed area is an area where two or more types of detection targets are mixed, and examples of multiple types of mixed areas include, for example, an area where buildings and water are mixed, an area where vehicles and water are mixed, and an area where buildings, vehicles, and water are mixed. The boundary area is an area where two types of detection targets are mixed, and examples of multiple types of boundary areas include, for example, an area where buildings and water are mixed, and an area where vehicles and water are mixed.

[0024] The definition unit 30 includes a target region definition unit 31, a mixed region definition unit 32, and a boundary region definition unit 33. The target region definition unit 31 performs a process of adding a label to a target region specified by the user in the learning image for each type of detection target, based on a labeling operation by the user on the operation unit 70. As a result, different labels are added to the regions of the multiple types of detection targets.

[0025] The mixed area definition unit 32 performs a process of adding labels to the mixed areas specified by the user in the learning image for each type of mixed area based on the user's labeling operation on the operation unit 70. As a result, different labels are added to each of the multiple types of mixed areas.

[0026] 2 is a diagram illustrating an example of labeling processing by the definition unit of the information processing device according to the first embodiment. In FIG. 2, an example is shown in which labeling processing is performed on a learning image including an image of a building and an image of water, and a target area of ​​the building, a target area of ​​the water, and a mixed area of ​​the building and water are identified by the user in the learning image. In the example shown in FIG. 2, the target area of ​​the building, the target area of ​​the water, and the mixed area of ​​the building and water are each colored differently by the user's labeling operation on the operation unit 70, and the difference in color is indicated by different hatching.

[0027] The target area definition unit 31 adds a vehicle area label, which is a label of the building target area, to each pixel of the building target area specified by the user in the training image, and adds a water area label, which is a label of the water target area, to each pixel of the water target area specified by the user in the training image. In addition, the mixed area definition unit 32 adds a building / water mixture label, which is a label of the mixed area where buildings and water are mixed, to each pixel of the building / water mixture area specified by the user in the training image.

[0028] As shown in Figure 2, there are many buildings in the training image. Therefore, if there is no function to add building-water boundary labels to areas where buildings and water coexist, attempting to separate submerged buildings from water requires tedious labeling of small, complex shapes.

[0029] On the other hand, as described above, the definition unit 30 has a function of adding a building / water mixed label to a mixed area of ​​buildings and water. Therefore, as shown in Fig. 2, the user can roughly label the mixed area of ​​buildings and water, and can easily perform labeling using the information processing device 1 even when there is a boundary between the building and the water.

[0030] 1 performs a process of adding labels to the boundary areas specified by the user in the learning image for each type of boundary area based on the user's labeling operation on the operation unit 70. As a result, different labels are added to each of the multiple types of boundary areas.

[0031] 3 is a diagram illustrating another example of labeling processing by the definition unit of the information processing device according to the first embodiment. In FIG. 3, an example is shown in which labeling processing is performed on learning images including an image of a vehicle and an image of water, and a target region of the vehicle, a target region of the water, and a boundary region between the vehicle and the water are identified by a user in the learning images. In the example illustrated in FIG. 3, the target region of the vehicle, the target region of the water, and the boundary region between the vehicle and the water are each colored differently by a labeling operation performed by the user on the operation unit 70, and the differences in color are indicated by different hatching.

[0032] The target area definition unit 31 adds a vehicle area label, which is a label of the target area of ​​the vehicle, to each pixel of the target area of ​​the vehicle specified by the user in the training image, and adds a water area label, which is a label of the target area of ​​the water, to each pixel of the target area of ​​the water specified by the user in the training image. In addition, the boundary area definition unit 33 adds a vehicle-water boundary label, which is a label of the boundary between the vehicle and the water, to each pixel of the boundary area between the vehicle and the water specified by the user in the training image.

[0033] Here, a case where there is no function to add a vehicle-water boundary label to the boundary region between the vehicle and water will be described. Fig. 4 is a diagram showing an example of labeling processing when the definition unit of the information processing device according to the first embodiment does not have the function to add a vehicle-water boundary label to the boundary region between the vehicle and water. As shown in Fig. 4, when there is no function to add a vehicle-water boundary label to the boundary region between the vehicle and water, the user must carefully distinguish between the water region and the vehicle region, and the boundary between the water region and the vehicle region must be labeled with detailed and complex shapes, for example, by labeling splashes as water.

[0034] On the other hand, as described above, the definition unit 30 of the information processing device 1 according to the first embodiment has a function of adding a vehicle-water boundary label to the boundary area between the vehicle and water. Therefore, as shown in Fig. 3, the user can roughly label the boundary area between the vehicle and water, and even if there is a boundary between the vehicle and water, the user can easily perform labeling using the information processing device 1.

[0035] Fig. 5 is a diagram showing yet another example of labeling processing by the definition unit of the information processing device according to the first embodiment. Fig. 5 shows an example in which labeling processing is performed on learning images including an image of a building and an image of water, and a target area of ​​the building, a target area of ​​the water, and a boundary area between the building and the water are identified by the user in the learning images. In the example shown in Fig. 5, the target area of ​​the building, the target area of ​​the water, and the boundary area between the building and the water are each colored differently by the user's operation on the operation unit 70, and the difference in color is indicated by different hatching.

[0036] 5, the user can roughly label the boundary area between the building and the water. Therefore, even when there is a boundary between the building and the water, the user can easily perform labeling using the information processing device 1.

[0037] The definition unit 30 outputs a plurality of pieces of training information, each of which includes a training image and label data, to the training unit 40. The label data is data that distinguishes a plurality of regions in the training image, including a target region, a mixed region, and a boundary region, and includes label data for each pixel of the training image.

[0038] For example, in the example shown in Figure 2, the label data includes a building area label associated with each pixel of the building target area identified by the user in the training image, a water area label associated with each pixel of the water target area identified by the user in the training image, and a building-water mixture label associated with each image of the building-water mixture area identified by the user in the training image.

[0039] In the example shown in Figure 3, the label data includes a vehicle area label associated with each pixel of the vehicle target area identified by the user in the training image, a water area label associated with each pixel of the water target area identified by the user in the training image, and a vehicle-water boundary label associated with each image of the boundary area between the vehicle and water identified by the user in the training image.

[0040] Returning to FIG. 1 , the learning unit 40 will be described. The learning unit 40 generates a trained model that detects a target region, a boundary region, and a mixed region based on the multiple pieces of learning information output from the definition unit 30. The trained model is a trained model that performs semantic segmentation. Such a trained model is, for example, a neural network such as a convolutional neural network or a recurrent neural network, and is generated by deep learning.

[0041] The learning unit 40 includes an error value calculation unit 41 and a parameter learning unit 42. The error value calculation unit 41 inputs each of a plurality of learning images to a learning model and obtains an inference score for each pixel output from the learning model. The inference score ranges from 0 to 1, for example, and includes a score for each label.

[0042] The error value calculation unit 41 compares the inference score with the label data indicated by the label data for each pixel, and calculates an error value E. The label data indicated by pixel in the label data is so-called correct answer data. The error value calculation unit 41 calculates the error value E using cross entropy or square sum error, but the method of calculating the error value E is not limited to cross entropy and square sum error.

[0043] The parameter learning unit 42 determines the parameters of the trained model by repeating the process of adjusting the parameters of the training model so as to reduce the error value E calculated by the error value calculation unit 41. The parameter learning unit 42 sets the determined parameters in the inference unit 50, thereby generating the trained model.

[0044] The error value calculation unit 41 sets the error value E when a region in the learning image that is shown as a boundary region or a mixed region is mistakenly detected as the target region of one of two or more types of detection targets related to this region to be smaller than the error value E when the target region is mistakenly detected as a different region.

[0045] For example, when a mixed area of ​​buildings and water is erroneously detected as a building target area or a water target area, the error value calculation unit 41 reduces the error value E compared to when the building target area or the water target area is erroneously detected as a different area. Also, when a boundary area between a vehicle and water is erroneously detected as a vehicle target area or a water target area, the error value calculation unit 41 reduces the error value E compared to when the vehicle target area or the water target area is erroneously detected as a different area.

[0046] Here, an example will be described in which the error value calculation unit 41 calculates the error value E using cross entropy. The error value calculation unit 41 can calculate the error value E using the following formula (1) to calculate the cross entropy. In the following formula (1), "k" is an index of the classification class, and "t k " is the label of the region indicated by index k, "log" is the natural logarithm, and "y' k " is the error value of the region indicated by index k. E=-Σ(t k ×log(y' k )) ···(1)

[0047] Next, an example will be described in which the error value calculation unit 41 calculates the error value E using the square sum error. The error value calculation unit 41 can calculate the error value E using the following equation (2) as the calculation of the square sum error. E=1 / 2×{Σ(y' k -t k ) 2} ···(2)

[0048] Here, the index of the target region of the first detection target is 2, the index of the target region of the second detection target is 5, the index of the mixed region or boundary region of the first detection target and the second detection target is 25, and the output of the learning model is "y k The first detection target is, for example, a building or a vehicle, and the second detection target is, for example, water.

[0049] In this case, when k≠25, the error value calculation unit 41 calculates y' k= y k Furthermore, when k=25, the error value calculation unit 41 calculates y' 25 =y 25 +y2+y5 or y' 25 =y 25 +α(y2+y5), and then the above formula (1) or (2) is calculated. Note that α is a value expressed as 0<α<1. This makes it possible to make the error value E smaller when a mixed region or boundary region of the first detection target and the second detection target is mistakenly detected as the target region of the first detection target or the target region of the second detection target than the error value E when the target regions are mistakenly detected as different regions.

[0050] The error value calculation unit 41 only needs to be able to perform processing that makes the error value E smaller when an area indicated as a boundary area or a mixed area is mistakenly detected as the target area of ​​one of two types of detection targets related to this area than the error value E when the target area is mistakenly detected as a different area, and the method of making the error value E smaller is not limited to the method described above.

[0051] In addition to the process for reducing the error value E, a process for reducing the error value E when two types of detection objects are mistakenly detected as a boundary area or a mixed area may be added. In the case of k=2, y'2=y2+y 25 or y'2=y2+β(y 25 ) where β is a value expressed as 0<β<1.

[0052] 1, the inference unit 50 and the display processing unit 60 will be described. The inference unit 50 detects a target region, a boundary region, and a mixed region from the input image acquired by the input image acquisition unit 22, using a trained model in which parameters determined by the training unit 40 are set.

[0053] The display processing unit 60 displays an image on the display unit 80 in which the boundary region, one of the regions of the two types of detection targets corresponding to the boundary region, and the mixed region are each highlighted as abnormal regions. For example, the highlighted region of one of the two types of detection targets corresponding to the boundary region is the region of the vehicle that is partially submerged in water in the case of a boundary region between a vehicle and water, and is the region of the building that is partially submerged in water in the case of a boundary region between a building and water.

[0054] 6 is a diagram illustrating an example of a detection result of the inference unit and an image displayed on the display unit by the display processing unit of the information processing device according to the first embodiment. The input image shown in FIG. 6 is the same as the learning image shown in FIG.

[0055] The inference unit 50 detects a target area of ​​a building, a target area of ​​water, and a mixed area of ​​a building and water in the input image by inputting the input image shown in Fig. 6 into the trained model. The display processing unit 60 causes the display unit 80 to display a highlighted image in which the mixed area of ​​a building and water detected by the inference unit 50 in the input image is highlighted as an abnormal area.

[0056] As a result, for example, the information processing device 1 can provide a user of the information processing device 1 with a highlighted image that allows the user to easily grasp a flooded area, which is an area where a building is flooded in the event of a disaster caused by heavy rain, etc. Therefore, the user of the information processing device 1 can easily grasp the flooding of a wide area, etc., based on an aerial image captured by the imaging device 2, etc.

[0057] 7 is a diagram illustrating another example of the detection result of the inference unit and the image displayed on the display unit by the display processing unit of the information processing device according to the first embodiment. The input image shown in FIG. 7 is the same as the learning image shown in FIG.

[0058] 7 to the trained model, the inference unit 50 detects the target region of the vehicle, the target region of the water, and the boundary region between the vehicle and the water in the input image. The display processing unit 60 causes the display unit 80 to display a highlighted image in which the target region of the vehicle and the boundary region between the vehicle and the water detected by the inference unit 50 in the input image are highlighted as abnormal regions.

[0059] This allows the information processing device 1 to provide the user of the information processing device 1 with a highlighted image that allows the user to easily identify vehicles that are submerged in water during a disaster such as heavy rain, for example.

[0060] Next, a description will be given of a process performed by the information processing device 1 using a flowchart. Fig. 8 is a flowchart showing an example of a process performed by the information processing device according to the first embodiment.

[0061] 8, the information processing device 1 determines whether or not a labeling operation has been performed by the user (step S10). If the information processing device 1 determines that a labeling operation has been performed by the user (step S10: Yes), it generates label data based on the labeling operation by the user (step S11).

[0062] When the process of step S11 is completed or when it is determined that the user has not performed a labeling operation (step S10: No), the information processing device 1 determines whether or not it is time to learn (step S12). For example, the information processing device 1 determines that it is time to learn when it has generated a predetermined number or more of learning information including new label data.

[0063] When the information processing device 1 determines that the timing for learning has arrived (step S12: Yes), the information processing device 1 generates a trained model based on a plurality of pieces of learning information including new learning information (step S13).

[0064] When the process of step S13 is completed or when it is determined that the learning timing has not arrived (step S12: No), the information processing device 1 determines whether or not an input image is present (step S14). When it is determined that an input image is present (step S14: Yes), the information processing device 1 inputs the input image into the trained model and performs a region detection process to detect a plurality of regions, including a target region, a mixed region, and a boundary region, contained in the input image (step S15).

[0065] Next, the information processing device 1 performs an abnormal region highlighting process to highlight, as abnormal regions, a mixed region included in the input image and a region including a boundary region included in the input image and one of two types of detection targets corresponding to the boundary region (step S16).Then, the information processing device 1 performs a display process to display, on the display unit 80, a highlighted image that is the input image that has been subjected to the abnormal region highlighting process (step S17).

[0066] When the process of step S17 is completed or when it is determined that there is no input image (step S14: No), the information processing device 1 determines whether it is time to end the operation (step S18). For example, when it is determined that the power supply (not shown) of the information processing device 1 has been turned off or when it is determined that an operation to end the operation has been performed on the operation unit 70, the information processing device 1 determines that it is time to end the operation.

[0067] If the information processing device 1 determines that it is not time to end the operation (step S18: No), it proceeds to step S10, and if it determines that it is time to end the operation (step S18: Yes), it terminates the processing shown in Figure 8.

[0068] 9 is a diagram illustrating an example of a hardware configuration of the information processing device according to the first embodiment. As illustrated in FIG. 9, the information processing device 1 includes a computer including a processor 101, a memory 102, a communication device 103, an input / output device 104, and a bus 105.

[0069] The processor 101, memory 102, communication device 103, and input / output device 104 can transmit and receive information to and from each other via a bus 105. The processor 101 executes the functions of an acquisition unit 20, a definition unit 30, a learning unit 40, an inference unit 50, and a display processing unit 60 by reading and executing a program stored in the memory 102. The processor 101 is, for example, an example of a processing circuit, and includes one or more of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a system LSI (Large Scale Integration).

[0070] The memory 102 includes one or more of a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), and an EEPROM (Electrically Erasable Programmable Read Only Memory). The memory 102 also includes a recording medium on which a computer-readable program is recorded. Such a recording medium includes one or more of a non-volatile or volatile semiconductor memory, a magnetic disk, a flexible memory, an optical disk, a compact disk, and a DVD (Digital Versatile Disc). The information processing device 1 may also include an integrated circuit such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array).

[0071] As described above, the information processing device 1 according to the first embodiment includes the acquisition unit 20, the definition unit 30, and the learning unit 40. The acquisition unit 20 acquires a training image including target regions, which are regions for each of multiple types of detection targets, and a mixed region, which is a region where two or more types of detection targets are mixed. The definition unit 30 generates label data for distinguishing multiple regions, including the target regions and the mixed region, in the training image based on a labeling operation by a user on the training image. The learning unit 40 generates a trained model that detects the target regions and the mixed region from the input image based on the label data and the training image. This allows the information processing device 1 to generate a trained model that can detect the mixed region in addition to the target region with a simple labeling operation.

[0072] Furthermore, the learning unit 40 generates a trained model by making the error value E smaller when a region in the training image indicated as a mixed region is erroneously detected as a target region of one of two types of detection targets than the error value E when the target region is erroneously detected as a different region. This allows the information processing device 1 to accurately generate a trained model that can detect mixed regions in addition to target regions with a simple labeling operation.

[0073] Here, easy labeling operation means that there is no need to strictly define the range of the mixed region and to label even small regions without error, and the region division is automatically optimized during the learning process based on the overall labeled information. In other words, even if a region that should originally be labeled as a target region of one of the two types of detection targets is labeled as a mixed region, it will be correctly labeled during the learning process.

[0074] The multiple regions also include a boundary region that is a region at the boundary between two types of detection targets among the multiple types of detection targets. The learning unit 40 generates a trained model by reducing the error value E when a region in the learning image indicated as a boundary region is erroneously detected as a target region of one of the two types of detection targets compared to the error value E when the target region is erroneously detected as a different region. This allows the information processing device 1 to generate a trained model that can detect boundary regions in addition to target regions and mixed regions with a simple labeling operation. The effects of the simple labeling operation are as described above.

[0075] If a process is added to reduce the error value E when two types of detection targets are mistakenly detected as a boundary region or a mixed region, the region division will be automatically optimized during the learning process, as described above, in cases where the boundary region or mixed region is labeled narrower than the ideal correct labeling.

[0076] The information processing device 1 also includes an acquisition unit 20 that acquires an input image, and an inference unit 50 that uses the trained model generated by the learning unit 40 to detect a target region and a mixed region from the input image acquired by the acquisition unit 20. This allows the information processing device 1 to detect a mixed region in addition to a target region.

[0077] The information processing device 1 also includes an acquisition unit 20 that acquires an input image, and an inference unit 50 that uses the trained model generated by the learning unit 40 to detect a target region, a mixed region, and a boundary region from the input image acquired by the acquisition unit 20. This allows the information processing device 1 to detect a mixed region and a boundary region in addition to the target region.

[0078] The information processing device 1 also includes a definition unit 30 that generates label data based on a labeling operation performed by a user on a training image. The learning unit 40 generates a trained model based on the label data generated by the definition unit 30. This enables the information processing device 1 to generate a trained model that can detect a mixed region in addition to a target region with a simple labeling operation.

[0079] The information processing device 1 also includes a display processing unit 60 that displays an image on the display unit 80 in which the mixed region detected by the inference unit 50 in the input image is highlighted as an abnormal region. This allows the information processing device 1 to present to the user an image in which the mixed region in the input image can be easily recognized as an abnormal region.

[0080] The information processing device 1 also includes a display processing unit 60 that causes the display unit 80 to display an image in which the boundary region detected by the inference unit 50 in the input image, the target region of one of the two types of detection targets corresponding to the boundary region, and the mixed region are each highlighted as abnormal regions. This allows the information processing device 1 to present to the user an image in which the region including the boundary region and the target region of one of the two types of detection targets, and the mixed region in the input image can be easily recognized as abnormal regions.

[0081] The information processing device 1 also includes an acquisition unit 20 that acquires an input image, and an inference unit 50 that detects target regions and mixed regions from the input image acquired by the acquisition unit 20 using a trained model that detects target regions, which are regions of each of the multiple types of detection targets, and mixed regions, which are a mixture of two or more of the multiple types of detection targets. The trained model is a trained model generated by making the error value E smaller when a mixed region in a training image is erroneously detected as a target region of one of the two types of detection targets than the error value E when other regions are erroneously detected as different regions. This allows the information processing device 1 to detect mixed regions in addition to target regions using the trained model generated by a simple labeling operation.

[0082] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, and parts of the configurations may be omitted or modified without departing from the spirit of the invention. [Explanation of symbols]

[0083] 1 Information processing device, 2 Imaging device, 3 Data server, 4 Communication network, 10 Communication unit, 20 Acquisition unit, 21 Learning image acquisition unit, 22 Input image acquisition unit, 30 Definition unit, 31 Target area definition unit, 32 Mixed area definition unit, 33 Boundary area definition unit, 40 Learning unit, 41 Error value calculation unit, 42 Parameter learning unit, 50 Inference unit, 60 Display processing unit, 70 Operation unit, 80 Display unit.

Claims

1. an acquisition unit that acquires a learning image including a target region that is a region of each of a plurality of types of detection targets and a mixed region that is a region in which two or more types of detection targets are mixed; a definition unit that generates label data for distinguishing between a plurality of regions in the learning image, including the target region and the mixed region, based on a labeling operation by a user on the learning image; a learning unit that generates a trained model that detects the target region and the mixed region from an input image based on the label data and the training image, The learning unit The trained model is generated by making an error value smaller when a region in the training image that is indicated as the mixed region is erroneously detected as a target region of one of the two types of detection targets than when the target region is erroneously detected as a different region.

1. An information processing device comprising:

2. The plurality of regions are: a boundary region that is a region of a boundary between two types of detection targets among the plurality of types of detection targets, The learning unit The trained model is generated by making an error value smaller when a region indicated as the boundary region in the training image is erroneously detected as a target region of one of the two types of detection targets than when the target region is erroneously detected as a different region.

2. The information processing apparatus according to claim 1, wherein:

3. an acquisition unit that acquires an input image; an inference unit that detects the target region and the mixed region from the input image acquired by the acquisition unit using the trained model generated by the training unit.

2. The information processing apparatus according to claim 1, wherein:

4. an acquisition unit that acquires an input image; an inference unit that detects the target region, the mixed region, and the boundary region from the input image acquired by the acquisition unit using the trained model generated by the training unit.

3. The information processing apparatus according to claim 2, wherein:

5. a display processing unit that displays on a display unit an image in which the mixed region detected by the inference unit in the input image is highlighted as an abnormal region; 4. The information processing apparatus according to claim 3,

6. a display processing unit that displays on a display unit an image in which the boundary region detected by the inference unit in the input image, a region of one of the two types of target regions corresponding to the boundary region, and the mixed region are each highlighted as an abnormal region; 5. The information processing apparatus according to claim 4,

7. an acquisition unit that acquires an input image; an inference unit that detects, from the input image acquired by the acquisition unit, a target region that is a region of each of a plurality of types of detection targets and a mixed region in which two or more types of detection targets are mixed, using a trained model that detects the target region and the mixed region; The trained model is The trained model is generated by making the error value when the mixed region in the training image is erroneously detected as a target region of one of the two types of detection targets smaller than the error value when the target region is erroneously detected as a different region.

1. An information processing device comprising:

8. 1. A computer-implemented information processing method, comprising: an acquisition step of acquiring a learning image including a target region that is a region of each of a plurality of types of detection targets and a mixed region that is a region in which two or more types of detection targets are mixed; a defining step of generating label data for distinguishing a plurality of regions in the learning image, including the target region and the mixed region, based on a labeling operation by a user on the learning image; a learning step of generating a trained model that detects the target region and the mixed region from an input image based on the label data and the training image, The learning step The trained model is generated by making an error value smaller when a region in the training image that is indicated as the mixed region is erroneously detected as a target region of one of the two types of detection targets than when the target region is erroneously detected as a different region. An information processing method comprising:

9. 1. A computer-implemented information processing method, comprising: an acquisition step of acquiring an input image; an inference step of detecting, from the input image acquired by the acquisition step, a target region that is a region of each of a plurality of types of detection targets and a mixed region in which two or more types of detection targets are mixed, using a trained model that detects the target region and the mixed region; The trained model is The trained model is generated by making the error value when the mixed region in the training image is erroneously detected as a target region of one of the two types of detection targets smaller than the error value when the target region is erroneously detected as a different region. An information processing method comprising:

10. an acquisition step of acquiring a learning image including a target region that is a region of each of a plurality of types of detection targets and a mixed region that is a region in which two or more types of detection targets are mixed; a defining step of generating label data for distinguishing a plurality of regions in the learning image, including the target region and the mixed region, based on a labeling operation by a user on the learning image; a learning step of generating a trained model that detects the target region and the mixed region from an input image based on the label data and the training image; The learning step The trained model is generated by making an error value smaller when a region in the training image that is indicated as the mixed region is erroneously detected as a target region of one of the two types of detection targets than when the target region is erroneously detected as a different region. An information processing program characterized by:

11. an acquisition step of acquiring an input image; an inference step of detecting, from the input image acquired by the acquisition step, a target region that is a region of each of a plurality of types of detection targets and a mixed region in which two or more types of detection targets are mixed, using a trained model that detects the target region and the mixed region; The trained model is The trained model is generated by making the error value when the mixed region in the training image is erroneously detected as a target region of one of the two types of detection targets smaller than the error value when the target region is erroneously detected as a different region. An information processing program characterized by:

Images