Information processing method, program and information processing device

Abstract

To improve detection accuracy of characteristics of a determination object.SOLUTION: An information processing method that is executed by a computer includes: an acquisition step for acquiring determination image data being image data including an object structure being a structure of an object whose prescribed characteristics are detected, and a background; a separation step for generating separation image data being image data which is obtained by separating the background being a subject other than the object structure included in the determination image data and includes the object structure; and an output step for outputting an area in which the prescribed characteristics included in the determination image data are detected by inputting the separation image data to a learned model that performs learning so as to detect the prescribed characteristics including the input data when the image data including the object structure is made to be the input data.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing method, a program, and an information processing device. [Background technology]

[0002] 2. Description of the Related Art There is known a device that diagnoses the deterioration state of equipment by photographing images of rust that has developed on the equipment (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-189523 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has a problem in that rust on equipment may be erroneously detected.

[0005] Therefore, the present invention has been made in consideration of these points, and has an object to improve the detection accuracy of features of a determination target. [Means for solving the problem]

[0006] The information processing method of the first aspect of the present invention includes an acquisition step executed by a computer to acquire judgment image data, which is image data including a target structure, which is a structure for which predetermined features are to be detected, and a background; a separation step to generate separated image data, which is image data in which the background, which is a subject other than the target structure, contained in the judgment image data is separated, and which is image data including the target structure; and an output step to output an area in which the predetermined features contained in the judgment image data are detected by inputting the separated image data into a trained model that has been trained to detect the predetermined features contained in input data when image data including the target structure is used as input data.

[0007] The method may further include a division step of dividing the separated image data, and in the output step, the separated image data divided in the division step may be input to the trained model to output an area in which the specified feature contained in the judgment image data is detected.

[0008] The method may further include a division step of dividing the separated image data and generating a plurality of divided image data by resizing the divided image data to a predetermined number of pixels, and in the output step, the divided image data may be input to the trained model to output an area in which the predetermined feature contained in the judgment image data is detected.

[0009] The method may further include a receiving step of receiving a selection of a criterion for dividing the image data in the dividing step, and the dividing step may divide the separated image data based on the criterion received in the receiving step.

[0010] The trained model may be characterized in that it has been trained using image data including a structure in which the specified features can be detected and a background, and an area of ​​the specified features in the structure as training data.

[0011] The trained model may be characterized in that it has further learned, as training data, image data in which background, which is a subject other than a structure contained in the image data, is separated from the image data, and an area of ​​the specified feature in the structure.

[0012] The trained model may further be characterized in that it has learned, as training data, image data obtained by dividing image data including a structure in which the specified feature can be detected and a background, and an area of ​​the specified feature in the structure included in the image data.

[0013] In the program of the second aspect of the present invention, the computer is caused to execute an acquisition step of acquiring judgment image data, which is image data including a target structure, which is a structure to be detected for a predetermined feature, and a background; a separation step of generating separated image data, which is image data including the target structure, in which the background, which is a subject other than the target structure, contained in the judgment image data is separated; and an output step of outputting an area in which the predetermined feature contained in the judgment image data is detected by inputting the separated image data into a trained model that has been trained to detect the predetermined feature contained in input data when image data including the target structure is used as input data.

[0014] An information processing device of a third aspect of the present invention has a memory unit that stores a trained model that has been trained to detect specified features contained in input data when the input data includes image data that includes a target structure, which is a structure that is a target for detecting specified features, the input data; an acquisition unit that acquires judgment image data, which is image data that includes the target structure and a background; a background separation unit that generates separated image data, which is image data in which the background, which is a subject other than the target structure included in the judgment image data, is separated, the separated image data being image data that includes the target structure; and an output unit that inputs the separated image data to the trained model and outputs an area in which the specified features included in the judgment image data are detected. Effect of the Invention

[0015] The present invention has an effect of improving the detection accuracy of the feature of the determination target. [Brief description of the drawings]

[0016] [Figure 1] 1 is a diagram for explaining an overview of an information processing system S according to an embodiment. [Diagram 2] 2 is a diagram for explaining processing in an information processing system S according to an embodiment. FIG. [Diagram 3] 1 is a block diagram showing a configuration of an information processing device 1. FIG. [Figure 4] 2 is a diagram for explaining processing in an information processing system S according to an embodiment. FIG. [Diagram 5] 3 is a flowchart showing a process flow in the information processing device 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] <First embodiment> FIG. 1 is a diagram for explaining an overview of an information processing system S according to an embodiment. The information processing system S is a system for determining the state of a structure to be inspected. The information processing system S has an information processing device 1 and an information terminal 2. The information processing device 1 and the information terminal 2 are communicatively connected via a network. The information processing system S may have multiple information terminals 2.

[0018] The information processing device 1 is a device that judges whether or not a predetermined feature is detected in a subject included in image data. Examples of the predetermined feature include dirt, mold, moss, and other attachments to a structure, and features that are recognized when a structure is deteriorated. Specifically, the predetermined feature is rust that occurs on a subject included in image data, but is not limited thereto. The predetermined feature may be, for example, scratches, cracks, discoloration, deterioration of markings, and the like that occur on a subject included in image data. Subjects to be judged in the information processing device 1 are structures such as steel towers, bridges, roads, railroad tracks, and utility poles. The information processing device 1 is, for example, a server. The information processing device 1 stores a trained model for judging the presence or absence of a predetermined feature. The trained model will be described later.

[0019] The information terminal 2 is a terminal used by a user of the information processing system S. The user of the information processing system S is, for example, a person in charge of maintaining the structure to be determined. The information terminal 2 transmits image data including the structure to be determined as a subject to the information processing device 1, and displays a screen showing the determination result acquired from the information processing device 1. The information terminal 2 is, for example, a smartphone, a tablet, or a personal computer.

[0020] An overview of the processing in the information processing system S will be described with reference to FIG. 2. First, in the learning phase, the information processing device 1 generates a trained model trained using judgment image data D1 for learning and supervised data D2 corresponding to the judgment image data D1 as teacher data. The judgment image data D1 is image data including a target structure. The judgment image data D1 may further include the background of the target structure, which is the subject. The supervised data D2 is information indicating an area having a predetermined feature in the target structure included in the judgment image data D1. When image data including the target structure is input, the trained model trained in this way outputs an area that detects the predetermined feature included in the input data.

[0021] In the inference phase, the information processing device 1 separates the background included in the determination image data D3 of the inference target, and generates separated image data D4. The separated image data D4 is image data obtained by separating the background, which is a subject other than the target structure included in the determination image data D3, and is image data including the target structure.

[0022] The information processing device 1 inputs the generated separated image data D4 into the trained model and outputs the judgment result. The judgment result includes information indicating an area in which a predetermined feature is detected in the target structure included in the judgment image data D3. The judgment result may further include information indicating the degree to which the predetermined feature is detected or the quality of the predetermined feature. As an example, the judgment result may include information indicating the density of the chorus or the type of the chorus for each pixel or area of ​​the judgment image data D3.

[0023] The information processing system S configured in this way has the effect of improving the detection accuracy of the predetermined feature. That is, in the information processing system S, by determining the predetermined feature based on image data from which the background has been separated, it is possible to reduce the problem of erroneously detecting a subject other than the subject to be determined that appears in the background as the subject to be determined.

[0024] [Configuration of information processing device 1] 3 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 has a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 has a learning unit 131, an acquisition unit 132, a background separation unit 133, an output unit 134, a division unit 135, and a reception unit 136.

[0025] The communication unit 11 is a communication interface for transmitting and receiving data to and from other devices via a network. The storage unit 12 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), a hard disk drive, etc. The storage unit 12 stores in advance a program to be executed by the control unit 13.

[0026] The memory unit 12 stores a trained model that has been trained to detect specified features contained in input data when the input data is image data including a target structure, which is a structure for which specified features are to be detected.

[0027] The control unit 13 is a processor such as a CPU (Central Processing Unit), etc. The control unit 13 executes a program stored in the storage unit 12, thereby functioning as a learning unit 131, an acquisition unit 132, a background separation unit 133, an output unit 134, a division unit 135, and a reception unit 136.

[0028] The learning unit 131 acquires judgment image data D1 for learning and correct answer data D2 corresponding to the judgment image data D1. The judgment image data D1 for learning and the correct answer data D2 may be stored in the storage unit 12. The learning unit 131 may acquire the judgment image data D1 for learning and the correct answer data D2 from the information terminal 2 or an external device (not shown). The learning unit 131 trains a machine learning model on the judgment image data D1 for learning and the correct answer data D2, generates a trained model, and stores it in the storage unit 12.

[0029] The acquisition unit 132 acquires the judgment image data D3 of the inference target. The acquisition unit 132 may acquire the judgment image data D3 from the information terminal 2, or may acquire the judgment image data D3 from an external device (not shown).

[0030] The background separation unit 133 generates separated image data D4 based on the determination image data D3. The background separation unit 133 separates the background included in the determination image data D3 from the determination image data D3 using a known background separation method, and generates separated image data D4.

[0031] The output unit 134 inputs the separated image data D4 to the trained model, and outputs an area in which a predetermined feature included in the judgment image data D3 is detected. As an example, the output unit 134 may cause the information terminal 2 to display a screen in which an area in which a predetermined feature indicated by the output result output by the trained model is detected is superimposed on the judgment image data D3. As an example, the output unit 134 may cause the information terminal 2 to display in a different manner depending on the degree or quality of the predetermined feature included in the judgment result.

[0032] The information processing device 1 configured in this way has the effect of improving the detection accuracy of a predetermined feature.

[0033] The information processing device 1 may be configured to separate the background, divide the image after the separation, and determine a predetermined feature based on the divided image data. The process in this case will be described with reference to FIG.

[0034] The division unit 135 divides the separated image data D4 to generate a plurality of divided image data D5. The division unit 135 divides the separated image data D4 into a plurality of image data by a known image division technique. The division unit 135 divides the separated image data D4 so that the number of pixels of the divided image data D5 is equal to or less than the number of pixels that can be input to the trained model (e.g., the number of input layers).

[0035] The output unit 134 inputs the divided image data D5 to the trained model, and outputs the area where the predetermined feature included in the judgment image data D3 is detected. The output unit 134 inputs the divided image data D5 to the trained model instead of or in addition to the judgment image data D3, and outputs the area where the predetermined feature is detected.

[0036] By configuring the information processing device 1 in this way, it is possible to improve the detection accuracy when the area where a predetermined feature is detected is small compared to the size of the structure to be judged that appears in the judgment image data. In particular, when an image is reduced, the shape of the feature area approaches a point, and by performing image division on a small area that is indistinguishable from noise, the shape of the area is maintained, which is effective in improving the detection accuracy.

[0037] The information processing device 1 may be configured to determine a predetermined characteristic based on an image obtained by resizing the image data after dividing the image data.

[0038] The division unit 135 divides the separated image data D4, and generates a plurality of divided image data D5 by resizing the divided image data to a predetermined number of pixels. As an example, the division unit 135 resizes the divided image data D5 after division so that the number of pixels constituting the divided image data D5 is greater than the number of pixels of the area corresponding to the pixels constituting the divided image data D5 in the separated image data D4 before division. The division unit 135 generates the divided image data D5 resized so that the number of pixels after resizing is equal to or less than the number of pixels that can be input to the trained model. The output unit 134 inputs the divided image data D5 to the trained model, and outputs the area in which a predetermined feature included in the judgment image data is detected.

[0039] By configuring the information processing device 1 in this way, it is possible to improve the detection accuracy for a predetermined feature that appears small in the determination image data, compared to the case where the image is simply divided.

[0040] The information processing device 1 may be configured to accept criteria for dividing an image.

[0041] Receiving unit 136 receives a selection of a criterion for dividing image data in dividing unit 135. Examples of the criterion for dividing image data include a maximum or minimum number of pixels, an aspect ratio, a number of divisions, a maximum data size, etc. Receiving unit 136 displays a screen for receiving the selection of a criterion for dividing image data on information terminal 2, and acquires information indicating the criterion selected on information terminal 2.

[0042] The division unit 135 divides the separated image data D4 based on the criterion accepted in the accepting step. As an example, when the criterion accepted by the accepting unit 136 indicates the maximum number of pixels, the dividing unit 135 divides the separated image data D4 to generate divided image data D5 such that the number of pixels of the divided image data does not exceed the maximum number of pixels accepted by the accepting unit 136.

[0043] By configuring the information processing device 1 in this way, it is possible to divide the image into sizes appropriate for the structure to be determined, which has the effect of improving the accuracy of the determination result.

[0044] By learning the target image data D1 as target image data for learning without separating the background from the target image data D1, it is possible to improve the detection accuracy of a specific region regardless of the background separation technology employed. That is, even if the background separation technology cannot completely remove the background and a small amount of background image is mixed in, it is possible to reduce erroneous detection of a specific region. In this case, the learning unit 131 generates a trained model using the learning judgment image data D1 including the target structure and the background and the correct answer data D2 as training data. That is, the trained model in this case is characterized by having been trained using image data including a structure in which a predetermined feature can be detected and the background, and a region of the predetermined feature in the structure as training data.

[0045] The trained model is generated by learning from image data including a structure, from which the background has been separated, as training data, thereby improving the detection accuracy of the feature of the determination target. In this case, the learning unit 131 generates the trained model by training training data to which separated image data for training is further added. The separated image data for training is image data generated by separating the background from the determination image data D1 for training, which includes the target structure and the background. In this case, the trained model is characterized by having further learned from the image data, from which the background, which is a subject other than the structure included in the image data, has been separated, and an area of ​​a predetermined feature in the structure, as training data.

[0046] In the above example, an example was described in which a trained model is generated by training separated image data for learning in addition to the judgment image data for learning D1, but the information processing device 1 may be configured to generate a trained model by training separated image data for learning in place of the judgment image data for learning D1.

[0047] The information processing device 1 may be configured to further learn the divided image data as teacher data. By configuring the information processing device 1 in this way, it is possible to improve the detection accuracy of a predetermined feature detected in a small area. In this case, the learning unit 131 generates a trained model using the divided image data for learning and the correct answer data D2 as teacher data. The divided image data for learning is image data generated by dividing the judgment image data for learning D1. The learning unit 131 trains the divided image data for learning instead of or in addition to the judgment image data D1. The trained model in this case is characterized in that it learns image data obtained by dividing image data including a structure in which a predetermined feature can be detected and a background, and an area of ​​a predetermined feature in the structure included in the image data as teacher data.

[0048] [Processing flow in information processing device 1] Fig. 5 is a flowchart showing the flow of processing in the information processing device 1. The flowchart in Fig. 5 starts from the point in time when an operation for starting judgment of a predetermined characteristic is performed.

[0049] The acquisition unit 132 acquires the determination image data (S01). The background separation unit 133 separates the background included in the acquired determination image data D3 and generates separated image data D4 (S02).

[0050] The division unit 135 divides the separated image data D4 to generate divided image data D5. The output unit 134 inputs the divided image data D5 to the trained model and causes the determination result to be output (S04). The output unit 134 may also input the separated image data D4 to the trained model and causes the determination result to be output. The output unit 134 causes the information terminal 2 to display a screen showing the output determination result (S05). The information processing device 1 ends the process.

[0051] [Effects of information processing device 1] By configuring the information processing device 1 as described above, it is possible to achieve the effect of improving the detection accuracy of the feature of the determination target.

[0052] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination combines the effect of the original embodiment. [Explanation of symbols]

[0053] 1. Information processing device 2. Information terminal 11 Communications Department 12 Storage section 13 Control section 131 Learning Department 132 Acquisition Department 133 Background separation section 134 Output section 135 Division 136 Reception

Claims

1. The computer executes An acquisition step of acquiring judgment image data that is image data including a target structure that is a target structure for detecting a predetermined feature and a background; a separation step of generating separated image data, which is image data obtained by separating a background that is a subject other than the target structure included in the determination image data, and which is image data including the target structure; an output step of inputting the separated image data into a trained model trained to detect the predetermined feature included in input data when the input data includes the target structure, thereby outputting an area in which the predetermined feature included in the determination image data is detected; An information processing method comprising the steps of:

2. The method further includes a division step of dividing the separated image data, In the output step, the separated image data divided in the division step is input to the trained model to output an area in which the predetermined feature included in the determination image data is detected. The information processing method according to claim 1 .

3. The method further includes a division step of dividing the separated image data and generating a plurality of divided image data by resizing the divided image data to a predetermined number of pixels, In the output step, the divided image data is input to the trained model to output an area in which the predetermined feature included in the determination image data is detected. The information processing method according to claim 1 .

4. The method further includes a receiving step of receiving a selection of a criterion for dividing the image data in the dividing step, The dividing step divides the separated image data based on the criteria accepted in the accepting step.

4. The information processing method according to claim 2 or 3.

5. The trained model is characterized in that it has been trained using image data including a structure in which the predetermined feature can be detected and a background, and an area of ​​the predetermined feature in the structure as training data. The information processing method according to claim 1 .

6. The trained model further learns image data obtained by separating a background, which is a subject other than a structure contained in the image data, from the image data, and an area of ​​the predetermined feature in the structure as training data. The information processing method according to claim 5.

7. The trained model further learns image data obtained by dividing image data including a structure in which the predetermined feature can be detected and a background, and an area of ​​the predetermined feature in the structure included in the image data, as training data. The information processing method according to claim 5.

8. On the computer, An acquisition step of acquiring judgment image data that is image data including a target structure that is a target structure for detecting a predetermined feature and a background; a separation step of generating separated image data, which is image data obtained by separating a background that is a subject other than the target structure included in the determination image data, and which is image data including the target structure; an output step of inputting the separated image data into a trained model trained to detect the predetermined feature included in input data when the input data includes the target structure, thereby outputting an area in which the predetermined feature included in the determination image data is detected; A program that executes the following.

9. A storage unit that stores a trained model that has been trained to detect a predetermined feature included in input data when image data including a target structure that is a structure for which a predetermined feature is to be detected is used as input data; and An acquisition unit that acquires determination image data that is image data including a target structure and a background; a background separation unit that generates separated image data, which is image data that includes the target structure and is obtained by separating a background that is a subject other than the target structure included in the determination image data; an output unit that outputs an area in which the predetermined feature included in the determination image data is detected by inputting the separated image data to the trained model; An information processing device having the above configuration.

Images