Information processing device, method for generating teacher data, method for generating trained model, and program

Abstract

To generate teacher data including a label indicating a position and a range of a detection object correctly.SOLUTION: An information processing device (1) comprises: a teacher image generation unit (101) that generates a teacher image by applying conversion that enlarges or shrinks an original image on which a detection object is photographed in a height direction and a width direction; a coordinate conversion unit (102) that converts a coordinate of each apex indicating a position and a range of the detection object photographed on the original image into a coordinate of each apex indicating a position and a range of the detection object photographed on the teacher image based on the magnification of the conversion; and a teacher data generation unit (103) that generates a label indicating the position and the range of the detection object photographed on the teacher image based on the converted coordinate to make it associated with the teacher image to be teacher data (112).SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device that generates training data for use in machine learning. [Background technology]

[0002] Machine learning using training data has been used for some time. Generally, machine learning requires a large amount of training data, but it can be difficult to obtain a large amount of training data. A technique for solving this problem is known that generates new training data based on existing training data to increase the total amount of training data.

[0003] For example, Patent Document 1 listed below discloses a machine learning system that classifies training images into multiple patterns, identifies patterns that are few in number, and generates new training images that belong to the missing patterns by spatially inverting the training images or changing their color tones. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2018-169672 Summary of the Invention [Problem to be solved by the invention]

[0005] In machine learning for generating a trained model for detecting a detection target appearing in an image, a teacher image in which the detection target appears is associated with a label indicating the position and range of the detection target, and this is used as the teacher data. In general, the representative coordinates of a rectangular area surrounding the detection target (e.g., the coordinates of the center position of the rectangular area) and the width and height of the rectangular area are associated with the teacher image as labels. Note that the width and height can also be interpreted as the horizontal length and vertical length.

[0006] Fig. 4 is a diagram showing a conventional technique, and is a diagram showing an example of setting a label indicating the position and range of a detection target in a teacher image. A detection target OB1 is shown in the teacher image IMG1 shown on the left side of Fig. 4, and a rectangular area R1 surrounding this detection target OB1 is set. The representative coordinates, width, and height of this rectangular area R1 are associated with the teacher image IMG1 as a label.

[0007] Here, when the detection target OB1 is a rod-shaped object and is photographed at an angle, as in the teacher image IMG1 in Fig. 4, the ratio of the background area (upper right and lower left areas of the rectangular area R1) in which the detection target OB1 is not photographed becomes high in the rectangular area R1. If learning is performed using such a label, the influence of the background area of ​​the detection target OB1 becomes large, which is not preferable. For example, in the teacher image IMG1 in Fig. 4, part of a plate-shaped object is photographed in the rectangular area R1, which may affect the learning results.

[0008] Therefore, it is possible to set an inclined rectangular region R2, as in the teacher image IMG2 on the right side of Figure 4. This makes it possible to significantly reduce the proportion of the background region compared to the rectangular region R1. In addition to the representative coordinates, width, and height of the rectangular region R2, the inclination angle α can be associated with the teacher image IMG2 as a label.

[0009] However, when the rectangular area is tilted, if a transformation is performed to change the aspect ratio (ratio of height to width; synonymous with aspect ratio) of the original image in order to increase the variety of teacher images, there is a problem that the position and range of the detection target indicated by the label in the teacher image generated by the transformation may differ from the position and range of the detection target shown in the teacher image.

[0010] This will be described with reference to Fig. 5. Fig. 5 is a diagram for explaining the problems of the conventional technology. In the example of Fig. 5, like the above-mentioned teacher images IMG1 and IMG2, a part of the cut-out area indicated by the dashed line is cut out from an image IMG3 in which a detection object OB1 appears. Then, the part of the cut-out area is used as an original image IMG3', and a teacher image IMG4 is generated by adjusting the size of the original image IMG3'.

[0011] More specifically, image IMG3 has a width and height of 608 pixels, and a rectangular region R3 inclined at an angle α is set on object OB1 shown in image IMG3, similar to IMG2 in Fig. 4. Original image IMG3' cut out from image IMG3 has a width of 566 pixels and a height of 383 pixels. The bottom edge of original image IMG3' extends beyond image IMG3, but such a cutout method is also possible when generating a teacher image.

[0012] The original image IMG3' is then enlarged and reduced to generate teacher image IMG4, which has a width and height of 544 pixels. Specifically, teacher image IMG4 is generated by multiplying original image IMG3' by (544 / 566) in the width direction and (544 / 383) in the height direction. Because the enlargement rates in the width and height directions are different, the aspect ratios of original image IMG3' and teacher image IMG4 change. Changing the aspect ratio is itself desirable from the perspective of increasing the variety of teacher images.

[0013] In addition, in IMG4, rectangular region R4 indicating the position and range of detection target OB1 is obtained by multiplying the width of rectangular region R2 by (544 / 566) and the height by (544 / 383). In this way, if rectangular region R4 is set by enlarging and reducing the width and height of rectangular region R3 in the same way as in original image IMG3', part of detection target OB1 may protrude from rectangular region R4 as shown in Figure 5.

[0014] This is because the inclination angle of the detection object OB1 has changed due to a change in the aspect ratio of the original image IMG3', whereas the inclination angle α of the rectangular region R4 is the same as the inclination angle of the rectangular region R3 in the original image IMG3'.

[0015] In this way, if the rectangular area set in the original image is inclined and a teacher image is generated that has an aspect ratio different from that of the original image, a problem occurs in that the label associated with the teacher image does not correctly indicate the position and range of the detection target.

[0016] One aspect of the present invention aims to realize an information processing device, etc. that is capable of generating teacher data including labels that correctly indicate the position and range of a detection target in a teacher image even in the above-mentioned case. [Means for solving the problem]

[0017] In order to solve the above problems, an information processing device according to one embodiment of the present invention comprises a teacher image generation unit that generates a teacher image by applying a transformation to an original image in which a detection target is to be detected, to enlarge or reduce the image in at least one of the height and width directions; a coordinate transformation unit that transforms, based on the magnification of the transformation, the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the original image into the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the teacher image; and a teacher data generation unit that generates a label indicating the position and range of the detection target in the teacher image based on the coordinates after transformation by the coordinate transformation unit, and associates the label with the teacher image to serve as teacher data.

[0018] In order to solve the above problems, a teacher data generation method according to one embodiment of the present invention is a teacher data generation method executed by one or more information processing devices, and includes a teacher image generation step of generating a teacher image by applying a transformation to an original image containing a detection target to enlarge or reduce it in at least one of the height and width directions, a coordinate transformation step of converting the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the original image based on the magnification of the transformation to the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the teacher image, and a teacher data generation step of generating a label indicating the position and range of the detection target in the teacher image based on the coordinates after transformation by the coordinate transformation step, and associating it with the teacher image to use as teacher data.

[0019] In order to solve the above problem, a trained model generation method according to one embodiment of the present invention is a trained model generation method executed by one or more information processing devices, and includes: a trained image generation step of generating a trained image by applying a transformation to an original image containing a detection target to enlarge or reduce it in at least one of a height direction and a width direction; a coordinate transformation step of converting the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the original image into the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the trainer image based on the magnification of the transformation; a trained data generation step of generating a label indicating the position and range of the detection target in the trainer image based on the coordinates after transformation by the coordinate transformation step, and associating it with the trainer image as trainer data; and a learning step of generating a trained model for detecting the detection target from an image by machine learning using the trainer data generated in the trainer data generation step. Effect of the Invention

[0020] According to one aspect of the present invention, even if a teacher image having an aspect ratio different from that of an original image in which an inclined rectangular area is set is generated from the original image, teacher data including labels that correctly indicate the position and range of the detection target in the teacher image can be generated. [Brief description of the drawings]

[0021] [Figure 1] 1 is a block diagram showing an example of a configuration of a main part of an information processing device according to an embodiment of the present invention; [Diagram 2] 3A to 3C are diagrams illustrating an example of generation of teacher images and labels by the information processing device. [Diagram 3] 4 is a flowchart illustrating an example of a process executed by the information processing device. [Figure 4] FIG. 13 is a diagram illustrating a conventional technique, showing an example of setting labels indicating the position and range of a detection target in a teacher image. [Diagram 5] FIG. 1 is a diagram illustrating a problem with the conventional technology. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] [Device configuration] The configuration of an information processing device 1 according to an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of a main configuration of the information processing device 1. The information processing device 1 may be, for example, a personal computer, a server, or a workstation.

[0023] As shown in the figure, the information processing device 1 includes a control unit 10 that controls each unit of the information processing device 1, and a storage unit 11 that stores various data used by the information processing device 1. The information processing device 1 also includes a communication unit 12 that enables the information processing device 1 to communicate with other devices, an input unit 13 that accepts input of various data to the information processing device 1, and an output unit 14 that enables the information processing device 1 to output various data.

[0024] The control unit 10 also includes a teacher image generation unit 101, a coordinate conversion unit 102, a teacher data generation unit 103, a learning unit 104, and an inference unit 105. The storage unit 11 stores an image 111, teacher data 112, and a trained model 113.

[0025] Image 111 is an image in which a detection target is shown. A label indicating the position and range of the detection target shown in image 111 is associated with image 111. This label may indicate, for example, representative coordinates, width, height, and tilt angle of a rectangular area surrounding the detection target. Note that width and height can also be interpreted as horizontal length and vertical length. In addition, when generating a trained model that not only detects but also identifies the detection target, an identifier of the detection target may be added to the label in addition to the above-mentioned information.

[0026] The image 111 may be input via the input unit 13 or may be received via the communication unit 12. For example, the input unit 13 may be a USB (Universal Serial Bus) interface. In this case, the input unit 13 and a photographing device that photographs the detection target may be connected via the USB interface. The information processing device 1 may receive the image photographed by the photographing device via the input unit 13 and store it as the image 111. Alternatively, the image photographed by the photographing device may be transmitted via a LAN (Local-Area Network) or a wireless LAN, and the information processing device 1 may receive the image via the communication unit 12 and store it as the image 111.

[0027] The teacher image generating unit 101 generates a teacher image by performing a conversion to enlarge or reduce an original image cut out from an image 111 containing a detection target in at least one of a height direction and a width direction. Note that the teacher image may be generated by using the image 111 as it is as an original image. The method of generating the teacher image will be described in detail later with reference to FIG. 2.

[0028] The coordinate conversion unit 102 converts the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection target in the teacher image, based on the conversion magnification of the teacher image (magnification of enlargement / reduction in the height direction and width direction). The details of the coordinate conversion method will be described later with reference to FIG. 2.

[0029] The teacher data generation unit 103 generates labels indicating the position and range of the detection target appearing in the teacher image generated by the teacher image generation unit 101 based on the coordinates converted by the coordinate conversion unit 102, and associates the generated labels with the teacher image to generate teacher data. The teacher data generation unit 103 then stores the generated teacher data in the storage unit 11 as teacher data 112. The method of generating the labels will be described in detail below with reference to FIG. 2.

[0030] The learning unit 104 generates a trained model for detecting a detection target from an image by machine learning using the teacher data 112 generated by the teacher data generation unit 103. Then, the learning unit 104 stores the generated trained model in the storage unit 11 as the trained model 113. The model may be, for example, a neural network model (including a deep neural network model). Furthermore, learning may be performed by repeatedly performing a process of inputting a teacher image included in the teacher data into the model under learning, calculating the error between the inference result obtained by inputting the teacher image included in the teacher data and the correct answer indicated in the teacher data, and updating the model by error backpropagation, while changing the teacher data (error backpropagation method).

[0031] The inference unit 105 detects a detection target from an image using the trained model 113. More specifically, the inference unit 105 first reads a target image. The image may be stored in advance in the storage unit 11, or may be input from a photographing device or the like via the communication unit 12 or the input unit 13. The inference unit 105 then inputs the read image to the trained model 113, and causes the output unit 14 to output, from among the obtained detection results, detection results with a certainty equal to or higher than a threshold value.

[0032] Since the trained model 113 is generated by learning using the teacher data 112 including labels that correctly indicate the position and range of the detection target, the inference unit 105 that performs detection using the trained model 113 can perform highly accurate object detection. The detection result indicates the position and range of the detection target appearing in the image. As the detection result, the representative coordinates, width, height, and tilt angle of the area surrounding the detection target may be output, or the detection result may be output in the form of a rectangle that indicates the above area being drawn on the target image.

[0033] As described above, the information processing device 1 comprises a teacher image generation unit 101 that generates a teacher image by performing a transformation on an original image in which the detection target is to be detected, enlarging or reducing it in at least one of the height and width directions, a coordinate transformation unit 102 that transforms, based on the magnification of the transformation, the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection target in the teacher image, and a teacher data generation unit 103 that generates a label indicating the position and range of the detection target in the teacher image based on the coordinates after transformation by the coordinate transformation unit 102, and associates it with the teacher image to generate teacher data 112.

[0034] According to the above configuration, a teacher image is generated by performing a transformation to enlarge or reduce the original image showing the detection target in at least one of the height direction and the width direction. This makes it possible to generate a wide variety of teacher images, including those with a different aspect ratio (the ratio of height to width, synonymous with the aspect ratio) from the original image.

[0035] According to the above configuration, a label to be associated with the generated teacher image is generated based on the coordinates of the four vertices of a quadrangular area obtained by converting the coordinates of the four vertices of the rectangular area of ​​the original image based on the magnification of the conversion applied when the teacher image was generated. Then, the label is associated with the teacher image to create teacher data.

[0036] The coordinates of the four vertices of the quadrangular region are calculated based on the magnification of the transformation applied when the teacher image was generated, and therefore correctly indicate the position and range of the detection target in the teacher image. This makes it possible to generate teacher data 112 associated with a label that correctly indicates the position and range of the detection target in the teacher image.

[0037] As described above, the above configuration has the effect of generating teacher data 112 including labels that correctly indicate the position and range of the detection target in the teacher image, even when the rectangular area set in the original image is inclined and a teacher image having an aspect ratio different from that of the original image is generated.

[0038] The information processing device 1 also includes a learning unit 104 that generates a trained model 113 for detecting a detection target from an image by machine learning using the teacher data 112 generated by the teacher data generation unit 103. With this configuration, the trained model 113 can be automatically generated.

[0039] [Example of generating training images and labels] FIG. 2 is a diagram showing an example of teacher images and labels generated by the information processing device 1. In FIG. 2, a portion of the cutout area indicated by the dashed line is cut out from an image IMG5, both of which have a width and height of 608 pixels, to generate an original image IMG5', which is then enlarged and reduced to generate a teacher image IMG6, both of which have a width and height of 544 pixels. The width and height of the teacher image IMG6 may be determined by using random numbers, etc. Also, a teacher image may be generated by using an original image in which not the entirety of the detection target OB1 but a part of it is cut out. By performing learning using such a teacher image, it becomes possible to detect the detection target OB1 from an image in which only a part of the detection target OB1 is captured.

[0040] In this way, the teacher image generating unit 101 performs a conversion to enlarge or reduce the original image IMG5' cut out from the image IMG5 containing the detection target in at least one of the height direction and the width direction to generate the teacher image IMG6. The teacher image generating unit 101 may determine the position and range of the cut-out region and the conversion magnification by, for example, random numbers.

[0041] When the teacher image IMG6 is generated, the coordinate conversion unit 102 determines the coordinates of each of the vertices P5a to P5d of the rectangular area R5 that indicates the position and range of the detection object OB1 appearing in the original image IMG5', based on the conversion magnification used when generating the teacher image IMG6. These coordinates are calculated from the representative coordinates, width, height, and tilt angle of the rectangular area R5, which are indicated in the label associated with the image IMG5.

[0042] Then, the coordinate conversion unit 102 converts the coordinates of the vertices P5a to P5d into the coordinates of each of the vertices P6a to P6d of a rectangular region R6 that indicates the position and range of the detection target OB1 appearing in the teacher image IMG 6. For example, the coordinate conversion unit 102 performs the above conversion by multiplying the x coordinates of the vertices P5a to P5d by a conversion magnification in the width direction when generating the teacher image IMG 6, and by multiplying the y coordinates of the vertices P5a to P5d by a conversion magnification in the height direction when generating the teacher image IMG 6.

[0043] The aspect ratios of the original image IMG5' and the teacher image IMG6 are different because the magnifications of the transformation in the height and width directions applied when generating the teacher image IMG6 are different. In this case, the balance between the height and width of the detection target OB1 shown in the teacher image IMG6 has changed, resulting in a distorted shape. In this case, the rectangular area R6 corresponding to the inclined rectangular area R5 becomes a parallelogram area along the outer edge of the distorted detection target OB1. If the aspect ratio does not change between the original image and the teacher image, the detection target shown in the teacher image is enlarged or reduced without distortion. In this case, the rectangular area of ​​the original image is enlarged or reduced without distortion, so the area after transformation is also rectangular.

[0044] Next, the teacher data generation unit 103 generates a label indicating the position and range of the detection target OB1 appearing in the teacher image IMG6 based on the coordinates converted by the coordinate conversion unit 102, i.e., the coordinates of the vertices P6a to P6d. Specifically, the teacher data generation unit 103 calculates the following four values ​​from the coordinates of the vertices P6a to P6d.

[0045] (1) Representative coordinates of rectangular area R6 (2) The length of the long side of the rectangular area R6 (3) The inclination angle of the long side of the rectangular area R6 (4) The distance between the opposing long sides of the rectangular region R6 The above (1) is information indicating the position of the detection target OB1 appearing in the teacher image IMG6. The representative coordinates may be any coordinates that can identify the position of the rectangular region R6. For example, the teacher data generation unit 103 may calculate the coordinates of the intersection P6e of the diagonal lines of the rectangular region R6 as the representative coordinates. Also, for example, the teacher data generation unit 103 may calculate the coordinates of the upper left vertex of the rectangular region R6, that is, the coordinates of the vertex P6a, as the representative coordinates.

[0046] The above (2) is information indicating the height H of the detection target appearing in the teacher image IMG6. The teacher data generating unit 103 may calculate the distance between vertices P6a and P6d, or the distance between vertices P6b and P6c. Note that instead of the length of the long side of the quadrangular region R6, the length of the short side of the quadrangular region R6 may be calculated.

[0047] The above (3) is information indicating the tilt angle of the detection target shown in the teacher image IMG6. The teacher data generating unit 103 may calculate the angle β that the line connecting vertices P6a and P6d makes with respect to the width direction of the teacher image IMG6, or the angle that the line connecting vertices P6b and P6c makes with respect to the width direction of the teacher image IMG6. Note that in the above (2), if the length of the short side of the quadrangular region R6 is calculated instead of the length of the long side of the quadrangular region R6, the tilt angle of the short side may be calculated.

[0048] The above (4) is information indicating the width W of the detection target appearing in the teacher image IMG6. The teacher data generation unit 103 may calculate the distance between the line connecting vertices P6a and P6d and the line connecting vertices P6b and P6c. In the above (2), if the length of the short side of the quadrangular region R6 is calculated instead of the length of the long side of the quadrangular region R6, the distance between the short sides may be calculated.

[0049] Although the width of the rectangular region R6 is slightly wider than the actual width of OB1, the teacher data generation unit 103 may calculate the distance between vertices P6a and P6b or the distance between vertices P6c and P6d as the width of the rectangular region R6.

[0050] Also, as shown in FIG. 2, the teacher data generating unit 103 may correct the vertices P6a to P6d to P6a' to P6d', specify a rectangle having P6a' to P6d' as four vertices, and generate a label indicating the position and range of the rectangle. Specifically, the teacher data generating unit 103 corrects the positions of the vertices P6a to P6d so that the area is the same as that of a parallelogram having P6a to P6d as four vertices, and the intersection point of the diagonal lines is P6e. For example, the long side of the rectangle may be the same length as the long side of the parallelogram, and the short side of the rectangle may be the same length as the height of the parallelogram. In this way, a rectangle having P6a' to P6d' as four vertices as shown in FIG. 2 is specified. In this case, for example, the coordinates of P6a' to P6d' may be used as a label indicating the position and range of the rectangle. Of course, the short side of the rectangle may be the same length as the short side of the parallelogram, and the long side of the rectangle may be the same length as the height of the parallelogram.

[0051] Finally, the teacher data generating unit 103 associates labels indicating the above calculated four values ​​with the teacher image IMG6 to set them as teacher data. In this manner, the teacher data generating unit 103 may, from the coordinates of each vertex obtained by the coordinate conversion unit 102, (1) calculate representative coordinates of the quadrangular area as information indicating the position of the detection target, (2) calculate the length of one side of the quadrangular area as information indicating the height of the detection target, (3) calculate the inclination angle of the one side as information indicating the inclination angle of the detection target, and (4) calculate the distance between the one side and the side opposite to the one side as information indicating the width of the detection target.

[0052] According to the above configuration, it is possible to generate training data that corresponds to labels indicating inclination angles that are useful when learning about rod-shaped detection targets in addition to representative coordinates, height, and width, which are common labels for object detection.

[0053] [Processing flow] The flow of processing executed by the information processing device 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of processing executed by the information processing device 1. It is assumed that the reading of setting information necessary for learning has been completed before the processing of Fig. 3 starts. Examples of the setting information include a storage location for the images 111, a range of random numbers used when generating a teacher image, the number of images 111 to be read at one time, and a condition for ending learning.

[0054] In S1, the teacher image generating unit 101 acquires images 111 from the storage unit 11. The number of images 111 to be acquired may be set in the setting information as described above. The number of images 111 to be acquired may be one or more. The teacher image generating unit 101 may also determine which image 111 to acquire by using a random number.

[0055] In S2, the teacher image generating unit 101 determines the cut-out range of the image 111 acquired in S1. For example, the teacher image generating unit 101 may determine the position, width, and height of the cut-out range using random numbers.

[0056] In S3 (teacher image generation step), the teacher image generation unit 101 cuts out the image 111 read out in S1 within the cutout range determined in S2 to generate an original image that is the basis of the teacher image. The teacher image generation unit 101 then performs a conversion on the original image to enlarge or reduce it in at least one of the height direction and the width direction to generate a teacher image. The magnification of the enlargement or reduction is determined by the height direction and width direction sizes of the teacher image to be generated and the height direction and width direction sizes of the original image. The teacher image generation unit 101 may determine the height direction and width direction sizes of the teacher image by random numbers.

[0057] In S4 (coordinate conversion step), first, the coordinate conversion unit 102 calculates the coordinates of four vertices that indicate the range of the detection target in the original image. Then, the coordinate conversion unit 102 converts the calculated coordinates based on the magnification of the conversion performed on the original image, and calculates the coordinates of four vertices that indicate the range of the detection target in the teacher image generated in S3. The method of calculating the coordinates of each vertex is as described with reference to FIG. 2, and therefore will not be described again here.

[0058] In S5, the teacher data generation unit 103 calculates the representative coordinates, width, height, and tilt angle of the detection target as information indicating the position and range of the detection target appearing in the teacher image generated in S3 from the coordinates of each vertex calculated in S4 by the coordinate conversion unit 102. The method of calculating this information is as described with reference to FIG. 2, and therefore will not be described again here.

[0059] In S6 (teacher data generation step), the teacher data generation unit 103 generates labels from each piece of information calculated in S5, and associates them with the teacher image generated in S3 to generate teacher data. The teacher data generation unit 103 then stores the generated teacher data in the storage unit 11 as teacher data 112. The method of generating labels from the transformed coordinates is as described with reference to FIG. 2, and therefore will not be described again here.

[0060] In S7 (learning step), the learning unit 104 performs machine learning using the teacher data 112 generated in S6. For example, when learning a neural network model, the learning unit 104 inputs a teacher image included in the teacher data 112 to the model being learned, and calculates the error between the output value output from the model and the value of the label associated with the teacher image. Then, the learning unit 104 updates the weight values ​​of the neural network model by the error backpropagation method based on the calculated error.

[0061] In S8, the learning unit 104 judges whether or not to end the learning. If it is judged that the learning is to be ended (YES in S8), the learning unit 104 stores the latest updated model in the storage unit 11 as the trained model 113, and the process of FIG. 3 ends. On the other hand, if it is judged that the learning is not to be ended (NO in S8), the process returns to S1, and a new image 111 is read out. The end condition of S8 may be determined in advance as setting information. For example, the end condition may be that the model has been updated a predetermined number of times or more, or that the error is equal to or less than a threshold value.

[0062] If a plurality of images 111 are read out in S1, then by the processes of S2 to S6, training data is generated from each of the plurality of images 111 read out in S1 and stored as training data 112. Then, in S7, learning is performed using the training data 112.

[0063] The processes of S1 to S6 in Fig. 3 are a method for generating teacher data. Moreover, S1 to S8, which are the method for generating teacher data plus the processes of S7 to S8 for generating a trained model by machine learning, are a method for generating a trained model. Note that the processes of S1 to S6 and S7 to S8 do not necessarily need to be performed consecutively. In S1, the number of images 111 required for learning may be acquired, and after the number of teacher data 112 required for learning is generated from these images 111 by the processes of S2 to S6, the processes of S7 to S8 may be performed.

[0064] In addition, in generating the training data, the color and rotation angle of the training image may be changed from the original image in order to increase the variation of the training data. As a method of color conversion and rotation conversion for increasing the variation of the training data, various conventional methods can be applied.

[0065] As described above, the method for generating teacher data executed by the information processing device 1 includes a teacher image generating step (S3) for generating a teacher image by performing a transformation for enlarging or reducing the original image in which the detection target is shown in at least one of the height direction and the width direction, a coordinate transformation step (S4) for transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection target shown in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection target shown in the teacher image based on the transformation magnification, and a teacher data generating step (S6) for generating a label indicating the position and range of the detection target shown in the teacher image based on the coordinates after transformation in the coordinate transformation step, and associating it with the teacher image to generate teacher data. This makes it possible to generate teacher data including a label that correctly indicates the position and range of the detection target in the teacher image.

[0066] As described above, the method for generating a trained model executed by the information processing device 1 includes a trained image generating step (S3) for generating a trained image by performing a transformation for enlarging or reducing at least one of the height direction and the width direction on an original image in which a detection target is shown, a coordinate transformation step (S4) for transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection target shown in the original image into the coordinates of each vertex of a rectangular area indicating the position and range of the detection target shown in the trained image based on the transformation magnification, a trained data generating step (S6) for generating a label indicating the position and range of the detection target shown in the trained image based on the coordinates after transformation by the coordinate transformation step, and associating it with the trained image as trained data, and a learning step (S7) for generating a trained model for detecting a detection target from an image by machine learning using the trained data generated in the trained data generating step. As a result, a trained model for detecting a detection target from an image can be automatically generated using trained data including a label correctly indicating the position and range of the detection target.

[0067] [Modifications] The execution subject of each process described in the above embodiment is arbitrary and is not limited to the above example. In other words, an information processing system having the same function as the information processing device 1 can be constructed by a plurality of information processing devices capable of communicating with each other. For example, the generation of the teacher image, the conversion of coordinates, the generation of the teacher data, learning, and inference may be respectively executed by different information processing devices. Also, for example, an information processing system including an information processing device that executes the process from the generation of the teacher image to the generation of the teacher data, that is, the method of generating the teacher data, an information processing device that generates a trained model using the generated teacher data, and an information processing device that performs inference using the generated trained model can be constructed. In this way, the above-mentioned method of generating the teacher data can be realized by one or more information processing devices. The same applies to the above-mentioned method of generating the trained model. Also, a photographing device that photographs the detection target may be included in the components of the information processing system.

[0068] [Software implementation example] The functions of the information processing device 1 (hereinafter referred to as the "device") can be realized by a program for causing a computer to function as the device, and a program for causing a computer to function as each control block of the device (particularly each part included in the control unit 10).

[0069] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program to realize each function described in each of the above embodiments.

[0070] The control device may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), or a combination of these. The storage device may include a high-speed storage unit capable of writing and reading data at high speed, and a large-capacity storage unit having a larger data storage capacity than the high-speed storage unit. The high-speed storage unit may be, for example, a high-speed access memory such as a synchronous dynamic random-access memory (SDRAM). The large-capacity storage unit may be, for example, a hard disk drive (HDD), a solid-state drive (SSD), a secure digital (SD) card, or an embedded multi-media controller (eMMC).

[0071] The program may be non-transitory and may be recorded in one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be provided to the device via any wired or wireless transmission medium.

[0072] In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit formed in an integrated circuit (IC chip) or the like. For example, an integrated circuit in which a logic circuit functioning as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of each of the control blocks can be realized by, for example, a quantum computer.

[0073] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0074] 1. Information processing device 101 Teacher image generation unit 102 Coordinate conversion section 103 Teacher Data Generation Unit 104 Learning Department

Claims

1. a teacher image generating unit that performs a transformation to enlarge or reduce an original image including a detection target in at least one of a height direction and a width direction to generate a teacher image; a coordinate conversion unit that converts, based on the conversion magnification, the coordinates of each vertex of a rectangular area indicating the position and range of the detection object in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection object in the teacher image; a teacher data generation unit that generates labels indicating a position of the detection target shown in the teacher image, a range of the detection target shown in the teacher image, and an inclination angle of the detection target shown in the teacher image based on the coordinates converted by the coordinate conversion unit, and associates the labels with the teacher image to set as teacher data; The teacher data generation unit calculates an inclination angle of one side of the rectangular area as information indicating the inclination angle of the detection object in the label from the coordinates of each vertex of the rectangular area.

2. The teacher data generation unit calculates, from the coordinates of each vertex of the quadrangular region, (1) calculating representative coordinates of the rectangular area as information indicating the position of the detection target on the label; (2) The information processing device according to claim 1, further comprising: calculating a length of one side of the rectangular area and a distance between the one side and an opposite side to the one side as information indicating the range of the detection target on the label.

3. The information processing device according to claim 1 , further comprising a learning unit configured to generate a trained model for detecting the detection target from an image by machine learning using the training data generated by the training data generation unit.

4. A method for generating teacher data executed by one or more information processing devices, comprising: a teacher image generating step of generating a teacher image by performing a transformation such that an original image including a detection target is enlarged or reduced in at least one of a height direction and a width direction; a coordinate transformation step of transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection object in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection object in the teacher image based on the magnification of the transformation; a teacher data generating step of generating labels indicating a position of the detection target shown in the teacher image, a range of the detection target shown in the teacher image, and an inclination angle of the detection target shown in the teacher image based on the coordinates converted by the coordinate conversion step, and associating the labels with the teacher image to obtain teacher data; A method for generating teacher data, in which the teacher data generating step calculates an inclination angle of one side of the rectangular area from the coordinates of each vertex of the rectangular area as information indicating the inclination angle of the detection object in the label.

5. A method for generating a trained model executed by one or more information processing devices, a teacher image generating step of generating a teacher image by performing a transformation such that an original image including a detection target is enlarged or reduced in at least one of a height direction and a width direction; a coordinate transformation step of transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection object in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection object in the teacher image based on the magnification of the transformation; a teacher data generation step of generating labels indicating a position of the detection target shown in the teacher image, a range of the detection target shown in the teacher image, and an inclination angle of the detection target shown in the teacher image based on the coordinates converted by the coordinate conversion step, and associating the labels with the teacher image to set as teacher data; A learning step of generating a trained model for detecting the detection target from an image by machine learning using the training data generated in the training data generation step, A method for generating a trained model, in which the teacher data generation step calculates an inclination angle of one side of the rectangular area from the coordinates of each vertex of the rectangular area as information indicating the inclination angle of the detection object in the label.

6. 2. A program for causing a computer to function as the information processing device according to claim 1, the program causing a computer to function as the teacher image generating section, the coordinate conversion section, and the teacher data generating section.

Images