Learning systems and learning methods

Abstract

This provides a learning system that can improve the accuracy of detecting small objects in images. [Solution] The data acquisition unit acquires input image data and training image data. The estimation unit inputs the input image data into the estimation model to acquire estimated image data that shows the estimated image region. The size ratio calculation unit calculates the size ratio, which is the ratio of the size of the ground truth image region to the size of the entire image. The weight coefficient calculation unit calculates weight coefficients such that they increase as the size ratio approaches the threshold, in both the case where the size ratio is greater than a threshold and the case where the size ratio is less than a threshold. The loss value calculation unit calculates the loss value based on a first loss function that defines the degree of deviation between the estimated image region and the ground truth image region, and the weight coefficients. The learning unit performs learning processing on the estimation model so that the loss value is reduced.

Description

[Technical Field]

[0001] This disclosure relates to a learning system and a learning method. [Background technology]

[0002] Patent Document 1 discloses a method for training an object detector. The method according to Patent Document 1 optimizes an object detector neural network by minimizing a distance-based loss function between each cropped image representation and a sample representation of an image. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-126130 [Overview of the project] [Problems that the invention aims to solve]

[0004] In the technology described in Patent Document 1, the influence of small objects may be small in the training of the object detector neural network. Therefore, the technology described in Patent Document 1 may not be able to achieve high accuracy in detecting small objects in an image.

[0005] This disclosure provides a learning system and a learning method that can improve the accuracy of detecting small objects in an image. [Means for solving the problem]

[0006] The learning system relating to this disclosure is a learning system for training an estimation model by machine learning, wherein the estimation model is trained to take image data as input and output estimated image data which is an estimation result that estimates an image region corresponding to one or more objects to be detected included in the image shown by the input image data, and includes a data acquisition unit that acquires input image data to be input to the estimation model and teaching image data which shows the correct image region which is the correct image region of the object to be detected in the input image data, and an estimation unit that acquires estimated image data which shows the estimated image region which is the image region estimated to correspond to the object to be detected, output from the estimation model by inputting the input image data to the estimation model, and The system includes: a size ratio calculation unit that uses training image data to calculate a size ratio for each object to be detected, which is the ratio of the size of the ground truth image region to the total size of the image shown in the training image data; a weight coefficient calculation unit that calculates a weight coefficient for each object to be detected, such that the size ratio increases as it approaches the threshold, in cases where the size ratio is greater than a predetermined threshold and cases where the size ratio is less than the threshold; a loss value calculation unit that calculates a loss value based on a first loss function that defines the degree of deviation between the estimated image region in the estimated image data and the ground truth image region in the training image data, and the weight coefficients; and a learning unit that performs learning processing on the estimation model so that the loss value becomes smaller. [Effects of the Invention]

[0007] This disclosure provides a learning system and learning method that can improve the accuracy of detecting small objects in an image. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of the learning system according to Embodiment 1. [Figure 2] This flowchart shows the learning method performed by the learning system according to Embodiment 1. [Figure 3]This flowchart shows the learning method performed by the learning system according to Embodiment 1. [Figure 4] This figure illustrates the input image and teaching image acquired by the data acquisition unit according to Embodiment 1. [Figure 5] This figure illustrates a curve showing the relationship between size ratio and weight coefficient. [Modes for carrying out the invention]

[0009] This embodiment will be described below with reference to the drawings. However, the present invention is not limited to the following embodiments. Also, for clarity of explanation, the following description and drawings have been simplified as appropriate.

[0010] (Embodiment 1) Figure 1 shows the configuration of the learning system 100 according to Embodiment 1. The learning system 100 according to Embodiment 1 is, for example, a computer such as a server. The learning system 100 may be implemented, for example, by cloud computing. The learning system 100 may also be implemented by multiple computers. In this case, the multiple components of the learning system 100, which will be described later, may each be implemented on physically different computers. The learning system 100 may also have an imaging device such as a camera for photographing objects. The learning system 100 may also include a storage device for storing various types of data. The learning system 100 is configured to train an estimation model by machine learning. Details of the estimation model will be described later.

[0011] The learning system 100 has a main hardware configuration consisting of a control unit 102, a storage unit 104, a communication unit 106, and an interface unit 108 (IF). The control unit 102, storage unit 104, communication unit 106, and interface unit 108 are interconnected via a data bus or the like. If the learning system 100 is implemented on multiple computers, each of the computers may have the hardware configuration shown in Figure 1.

[0012] The control unit 102 is a processor such as a CPU (Central Processing Unit). The control unit 102 has a function as an arithmetic unit that performs control processing, arithmetic processing, and the like. Note that the control unit 102 may have a plurality of processors. The storage unit 104 is a storage device such as a memory or a hard disk. The storage unit 104 is, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 104 has a function of storing a control program, an arithmetic program, and the like executed by the control unit 102. That is, the storage unit 104 (memory) stores one or more instructions. The storage unit 104 also has a function of temporarily storing processing data and the like. The storage unit 104 may include a database. The storage unit 104 may also have a plurality of memories.

[0013] The communication unit 106 performs processing necessary for communicating with other devices via a network. The communication unit 106 may include a communication port, a router, a firewall, and the like. The interface unit 108 is, for example, a user interface (UI). The interface unit 108 has an input device such as a keyboard, a touch panel, or a mouse, and an output device such as a display or a speaker. The interface unit 108 may be configured such that the input device and the output device are integrated, such as a touch screen (touch panel). The interface unit 108 receives an operation of inputting data by a user (operator) and outputs information to the user.

[0014] Further, the learning system 100 according to Embodiment 1 includes, as components, an estimation model storage unit 112, a data acquisition unit 120, an estimation unit 130, a size ratio calculation unit 140, a weight coefficient calculation unit 150, a loss value calculation unit 160, and a learning unit 170. The data acquisition unit 120 includes an input image data acquisition unit 122 and a teaching image data acquisition unit 124.

[0015] Each of the above-described components can be realized, for example, by causing a program to be executed under the control of the control unit 102. More specifically, each component can be realized by the control unit 102 executing a program (instruction) stored in the storage unit 104. Also, it is possible to record the necessary program on an arbitrary non-volatile recording medium and install it as needed to realize each component. Further, each component is not limited to being realized by software based on a program, and may be realized by any combination of hardware, firmware, and software, etc. Also, each component may be realized using a user-programmable integrated circuit such as, for example, an FPGA (field-programmable gate array) or a microcomputer. In this case, a program composed of the above-described components may be realized using this integrated circuit.

[0016] The estimation model storage unit 112 stores the estimation model to be learned by the learning system 100. The estimation model is a trained model learned by a machine learning algorithm. The estimation model is generated by a machine learning algorithm such as a neural network, but is not limited to this. The estimation model may be, for example, a segmentation network model. The estimation model detects objects to be detected in an image. In other words, the estimation model estimates the position of objects to be detected in an image. Specifically, the estimation model estimates which object each pixel in the image corresponds to. The estimation model is trained to take image data as input and output estimated image data, which is the result of estimating the image regions corresponding to one or more objects to be detected in the image shown by the input image data. As a result, the estimation model outputs estimated image data that segments the image regions corresponding to one or more objects to be detected in the image shown by the input image data. Note that objects to be detected are objects that are the target of detection. Objects to be detected are also called, for example, "classes" or "labels". For example, if the environment to which the estimation model is applied is a manufacturing plant for industrial products, the objects to be detected may include various workpieces such as "bolts," "nuts," "parts," or "finished products."

[0017] Furthermore, the term "image" below also refers to "image data representing an image" as the object of processing in information processing. Pixels are components of an image. The images handled by the estimation model may be two-dimensional images such as RGB images, or three-dimensional images such as RGB-D images. If the image is two-dimensional, pixels may be voxels. If the image is three-dimensional, pixels may be voxels. If the three-dimensional image is represented by point cloud data, pixels may be points in the point cloud data. An "image region" is a region in an image composed of multiple pixels. A region corresponding to a detection target is also called a segment region. For example, the image region (segment region) corresponding to the detection target "bolt" is the region composed of pixels corresponding to "bolt". Note that if the image is two-dimensional, the image region is a two-dimensional region. If the image is three-dimensional, the image region is a three-dimensional region.

[0018] Figures 2 and 3 are flowcharts showing the learning method executed by the learning system 100 according to Embodiment 1. The functions of the components of the learning system 100 will be explained below using the flowcharts shown in Figures 2 and 3. First, before starting the learning method, the learning count m is reset to 0. The overall processing of the learning method will be explained using Figure 2.

[0019] The data acquisition unit 120 acquires input image data and teaching image data corresponding to the input image data (step S102). Specifically, the input image data acquisition unit 122 acquires input image data. The teaching image data acquisition unit 124 acquires teaching image data. The data acquisition unit 120 may also receive pairs of input image data and teaching image data corresponding to the input image data from another device that is communicably connected to the learning system 100. Alternatively, the data acquisition unit 120 may generate input image data and teaching image data corresponding to the input image data.

[0020] Here, input image data refers to the input image that is input to the estimation model. For example, the input image data acquisition unit 122 may acquire input image data by taking pictures of the surrounding environment using an imaging device. Alternatively, for example, the input image data acquisition unit 122 may acquire input image data by taking pictures (rendering) of the virtual space obtained by running a simulation.

[0021] Furthermore, the teaching image data represents a teaching image for the input image shown in the input image data. The teaching image data represents the ground truth image region in the input image data. The ground truth image region corresponds to the correct image region of the object to be detected in the input image data. The ground truth image region is a region where pixels corresponding to the object to be detected are labeled. Each pixel in the ground truth image region of the teaching image data is assigned a label (class) that indicates the corresponding object to be detected. For example, if the object to be detected is a "bolt," the teaching image data represents the image region corresponding to "bolt" in the input image data as the ground truth image region. The teaching image data acquisition unit 124 may acquire the teaching image data by having an operator or the like perform annotation on the input image data obtained using the imaging device. Also, the class (label) of an object placed in a virtual space obtained by simulation is clear in advance. Therefore, if the input image data is obtained by simulation, the teaching image data acquisition unit 124 may acquire the teaching image data by setting the region of the object corresponding to the object to be detected in the input image data as the ground truth image region.

[0022] Figure 4 is a diagram illustrating the input image and teaching image acquired by the data acquisition unit 120 according to Embodiment 1. The input image ImI is, for example, an RGB image. The input image ImI includes object CL1, which is an image of detection target object #1, and object CL2, which is an image of detection target object #2.

[0023] The training image ImT includes a ground truth image region Ar1 corresponding to object CL1 in the input image ImI, and a ground truth image region Ar2 corresponding to object CL2 in the input image ImI. Here, it is assumed that "label #1" is associated with object #1 and "label #2" is associated with object #2. In this case, "label #1" is associated with each pixel of the ground truth image region Ar1. Similarly, "label #2" is associated with each pixel of the ground truth image region Ar2. Furthermore, a mask corresponding to "label #1" is applied to the ground truth image region Ar1, and a mask corresponding to "label #2" is applied to the ground truth image region Ar2.

[0024] The vertical length of the input image ImI and the training image ImT is H. The horizontal length of the input image ImI and the training image ImT is W. Length H can be defined by the number of pixels in the vertical direction of the input image ImI and the training image ImT. Length W can be defined by the number of pixels in the horizontal direction of the input image ImI and the training image ImT. The total area H × W of the input image ImI and the training image ImT corresponds to the resolution of the input image ImI and the training image ImT.

[0025] The estimation unit 130 inputs the input image data into the estimation model and obtains estimated image data (step S104). Specifically, the estimation unit 130 obtains estimated image data output from the estimation model by inputting the input image data into the estimation model. The estimated image data represents the estimated image corresponding to the input image shown in the input image data. The estimated image data represents the estimated image region in the input image data. The estimated image region corresponds to the image region in the input image data that is estimated to correspond to the object to be detected. The estimated image region is a region where a label (class) associated with the object to be detected is assigned to the pixels estimated to correspond to the object to be detected. The estimated image region corresponds to a segmentation region that has been segmented by being estimated to correspond to the object to be detected. For example, if the object to be detected is a "bolt", the estimated image data represents the image region in the input image data that is estimated to be a "bolt" as the estimated image region. The estimation unit 130 may also perform segmentation such as instance segmentation using the estimation model. The estimated image region may also be, for example, a segmentation mask. In other words, the estimation unit 130 may use an estimation model to add a segmentation mask to the image region in the input image data that is estimated to correspond to the object to be detected.

[0026] The learning system 100 calculates a loss value L using the estimated image data and the training image data (step S110). The loss value L corresponds to the loss of the estimation model with respect to the input image data. The specific processing of S110 will be explained below using Figure 3. The learning system 100 determines the object to be detected #i (step S112). That is, the learning system 100 determines the label #i to be processed. Here, i is the index of the label (class). i is an integer from 1 to N. N is the number of objects to be detected, i.e., the number of labels (number of classes). In the example in Figure 4, the learning system 100 determines, for example, that object #1 is the object to be processed #i. That is, the learning system 100 determines i=1. Then, the learning system 100 performs the following processing on object CL1, which is object #1.

[0027] The size ratio calculation unit 140 calculates the size ratio for the object to be detected #i (step S114). The size ratio calculation unit 140 calculates the size ratio for each object to be detected. The size ratio calculation unit 140 calculates the size ratio for the object to be detected #i using the training image data. The size ratio is the ratio of the size of the ground truth image region corresponding to the object to be detected #i to the total size of the training image. If the training image data contains multiple objects of the same object to be detected #i, the size of the ground truth image region corresponding to the object to be detected #i can be calculated by summing the sizes of the multiple ground truth image regions corresponding to the same object to be detected #i.

[0028] The size ratio of the detected object #i (label #i) is determined by AMR. i Therefore, AMR i This is expressed by the following equation (1). Note that label_area i `#i` indicates the size of the ground truth image region corresponding to label #i. `res` indicates the total size of the training image data. In the example in Figure 4, if i=1, then `label_area` i corresponds to the size of the ground truth image region Ar1. Also, res corresponds to the overall size H×W of the training image ImT.

number

[0029] Note that "size" corresponds to the size of the region. If the training image data is a two-dimensional image, the size of the region may, for example, correspond to the area of ​​the region. In this case, the size ratio is the area ratio of the area of ​​the ground truth image region to the total area of ​​the training image. Note that if the size of the region corresponds to the area of ​​the region, the size of the region may also correspond to the number of pixels contained in that region. In this case, the size ratio is the ratio of the number of pixels in the ground truth image region to the total number of pixels (i.e., the image resolution) of the training image. Alternatively, for example, the size of the region may correspond to the length of the region's contour. In this case, the size ratio may be the ratio of the length of the contour of the ground truth image region to the total perimeter of the training image. Alternatively, the length of the contour may correspond to the number of pixels corresponding to the contour. In this case, the size ratio may be the ratio of the number of pixels corresponding to the contour of the ground truth image region to the total number of pixels of the training image.

[0030] Furthermore, if the training image data is a three-dimensional image, the size of the region may correspond to the volume of the region. In this case, the size ratio is the volume ratio of the volume of the ground truth image region to the total volume of the training image. The volume of the region may also correspond to the number of voxels or the number of point clouds. Additionally, if the training image data contains multiple objects of the same detection target #i, the size of the ground truth image region corresponding to detection target #i may be the average of the sizes of the multiple ground truth image regions corresponding to the same detection target #i.

[0031] The weight coefficient calculation unit 150 calculates a weight coefficient for the detected object #i (step S116). Specifically, for each detected object, the weight coefficient calculation unit 150 calculates a weight coefficient such that the closer the size ratio is to the threshold AT, in both cases where the size ratio is greater than a predetermined threshold AT and where the size ratio is smaller than the threshold AT. Here, the threshold AT corresponds to the size ratio of the detected object for which the accuracy of estimation by the estimation model is to be increased. If the accuracy of estimation of small detected objects in the image is to be increased, the threshold AT is set to a small value. In this case, for example, the threshold AT may be 0.1 to 0.15. The threshold AT may be predetermined for each detected object #i. For example, the threshold AT may be the minimum value of the size ratio of the detected object #i in the image. Alternatively, for example, the threshold AT may be the average value of the size ratio of the detected object #i in the image.

[0032] The weighting coefficient for detected object #i (label #i) is set to AMW. i For example, AMW i This is expressed by the following equation (2). Note that α and β are coefficients that can be adjusted as appropriate. α and β may be adjusted by trial and error. Also, α and β may be adjusted according to the progress of learning. Also, α and β may be adjusted according to the value of the loss value L, which will be described later.

number

[0033] Figure 5 illustrates a curve showing the relationship between the size ratio and the weight coefficient. Figure 5 illustrates a graph showing the relationship between the size ratio and the weight coefficient. In the graph illustrated in Figure 5, the horizontal axis represents the size ratio AMR, and the vertical axis represents the weight coefficient AMW. The first equation of equation (2) corresponds to the curve to the left of the threshold AT in Figure 5, that is, the portion where the size ratio AMR is smaller than the threshold AT. The second equation of equation (2) corresponds to the curve to the right of the threshold AT in Figure 5, that is, the portion where the size ratio AMR is larger than the threshold AT. As shown in Figure 5, the weight coefficient AMR reaches its maximum value of 1.00 when the size ratio AMR is equal to the threshold AT. Furthermore, the weight coefficient AMW increases as the size ratio AMR approaches the threshold AT, both when the size ratio AMR is larger than the threshold AT and when the size ratio AMR is smaller than the threshold AT.

[0034] Furthermore, by appropriately adjusting α and β, the gradient of the weight coefficient AMW with respect to the size ratio AMR can be adjusted. In other words, the weight coefficient AMW is defined by coefficients α and β, which allow the gradient of the weight coefficient AMW to be adjusted. Specifically, by appropriately adjusting α and β, the gradient of the weight coefficient AMW can be made gentler in the vicinity of the threshold AT. Also, by adjusting α and β separately, the gradient can be adjusted separately for the portion where the size ratio AMR is greater than the threshold AT and the portion where it is less than the threshold AT. In other words, the weight coefficient AMW is defined by multiple coefficients α and β, which allow the gradient of the weight coefficient AMW to be adjusted separately for the cases where the size ratio AMR is greater than the threshold AT and the case where the size ratio AMR is less than the threshold AT. Specifically, by adjusting α, the gradient of the weight coefficient AMW can be made steeper in the portion where the size ratio AMR is less than the threshold AT, as shown by arrow A1 in Figure 5. Furthermore, by adjusting β, the gradient of the weight coefficient AMW can be made gentler in the region where the size ratio AMR is greater than the threshold AT, as shown by arrow A2 in Figure 5.

[0035] The loss value calculation unit 160 calculates the first loss function F and the weight coefficient AMW for each detected object #i. i Based on this, the object loss value L is the loss value related to the detected object #i.i Calculate (S118, S120). Specifically, the loss value calculation unit 160 calculates the first loss value for each detection target #i using the first loss function F (step S118). Here, the first loss function F is a function that defines the degree of divergence (error) between the estimated image region in the estimated image data and the correct image region in the teaching image data. The first loss value is the output value of the first loss function F. The first loss function F is a function such that the larger the error between the correct image region and the estimated image region, the larger the first loss value. The first loss function F may be a Distribution based Loss such as Cross entropy loss or Focal loss, for example. The first loss function F may also be a Region based Loss such as Dice loss, IoU Loss, or Tversky Loss, for example. The first loss function F may also be, for example, a Focal Tversky Loss obtained by applying Focal Loss to Tversky Loss.

[0036] i For example, when the first loss function F is Focal loss, the first loss function F is represented by the following formula (3). Here, P i | γ | indicates the probability (estimated probability; probability distribution) of the estimation result of the detection target #i. In formula (3), since the first loss value becomes smaller as the estimated probability increases due to "|1 - P

Equation

[0037] The loss value calculation unit 160 calculates the object loss value L of the detection target #i using the weight coefficient and the first loss value (step S120). Specifically, the loss value calculation unit 160 multiplies the weight coefficient and the first loss value for each detection target #i to obtain the loss value (object loss value L i i ​Calculates the object loss value L. i This corresponds to the loss (error) in the input image data input to the estimation model for the detected object #i. More specifically, the loss value calculation unit 160 calculates the object loss value L of the detected object #i using the following equation (4). i Calculate F. i This indicates the first loss value.

number

[0038] The learning system 100 determines whether processing has been completed for all detection targets #i (step S122). In other words, the learning system 100 determines whether i=N. If there are detection targets #i whose processing has not been completed (NO in S122), the processing flow returns to S112. Then, processing S112~S120 is performed for the detection targets #i whose processing has not been completed. In the example in Figure 4, the learning system 100 determines that i=2 and performs the above processing for object CL2, which is detection target #2.

[0039] On the other hand, when processing is completed for all detected objects #i (YES in S122), the loss value calculation unit 160 calculates the object loss value L i The average value is calculated as the overall image loss value L (step S124). Specifically, the loss value calculation unit 160 calculates the object loss value L of all detected objects #i (i=1~N) as shown in the following equation (5). i The values ​​are summed up, and the resulting value is divided by the number of objects to be detected, N, to calculate the overall image loss value L.

number

[0040] Returning to the explanation of Figure 2, the learning unit 170 performs learning processing using the calculated loss value L (step S130). Specifically, the learning unit 170 performs learning processing on the estimated model so as to reduce the loss value L. More specifically, the learning unit 170 updates the parameters in the estimated model so as to reduce the loss value L. The parameters in the estimated model are, for example, the parameters that constitute neurons in a neural network (weights and activation functions, etc.), but are not limited to these.

[0041] The learning unit 170 increments the learning count m by one (step S132). Then, the learning unit 170 determines whether the learning count m exceeds a predetermined maximum learning count M (step S134). If the learning count m is less than or equal to the maximum learning count M (NO in S134), the process returns to S104, and the learning system 100 performs the processes S104 to S132 again. On the other hand, if the learning count m exceeds the maximum learning count M (YES in S134), the learning system 100 terminates the process.

[0042] The first loss function F tends to produce a smaller first loss value when the size of the object to be detected in the image is small, regardless of the estimation accuracy. Therefore, if the learning process is performed using only the first loss function F when estimating small objects, the first loss value may become small even if the estimation accuracy is not high, and the learning process may not proceed properly. Consequently, even if the learning process is performed using only the first loss function F, it may not be possible to properly learn for small objects and may not be able to perform estimation with good accuracy.

[0043] In contrast, the learning system 100 according to Embodiment 1 described above calculates a loss value L using weight coefficients and a first loss function F, and performs learning processing using this loss value L. Here, the weight coefficients reach their maximum value when the size ratio is equal to the threshold AT. By setting the value of the threshold AT defined by the weight coefficients to a value corresponding to the size ratio of small detection targets, it is possible to suppress the loss value L from becoming small even if the estimation accuracy is not high. Therefore, learning can be carried out appropriately even for small detection targets. Consequently, it is possible to improve the detection accuracy even for small detection targets in images.

[0044] Furthermore, the weight coefficient AMW is defined by coefficients α and β, which allow adjustment of the gradient of the weight coefficient. As described above, by appropriately adjusting coefficients α and β, the gradient of the weight coefficient can be made gentler in the vicinity of the threshold AT. This suppresses the occurrence of gradient vanishing during the learning process for detection targets based on their size ratio in the vicinity of the threshold AT. Therefore, learning can proceed appropriately.

[0045] Furthermore, the weight coefficient AMW is defined by coefficients α and β, which allow for separate adjustment of the gradient of the weight coefficient for cases where the size ratio is greater than the threshold and cases where the size ratio is less than the threshold. Here, for example, if an operator labels the ground truth image region in the training image data by annotation, noise may be generated due to incorrect annotation of minute image regions. In this case, it is undesirable to perform learning on such minute noise. In contrast, in Embodiment 1, as described above, by adjusting coefficient α, the gradient of the weight coefficient can be made steeper for parts where the size ratio is smaller than the threshold AT. This suppresses the progress of learning on noise. On the other hand, by adjusting β, the gradient of the weight coefficient AMW can be made gentler for parts where the size ratio AMR is greater than the threshold AT. This allows learning to proceed appropriately for size ratios that are possible for detection targets. Also, as described above, by adjusting α and β, the curve of the weight coefficient AMW can be made continuous, that is, differentiable. Therefore, the occurrence of gradient vanishing can be suppressed, and learning can be performed appropriately.

[0046] (modified version) It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. For example, the order of the flowchart described above can be changed as appropriate. Also, one or more of the processes in the flowchart described above can be omitted as appropriate.

[0047] The program described above, when loaded into a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disk (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include electrical, optical, acoustic or other forms of propagating signals. The program may include a program product. [Explanation of Symbols]

[0048] 100...Learning system, 112...Estimation model storage unit, 120...Data acquisition unit, 122...Input image data acquisition unit, 124...Teaching image data acquisition unit, 130...Estimation unit, 140...Size ratio calculation unit, 150...Weight coefficient calculation unit, 160...Loss value calculation unit, 170...Learning unit

Claims

1. A learning system for training an estimation model using machine learning, The estimation model is trained to take image data as input and output estimated image data, which is an estimation result obtained by estimating image regions corresponding to one or more objects to be detected in the image shown by the input image data. A data acquisition unit that acquires input image data to be input to the estimation model and teaching image data that shows the correct image region, which is the correct image region of the object to be detected in the input image data, An estimation unit inputs the aforementioned input image data into the estimation model to obtain estimated image data that shows an estimated image region, which is an image region estimated to correspond to the object to be detected, output from the estimation model. A size ratio calculation unit calculates a size ratio for each object to be detected, which is the ratio of the size of the correct image region to the total size of the image shown in the teaching image data, using the teaching image data. A weighting coefficient calculation unit calculates a weighting coefficient such that, for each of the above-mentioned objects to be detected, the size ratio increases as it approaches the threshold, in cases where the size ratio is greater than a predetermined threshold and in cases where the size ratio is less than the threshold. A loss value calculation unit calculates a loss value based on a first loss function that defines the degree of deviation between the estimated image region in the estimated image data and the ground truth image region in the teaching image data, and the weight coefficient. A learning unit that performs learning processing on the estimation model so as to reduce the loss value, A learning system that has the following features.

2. The loss value calculation unit calculates the loss value for each detected object by multiplying the weight coefficient by the output value of the first loss function. The learning system according to claim 1.

3. The weighting coefficient is defined by a coefficient whose gradient can be adjusted. The learning system according to claim 1.

4. The weighting coefficient is defined by a plurality of coefficients that allow for separate adjustment of the slope of the weighting coefficient for the cases where the size ratio is greater than the threshold and where the size ratio is less than the threshold. The learning system according to claim 3.

5. A learning method for training an estimation model using machine learning, The estimation model is trained to take image data as input and output estimated image data, which is an estimation result obtained by estimating image regions corresponding to one or more objects to be detected in the image shown by the input image data. The estimation model obtains input image data and teaching image data that shows the correct image region of the object to be detected in the input image data. By inputting the aforementioned input image data into the estimation model, estimated image data is obtained that represents an estimated image region, which is an image region estimated to correspond to the object to be detected, output from the estimation model. Using the aforementioned training image data, for each object to be detected, a size ratio is calculated, which is the ratio of the size of the ground truth image region to the total size of the image shown in the training image data. For each of the objects to be detected, a weighting coefficient is calculated such that the closer the size ratio is to the threshold, in both cases where the size ratio is greater than a predetermined threshold and where the size ratio is less than the threshold. Based on a first loss function that defines the degree of discrepancy between the estimated image region in the estimated image data and the ground truth image region in the teaching image data, and the weight coefficient, the loss value is calculated. The learning process is performed on the estimation model so that the loss value is reduced. Learning methods.

Images