Learning systems and learning methods

Abstract

To provide a learning system that improves the accuracy of identifying objects in an image area that are difficult to judge, while simultaneously suppressing the influence of minute objects. [Solution] The learning system comprises an image acquisition unit, a model learning unit, and an output unit. The image acquisition unit acquires training image data that defines an image and an image region corresponding to an object to be detected contained in the image. The model learning unit calculates the degree of deviation by comparing estimated image data and training image data for each pixel of the image region, and performs model learning using an evaluation function based on the degree of deviation. The model learning unit sets the weight applied to the evaluation function to be smaller as the degree of deviation decreases, and determines the evaluation function to be either a first function or a second function based on the comparison result of the degree of deviation and a predetermined threshold. The magnitude of the degree of deviation to which the first function is applied is smaller than the magnitude of the degree of deviation to which the second function is applied. The rate of change of weights with respect to changes in the degree of deviation within the applicable range of the second function is more gradual than when the first function is applied.

Description

[Technical Field]

[0001] The present invention relates to a learning system and a learning method. [Background technology]

[0002] In the automation of machine control or inspection, detecting objects within an image is a highly important process. When training a model that outputs image regions corresponding to objects to be detected in an image, using a Focal Loss function or Focal Dice function in the evaluation function can enhance the adjustment of weights for pixels where the model is unsure whether they are within the range of an object, thereby increasing the influence on learning relatively difficult-to-distinguish objects in an image. Patent Document 1 describes a method for training an object detector with a small amount of labeled data, achieved using a self-supervised learning representation. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-126130 [Overview of the project] [Problems that the invention aims to solve]

[0004] The invention described in Patent Document 1 avoids the use of per-image labels by using an embedded network and a cropping module. This allows users to train a general-purpose object detection system without needing per-image spatial labels, significantly reducing the cost of labeling training data in machine learning. However, the invention described in Patent Document 1, by using a Focal Loss function or the like, may increase the influence on learning of minute objects that should be treated as noise in the image, potentially reducing the output accuracy of the image region.

[0005] In light of the above-mentioned problems, this disclosure provides a learning system and learning method that improves the accuracy of identifying objects that are difficult to determine in an image region, while simultaneously suppressing the influence of minute objects that should be treated as noise. [Means for solving the problem]

[0006] A learning system according to one aspect of this disclosure comprises an image acquisition unit, a model learning unit, and an output unit. The image acquisition unit acquires training image data that defines an image and an image region corresponding to an object to be detected contained in the image. The model learning unit calculates a deviation degree by comparing estimated image data and training image data for each pixel of the image, and performs model learning using an evaluation function based on the deviation degree. The model learning unit sets the weight applied to the evaluation function to be smaller as the deviation degree decreases, and determines the evaluation function to be a first function or a second function based on the deviation degree and a predetermined threshold comparison result. The magnitude of the deviation degree to which the first function is applied is smaller than the magnitude of the deviation degree to which the second function is applied. The rate of change of weights with respect to changes in the deviation degree within the application range of the second function is more gradual than when the first function is applied. The output unit outputs the learned model.

[0007] In the learning system described above, the second function may have a lower degree than the first function.

[0008] In the learning system described above, the evaluation function is the function shown in equation (1) below,

number

[0009] In the learning system described above, the learned model may be used for object detection for automatic control of a machine.

[0010] A learning method according to one aspect of the present disclosure includes a computer executing the following processes. The computer acquires teacher image data defining an image and an image region corresponding to a detection object included in the image. The computer calculates a divergence degree by comparing estimated image data obtained by estimating whether each pixel of the image belongs to the image region with the teacher image data, and executes learning of the model using an evaluation function based on the divergence degree. In learning the model, the computer sets a smaller weight for the evaluation function as the divergence degree becomes lower, and determines the evaluation function as a first function or a second function based on the divergence degree and a comparison result with a predetermined threshold value. The magnitude of the divergence degree to which the first function is applied is smaller than the magnitude of the divergence degree to which the second function is applied. The second function has a gentler rate of change of the weight with respect to a change in the divergence degree within the applicable range than when the first function is applied. The computer outputs the learned model.

Advantages of the Invention

[0011] According to the present disclosure, it is possible to provide a learning system and a learning method that improve the determination accuracy of an object for which it is difficult to determine an image region and at the same time suppress the influence of minute objects that are to be treated as noise.

Brief Description of the Drawings

[0012] [Figure 1] It is a block diagram of a learning system according to Embodiment 1. [Figure 2] It is a flowchart in the learning method of the learning system according to Embodiment 1. [Figure 3] It is a flowchart in the learning method of the learning system according to Embodiment 2. [Figure 4] It is a flowchart of the application of the loss function of the learning system according to Embodiment 2. [Figure 5] It is a graph showing a schematic of the weight of the proposed loss function according to Embodiment 2. [Figure 6] It is a block diagram illustrating the hardware configuration of a computer.

Modes for Carrying Out the Invention

[0013] The present invention will be described below through embodiments of the invention, but the invention claimed is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential as means of solving the problem. For clarity of explanation, the following descriptions and drawings have been omitted and simplified as appropriate. In each drawing, the same elements are denoted by the same reference numerals, and redundant explanations have been omitted where necessary.

[0014] <Embodiment 1> Referring to Figure 1, the learning method of the learning system 10 according to Embodiment 1 will be described. Figure 1 is a block diagram of the learning system 10 according to Embodiment 1. The learning system 10 relates to learning a detection model for objects to be detected contained in an image. The detection model is, for example, segmentation, but is not limited to this. Segmentation is a technique that divides an image into pixel units and identifies objects within the image. The learning system 10 comprises an image acquisition unit 101, a model learning unit 102, and an output unit 103.

[0015] The image acquisition unit 101 acquires image data that defines the image and the image region corresponding to the object to be detected contained in the image. The image acquisition unit 101 acquires the image and the image data from, for example, a connected storage device (not shown), a computer, or an external server. The image acquisition unit 101 may also acquire images captured by connecting to an imaging device (not shown), such as a camera. The image data is created, for example, by computer simulation based on the arrangement of the object to be detected. The image data may also be created by manually specifying the image region corresponding to the object to be detected in the image.

[0016] The model learning unit 102 estimates the image region for each pixel of the image and generates estimated image data. The model learning unit 102 calculates the degree of deviation by comparing the estimated image data with the training image data. The model learning unit 102 performs model learning using an evaluation function based on the degree of deviation. Here, the model learning unit 102 sets the weight applied to the evaluation function to be smaller as the degree of deviation decreases. As a result, the learning system 10 can suppress the impact on learning even when estimating objects that occupy a large proportion of the image, as long as the degree of deviation is small. Therefore, the learning system 10 can appropriately evaluate and learn the degree of deviation without burying the estimation results for objects that occupy a small proportion of the image.

[0017] The model learning unit 102 determines the evaluation function to be either the first function or the second function based on the comparison result of the deviation degree and a predetermined threshold. The magnitude of the deviation to which the first function is applied is smaller than the magnitude of the deviation to which the second function is applied.

[0018] The model learning unit 102, for example, uses a first evaluation function if the magnitude of the deviation is smaller than a predetermined threshold, and a second evaluation function otherwise. Here, the second function has a slower rate of change in weights with respect to changes in the deviation within the applicable range than when the first function is applied. As a result, the learning system 10 can suppress the evaluation function value from becoming abnormally high when the deviation is high, thereby preventing learning from being hindered or overfitting from occurring. Here, the predetermined threshold is determined according to the target and purpose of learning. The output unit 103 outputs the learned model.

[0019] Figure 2 is a flowchart of the learning method of the learning system 10 according to Embodiment 1. The learning method of the learning system 10 includes steps S11 to S19.

[0020] In step S11, the image acquisition unit 101 acquires the image and training image data. The training image data is training data for the detection model that defines the image regions corresponding to the objects to be detected contained in the image. In step S12, the model learning unit 102 estimates whether each pixel corresponds to an image region corresponding to an object to be detected and generates estimated image data. The estimated image data is, for example, a matrix the same size as the pixel size of the image, and has values ​​only for components corresponding to the coordinates of the image region corresponding to the object to be detected. The estimated image data is not limited to this, and may also be a list of coordinates of the image region corresponding to the object to be detected.

[0021] In step S13, the model learning unit 102 calculates the degree of deviation by comparing the estimated image data and the training image data. In step S14, the model learning unit 102 determines whether the degree of deviation is below a predetermined threshold. If the degree of deviation is below the threshold, the learning system 10 starts step S15. Otherwise, the learning system 10 starts step S16.

[0022] In step S14, the model learning unit 102 may make a determination using another method based on the comparison result of the deviation degree and a predetermined threshold. In this case, the learning system 10 makes the magnitude of the deviation degree when proceeding to step S15 smaller than the magnitude of the deviation degree when proceeding to step S16.

[0023] In step S15, the model learning unit 102 evaluates the detection model using an evaluation function based on the degree of deviation, with the evaluation function being the first function. The first function set as the evaluation function is weighted so that the lower the degree of deviation, the smaller the weight applied to the evaluation function. After the completion of step S15, the learning system 10 starts step S17.

[0024] In step S16, the model learning unit 102 uses a second evaluation function and evaluates the detection model using an evaluation function based on the degree of deviation. The second function, which is set as the evaluation function, sets smaller weights on the evaluation function as the degree of deviation decreases. Here, the rate of change of weights in response to changes in the degree of deviation within the applicable range of the second function is slower than when the first function is applied. After the completion of step S16, the learning system 10 starts step S17.

[0025] In step S17, the model learning unit 102 adjusts the estimated parameters. The model learning unit 102 adjusts the estimated parameters, for example, based on evaluation values. The model learning unit 102 may adjust the estimated parameters in any other predetermined way.

[0026] In step S18, the model learning unit 102 determines whether the learning termination condition has been met. The learning termination condition is, for example, whether the process from step S12 to step S17 has been repeated a predetermined number of times. The learning termination condition may also be whether the evaluation value falls within a predetermined range. The learning termination condition is not limited to these and may be set as needed. If the learning termination condition has been met, the learning system 10 starts step S19. Otherwise, the learning system 10 resumes processing from step S12. In step S19, the output unit 103 outputs the learned detection model.

[0027] As explained above, the learning system 10 learns the detection model by adjusting the influence of the degree of discrepancy in the evaluation function based on the degree of discrepancy between the estimated image data and the training image data. Specifically, the learning system 10 reduces the weight applied to the evaluation function as the degree of discrepancy decreases. This allows the learning system 10 to suppress learning for objects whose image region is easy to judge and to favorably advance learning for objects whose image region is difficult to judge.

[0028] Furthermore, if the degree of deviation exceeds a predetermined threshold, the learning system 10 uses a second function as the evaluation function, which has a slower rate of increase in weights in response to an increase in the degree of deviation than the first function. This allows the learning system 10 to suppress the occurrence of outliers in evaluation values ​​even when the degree of deviation is high.

[0029] Therefore, the learning system 10 can improve the accuracy of identifying objects in image regions that are difficult to judge, while simultaneously suppressing the influence of minute objects that should ideally be treated as noise.

[0030] The learning system 10 may also have a processor and a memory device, although these are not shown in the diagram. The memory device of the learning system 10 may include, for example, a memory device that includes non-volatile memory such as flash memory or an SSD (Solid State Drive). In this case, the memory device stores a computer program (hereinafter also referred to simply as a program) for executing the above-described method. The processor loads the computer program from the memory device into a buffer memory such as DRAM (Dynamic Random Access Memory) and executes the program.

[0031] Each component of the learning system 10 may be implemented with dedicated hardware. Furthermore, some or all of each component may be implemented by general-purpose or dedicated circuits, processors, or combinations thereof. These may be implemented by a single chip or by multiple chips connected via a bus. Some or all of each component of each device may be implemented by a combination of the aforementioned circuits and programs. Processors include CPUs (Central Processing Units), GPUs (Graphics Processing Units), FPGAs (Field-Programmable Gate Arrays), etc. Also, at least a portion of the processing performed by the learning system 10 may be provided as SaaS (Software as a Service). The descriptions of the configurations described herein may also apply to other systems described below in this disclosure.

[0032] <Embodiment 2> Figure 3 is a flowchart of the learning method of the learning system according to Embodiment 2. The learning system according to Embodiment 2 has the same configuration as the learning system 10 described with reference to Figure 1. Therefore, redundant explanations of the learning system's configuration are omitted. The learning method of the learning system includes steps S21 to S27.

[0033] In step S21, the learning system acquires image and training image data to create a dataset. The dataset includes image information and ground truth label information for each pixel of the image. Here, the label is, for example, an identification number for the object detected by the detection model. The ground truth label indicates the correct label for each pixel of the image, indicating which object is displayed in that pixel.

[0034] In step S22, the learning system trains a detection model for objects using a segmentation network. The segmentation network is a network-based deep learning method that performs segmentation. The segmentation is, for example, semantic segmentation. The segmentation is not limited to this, and may also be instance segmentation or panoptic segmentation. The network used for training may be, for example, a fully convolutional network, a convolutional neural network, or a feature pyramid network. The network is not limited to these, and depending on the purpose, a recurrent neural network or R-CNN (Region Convolutional Neural Network) may be used.

[0035] In step S23, the learning system evaluates the model trained using the segmentation network using a loss function. Here, the learning system calculates the degree of deviation between the estimated image data, obtained by the trained segmentation network estimating labels for each pixel of the image, and the training image data. If the degree of deviation is below a predetermined threshold, the first loss function is used. Otherwise, the second loss function is used.

[0036] Here, the second function is set to a function with a lower degree than the first function. This allows the learning system to easily determine the second function and suitably set the first and second functions.

[0037] Note that the first and second functions do not need to be continuous with respect to a predetermined threshold. The first and second functions only need to have an output that is fixed to a single value for each degree of deviation. The second function may be a constant and does not need to be continuous. Specifically, the second function may be a floor function or the like.

[0038] The process in step S23 will be described in detail with reference to Figure 4. Figure 4 is a flowchart of the loss function application of the learning system according to Embodiment 2. Step S23 includes substeps S231 to S233.

[0039] In sub-step S231, the learning system calculates P which is a Distributed based Loss based on the estimated image data and the teacher image data. C Here, P C is specifically calculated using the CrossEntropy Loss function. Also, the learning system calculates P which is a Region based Loss based on the estimated image data and the teacher image data. D Here, P D is specifically calculated using the Tversky Loss function.

[0040] In sub-step S232, the learning system calculates the composite loss function P which is the estimated probability of the detection model from P C and P D . Specifically, the learning system calculates P E as P E = βP C +γP D . Here, P E is calculated for each label. Also, P E may be outside the range from 0 to 1. Here, β and γ are weights, and β + γ = 1 holds. β and γ may be different values for each label. E

[0041] By using the composite loss function P E , the learning system can simultaneously evaluate the segment region and the label evaluation. Also, the learning system can converge the result of the proposed loss function even when the error becomes large during the learning process by adjusting β and γ.

[0042] In sub-step S233, the learning system substitutes P E into the proposed loss function to evaluate the detection model. The proposed loss function is the function shown in Equation (1).

Equation

[0043] Figure 5 is a graph showing the general shape of the weights y of the proposed loss function according to Embodiment 2. Here, the weights of the proposed loss function are -log(P) in the proposed loss function. E This is the coefficient applied to ).

[0044] In Figure 5, the vertical axis represents the magnitude of the weights in the proposed loss function, and the horizontal axis represents |1-P E The estimated probability error is represented by |. The weight represented by y is α > |1-P E It is a quadratic function within the range |. The learning system is such that α > |1-P E By adjusting the degree of the weight calculation formula within the range of |, gradient loss can be avoided and overfitting can be prevented. Also, the weights are α ≤ |1-P E It is linear within the range |. α ≤ |1-P E By making the weights a gentle function within the range of |, the estimated probability error |1-P E Even when | becomes rapidly large, the loss function does not take abnormal values. Therefore, the learning system can stably converge the result of the loss function.

[0045] Note that α is determined according to the purpose and application. For example, if the user of the learning system wants to rapidly learn to estimate the object to be detected, they may set α to a high value. If the user wants to prevent overfitting, they may set α to a low value. Also, α may be changed during the learning process. For example, the user may set α to a high value in the first half of learning and to a low value in the second half. In this case, the learning system can learn quickly in the first half and learn while avoiding overfitting in the second half.

[0046] As described above, by executing step S23, which includes substeps S231 to S233, the learning system can suitably evaluate the estimated probability of the detection model.

[0047] Referring again to Figure 3, in step S24, the learning system updates the parameters of the detection model. The learning system may update the parameters based on the evaluation value of the learning model obtained by the loss function.

[0048] In step S25, the learning system increments the number of epochs by 1. With this, the learning system completes one training run for the input image. In step S16, the learning system determines whether the detection model has completed a predetermined number of epochs N. If the number of epochs is less than the predetermined number of epochs N, the learning system restarts processing from step S22. If the number of epochs reaches the predetermined number of epochs N, the learning system starts step S27.

[0049] In step S27, the learning system outputs the completed detection model. As described above, the learning system according to this embodiment can improve the accuracy of judging objects that are difficult to determine in an image region, while suppressing the influence of minute objects that should be treated as noise, and can suitably train the detection model.

[0050] The order of execution of the processes may be changed as needed. For example, the learning system may perform the parameter update in step S24 between the judgment in step S26 and the start of step S22. In addition, to confirm the learning process, the learning system may output the detection model in progress between the judgment in step S26 and the start of step S22 once a predetermined number of learning iterations have been completed.

[0051] The user of the learning system may configure the variables α, β, and γ to be changed during the learning process. Furthermore, the user may change α, β, and γ based on the evaluation of the trained detection model and train the detection model again.

[0052] The trained detection model can be used, for example, for object detection in the automatic control of machinery. Specifically, the detection model trained by the learning system according to this embodiment is effective in situations such as robot-assisted parts assembly, autonomous vehicle driving, detection of empty spaces in parking lots, or detection of people in images captured by drones.

[0053] <Example hardware configuration> The following describes examples of how each functional configuration of the information processing device in this disclosure is realized through a combination of hardware and software.

[0054] Figure 6 is a block diagram illustrating the hardware configuration of a computer. The information processing device in this disclosure can realize the above-described functions using a computer 500 including the hardware configuration shown in the figure. The computer 500 may be a portable computer such as a smartphone or tablet terminal, or a stationary computer such as a PC. The computer 500 may be a dedicated computer designed to realize each device, or it may be a general-purpose computer. The computer 500 can realize the desired functions by installing a predetermined application.

[0055] Computer 500 includes a bus 502, a processor 504, memory 506, a storage device 508, an input / output interface (I / F) 510, and a network interface (I / F) 512. Bus 502 is a data transmission path for the processor 504, memory 506, storage device 508, input / output interface 510, and network interface 512 to send and receive data to and from each other. However, the method of connecting the processor 504 and other components to each other is not limited to bus connection.

[0056] Processor 504 is a processor such as a CPU, GPU, or FPGA. Memory 506 is main memory implemented using RAM (Random Access Memory), etc.

[0057] The storage device 508 is an auxiliary storage device implemented using a hard disk, SSD, memory card, or ROM (Read Only Memory). The storage device 508 stores a program for realizing a desired function. The processor 504 reads this program into memory 506 and executes it to realize each functional component of each device.

[0058] The input / output interface 510 is an interface for connecting the computer 500 to input / output devices. For example, input devices such as keyboards and output devices such as display devices are connected to the input / output interface 510. The network interface 512 is an interface for connecting the computer 500 to a network.

[0059] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. For example, various modifications can be made to the configuration and details of this disclosure that can be understood by those skilled in the art within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate. [Explanation of Symbols]

[0060] 10 Learning Systems 101 Image acquisition unit 102 Model Learning Department 103 Output section N Number of epochs α threshold 500 Computers Bus 502 504 Processors 506 memory 508 Storage Devices 510 Input / Output Interfaces 512 Network Interfaces

Claims

1. An image acquisition unit that acquires an image and training image data that defines an image region corresponding to an object to be detected contained in the image, A model learning unit calculates a degree of deviation by comparing estimated image data estimated from the image region with the training image data for each pixel of the aforementioned image, and performs model learning using an evaluation function based on the degree of deviation. An output unit that outputs the learned model, Equipped with, The aforementioned model learning unit, The lower the degree of deviation, the smaller the weight applied to the evaluation function. Based on the comparison result of the aforementioned deviation degree and a predetermined threshold, the evaluation function is determined to be either the first function or the second function. The magnitude of the deviation to which the first function is applied is smaller than the magnitude of the deviation to which the second function is applied. The second function has a slower rate of change in the weights with respect to the change in the degree of deviation within the applicable range than when the first function is applied. Learning system.

2. The learning system according to claim 1, wherein the second function has a lower degree than the first function.

3. The aforementioned evaluation function is the function shown in equation (1), [Math 1] Here, P Ｅ This is the estimated probability of the aforementioned model, |1-P Ｅ | represents the degree of deviation, and α is the threshold. The learning system according to claim 2.

4. The learning system according to any one of claims 1 to 3, wherein the learned model is used for object detection for automatic control of a machine.

5. Computers A training image data is obtained that defines an image and an image region corresponding to the object to be detected contained in the image. The degree of deviation is calculated by comparing the estimated image data estimated from the image region with the training image data for each pixel of the aforementioned image, and the model is trained using an evaluation function based on the degree of deviation. Output the trained model, In the learning of the aforementioned model, The lower the degree of deviation, the smaller the weight applied to the evaluation function. Based on the comparison result of the aforementioned deviation degree and a predetermined threshold, the evaluation function is determined to be either the first function or the second function. The magnitude of the deviation to which the first function is applied is smaller than the magnitude of the deviation to which the second function is applied. The second function has a slower rate of change in the weights with respect to the change in the degree of deviation within the applicable range than when the first function is applied. Learning methods.

Images