Learning system and learning method

Abstract

The present invention addresses the problem of providing a learning system or the like that suppresses the influence of a minute object while improving the accuracy of determination of an object that is difficult to determine in an image region. A learning system includes an image acquisition unit, a model learning unit, and an output unit. The image acquisition unit acquires an image and teacher image data defining an image region corresponding to a detection object included in the image. The model learning unit calculates a degree of deviation by comparing estimated image data in which whether or not each pixel of an image is an image region with teacher image data, and performs learning of a model using an evaluation function based on the degree of deviation. In the model learning unit, the lower the degree of deviation is, the smaller the weight assigned to the evaluation function is, and the evaluation function is determined as a first function or a second function on the basis of the result of comparison between the degree of deviation and a predetermined threshold value.

Description

Learning systems and learning methods Technical Field

[0001] This invention relates to a learning system and a learning method. Background Technology

[0002] In the automation of machine control or inspection, detecting objects within an image is a crucial process. By training a model that corresponds to the image region containing the detected objects in the image, the use of Focal Loss or Focal Dice functions in the evaluation function can enhance the adjustment of weights for pixels lacking confidence in their range of detection, thereby improving the impact of relatively difficult-to-distinguish objects in the image on the learning process. Patent Document 1 illustrates a method for training an object detector using a small amount of labeled data, implemented with self-supervised learning representations.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2023-126130 The invention described in Patent Document 1 avoids the need for individual image labels by using an embedded network and a cropping module. This allows users to train a general object detection system without spatial labels for each image, significantly reducing the cost of training data labeling in machine learning. However, the invention described in Patent Document 1, by using a loss function such as FocalLoss, may increase the learning effect on small objects in the image that were originally intended to be noise, thereby reducing the output accuracy of the image region.

[0004] In view of the above-mentioned problems, the present invention provides a learning system and learning method that improves the accuracy of judging difficult-to-judge objects in an image region while suppressing the influence of tiny objects that are intended to be treated as noise.

[0005] The learning system according to one aspect of the present invention includes an image acquisition unit, a model learning unit, and an output unit. The image acquisition unit acquires an image and teacher image data defining an image region corresponding to a detection object contained in the image. The model learning unit calculates the deviation by comparing estimated image data (whether each pixel estimate of the image corresponds to an image region) with the teacher image data, and performs model learning using an evaluation function based on the deviation. In the model learning unit, the lower the deviation, the smaller the weight assigned to the evaluation function. Based on the comparison result of the deviation and a defined threshold, the evaluation function is determined to be either a first function or a second function. The deviation magnitude applicable to the first function is smaller than that applicable to the second function. In the second function, the rate of change of the weight relative to the change in deviation within the applicable range is slower than when the first function is applied. The output unit outputs the learned model.

[0006] In the learning system described above, the degree of the second function can be less than the degree of the first function.

[0007] In the above learning system, the evaluation function is a function represented by the following equation (1).

[0008] Among them, P E This can be used to estimate the probability of the model, |1-P E | can represent the degree of deviation, and α can be the threshold.

[0009] In the aforementioned learning system, the learned model can be used for object detection in automatic control of machinery.

[0010] In one aspect of the learning method of the present invention, the following processing is performed by a computer. The computer acquires an image and teacher image data defining image regions corresponding to the detection objects contained in the image. The computer calculates the deviation by comparing the estimated image data (whether each pixel of the image is an image region) with the teacher image data, and performs model learning using an evaluation function based on the deviation. During model learning, the lower the deviation, the smaller the weight assigned to the evaluation function. Based on the comparison result of the deviation and a defined threshold, the evaluation function is determined to be either a first function or a second function. The magnitude of the deviation when using the first function is smaller than the magnitude of the deviation when using the second function. In the second function, the rate of change of the weight relative to the change in deviation within the applicable range is slower than when using the first function. The computer outputs the learned model.

[0011] According to the present invention, a learning system and method can be provided that improves the accuracy of judging objects that are difficult to judge in an image region while suppressing the influence of tiny objects that are to be treated as noise. Attached Figure Description

[0012] Figure 1 is a block diagram of the learning system involved in Implementation Method 1.

[0013] Figure 2 is a flowchart of the learning method of the learning system involved in Implementation 1.

[0014] Figure 3 is a flowchart of the learning method of the learning system involved in Implementation Method 2.

[0015] Figure 4 is a flowchart illustrating the applicable loss function for the learning system involved in Implementation Method 2.

[0016] Figure 5 is a graph showing the approximate shape of the weights of the proposed loss function involved in Implementation 2.

[0017] Figure 6 is a block diagram illustrating the hardware structure of a computer. Detailed Implementation

[0018] The present invention will now be described through embodiments thereof, but the invention as described in the claims is not limited to these embodiments. Furthermore, not all structures described in the embodiments are necessarily necessary as means of solving the problem. For clarity, appropriate omissions and simplifications have been made in the following descriptions and drawings. Additionally, in the drawings, the same symbols are used to denote the same elements, and repeated descriptions have been omitted as necessary.

[0019] <Embodiment 1> Referring to FIG1, the learning method of the learning system 10 according to Embodiment 1 will be described. FIG1 is a block diagram of the learning system 10 according to Embodiment 1. The learning system 10 is related to the learning of a detection model of the objects to be detected contained in an image. The detection model is, for example, segmentation, but is not limited to this. Segmentation refers to the technique of dividing an image into pixels and determining the objects within the image. The learning system 10 includes an image acquisition unit 101, a model learning unit 102, and an output unit 103.

[0020] The image acquisition unit 101 acquires images and teacher image data, specifying image regions corresponding to the detection objects contained in the images. The image acquisition unit 101 acquires images and teacher image data, for example, via a connected destination storage device (not shown), a computer, or an external server. The image acquisition unit 101 can be connected to a photographic device (not shown) such as a camera and acquire captured images. The teacher image data is created, for example, through computer simulation based on the configuration of the detection objects. The teacher image data can also be created by manually specifying image regions corresponding to the detection objects in the images.

[0021] The model learning unit 102 generates estimated image data by estimating the image region for each pixel of the image. The model learning unit 102 calculates the deviation by comparing the estimated image data with the teacher's image data. The model learning unit 102 performs model learning using an evaluation function based on the deviation. Here, the model learning unit 102 sets the weights in the evaluation function to be smaller as the deviation decreases. Therefore, even when the deviation is small, the learning system 10 can suppress the influence of estimations of objects that occupy a large proportion of the image on the learning process. Thus, the learning system 10 can appropriately evaluate the deviation and learn without masking the estimation results of objects that occupy a small proportion of the image.

[0022] The model learning unit 102 determines the evaluation function as either the first function or the second function based on a comparison between the deviation and a specified threshold. The deviation for which the first function is applied is smaller than the deviation for which the second function is applied.

[0023] The model learning unit 102 sets the evaluation function to the first function when the deviation is less than a predetermined threshold, and sets it to the second function otherwise. In the second function, the rate of change of the weights relative to changes in deviation within the applicable range is slower than when the first function is used. Therefore, the learning system 10 can suppress abnormally high values ​​of the evaluation function when the deviation is high, thereby preventing hindered learning or overlearning. Here, the predetermined threshold is determined according to the learning object or purpose. The output unit 103 outputs the learned model.

[0024] Figure 2 is a flowchart of the learning method of the learning system 10 according to Embodiment 1. The learning method of the learning system 10 includes steps S11 to S19.

[0025] In step S11, the image acquisition unit 101 acquires the image and teacher image data. The teacher image data is teacher data for a detection model that defines the image region corresponding to the detection object contained in the image. In step S12, the model learning unit 102 generates estimated image data by estimating whether each pixel corresponds to the image region corresponding to the detection object. The estimated image data is, for example, a matrix of the same size as the pixel size of the image, with components having values ​​only at the coordinates of the image region corresponding to the detection object. The estimated image data is not limited to this and may also be a list of coordinates of the image region corresponding to the detection object, etc.

[0026] In step S13, the model learning unit 102 calculates the deviation by comparing the estimated image data and the teacher image data. In step S14, the model learning unit 102 determines whether the deviation is below a predetermined threshold. If the deviation is below the threshold, the learning system 10 begins step S15. Otherwise, the learning system 10 begins step S16.

[0027] Alternatively, in step S14, the model learning unit 102 can also determine the deviation using other methods based on the comparison result between the deviation degree and a predetermined threshold. In this case, the learning system 10 makes the deviation degree at the time of transition to step S15 smaller than the deviation degree at the time of transition to step S16.

[0028] In step S15, the model learning unit 102 sets the evaluation function to the first function and uses the deviation-based evaluation function to evaluate the detection model. Furthermore, the weights in the evaluation function are set smaller as the deviation decreases, with the first function being the first function. After step S15 ends, the learning system 10 begins step S17.

[0029] In step S16, the model learning unit 102 sets the evaluation function to the second function and uses the deviation-based evaluation function to evaluate the detection model. Furthermore, the weights in the evaluation function are set smaller as the deviation decreases in the second function. Here, in the second function, the rate of change of weights relative to changes in deviation within the applicable range is slower than when the first function is applied. After step S16 ends, the learning system 10 begins step S17.

[0030] In step S17, the model learning unit 102 adjusts the estimation parameters. The model learning unit 102 adjusts the estimation parameters, for example, based on the evaluation values. However, the model learning unit 102 is not limited to this; it can also adjust the estimation parameters using a prescribed method.

[0031] In step S18, the model learning unit 102 determines whether the learning termination condition has been met. The learning termination condition may be, for example, whether the processing from steps S12 to S17 has been repeated a predetermined number of times. Alternatively, the learning termination condition may be whether the evaluation value is within a predetermined range.

[0032] The learning termination condition is not limited to this and can be set as needed. If the learning termination condition is met, the learning system 10 begins step S19. Otherwise, the learning system 10 restarts processing from step S12. In step S19, the output unit 103 outputs the learned detection model.

[0033] As explained above, the learning system 10 learns the detection model by adjusting the influence of deviation in the evaluation function based on the estimated deviation between the image data and the teacher's image data. Specifically, the learning system 10 sets smaller weights in the evaluation function as the deviation decreases. Thus, the learning system 10 suppresses the learning of objects that are easy to identify in image regions, and appropriately learns objects that are difficult to identify in image regions.

[0034] Furthermore, when the deviation exceeds a predetermined threshold, the learning system 10 uses a second function—which increases the deviation at a slower rate than the first function—as the evaluation function. Thus, the learning system 10 can suppress the evaluation value from becoming an outlier even when the deviation becomes high.

[0035] Therefore, the learning system 10 can improve the accuracy of judging objects in image regions that are difficult to judge, while suppressing the influence of tiny objects that were originally intended to be treated as noise.

[0036] Alternatively, the learning system 10 may have a processor and a storage device (not shown). The storage device in the learning system 10 may include, for example, a storage device containing non-volatile memory such as flash memory or a solid-state drive (SSD). In this case, the storage device stores a computer program (hereinafter also simply referred to as the program) for performing the above-described method. The processor then reads the computer program from the storage device into a buffer memory such as dynamic random access memory (DRAM) and executes the program.

[0037] Each structure of the learning system 10 can be implemented by dedicated hardware. Furthermore, some or all of each structural element can be implemented by general-purpose or special-purpose circuits, processors, or combinations thereof. They can be composed of a single chip or multiple chips connected via a bus. Some or all of the structural elements of each device can be implemented by a combination of the aforementioned circuits and programs. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), or the like.

[0038] Furthermore, at least a portion of the processing performed by the learning system 10 can be provided as Software as a Service (SaaS). Additionally, the description of the structure herein is also applicable to other systems described below in this invention.

[0039] <Second Embodiment> Figure 3 is a flowchart of the learning method of the learning system according to Embodiment 2. The learning system according to Embodiment 2 has the same structure as the learning system 10 described with reference to Figure 1. Therefore, repeated descriptions of the structure of the learning system are omitted. The learning method of the learning system includes steps S21 to S27.

[0040] In step S21, the learning system acquires image and teacher image data to create a dataset. The dataset includes image information and correct answer labels for each pixel of the image. Here, a label is, for example, a number used for object recognition detected by the detection model. A correct answer label represents the label of the correct answer for each pixel of the image, indicating which object is displayed at that pixel.

[0041] In step S22, the learning system uses a segmentation network to learn a detection model for the detected objects. The segmentation network is a deep learning method based on a network for segmentation. Segmentation can be, for example, semantic segmentation. Segmentation is not limited to this; it can also be instance segmentation or panoptic segmentation.

[0042] Networks used for learning include, for example, fully convolutional networks, convolutional neural networks, or feature pyramid networks. Networks are not limited to these; recurrent neural networks or region convolutional neural networks (R-CNN) can be used depending on the purpose.

[0043] In step S23, the learning system uses a loss function to evaluate the model learned using the segmentation network. Here, the learning system calculates the deviation between the estimated image data (provided by the segmentation network for each pixel of the image) and the teacher's image data. If the deviation is below a predetermined threshold, the first loss function is used. Otherwise, the second loss function is used.

[0044] Here, the second function is defined as a function with a degree less than that of the first function. Therefore, the learning system can easily determine the second function and appropriately define the first and second functions.

[0045] Furthermore, the first and second functions can be continuous without being bounded by a specified threshold. The output of the first and second functions relative to the deviation can be determined as a single value. The second function can be a constant or discontinuous. Specifically, the second function can be a floor function, etc.

[0046] Referring to Figure 4, the processing in step S23 will be described in detail. Figure 4 is a flowchart of the applicable loss function for the learning system according to Embodiment 2. Step S23 includes sub-steps S231 to S233.

[0047] In sub-step S231, the learning system calculates the distribution-based loss P based on the estimated image data and the teacher image data. C Here, P C Specifically, the cross-entropy loss function is used for calculation. Furthermore, the learning system calculates the region-based loss P based on the estimated image data and the teacher's image data. D Here, P D Specifically, the Tversky Loss function is used for calculation.

[0048] In sub-step S232, the learning system according to PC and P D To calculate the synthetic loss function P, which serves as the estimated probability of the detection model. E Specifically, the learning system acts as a P... E =βP C +γP D To calculate P E Additionally, P E Calculated per label. And, P E It can also be outside the range of 0 to 1. Here, β and γ are weights, and β + γ = 1 holds true. β and γ can be different values ​​for each label.

[0049] The learning system uses the synthetic loss function P E It can simultaneously evaluate segmented regions and labels. Furthermore, by adjusting β and γ, the learning system can ensure the convergence of the proposed loss function even when the error increases during the learning process.

[0050] In sub-step S233, the learning system will P E Substitute these values ​​into the proposal loss function to evaluate the detection model. The proposal loss function is the function shown in equation (1).

[0051]

[0052] Here, |1-P E | represents the estimated probability error, indicating the deviation of the estimated results in the detection model, where α is a specified threshold.

[0053] Figure 5 is a graph showing the approximate shape of the weights y of the proposal loss function according to Embodiment 2. Here, the weights of the proposal loss function are multiplied by -log(P) in the proposal loss function. E The coefficient of ).

[0054] In Figure 5, the vertical axis represents the magnitude of the weights of the proposal loss function, and the horizontal axis represents the weights of |1-P|. E | represents the estimated probability error. The weights, represented by y, are in the range α > |1-P. E The function is a quadratic function within the range of |1-P. The learning system learns by using α > |1-P. E Adjusting the number of times in the weight calculation formula within the range of |1-P| can avoid gradient loss and prevent overlearning. Furthermore, the weights are within the range of α≤|1-P ... E The expression is linear within the range of |1-P. E Within the range of |, the weights are set as a slow function, even if the estimated probability error is |1-P. E Even when the value increases dramatically, the loss function will not take out of proportion. Therefore, the learning system can make the loss function converge stably.

[0055] Furthermore, α is determined based on the purpose and intended use. For example, if the user of the learning system wants to quickly learn the estimation of the detected object, α can be set high. If the user wants to prevent overlearning, α can be set low. Moreover, α can be changed during the learning process. For example, the user can set α to a high value in the first half of the learning process and a low value in the second half. Thus, the learning system can learn quickly in the first half and avoid overlearning in the second half.

[0056] As described above, by executing step S23, which includes sub-steps S231 to S233, the learning system is able to appropriately evaluate the estimated probability of the detection model.

[0057] Referring again to Figure 3, in step S24, the learning system updates the parameters of the detection model. The learning system can update the parameters based on the evaluation value of the learning model obtained through the loss function.

[0058] In step S25, the learning system increments the learning count (Epoch) by 1. Thus, the learning system completes one learning iteration of the input image. In step S16, the learning system determines whether the detection model has completed the prescribed learning count N. If the prescribed learning count N has not been reached, the learning system restarts from step S22. If the prescribed learning count N has been reached, the learning system begins step S27.

[0059] In step S27, the learning system outputs the detection model after learning is complete. As explained above, the learning system according to this embodiment can improve the accuracy of judging objects in image regions that are difficult to judge, while suppressing the influence of tiny objects that are intended to be treated as noise, thereby appropriately learning the detection model.

[0060] Furthermore, the execution order of the processing can be changed as needed. For example, the learning system can perform the parameter update in step S24 during the period from the determination in step S26 to the start of step S22. And, to confirm the learning process, the learning system can output the learned detection model during the period from the determination in step S26 to the start of step S22 at the stage where a predetermined number of learning iterations have ended.

[0061] Users of the learning system can set α, β, and γ to be changed as variables during the learning process. Furthermore, users can change α, β, and γ based on the evaluation of the learned detection model and then relearn the detection model.

[0062] The learned detection model can be used, for example, for object detection in automated control of machinery. Specifically, the learned detection model in the learning system described in this embodiment is effective in scenarios such as robot assembly of parts, autonomous driving of vehicles, detection of empty spaces in parking lots, or detection of people in photographic images taken by drones.

[0063] <Examples of Hardware Structures> Hereinafter, examples of cases in which the functional structures of the information processing device in this invention are implemented by a combination of hardware and software will be described.

[0064] Figure 6 is a block diagram illustrating the hardware structure of a computer. The information processing device of the present invention can realize the above-mentioned functions through a computer 500 including the hardware structure shown in the figure. The computer 500 can be a portable computer such as a smartphone or tablet terminal, or a stationary computer such as a PC. The computer 500 can be a dedicated computer designed to implement various devices, or a general-purpose computer. By installing the prescribed application programs, the computer 500 can realize the desired functions.

[0065] Computer 500 includes a bus 502, a processor 504, a memory 506, a storage device 508, an input / output interface (I / F) 510, and a network interface (I / F) 512. The internal bus 502 serves as a data transmission path for the processor 504, memory 506, storage device 508, I / O interface 510, and network interface 512 to send and receive data. The method of connecting the processor 504, etc., is not limited to a bus connection.

[0066] Processor 504 is a processor such as a CPU, GPU, or FPGA. Memory 506 is a main storage device implemented using random access memory (RAM) or similar methods.

[0067] Storage device 508 is an auxiliary storage device implemented using a hard disk, SSD, memory card, or read-only memory (ROM). Storage device 508 stores programs for implementing desired functions. Processor 504 implements the various functional units of each device by reading these programs into memory 506 and executing them.

[0068] Input / output interface 510 is an interface for connecting computer 500 and input / output devices. For example, input devices such as keyboards or output devices such as displays can be connected to input / output interface 510. Network interface 512 is an interface for connecting computer 500 to a network.

[0069] Furthermore, the present invention is not limited to the embodiments described above, and appropriate modifications can be made without departing from the spirit of the invention. For example, the structure and details of the present invention can be modified in various ways that are understandable to those skilled in the art within the scope of the invention. Moreover, each embodiment can be appropriately combined with other embodiments.

[0070] Symbol Explanation: 10-Learning System, 101-Image Acquisition Unit, 102-Model Learning Unit, 103-Output Unit, N-Number of Learning Attempts, α-Threshold, 500-Computer, 502-Bus, 504-Processor, 506-Memory, 508-Storage Device, 510-Input / Output Interface, 512-Network Interface.

Claims

1. A learning system, characterized in that, It includes: an image acquisition unit that acquires an image and teacher image data that defines an image region corresponding to a detection object contained in the image; and a model learning unit that calculates a deviation by comparing the estimated image data for each pixel of the image with the teacher image data to determine whether the estimated image data corresponds to the image region, and performs model learning using an evaluation function based on the deviation. The model learning unit outputs the learned model. In the model learning unit, the lower the deviation, the smaller the weight assigned to the evaluation function. Based on the comparison result of the deviation and a predetermined threshold, the evaluation function is determined to be either a first function or a second function. The deviation applied to the first function is smaller than the deviation applied to the second function. In the second function, the rate of change of the weight relative to the change in the deviation within the applicable range is slower than when the first function is applied.

2. The learning system according to claim 1, characterized in that, The second function has a lower degree than the first function.

3. The learning system according to claim 2, characterized in that, The evaluation function is a function represented by equation (1). Among them, P E Let |1-P| be the estimated probability of the model. E | represents the deviation degree, and α is the threshold.

4. The learning system according to any one of claims 1 to 3, characterized in that, The learned model is used in object detection for automatic control of machinery.

5. A learning method, characterized in that, The computer performs the following processing: acquires an image and teacher image data that defines the image region corresponding to the detected object contained in the image; calculates the deviation by comparing the estimated image data of the image region with the teacher image data for each pixel of the image; and performs model learning using an evaluation function based on the deviation. The learned model is output; and in the learning of the model, the lower the deviation, the smaller the weight assigned to the evaluation function is set; the evaluation function is determined as a first function or a second function based on the comparison result of the deviation and a specified threshold, wherein the deviation applied to the first function is smaller than the deviation applied to the second function, and in the second function, the rate of change of the weight relative to the change of the deviation within the applicable range is slower than when the first function is applied.

Images