Learning device, learning method, and program

Abstract

To ensure the accuracy of a road area detected from an image by verifying the road area from a plurality of viewpoints.SOLUTION: A learning device for training a machine learning model that receives an image as an input and outputs a first pixel region representing road edges and a second pixel region representing a road area in the image includes: a determination unit configured to compare the first and second pixel regions to determine a degree of matching between the first and second pixel regions; a setting unit configured to set a gain for a first error between the output first pixel region and correct data representing the road edges and a second error between the output second pixel region and correct data representing the road area based on the degree of matching; and a learning unit configured to train the machine learning model so as to reduce a value of a loss function calculated based on the first and second errors for which the gain has been set.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a learning device, a learning method, and a program. [Background technology]

[0002] Conventionally, there are known techniques for detecting road areas included in an image in order to utilize the technique for driving assistance or autonomous driving of a vehicle. For example, Patent Document 1 discloses a technique for detecting a road area by scanning an input image in the x-direction, extracting edges, and detecting white lines based on color differences near the edges. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-038101 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 does not verify the accuracy of the white lines detected by scanning from a different perspective. As a result, the conventional technology may not be able to guarantee the accuracy of the road area detected from the image.

[0005] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a learning device, a learning method, and a program that can ensure the accuracy of road areas detected from an image by verifying them from multiple perspectives. [Means for solving the problem]

[0006] The learning device, learning method, and program according to the present invention employ the following configuration. (1): A learning device according to one embodiment of the present invention is a learning device that learns a machine learning model that takes an image as input and outputs a first pixel area representing a road edge and a second pixel area representing a road area in the image, and includes a determination unit that compares the first pixel area with the second pixel area and determines a degree of consistency between the first pixel area and the second pixel area based on the degree of consistency, a setting unit that sets gains on a first error between the outputted first pixel area and correct answer data representing the road edge and a second error between the outputted second pixel area and the correct answer data representing the road area, and a learning unit that trains the machine learning model so as to reduce the value of a loss function calculated based on the first error and the second error to which the gains have been set.

[0007] (2) In the above aspect (1), the determination unit determines that the degree of consistency is higher as the second pixel region is sandwiched between the first pixel regions without any gaps or overlaps.

[0008] (3) In the above aspect (1) or (2), the setting unit sets a larger value of the gain as the determined degree of consistency becomes lower.

[0009] (4): A learning method according to another aspect of the present invention is a learning method for training a machine learning model that uses an image as input and outputs a first pixel region representing a road edge and a second pixel region representing a road area in the image, in which a computer compares the first pixel region with the second pixel region to determine whether the second pixel region is sandwiched between the first pixel region and the correct data representing the road edge based on the degree of consistency, sets gains to a first error between the output first pixel region and correct data representing the road edge and a second error between the output second pixel region and the correct data representing the road area, and trains the machine learning model so as to reduce the value of a loss function calculated based on the first error and the second error to which the gains are set.

[0010] (5): A program according to another aspect of the present invention is a program for training a machine learning model that takes an image as input and outputs a first pixel region representing a road edge and a second pixel region representing a road area in the image, the program causing a computer to compare the first pixel region with the second pixel region and determine whether the second pixel region is sandwiched between the first pixel region and the correct data representing the road edge based on the degree of consistency, and sets gains to a first error between the outputted first pixel region and correct data representing the road edge and a second error between the outputted second pixel region and the correct data representing the road area, and trains the machine learning model so as to reduce the value of a loss function calculated based on the first error and the second error to which the gains have been set. Effect of the Invention

[0011] According to (1) to (5), the accuracy of the road area detected from the image can be ensured by verifying the road area from multiple viewpoints. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating a configuration of a learning device 100 according to the present embodiment. [Diagram 2] FIG. 1 is a diagram for explaining an overview of learning of a machine learning model 140B based on learning data 140A. [Diagram 3] 10 is a diagram for explaining a determination of the degree of consistency by a determining unit 110. FIG. [Figure 4] 4 is a flowchart showing an example of the flow of processing executed by the learning device 100 according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of a learning device, a learning method, and a program according to the present invention will be described with reference to the drawings.

[0014] [composition] FIG. 1 is a diagram showing the configuration of a learning device 100 according to this embodiment. The learning device 100 is an information processing device that learns a machine learning model that receives an image as an input and outputs a first pixel region representing a road edge and a second pixel region representing a road region in the image. The learning device 100 includes, for example, a determination unit 110, a setting unit 120, a learning unit 130, and a storage unit 140. The determination unit 110, the setting unit 120, and the learning unit 130 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). In addition, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (a storage device equipped with a non-transient storage medium) such as the HDD or flash memory of the learning device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the learning device 100 by mounting the storage medium (non-transient storage medium) in a drive device. The storage unit 140 stores, for example, learning data 140A and machine learning model 140B. The storage unit 140 is realized by, for example, a RAM, a flash memory, an SD card, or the like.

[0015] Fig. 2 is a diagram for explaining an overview of learning of the machine learning model 140B based on the learning data 140A. As shown in Fig. 2, the machine learning model 140B is a machine learning model that receives an image as input and outputs a first pixel region RE representing a road edge and a second pixel region RA representing a road area in the image. The machine learning model 140B is, for example, a convolutional neural network (CNN), which first expands the dimension (number of channels) in the convolution for an input image to extract the feature amount of the image, and then reduces the dimension to extract the first pixel region RE representing the road edge and the second pixel region RA representing the road area from the image.

[0016] The determination unit 110 compares the first pixel region RE and the second pixel region RA output from the machine learning model 140B to determine the degree of consistency between these pixel regions. That is, since a road region is generally sandwiched between road edges without gaps or overlaps, the determination unit 110 determines the degree of consistency based on whether the second pixel region RA representing the road region is sandwiched between the first pixel region RE representing the road edges without gaps or overlaps.

[0017] Fig. 3 is a diagram for explaining the determination of the degree of consistency by the determination unit 110. As shown in Fig. 3(a), the determination unit 110 compares a first pixel region RE and a second pixel region RA in an image, and if there is a gap region between the first pixel region RE and the second pixel region RA, it determines that the degree of consistency between these pixel regions is lower. Furthermore, as shown in Fig. 3(b), if there is an overlapping region between the first pixel region RE and the second pixel region RA, the determination unit 110 determines that the degree of consistency between these pixel regions is lower.

[0018] The setting unit 120 sets a gain for the error between the first pixel region RE and the second pixel region RA and the learning data 140A (correct answer data) based on the consistency determined by the determining unit 110. Here, as shown in FIG. 2, the learning data 140A is data in which a pixel region representing a road edge and a pixel region representing a road area are specified in advance in an image. The learning data 140A is generated, for example, by an administrator or an operator of the learning device 100 specifying the road edge and the road area for the pixels of the image in advance on his / her own terminal, and is stored in the storage unit 140. Alternatively, the learning device 100 may download the learning data 140A stored in an external server to the storage unit 140 via a network at the timing when the setting unit 120 executes learning.

[0019] The setting unit 120 sets (e.g., multiplies) a gain according to the degree of consistency for the first error between the first pixel region RE and the ground truth data of the road edge, and the second error between the second pixel region RA and the ground truth data of the road area. More specifically, the setting unit 120 sets a larger gain value for the first error and the second error as the determined degree of consistency is lower.

[0020] After the setting unit 120 sets gains to the first error and the second error, the learning unit 130 trains the machine learning model 140B so as to reduce the value of a loss function (e.g., the absolute sum of the first error and the second error) based on the first error and the second error to which the gains are set. In this case, any learning method such as back propagation may be used for the learning. The learning unit 130 acquires the machine learning model 140B for which learning has been completed as a trained model.

[0021] The acquired trained model may be used by a driving assistance device or a vehicle control device mounted on the vehicle. For example, the driving assistance device may input a forward image of the vehicle captured by a camera into the trained model, and display the road edge or road area output by the trained model on the HMI. Also, for example, the vehicle control device may input a forward image of the vehicle captured by a camera into the trained model, and control the vehicle to travel within the road edge or road area output by the trained model.

[0022] Thus, in the conventional technology, a machine learning model is generally trained to reduce the value of a loss function based on an error between the output of the machine learning model and ground truth data representing a road edge or road area. On the other hand, in the present embodiment, the machine learning model is trained to reduce the value of a loss function based on an error in which the gain is taken into account, after taking into account the consistency between a plurality of outputs of the machine learning model as a gain, for the error between the output of the machine learning model and the ground truth data. In other words, according to the present embodiment, the accuracy of the road area detected from the image can be ensured by verifying it from a plurality of viewpoints.

[0023] [Processing flow] Next, the flow of processing executed by the learning device 100 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of the flow of processing executed by the learning device 100 according to this embodiment.

[0024] First, the learning unit 130 acquires an image from the learning data 140A stored in the storage unit 140, and inputs the image to the machine learning model 140B (step S100). Next, the determination unit 110 determines the degree of consistency between the first pixel region RE and the second pixel region RA output from the machine learning model 140B (step S102).

[0025] Next, the setting unit 120 sets a gain to the first error between the first pixel region RE and the correct data and the second error between the second pixel region RA and the correct data based on the consistency (step S104). Next, the learning unit 130 learns the machine learning model 140B so as to reduce the value of the loss function calculated based on the first error and the second error for which the gains are set (step S106). This ends the process according to this flowchart.

[0026] As described above, in this embodiment, an image is used as input, a first pixel area representing a road edge in the image output by a machine learning model is compared with a second pixel area representing a road area, a consistency between the first pixel area and the second pixel area is determined, and a gain is set for a first error between the output first pixel area and correct answer data representing the road edge and a second error between the output second pixel area and correct answer data representing the road area based on the consistency, and the machine learning model is trained to reduce the value of a loss function calculated based on the first error and the second error for which a gain is set. In other words, by verifying the road area detected from an image from multiple viewpoints, its accuracy can be ensured.

[0027] The above-described embodiment can be expressed as follows. A storage device storing a program; a hardware processor; The hardware processor executes the program, Using an image as input, a first pixel region representing a road edge in the image and a second pixel region representing a road region output by a machine learning model are compared to determine a degree of consistency between the first pixel region and the second pixel region; setting a gain for a first error between the outputted first pixel area and the ground truth data representing the road edge and a second error between the outputted second pixel area and the ground truth data representing the road area based on the degree of consistency; training the machine learning model so as to reduce a value of a loss function calculated based on the first error and the second error for which the gain is set; The learning device is configured as follows.

[0028] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0029] 100 Learning Device 110 Judgment section 120 Setting section 130 Learning Department 140 Storage section 140A Training Data 140B Machine Learning Model

Claims

1. A learning device that learns a machine learning model that receives an image as input and outputs a first pixel area representing a road edge and a second pixel area representing a road area in the image, a determination unit that compares the first pixel region with the second pixel region to determine a degree of matching between the first pixel region and the second pixel region; a setting unit that sets a gain to a first error between the output first pixel area and the ground truth data representing the road edge and a second error between the output second pixel area and the ground truth data representing the road area based on the degree of consistency; A learning unit that learns the machine learning model so as to reduce a value of a loss function calculated based on the first error and the second error to which the gain is set. Learning device.

2. The determination unit determines that the degree of consistency is higher when the second pixel region is sandwiched between the first pixel regions without a gap and without overlap. The learning device according to claim 1 .

3. The setting unit sets a larger value of the gain as the determined consistency degree is lower. The learning device according to claim 1 or 2.

4. A learning method for learning a machine learning model that receives an image as input and outputs a first pixel area representing a road edge and a second pixel area representing a road area in the image, the method comprising: comparing the first pixel region with the second pixel region to determine a degree of matching between the first pixel region and the second pixel region; setting a gain for a first error between the outputted first pixel area and the ground truth data representing the road edge and a second error between the outputted second pixel area and the ground truth data representing the road area based on the degree of consistency; training the machine learning model so as to reduce a value of a loss function calculated based on the first error and the second error to which the gain is set; How to learn.

5. A program for training a machine learning model that receives an image as input and outputs a first pixel area representing a road edge and a second pixel area representing a road area in the image, the program comprising: comparing the first pixel region with the second pixel region to determine a degree of matching between the first pixel region and the second pixel region; setting gains for a first error between the output first pixel area and the ground truth data representing the road edge and a second error between the output second pixel area and the ground truth data representing the road area based on the degree of consistency; training the machine learning model so as to reduce a value of a loss function calculated based on the first error and the second error for which the gain is set; program.

Images