Learning device, learning method, and program

Abstract

To ensure the accuracy of a road area detected from an image by verifying the road area from a plurality of viewpoints.SOLUTION: A learning device for training a machine learning model that receives an image as an input and outputs a first pixel region representing road edges and a second pixel region representing a road area in the image includes: a determination unit configured to compare the first and second pixel regions to determine a degree of matching between the first and second pixel regions; a setting unit configured to set a gain for a first error between the output first pixel region and correct data representing the road edges and a second error between the output second pixel region and correct data representing the road area based on the degree of matching; and a learning unit configured to train the machine learning model so as to reduce a value of a loss function calculated based on the first and second errors for which the gain has been set.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a learning device, a learning method, and a program.

Background Art

[0002] Conventionally, a technique for detecting a road region included in an image has been known for use in vehicle driving support or autonomous driving. For example, Patent Document 1 discloses a technique for detecting a road region by scanning an input image in the x direction, extracting edges, and detecting a white line based on a color difference in the vicinity of the edges.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technique described in Patent Document 1, the accuracy of the white line detected by scanning is not verified from different viewpoints. As a result, in the prior art, there are cases where the accuracy of the road region detected from an image cannot be ensured.

[0005] The present invention has been made in consideration of such circumstances, and one of its objectives is to provide a learning device, a learning method, and a program that can ensure the accuracy of a road region detected from an image by verifying it from a plurality of viewpoints.

Means for Solving the Problems

[0006] The learning device, learning method, and program according to this invention employ the following configuration. (1) A learning device according to one aspect of the present invention is a learning device for learning a machine learning model that takes an image as input and outputs a first pixel region representing the edge of a road and a second pixel region representing a road region from the image, comprising: a determination unit that compares the first pixel region and the second pixel region and determines the degree of consistency between the first pixel region and the second pixel region; a setting unit that sets gains for a first error between the outputted first pixel region and the correct data representing the edge of a road, and a second error between the outputted second pixel region and the correct data representing the road region, based on the degree of consistency; and a learning unit that learns the machine learning model to reduce the value of a loss function calculated based on the first error and the second error to which the gains have been set.

[0007] (2) In the embodiment of (1) above, the determination unit determines that the degree of consistency is higher the more the second pixel region is sandwiched by the first pixel region without any gaps or overlaps.

[0008] (3) In the embodiment of (1) or (2) above, the setting unit increases the value of the gain to be set as the determined degree of matching decreases.

[0009] (4): Another learning method according to this invention is a learning method for learning a machine learning model that takes an image as input and outputs a first pixel region representing the edge of a road and a second pixel region representing a road region from the image, wherein the computer compares the first pixel region and the second pixel region to determine whether the second pixel region is sandwiched by the first pixel region, sets gains on a first error between the output first pixel region and the ground truth data representing the edge of the road, and a second error between the output second pixel region and the ground truth data representing the road region based on the degree of consistency, and learns the machine learning model to reduce the value of the loss function calculated based on the first and second errors to which the gains have been set.

[0010] (5): Another aspect of the present invention is a program that takes an image as input and trains a machine learning model that outputs a first pixel region representing the edge of a road and a second pixel region representing a road region from the image, wherein the program causes the computer to compare the first pixel region and the second pixel region to determine whether the second pixel region is sandwiched by the first pixel region, and based on the degree of consistency, sets gains for a first error between the output first pixel region and the ground truth data representing the edge of the road, and for a second error between the output second pixel region and the ground truth data representing the road region, and trains the machine learning model to reduce the value of the loss function calculated based on the first and second errors to which the gains have been set. [Effects of the Invention]

[0011] According to (1) to (5), the accuracy of the road area detected from the image can be ensured by verifying it from multiple perspectives. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows the configuration of the learning device 100 according to this embodiment. [Figure 2] This diagram illustrates the overview of training the machine learning model 140B based on training data 140A. [Figure 3] This diagram illustrates the determination of consistency by the determination unit 110. [Figure 4] This flowchart shows an example of the processing flow performed by the learning device 100 according to this embodiment. [Modes for carrying out the invention]

[0013] Hereinafter, embodiments of the learning device, learning method, and program of the present invention will be described with reference to the drawings.

[0014] [composition] Figure 1 shows the configuration of the learning device 100 according to this embodiment. The learning device 100 is an information processing device that takes an image as input and learns a machine learning model that outputs a first pixel region representing the road edge and a second pixel region representing the road region within the image. The learning device 100 includes, for example, a determination unit 110, a setting unit 120, a learning unit 130, and a storage unit 140. The determination unit 110, the setting unit 120, and the learning unit 130 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance in a storage device such as the HDD or flash memory of the learning device 100 (a storage device equipped with a non-transient storage medium), or it may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the learning device 100 when the storage medium (non-transient storage medium) is mounted on a drive device. The storage unit 140 stores, for example, learning data 140A and a machine learning model 140B. The storage unit 140 is implemented by, for example, RAM, flash memory, an SD card, etc.

[0015] Figure 2 is a diagram illustrating the overview of training machine learning model 140B based on training data 140A. As shown in Figure 2, machine learning model 140B is a machine learning model that takes an image as input and outputs a first pixel region RE representing the road edge and a second pixel region RA representing the road area from the image. Machine learning model 140B is, for example, a convolutional neural network (CNN), which first expands the dimensionality (number of channels) in the convolution of the input image to extract the features of the image, and then reduces the dimensionality to extract the first pixel region RE representing the road edge and the second pixel region RA representing the road area from the image.

[0016] The determination unit 110 compares the first pixel region RE and the second pixel region RA output from the machine learning model 140B to determine the degree of consistency between these pixel regions. In other words, since road regions are generally enclosed by road edges without gaps or overlaps, the determination unit 110 determines the degree of consistency based on whether the second pixel region RA representing the road region is enclosed by the first pixel region RE representing the road edge without gaps or overlaps.

[0017] Figure 3 is a diagram illustrating the determination of consistency by the determination unit 110. As shown in Figure 3(a), the determination unit 110 compares the first pixel region RE and the second pixel region RA in the image, and determines that the consistency between these pixel regions is lower if there is a gap between the first pixel region RE and the second pixel region RA. Furthermore, as shown in Figure 3(b), the determination unit 110 determines that the consistency between these pixel regions is lower if there is an overlapping region between the first pixel region RE and the second pixel region RA.

[0018] The setting unit 120 sets a gain for the error between the first pixel region RE and the second pixel region RA and the learning data 140A (correct data) based on the degree of matching determined by the determination unit 110. Here, as shown in FIG. 2, the learning data 140A is such that pixel regions representing the road edge and the road region in the image are specified in advance. The learning data 140A is generated, for example, by an administrator or operator of the learning device 100 specifying the road edge and the road region for the pixels of the image in advance on their own terminal and stored in the storage unit 140. Alternatively, the learning device 100 may download the learning data 140A stored in an external server to the storage unit 140 via a network at the timing of learning execution by the setting unit 120.

[0019] The setting unit 120 sets (for example, multiplies) a gain corresponding to the degree of matching for the first error between the first pixel region RE and the correct data of the road edge and the second error between the second pixel region RA and the correct data of the road region. More specifically, the setting unit 120 sets a larger value of gain for the first error and the second error as the determined degree of matching is lower.

[0020] After the gain is set for the first error and the second error by the setting unit 120, the learning unit 130 learns the machine learning model 140B so as to reduce the value of a loss function (for example, the absolute sum of the first error and the second error) based on the first error and the second error for which the gain is set. The learning in this case may use any learning method such as, for example, the backpropagation method. The learning unit 130 acquires the machine learning model 140B for which the learning is completed as a learned model.

[0021] The obtained learned model may be used by a driving support device or a vehicle control device mounted on a vehicle. For example, the driving support device may input a front image of the vehicle captured by a camera into the learned model, and display a road edge or a road area output by the learned model on an HMI. Further, for example, the vehicle control device may input a front image of the vehicle captured by a camera into the learned model, and control the vehicle to travel within a road edge or a road area output by the learned model.

[0022] As described above, in the prior art, generally, the machine learning model is learned so as to reduce the value of a loss function based on the error between the output of the machine learning model and the correct data representing the road edge or the road area. On the other hand, in the present embodiment, with respect to the error between the output of the machine learning model and the correct data, after considering the consistency between a plurality of outputs of the machine learning model as a gain, the machine learning model is learned so as to reduce the value of the loss function based on the error considering the gain. That is, according to the present embodiment, the accuracy can be ensured by verifying the road area detected from the image from a plurality of viewpoints.

[0023] [Flow of processing] Next, referring to FIG. 4, the flow of processing executed by the learning device 100 according to the present embodiment will be described. FIG. 4 is a flowchart showing an example of the flow of processing executed by the learning device 100 according to the present embodiment.

[0024] First, the learning unit 130 acquires an image from the learning data 140A stored in the storage unit 140, and inputs it into the machine learning model 140B (step S100). Next, the determination unit 110 determines the degree of consistency between the first pixel region RE and the second pixel region RA output from the machine learning model 140B (step S102).

[0025] Next, the setting unit 120 sets gains for the first error between the first pixel region RE and the ground truth data, and for the second error between the second pixel region RA and the ground truth data, based on the degree of consistency (step S104). Next, the learning unit 130 learns the machine learning model 140B to reduce the value of the loss function calculated based on the first and second errors to which the gains have been set (step S106). This completes the process related to this flowchart.

[0026] As described above, in this embodiment, an image is taken as input, and a machine learning model outputs a first pixel region representing the road edge and a second pixel region representing the road region within that image. The degree of consistency between the first and second pixel regions is determined, and based on the degree of consistency, gains are set for the first error between the output first pixel region and the ground truth data representing the road edge, and for the second error between the output second pixel region and the ground truth data representing the road region. The machine learning model is then trained to reduce the value of the loss function calculated based on the first and second errors to which the gains have been set. In other words, the accuracy of the road region detected from the image can be ensured by verifying it from multiple viewpoints.

[0027] The embodiments described above can be expressed as follows. A memory device that stores the program, Equipped with a hardware processor, The hardware processor executes the program, The machine learning model takes an image as input and compares the first pixel region representing the road edge and the second pixel region representing the road area of ​​the image output by the machine learning model to determine the degree of consistency between the first pixel region and the second pixel region. Based on the degree of consistency, gains are set for the first error between the outputted first pixel region and the ground truth data representing the road edge, and for the second error between the outputted second pixel region and the ground truth data representing the road region. The machine learning model is trained to reduce the value of the loss function calculated based on the first and second errors, with the gain set accordingly. A learning device configured in such a way.

[0028] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0029] 100 Learning Devices 110 Judgment section 120 Setting section 130 Learning Department 140 Storage section 140A Training Data 140B Machine Learning Model

Claims

1. A learning device for training a machine learning model that takes an image as input and outputs a first pixel region representing the edge of the road and a second pixel region representing the road area from the image, A determination unit that compares the first pixel region and the second pixel region to determine the degree of consistency between the first pixel region and the second pixel region, Based on the degree of consistency, a setting unit sets gains for a first error between the outputted first pixel region and the correct data representing the road edge, and for a second error between the outputted second pixel region and the correct data representing the road region. The machine learning model is trained to reduce the value of the loss function calculated based on the first and second errors, for which the gains have been set. Learning device.

2. The determination unit determines that the degree of consistency is higher the more the second pixel region is sandwiched between the first pixel region without any gaps or overlaps. The learning device according to claim 1.

3. The setting unit increases the value of the gain to be set, the lower the determined degree of matching. The learning device according to claim 1 or 2.

4. A learning method for training a machine learning model that takes an image as input and outputs a first pixel region representing the edge of the road and a second pixel region representing the road area from the image, wherein the computer The degree of consistency between the first pixel region and the second pixel region is determined by comparing them. Based on the degree of consistency, gains are set for the first error between the outputted first pixel region and the ground truth data representing the road edge, and for the second error between the outputted second pixel region and the ground truth data representing the road region. The machine learning model is trained to reduce the value of the loss function calculated based on the first and second errors, with the gain set accordingly. Learning methods.

5. A program for training a machine learning model that takes an image as input and outputs a first pixel region representing the edge of the road and a second pixel region representing the road area from the image, wherein the computer... The degree of consistency between the first pixel region and the second pixel region is determined by comparing them. Based on the degree of consistency, gains are set for the first error between the outputted first pixel region and the ground truth data representing the road edge, and for the second error between the outputted second pixel region and the ground truth data representing the road region. The machine learning model is trained to reduce the value of the loss function calculated based on the first and second errors, for which the gain is set. program.

Images