Learning apparatus, learning method, and storage medium

Abstract

The invention provides a learning device, a learning method, and a storage medium which can ensure the accuracy of a road region detected from an image by verifying the road region from a plurality of viewpoints. A learning apparatus that learns a machine learning model that outputs a first pixel region representing an end of a road and a second pixel region representing a road region in an image using the image as an input, the learning apparatus including: a determination unit that determines whether the image is a road region or not by comparing the first pixel region with the second pixel region; the matching degree of the first pixel region and the second pixel region is judged; a setting unit that sets a gain for a first error between the output first pixel region and the forward solution data indicating the road end, and a second error between the output second pixel region and the forward solution data indicating the road region, on the basis of the adaptation degree; and a learning unit that learns the machine learning model so that the value of the loss function calculated on the basis of the first error and the second error for which the gain has been set is reduced.

Description

Technical Field

[0001] The invention relates to a learning device, a learning method and a storage medium. Background Art

[0002] In the past, there is a known technology for detecting a road area included in an image for use in driving support or autonomous driving of a vehicle. For example, Patent Document 1 discloses a technology for detecting a road area by scanning an input image along the x direction to extract an edge and detecting a white line based on the difference in color near the edge.

[0003] Prior Art Literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-038101 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, the accuracy of the white lines detected by scanning is not verified from a different point of view in the technology described in Patent Document 1. As a result, the accuracy of the road area detected from the image may not be ensured in the conventional technology.

[0008] The present invention has been made in consideration of such circumstances, and one of its objects is to provide a learning device, a learning method, and a storage medium that can ensure the accuracy of a road area detected from an image by verifying the road area from multiple viewpoints.

[0009] Solutions to Solve Problems

[0010] The learning device, learning method, and storage medium according to the present invention employ the following structures.

[0011] (1): One embodiment of the present invention relates to a learning device that enables a machine learning model to learn, wherein the machine learning model takes an image as input and outputs a first pixel area representing a road end and a second pixel area representing a road area in the image, wherein the learning device comprises: a determination unit that compares the first pixel area with the second pixel area to determine the degree of fitness between the first pixel area and the second pixel area; a setting unit that sets a gain for a first error between the output first pixel area and correct data representing the road end, and a second error between the output second pixel area and the correct data representing the road area based on the fitness; and a learning unit that enables the machine learning model to learn in such a way that the value of a loss function calculated based on the first error and the second error with the gain set is reduced.

[0012] (2) In the above-mentioned embodiment (1), the determination unit determines that the degree of adaptation is higher when the second pixel region is sandwiched by the first pixel region without any gap or overlap.

[0013] (3) In the solution of (1) or (2) above, the setting unit increases the set value of the gain as the determined degree of adaptation is lower.

[0014] (4): Another embodiment of the present invention relates to a learning method for causing a machine learning model to learn, wherein the machine learning model takes an image as input and outputs a first pixel area in the image representing an end of a road and a second pixel area representing a road area, wherein the learning method causes a computer to perform the following processing: comparing the first pixel area with the second pixel area to determine the degree of fitness between the first pixel area and the second pixel area; setting a gain for a first error between the output first pixel area and correct data representing the end of the road, and a second error between the output second pixel area and the correct data representing the road area based on the fitness; and causing the machine learning model to learn in a manner that reduces the value of a loss function calculated based on the first error and the second error with the gain set.

[0015] (5): Another embodiment of the present invention relates to a storage medium storing a program that causes a machine learning model to learn, the machine learning model taking an image as input and outputting a first pixel region representing a road end and a second pixel region representing a road region in the image, wherein the program causes a computer to perform the following processing:

[0016] The first pixel area is compared with the second pixel area to determine the fitness of the first pixel area and the second pixel area; based on the fitness, a gain is set for a first error between the output first pixel area and the correct data representing the road end, and a second error between the output second pixel area and the correct data representing the road area; and the machine learning model is learned in a manner that reduces the value of the loss function calculated based on the first error and the second error with the gain set.

[0017] Effects of the Invention

[0018] According to (1) to (5), the accuracy of the road area detected from the image can be ensured by verifying the road area from multiple viewpoints. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1It is a diagram showing the configuration of the learning device 100 according to the present embodiment.

[0020] Figure 2 This is a diagram for explaining an overview of learning of the machine learning model 140B based on the learning data 140A.

[0021] Figure 3 It is a diagram for explaining the determination of the degree of adaptation performed by the determination unit 110 .

[0022] Figure 4 This is a flowchart showing an example of the flow of processing executed by the learning device 100 according to the present embodiment.

[0023] Description of reference numerals:

[0024] 100 learning devices

[0025] 110 Judgment Department

[0026] 120 Setting department

[0027] 130 Learning Department

[0028] 140 Storage

[0029] 140A Learning Data

[0030] 140B machine learning model. DETAILED DESCRIPTION

[0031] Hereinafter, embodiments of a learning device, a learning method, and a storage medium according to the present invention will be described with reference to the accompanying drawings.

[0032] [structure]

[0033] Figure 1: is a diagram showing the structure of the learning device 100 involved in the present embodiment. The learning device 100 is an information processing device that enables a machine learning model to learn, and the machine learning model takes an image as input and outputs a first pixel area representing a road end and a second pixel area representing a road area in the image. The learning device 100, for example, includes a determination unit 110, a setting unit 120, a learning unit 130, and a storage unit 140. The determination unit 110, the setting unit 120, and the learning unit 130 are respectively implemented by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). In addition, some or all of these components can be implemented by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc., or can be implemented by the coordinated cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD or flash memory of the learning device 100 (a storage device having a non-transitory storage medium), or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the learning device 100 by attaching the storage medium (non-transitory storage medium) to a drive device. The storage unit 140 stores, for example, learning data 140A and a machine learning model 140B. The storage unit 140 is implemented, for example, by a RAM, a flash memory, an SD card, or the like.

[0034] Figure 2 140A is a diagram for explaining the outline of the learning of the machine learning model 140B based on the learning data 140A. Figure 2 As shown, the machine learning model 140B is a machine learning model that takes an image as input and outputs a first pixel region RE representing a road end and a second pixel region RA representing a road area in the image. The machine learning model 140B is, for example, a convolutional neural network (CNN), which first expands the dimension (number of channels) in the convolution of the input image to extract the feature value of the image, and then reduces the dimension to extract the first pixel region RE representing the road end and the second pixel region RA representing the road area from the image.

[0035] The determination unit 110 compares the first pixel region RE and the second pixel region RA output from the machine learning model 140B to determine the degree of fit between these pixel regions. That is, generally, a road region is a region sandwiched by road ends without gaps or overlaps, so the determination unit 110 determines the degree of fit based on whether the second pixel region RA representing the road region is sandwiched by the first pixel region RE representing the road ends without gaps or overlaps.

[0036] Figure 3 1 is a diagram for explaining the determination of the degree of adaptation performed by the determination unit 110. Figure 3 As shown in (a), the determination unit 110 compares the first pixel region RE and the second pixel region RA in the image, and determines that the degree of fit between these pixel regions is low when there is a gap region between the first pixel region RE and the second pixel region RA. Figure 3 As shown in (b), when there is an overlapping area between the first pixel area RE and the second pixel area RA, the determination unit 110 determines that the degree of fit between these pixel areas is low.

[0037] The setting unit 120 sets a gain for the error between the first pixel region RE and the second pixel region RA and the learning data 140A (correct answer data) based on the degree of fit determined by the determination unit 110. Figure 2 As shown in FIG. 1 , a pixel area representing a road end and a pixel area representing a road area in the image are specified in advance. The learning data 140A is generated, for example, by the administrator or operator of the learning device 100 specifying the road end and the road area for the pixels of the image in advance on their own terminal, and is stored in the storage unit 140. Alternatively, the learning device 100 may download the learning data 140A stored in the external server to the storage unit 140 via the network at the execution timing of the learning performed by the setting unit 120.

[0038] The setting unit 120 sets (e.g., multiplies) a gain corresponding to the degree of adaptation for the first error between the first pixel region RE and the correct data of the road end, and the second error between the second pixel region RA and the correct data of the road region. More specifically, the setting unit 120 sets a larger gain for the first error and the second error as the determined degree of adaptation is lower.

[0039] After the setting unit 120 sets the gain for the first error and the second error, the learning unit 130 causes the machine learning model 140B to learn in such a way that the value of the loss function (e.g., the absolute sum of the first error and the second error) obtained based on the first error and the second error for which the gain is set is reduced. The learning in this case can use any learning method such as the error back propagation method. The learning unit 130 obtains the machine learning model 140B that has been learned as a learned model.

[0040] The acquired learned model can be used by a driving support device or a vehicle control device mounted on the vehicle. For example, the driving support device may input the image of the front of the vehicle captured by the camera into the learned model, and display the road end or road area output by the learned model on the HMI. In addition, for example, the vehicle control device may input the image of the front of the vehicle captured by the camera into the learned model, and control the vehicle to travel within the road end or road area output by the learned model.

[0041] Thus, in the prior art, generally, the machine learning model is learned in such a way that the value of the loss function obtained based on the error between the output of the machine learning model and the correct answer data representing the road end or road area is reduced. On the other hand, in the present embodiment, the machine learning model is learned in such a way that the value of the loss function obtained based on the error taking into account the gain is reduced on the basis of considering the adaptability between the multiple outputs of the machine learning model as the gain for the error between the output of the machine learning model and the correct answer data. That is, according to the present embodiment, the accuracy of the road area detected from the image can be ensured by verifying it from multiple viewpoints.

[0042] [Processing flow]

[0043] Next, refer to Figure 4 The flow of processing executed by the learning device 100 according to this embodiment will be described. Figure 4 This is a flowchart showing an example of the flow of processing executed by the learning device 100 according to the present embodiment.

[0044] First, the learning unit 130 obtains an image from the learning data 140A stored in the storage unit 140 and inputs it to the machine learning model 140B (step S100). Next, the determination unit 110 determines the degree of fit between the first pixel region RE and the second pixel region RA output from the machine learning model 140B (step S102).

[0045] Next, the setting unit 120 sets a gain for the first error between the first pixel region RE and the correct data, and the second error between the second pixel region RA and the correct data based on the degree of adaptation (step S104). Next, the learning unit 130 causes the machine learning model 140B to learn in such a way that the value of the loss function calculated based on the first error and the second error for which the gain is set is reduced (step S106). Thus, the processing involved in this flowchart ends.

[0046] As described above, in this embodiment, the first pixel region representing the road end in the image output by the machine learning model using the image as input is compared with the second pixel region representing the road area to determine the degree of fit between the first pixel region and the second pixel region, and a gain is set based on the degree of fit for the first error between the output first pixel region and the correct data representing the road end, and the second error between the output second pixel region and the correct data representing the road area, so that the machine learning model is learned in such a way that the value of the loss function calculated based on the first error and the second error with the set gain is reduced. That is, the accuracy of the road area detected from the image can be ensured by verifying it from multiple viewpoints.

[0047] The above-described embodiment can be expressed as follows.

[0048] A learning device comprising:

[0049] a storage device storing a program; and

[0050] Hardware processor,

[0051] The hardware processor executes the program to perform the following processing:

[0052] Comparing a first pixel region representing a road end in the image output by the machine learning model using the image as input with a second pixel region representing a road region to determine a degree of fit between the first pixel region and the second pixel region;

[0053] Based on the adaptability, a gain is set for a first error between the output first pixel region and the correct solution data representing the road end, and a second error between the output second pixel region and the correct solution data representing the road region; and

[0054] The machine learning model is caused to learn in such a manner that a value of a loss function calculated based on the first error and the second error with the gain set is reduced.

[0055] Although specific embodiments of the present invention have been described above using the embodiments, the present invention is not limited to such embodiments at all, and various modifications and substitutions can be made without departing from the gist of the present invention.

Claims

1. A learning device that enables a machine learning model to learn, the machine learning model taking an image as input and outputting a first pixel region representing a road end and a second pixel region representing a road region in the image, wherein: The learning device comprises: a determination unit, which compares the first pixel region with the second pixel region to determine a degree of fit between the first pixel region and the second pixel region; a setting unit that sets a gain for a first error between the output first pixel region and the correct answer data representing the road end, and a second error between the output second pixel region and the correct answer data representing the road region, based on the degree of adaptation; as well as A learning unit that causes the machine learning model to learn in such a manner that a value of a loss function calculated based on the first error and the second error with the gain set is reduced.

2. The learning device according to claim 1, wherein: The determination unit determines that the degree of adaptation is higher when the second pixel region is sandwiched by the first pixel region without any gap or overlap.

3. The learning device according to claim 1 or 2, wherein: The setting unit increases the set value of the gain as the determined degree of adaptation is lower.

4. A learning method, which enables a machine learning model to learn, the machine learning model taking an image as input and outputting a first pixel region representing a road end and a second pixel region representing a road region in the image, wherein: The learning method enables the computer to perform the following processing: Comparing the first pixel region with the second pixel region to determine the degree of fit between the first pixel region and the second pixel region; Based on the adaptability, setting a gain for a first error between the output first pixel region and the correct solution data representing the road end, and a second error between the output second pixel region and the correct solution data representing the road region; as well as The machine learning model is caused to learn in such a manner that a value of a loss function calculated based on the first error and the second error with the gain set is reduced.

5. A storage medium storing a program that causes a machine learning model to learn, the machine learning model taking an image as input and outputting a first pixel region representing a road end and a second pixel region representing a road region in the image, wherein: The program causes the computer to perform the following processing: Comparing the first pixel region with the second pixel region to determine the degree of fit between the first pixel region and the second pixel region; Based on the adaptability, setting a gain for a first error between the output first pixel region and the correct solution data representing the road end, and a second error between the output second pixel region and the correct solution data representing the road region; as well as The machine learning model is caused to learn in such a manner that a value of a loss function calculated based on the first error and the second error with the gain set is reduced.

Images