Learning apparatus, learning method, and storage medium

Abstract

The invention provides a learning device, a learning method, and a storage medium capable of generating a machine learning model for accurately detecting a road region from a remote region in a region included in an image. A learning device that learns a machine learning model that outputs a discrimination value indicating whether or not each of a plurality of pixels represents a road using an image including the plurality of pixels as an input, the learning device being provided with: a setting unit that sets an image including the plurality of pixels; a weight setting unit that sets a weight for an error between a discrimination value output by the machine learning model and learning data indicating whether each of the pixels represents a road; and a learning unit that learns the machine learning model so as to reduce a value of a loss function calculated on the basis of the error for which the weight is set, and the setting unit increases the weight from the center of the lower end of the image toward one or more predetermined directions.

Description

Technical Field

[0001] The invention relates to a learning device, a learning method and a storage medium. Background Art

[0002] In the past, there is a known technology for detecting a road area included in an image for use in driving support or autonomous driving of a vehicle. For example, Patent Document 1 discloses a technology for detecting a road area by scanning an input image along the x direction to extract an edge and detecting a white line based on the difference in color near the edge.

[0003] Prior Art Literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-038101 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, in the technology described in Patent Document 1, the accuracy of scanning may be reduced because the number of pixels decreases as the area included in the image becomes farther away. Thus, in the prior art, it is sometimes impossible to accurately detect the road area in the distant area included in the image.

[0008] The present invention was completed in consideration of such a situation, and one of its purposes is to provide a learning device, a learning method, and a storage medium that can generate a machine learning model that can accurately detect a road area from a distant area in an area included in an image.

[0009] Solutions to Solve Problems

[0010] The learning device, learning method, and storage medium according to the present invention employ the following structures.

[0011] (1): One embodiment of the present invention relates to a learning device that enables a machine learning model to learn, wherein the machine learning model takes an image including a plurality of pixels as input and outputs a discrimination value indicating whether each of the pixels represents a road, wherein the learning device comprises: a setting unit that sets a weight for an error between the discrimination value output by the machine learning model and learning data indicating whether each of the pixels represents a road; and a learning unit that enables the machine learning model to learn in such a manner that a value of a loss function calculated based on the error with the weight set is reduced, wherein the setting unit increases the weight from the center of the lower end of the image toward one or more specified directions.

[0012] (2) In the above-mentioned aspect (1), the setting unit increases the weight as the one or more predetermined directions go from the lower center of the image toward the upper end and the left and right ends.

[0013] (3): Another embodiment of the present invention relates to a learning method, which enables a machine learning model to learn, wherein the machine learning model takes an image including a plurality of pixels as input and outputs a discrimination value indicating whether each of the pixels represents a road, wherein the learning method enables a computer to perform the following processing: setting a weight for an error between the discrimination value output by the machine learning model and learning data indicating whether each of the pixels represents a road; and enabling the machine learning model to learn in such a manner that a value of a loss function calculated based on the error with the weight set is reduced, wherein the weight is increased from the center of the lower end of the image toward one or more specified directions.

[0014] (4): Another embodiment of the present invention relates to a storage medium storing a program that enables a machine learning model to learn, wherein the machine learning model takes an image containing multiple pixels as input and outputs a discrimination value indicating whether each of the pixels represents a road, wherein the program enables a computer to perform the following processing: setting a weight for an error between the discrimination value output by the machine learning model and learning data indicating whether each of the pixels represents a road; and enabling the machine learning model to learn in a manner that reduces a value of a loss function calculated based on the error with the weight set, wherein the weight is increased from the center of the lower end of the image toward one or more specified directions.

[0015] Effects of the Invention

[0016] According to (1) to (4), it is possible to generate a machine learning model that accurately detects a road area from a distant area in an area included in an image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a diagram showing the configuration of the learning device 100 according to the present embodiment.

[0018] Figure 2 130A is a diagram showing an example of the structure of the learning data 130A.

[0019] Figure 3 This is a diagram for explaining a method of setting weights by the setting unit 110 .

[0020] Figure 4 This is a flowchart showing an example of the flow of processing executed by the learning device 100 according to the present embodiment.

[0021] Description of reference numerals:

[0022] 100 learning devices

[0023] 110 Setting Department

[0024] 120 Learning Department

[0025] 130 Storage

[0026] 130A Learning Data

[0027] 130B machine learning model. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of a learning device, a learning method, and a storage medium according to the present invention will be described with reference to the accompanying drawings.

[0029] [structure]

[0030] Figure 1 : is a diagram showing the structure of the learning device 100 involved in the present embodiment. The learning device 100 is an information processing device that enables a machine learning model to learn, and the machine learning model takes an image containing multiple pixels as input and outputs a judgment value indicating whether each pixel represents a road. The learning device 100 includes, for example, a setting unit 110, a learning unit 120, and a storage unit 130. The setting unit 110 and the learning unit 120 are respectively implemented by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). In addition, some or all of these components can be implemented by hardware (including circuitry) such as LSI (LargeScale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc., or can be implemented by the coordinated cooperation of software and hardware. The program may be stored in advance in a storage device such as a HDD or flash memory of the automatic driving control device 100 (a storage device having a non-transitory storage medium), or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the automatic driving control device 100 by assembling the storage medium (non-transitory storage medium) in a drive device. The storage unit 130 stores, for example, learning data 130A and a machine learning model 130B. The storage unit 150 is implemented, for example, by a RAM, a flash memory, an SD card, or the like.

[0031] Figure 21 is a diagram showing an example of the structure of the learning data 130A. The learning data 130A is data in which a correspondence is established between pixels PX (pixel groups) of one or more images IM and label values ​​indicating whether the pixels represent a road. Figure 2 In the case of , the area sandwiched by the road dividing lines LRL and RRL represents a road area, so the pixel PX1 is associated with a label value 0 indicating that the pixel does not represent a road, and on the other hand, the pixels PX2 to 11 are associated with a label value 1 indicating that the pixel represents a road. The learning data 130A is generated, for example, by the administrator or operator of the learning device 100 specifying each pixel and label value of the image IM in advance on their own terminal, and is stored in the storage unit 150. Alternatively, the learning device 100 may download the learning data 130A stored in an external server to the storage unit 130 via a network at the execution time of the learning performed by the learning unit 120. The label value is an example of a "discrimination value" in the technical solution.

[0032] The machine learning model 130B is a machine learning model that takes an image IM as input and outputs a label value indicating whether a pixel of the image IM represents a road. The machine learning model 130B is, for example, a convolutional neural network (CNN), which extracts feature quantities of a pixel (pixel group) of the input image IM to output a label value indicating whether the pixel represents a road. It should be noted that the image IM input to the machine learning model 130B may be an original image captured by a camera mounted on a vehicle, or may be a planar projection image or a cylindrical projection image of a road obtained by transforming the original image.

[0033] Here, in the prior art, generally, the learning of the machine learning model is performed in such a way that the value of the loss function calculated based on the error between the label value output by the machine learning model and the learning data 130A indicating whether each pixel represents a road is reduced, and the final learning model is generated. Here, the loss function is defined as, for example, the absolute sum of the errors about each pixel. On the other hand, the farther away from the location where the image IM is taken, the smaller the number of pixels represents the area of ​​the same size. For example, Figure 2 As shown in the right part of , although the areas A1 and A2 have the same area in the actual road, the area A1 far from the shooting location is represented by a single pixel PX2, while the area A2 near the shooting location is represented by a plurality of pixels PX3 to PX11. As a result, the learned model generated by the conventional technology has a tendency to have a high output accuracy for the area near the shooting location in the image IM, while having a low output accuracy for the area far from the shooting location in the image IM.

[0034] With this as the background, the setting unit 110 sets (e.g., multiplies) a weight to the error between the label value output by the machine learning model 130B and the learning data 130A indicating whether each pixel represents a road. More specifically, the setting unit 110 increases the value of the weight set for the error from the lower center of the image IM toward one or more predetermined directions. That is, the setting unit 110 uses the weight to increase the value of the error related to the area farther from the shooting location compared to the error related to the area closer to the shooting location, thereby preventing the accuracy of machine learning for the distant area from decreasing.

[0035] Figure 3 1 is a diagram for explaining a method of setting weights by the setting unit 110. The setting unit 110 increases the weights as one or more predetermined directions go from the lower center of the image IM toward the upper end and the left and right ends. Figure 3 As shown in the pattern (a) of FIG. 1 , the setting unit 110 may increase the value of the weight set for the error from the lower center of the image IM toward the upper end, or may increase the value of the weight set for the error as shown in FIG. Figure 3 As shown in the pattern (b) of FIG. 1 , the setting unit 110 increases the value of the weight set for the error from the lower center of the image IM toward the right end. Figure 3 As shown in the pattern (c) of FIG. 1 , the setting unit 110 increases the value of the weight set for the error from the lower center of the image IM toward the left end. These patterns may also be applied simultaneously (for example, when the pixel is located in the upper left direction of the image IM, the setting Figure 3 (a) The weight and Figure 3 In the present embodiment, the setting unit 110 increases the value of the weight set for the error at least from the lower center of the image IM toward the upper end.

[0036] After the setting unit 110 sets a weight to the error between the label value output by the machine learning model 130B and the learning data 130A indicating whether each pixel of the image IM represents a road, the learning unit 120 causes the machine learning model 130B to learn in such a way that the value of the loss function (e.g., the absolute sum of the errors) obtained based on the error set with the weight is reduced. The learning in this case can use any learning method such as the error back propagation method. The learning unit 120 obtains the machine learning model 130B that has completed learning as a learning completed model.

[0037] The acquired learned model can be used by a driving support device or a vehicle control device mounted on the vehicle. For example, the driving support device may input the image of the front of the vehicle captured by the camera into the learned model, and display the road area output by the learned model on the HMI. In addition, for example, the vehicle control device may input the image of the front of the vehicle captured by the camera into the learned model, and control the vehicle to travel within the road area output by the learned model.

[0038] Thus, in the present embodiment, the setting unit 110 increases the value of the weight set for the error from the lower center of the image IM toward one or more predetermined directions, and the learning unit 120 causes the machine learning model 130B to learn so as to reduce the value of the error function obtained based on the error to which the weight is set. In this way, a machine learning model that accurately detects a road area from a distant area in an area included in an image can be generated.

[0039] [Processing flow]

[0040] Next, refer to Figure 4 The flow of processing executed by the learning device 100 according to this embodiment will be described. Figure 4 This is a flowchart showing an example of the flow of processing executed by the learning device 100 according to the present embodiment.

[0041] First, the learning unit 120 obtains an image from the learning data 130A stored in the storage unit 130 (step S100 ). Next, the learning unit 120 calculates the error between the correct label of the learning data 130A and the output of the machine learning model 130B based on each pixel of the obtained image (step S102 ).

[0042] Next, the setting unit 110 sets a weight for the error calculated by the learning unit 120 according to the distance from the lower end center of the image (step S104). Next, the learning unit 120 calculates a loss function based on the error with the set weight (step S106). Next, the learning unit 120 causes the machine learning model 130B to learn in such a way that the value of the calculated loss function decreases (step S108). Thus, the processing involved in this flowchart ends.

[0043] As described above, in this embodiment, a weight is set for the error between the discrimination value output by the machine learning model that takes an image including a plurality of pixels as an input and outputs a discrimination value indicating whether each pixel represents a road and the learning data indicating whether each pixel represents a road, so that the machine learning model is trained in such a manner that the value of the loss function calculated based on the error with the set weight decreases, and in this setting, the weight increases from the lower end center of the image toward one or more predetermined directions. Thus, a machine learning model that accurately detects a road area from a distant area in an area included in the image can be generated.

[0044] The above-described embodiment can be expressed as follows.

[0045] A learning device, wherein

[0046] The learning device is configured to include:

[0047] a storage device storing a program; and

[0048] Hardware processor,

[0049] The hardware processor executes the program to perform the following processing:

[0050] setting a weight for an error between a discrimination value output by a machine learning model that takes an image including a plurality of pixels as input and outputs a discrimination value indicating whether each of the pixels indicates a road and learning data indicating whether each of the pixels indicates a road;

[0051] causing the machine learning model to learn so that a value of a loss function calculated based on the error with the weight set is reduced; and

[0052] In the above setting, the weight is increased from the lower center of the image toward one or more predetermined directions.

[0053] Although specific embodiments of the present invention have been described above using the embodiments, the present invention is not limited to such embodiments at all, and various modifications and substitutions can be made without departing from the gist of the present invention.

Claims

1. A learning device that causes a machine learning model to learn, the machine learning model taking an image containing a plurality of pixels as input and outputting a discrimination value indicating whether each of the pixels represents a road, wherein: The learning device comprises: a setting unit that sets a weight for an error between a discrimination value output by the machine learning model and learning data indicating whether each of the pixels represents a road; as well as a learning unit that causes the machine learning model to learn in such a manner that a value of a loss function calculated based on the error with the weight set is reduced, The setting unit increases the weight from the lower end center of the image toward one or more predetermined directions.

2. The learning device according to claim 1, wherein: The setting unit increases the weight from the lower center of the image toward the upper end and the left and right ends as the one or more predetermined directions.

3. The learning device according to claim 1, wherein: The setting unit increases the weight from the lower center toward the upper end of the image as the one or more predetermined directions.

4. A learning method for learning a machine learning model that takes an image containing a plurality of pixels as input and outputs a discrimination value indicating whether each of the pixels represents a road, wherein: The learning method enables the computer to perform the following processing: setting a weight for an error between a discrimination value output by the machine learning model and learning data indicating whether each of the pixels represents a road; as well as The machine learning model is trained in such a manner that a value of a loss function calculated based on the error with the weight set is reduced, In the above setting, the weight is increased from the lower center of the image toward one or more predetermined directions.

5. A storage medium storing a program that causes a machine learning model to learn, the machine learning model taking an image including a plurality of pixels as input and outputting a determination value indicating whether each of the pixels represents a road, wherein: The program causes the computer to perform the following processing: setting a weight for an error between a discrimination value output by the machine learning model and learning data indicating whether each of the pixels represents a road; as well as The machine learning model is trained in such a manner that a value of a loss function calculated based on the error with the weight set is reduced, In the above setting, the weight is increased from the lower center of the image toward one or more predetermined directions.

Images