Automatic labeling method and automatic labeling device

Abstract

To provide an automatic labeling method capable of accurately labeling a light of an object.SOLUTION: An automatic labeling method includes: acquiring an image from a camera 5; determining a state of a light 51 in the image; inputting a determination result to a buffer area; and labeling the light of the image with the latest determination result as a correct answer value when the latest determination result is lighting and also the frequency of the determination result determined as lighting is equal to or more than a threshold. A process for inputting the determination result to the buffer area is executed at a predetermined sampling cycle. The sampling cycle is determined based on at least one of a light type and a position of a vehicle with respect to an object.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an automatic labeling method and an automatic labeling device.

Background Art

[0002] Patent Document 1 discloses an automatic labeling method for images for supervised machine learning. The automatic labeling method includes a step of acquiring an image of a roadside object using a camera attached to a vehicle, and a step of recording the position and orientation of the vehicle within a defined coordinate system while acquiring the image. The automatic labeling method further includes a step of recording position information regarding each roadside object using the defined coordinate system, and a step of correlating the position of each image of the roadside object with the position information of each roadside object in consideration of the recorded position and orientation of the vehicle. These images are labeled to identify the roadside objects in consideration of the relative positions of the acquired images of the roadside objects.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the conventional labeling method may not be able to accurately label the lights of objects whose states transition over time.

[0005] The present invention has been made in view of the above problems, and an object thereof is to accurately label the lights of an object.

Means for Solving the Problems

[0006] The automatic labeling method according to one aspect of the present invention includes acquiring an image from an imaging device, determining the state of a light in the image, inputting the determination result into a buffer area, and when the latest determination result is "lit" and the number of determination results determined to be "lit" is equal to or greater than a threshold value, labeling the light in the image from which the latest determination result was obtained as the correct value. The process of inputting the determination result into the buffer area is executed at a predetermined sampling period, and the sampling period is determined based on at least one of the type of the light and the position of the vehicle relative to the object.

Advantages of the Invention

[0007] According to the present invention, accurate labeling can be performed on the lights of an object.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0009] With reference to the drawings, this embodiment will be described. In the description of the drawings, the same parts are denoted by the same reference numerals and the description thereof will be omitted.

[0010] Referring to FIG. 1, the automatic labeling device 2 according to the present embodiment will be described. The automatic labeling device 2 forms a part of a vehicle system 1 mounted on a vehicle. The vehicle is a vehicle having an automatic driving function, but may also be a vehicle without an automatic driving function. Further, the vehicle may be a vehicle capable of switching between an automatic driving function and a manual driving function. The automatic driving function is a function that automatically controls a plurality of vehicle control functions such as a steering control function, a braking force control function, and a driving force control function to perform autonomous driving, but may also be a function that automatically controls only some of the plurality of vehicle control functions to assist the driver's driving.

[0011] The vehicle system 1 includes a camera 5, a map database (map DB) 6, a GPS receiver 7, and a sensor 8.

[0012] The camera 5 images the front of the vehicle and outputs the captured image. The camera 5 includes a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 5 is arranged to face the front of the vehicle. The image captured by the camera 5 is, for example, a color image. The camera 5 images a predetermined range in front of the vehicle by setting parameters such as the focal length, the angle of view of the lens, and the attitude in the vertical and horizontal directions. The camera 5 performs imaging at a predetermined imaging cycle.

[0013] The map DB 6 is a storage device that stores high-precision map data. The high-precision map data is map data that shows information about roads and structures attached to the roads with high accuracy. The information about the road includes, for example, the three-dimensional shape of the road and information such as lanes. Further, the information about the structure includes information such as the position and type of road surface markings such as stop lines at intersections, and information such as the position, shape, and type of three-dimensional structures. The three-dimensional structures include traffic signs and traffic lights.

[0014] Referring to FIG. 2, the information of the traffic signal 50 included in the high-precision map data will be described. The traffic signal 50 shown in FIG. 2 is a three-light type vehicle traffic signal including a red light 51, a yellow light 52, and a blue light 53, and is provided at an intersection. The lights 51 to 53 of each color are arranged horizontally. The red light 51 is located at the right end, the yellow light 52 is located in the center, and the blue light 53 is located at the left end. When the lights 51 to 53 of each color are lit, signals of each color are represented. However, the lights 51 to 53 of each color may be arranged vertically. The shapes of the lights 51 to 53 of each color are circular, but may also be polygonal shapes such as square shapes.

[0015] In addition, the traffic signal 50 further includes a right arrow light 54, an up arrow light 55, and a left arrow light 56. The arrow lights 54 to 56 are arranged horizontally in the same manner as the lights 51 to 53 of each color. The right arrow light 54 is located at the right end, the up arrow light 55 is located in the center, and the left arrow light 56 is located at the left end. When the arrow lights 54 to 56 are lit, signals of each arrow are represented. However, the traffic signal 50 does not necessarily need to include all three arrow lights 54 to 56, and it may be sufficient to include at least one arrow light 54 to 56 according to the right-of-way given by the traffic signal 50 to traffic. Also, the traffic signal 50 may be configured not to include any of the arrow lights 54 to 56.

[0016] The information of the traffic signal 50 included in the high-precision map data includes the information of each of the lights 51 to 56. Specifically, the information of the red light 51 includes an identifier (ID), the type of the light, position information, and the width Wa of the light. The position information is global coordinates defined by latitude, longitude, and altitude, or the X coordinate (Xa), Y coordinate (Ya), and Z coordinate (Za). Such information is also defined for the yellow and blue lights 52 and 53. Similarly, the information of the right-arrow light 54 includes an identifier (ID), the type of the light, position information, and the width Wb of the light. The position information is global coordinates such as latitude, longitude, and altitude, or the X coordinate (Xb), Y coordinate (Yb), and Z coordinate (Zb). Such information is also defined for the up-arrow and left-arrow lights 55 and 56. The automatic labeling device 2 described later can identify each of the lights 51 to 56 based on the ID assigned to each of the lights 51 to 56.

[0017] As shown in FIG. 1, the GPS receiver 7 receives radio waves from a plurality of GPS satellites in order to calculate the position of the vehicle. The sensor 8 detects the state of the vehicle such as the vehicle speed, pitch rate, and yaw rate.

[0018] The automatic labeling device 2 is a general-purpose microcomputer including a CPU (Central Processing Unit), a memory, and an input / output unit. A computer program for causing the microcomputer to function as the automatic labeling device 2 is installed in the microcomputer. By executing the computer program, the microcomputer functions as a plurality of information processing circuits included in the automatic labeling device 2.

[0019] In the present embodiment, an example in which a plurality of information processing circuits included in the automatic labeling device 2 are realized by software is shown. Of course, it is also possible to configure a plurality of information processing circuits for executing each of the information processes shown below by dedicated hardware. Also, the plurality of information processing circuits may be configured by individual hardware.

[0020] The automatic labeling device 2 includes, as a plurality of information processing circuits, an image acquisition unit 10, an object position calculation unit 11, a self-position calculation unit 12, a labeling processing unit 20, and a data registration unit 30.

[0021] The image acquisition unit 10 acquires an image from the camera 5. An image corresponding to one frame acquired by the image acquisition unit 10 is input to the labeling processing unit 20.

[0022] The object position calculation unit 11 acquires the position of the traffic signal 50 existing in front of the vehicle from the map DB6. The object position calculation unit 11 refers to the information of a navigation system (not shown) and acquires the position of the traffic signal 50 existing at the position closest to the vehicle on the future driving route of the vehicle. The position of the traffic signal 50 includes the positions of the respective lights 51 to 56.

[0023] The self-position calculation unit 12 calculates the current position of the vehicle based on the information received by the GPS receiver 7. The position of the vehicle calculated by the self-position calculation unit 12 is expressed in the same global coordinates as the high-precision map data of the map DB6. Note that the calculation of the position of the vehicle may be performed by a method using a distance measurement sensor such as a millimeter-wave radar or a LiDAR [Light Detection And Ranging].

[0024] The labeling processing unit 20 labels the lights 51 to 56 of the traffic signal 50 based on the image captured by the camera 5. The labeling processing unit 20 includes a period setting unit 21, a coordinate conversion unit 22, a light determination unit 23, a tracking unit 24, a determination result storage unit 25, and a correct answer assignment unit 26. The description of each function included in the labeling processing unit 20 will be described later.

[0025] The data registration unit 30 registers the learning data correctly labeled by the labeling processing unit 20 as a data set for machine learning. The registered data set is used as teacher data for machine learning of the learning model.

[0026] The vehicle system 1 further includes an object recognition device 40 and a vehicle control device 45.

[0027] The object recognition device 40 is provided with a learning model that has been pre-trained by machine learning. The object recognition device 40 performs object recognition based on the image captured by the camera 5 by using the learning model. The learning model included in the object recognition device 40 has been machine-learned using a dataset including the learning data collected by the automatic labeling device 2 as teacher data.

[0028] The vehicle control device 45 controls various actuators that control the behavior of the vehicle, such as a steering actuator, a braking force actuator, and a driving force actuator, based on the recognition result of the object recognition device 40.

[0029] Referring to FIG. 3, the automatic labeling method executed by the automatic labeling device 2 will be described. Hereinafter, as shown in FIG. 4, the automatic labeling method will be described by taking as an example the situation where the vehicle V is going straight through the intersection 100 where the traffic signal 50 is installed.

[0030] In step S20 of FIG. 3, the period setting unit 21 sets a sampling period. The sampling period is determined based on any one of the speed of the vehicle V, the types of the lit lights 51 to 56, and the position (distance D) of the vehicle V with respect to the traffic signal 50.

[0031] When the speed of the vehicle V is high, the number of times the traffic signal 50 can be imaged before the vehicle V passes the traffic signal 50 is less than when the speed of the vehicle V is low. Also, in a general traffic signal 50, the lighting frequency of the yellow light 52 is lower than that of the blue light 53. The lighting frequencies of the arrow lights 54 to 56 are lower than that of the blue light 53. In particular, in the case of driving on the right side, the lighting frequency of the right arrow light 54 is lower than that of the blue light 53. In addition, among the driving situations of the vehicle V, the frequency of driving inside the intersection 100 is lower than the frequency of driving in the environment other than the intersection 100. For this reason, the frequency at which the traffic signal 50 can be imaged from the vicinity is low. Therefore, if the processing is performed in the same sampling period, in the above situation, the frequency at which the determination result is input to the determination result storage unit 25, which is the buffer area, will be low. In this case, the collection efficiency of the learning data will decrease.

[0032] Therefore, the period setting unit 21 determines the sampling period based on at least one of the speed of the vehicle V, the types of the lit lights 51 to 56, and the position of the vehicle V relative to the traffic signal 50.

[0033] Focusing on the speed of the vehicle V, the period setting unit 21 sets the sampling period shorter as the speed is higher. Thereby, the number of processing times per unit time can be increased before the vehicle V passes the traffic signal 50. At this time, the period setting unit 21 may divide the speed into a plurality of sections and determine the sampling period for each section so that the sampling period changes step by step. The period setting unit 21 can refer to the vehicle speed detected by the sensor 8.

[0034] When paying attention to the types of lights 51 to 56, the cycle setting unit 21 sets a shorter sampling cycle when the yellow lights 52 and the arrow lights 54 to 56 are lit than when the blue light 53 is lit. Thereby, the number of processing times per unit time when the yellow lights 52 and the arrow lights 54 to 56 are lit can be increased. The cycle setting unit 21 may recognize the types of the lit lights 51 to 56 based on, for example, the states of the lights 51 to 56 determined in the previous processing cycle. Also, in the first processing cycle, for convenience, a preset design value may be determined as the sampling cycle.

[0035] When paying attention to the position of the vehicle V with respect to the traffic signal 50, the cycle setting unit 21 sets a shorter sampling cycle when the vehicle V is within the intersection 100 or near the stop line of the intersection 100 than in other cases. Thereby, the number of processing times per unit time in a situation where the traffic signal 50 can be imaged from the vicinity can be increased. The cycle setting unit 21 may recognize the positional relationship between the two based on, for example, the position of the vehicle V calculated in the previous processing cycle and the position of the traffic signal 50. Also, in the first processing cycle, for convenience, a preset design value may be determined as the sampling cycle.

[0036] The cycle setting unit 21 may determine the sampling cycle by comprehensively considering each element of the speed of the vehicle V, the types of the lit lights 51 to 56, and the distance D to the traffic signal 50. Note that the sampling cycle is determined to be at least equal to or longer than the imaging cycle of the camera 5.

[0037] When the sampling timing determined by the sampling cycle arrives (step S21: Yes), the process proceeds to the processes after step S22. In step S22, the image acquisition unit 10 acquires an image captured by the camera 5. In step S23, the self-position calculation unit 12 calculates the current position of the vehicle V based on the information received by the GPS receiver 7.

[0038] In step S24, the coordinate conversion unit 22 performs coordinate conversion based on the positions of the lights 51 to 56 of the traffic signal 50 acquired by the object position calculation unit 11 and the position of the vehicle V calculated by the self-position calculation unit 12. Thereby, the coordinate conversion unit 22 specifies the positions of the lights 51 to 56 in the image.

[0039] When the traffic signal 50 includes a plurality of lights 51 to 56, the processes after step S24 are executed for each of the lights 51 to 56. Hereinafter, the red light 51 will be described as an example.

[0040] In step S25, the light determination unit 23 determines the state of the red light 51 based on the position of the identified red light 51. The state of the red light 51 is determined as being on or off.

[0041] In step S26, the tracking unit 24 inputs the determination result of the light determination unit 23 to the determination result storage unit 25.

[0042] As shown in FIG. 5, the determination result storage unit 25 includes a buffer area having a capacity capable of storing not only the latest determination result but also the determination results for the past n times. When the latest determination result is input, the oldest determination result is deleted from the determination result storage unit 25, and it is possible to store the determination results for a total of n + 1 times. In FIG. 5, the area indicated by "t" is the area where the latest determination result is stored, and the area indicated by "t - n" is the area where the determination result n times before the latest determination result is stored. The determination result storage unit 25 can store the determination results for n + 1 times (a predetermined number of times) in chronological order. In the figure, the white area indicates that the determination result is "off", and the hatched area indicates that the determination result is "on".

[0043] The buffer area of the determination result storage unit 25 is prepared for each of the lights 51 to 56 of the traffic signal 50. The tracking unit 24 inputs the determination result to the buffer area corresponding to each of the lights 51 to 56, thereby tracking the determination results for each of the lights 51 to 56 in chronological order.

[0044] As shown in FIG. 3, in step S27, the correct assignment unit 26 refers to the latest determination result in the determination result storage unit 25 and determines whether the red light 51 is lit. If the red light 51 is lit (step S27: Yes), the correct assignment unit 26 refers to all the determination results in the determination result storage unit 25 and counts the number of determination results determined to be lit. In step S28, the correct assignment unit 26 substitutes the counted count value into the control variable N.

[0045] In step S29, the correct assignment unit 26 determines whether the control variable N is greater than or equal to the threshold value Th. The threshold value Th is a value for determining that the red light 51 has been continuously lit for a certain period in the past, and is set to a value of 2 or more and less than or equal to the upper limit number of determination results that can be stored in the determination result storage unit 25. For example, the threshold value Th is an integer value that is 60% or more of the upper limit number (predetermined number) of determination results that can be stored in the determination result storage unit 25.

[0046] As shown in FIG. 6(c), when the state of the latest red light 51 is lit and the control variable N is greater than or equal to the threshold value Th (step S29: Yes), the process proceeds to step S30. The correct assignment unit 26 labels the red light 51 in the image where the latest determination result is obtained as the correct value.

[0047] On the other hand, as shown in FIG. 6(a), even if the state of the latest red light 51 is lit, if the control variable N is smaller than the threshold value Th (step S29: No), the process proceeds to step S31. Or, as shown in FIG. 6(b), even when the state of the latest red light 51 is off (step S27: No), the process proceeds to step S31. The correct assignment unit 26 does not label the red light 51 in the image where the latest determination result is obtained as the correct value.

[0048] Note that even when the control variable N is greater than or equal to the threshold value Th, it is preferable that the correct assignment unit 26 does not label as the correct value when the following first condition or second condition is satisfied.

[0049] The first condition is when a state quantity indicating the behavior of the vehicle V, such as the pitch rate and yaw rate, is equal to or greater than a determination value. When the behavior of the vehicle V, such as the pitch rate and yaw rate, is large, it may lead to camera 5 shaking during imaging, causing the red traffic light 51 in the image to shake or appear double. Therefore, when the first condition is met, even if the control variable N is equal to or greater than the threshold Th, it is preferably not labeled as the correct value. The state quantity of the behavior of the vehicle V, such as the pitch rate and yaw rate, can be detected by the sensor 8, or the state quantity of the behavior of the vehicle V may be specified using well-known image processing techniques.

[0050] The second condition is when the distance D from the vehicle V to the traffic signal 50 is equal to or greater than a determination value. When the vehicle V is far from the traffic signal 50, the number of pixels corresponding to the red traffic light 51 decreases, and it may not be possible to accurately determine the state of the red traffic light 51. Therefore, when the second condition is met, even if the control variable N is equal to or greater than the threshold Th, it is preferably not labeled as the correct value. The determination value is, for example, 200 m, and is preset as the distance from the vehicle V to the traffic signal 50 such that each traffic light 51 - 56 of the traffic signal 50 can be recognized as a region with a predetermined number of pixels or more in the image.

[0051] In step S32, the data registration unit 30 registers the learning data (image data) labeled with the correct value, which is collected by the labeling processing unit 20, as a data set for machine learning. The data registration unit 30 may register the learning data in a storage device (not shown) mounted on the vehicle system 1, or may register it in a storage device on the cloud via a network.

[0052] The dataset containing the training data registered by the data registration unit 30 is used as teacher data for machine learning of the learning model provided in the object recognition device 40. The machine learning of the learning model may be performed offline using an external system, or may be performed within the vehicle system 1. Further, the dataset for machine learning may include training data collected by other vehicles other than the host vehicle V.

[0053] The object recognition device 40 performs object recognition including the traffic signal 50 based on the image captured by the camera 5 by using the learning model. The object recognition result by the object recognition device 40 is output to the vehicle control device 45. The vehicle control device 45 controls an actuator that controls the behavior of the vehicle V based on the object recognition result.

[0054] As described above, according to the automatic labeling device 2 and the automatic labeling method according to the present embodiment, the following operations and effects are achieved. The lights of the traffic signal 50 repeatedly turn on and off periodically. Further, in the case of a light using an LED, even when it is in the on state, it blinks repeatedly. Therefore, depending on the image capture timing, the image may be captured in a state where the light is off. Further, when switching from the red light on to the blue light on, the red and blue lights may be on at the same time. For this reason, there may be a case where a light that is not on and a light that is on at a timing when labeling should not be performed are labeled as correct values.

[0055] In this regard, according to the automatic labeling device 2 and the automatic labeling method of the present embodiment, in addition to the fact that the most recent determination result is lighting, considering the number of determination results determined to be lighting in the past, it is determined whether to label with the correct value. Therefore, it is possible to suppress labeling a non-lighting lamp and a lamp that lights at a timing when it should not be labeled as the correct value. For this reason, it is possible to accurately label the lamps of an object. In addition, it is possible to suppress the inclusion of mislabeled learning data in the data set for machine learning, so that it is possible to prevent a decrease in the detection performance by the learning model.

[0056] Also, since the sampling period is determined in consideration of the type of lamp or the position of the vehicle V with respect to the traffic signal 50, the number of determination results input to the buffer area can be controlled. Thereby, even in a situation where the occurrence frequency is low, the number of determination results input to the buffer area can be increased. As a result, the number of learning data labeled as the correct value can be relatively increased, so that the data set for machine learning can be efficiently collected.

[0057] According to the present embodiment, since the sampling period can be determined in consideration of the speed of the vehicle V, when the speed of the vehicle V is high, the number of determination results input to the buffer area can be increased. Thereby, the number of labeled learning data can be relatively increased, so that the data set for machine learning can be efficiently collected.

[0058] According to the present embodiment, even when the traffic signal 50 includes a plurality of lamps, each lamp can be labeled.

[0059] According to the method of the present embodiment, the position of the lamp in the image can be specified by coordinate conversion. Thereby, the state of the lamp can be appropriately determined.

[0060] According to the method of the present embodiment, even the lights of the traffic signal 50 whose state changes over time can be accurately labeled.

[0061] According to the method of the present embodiment, since the sampling period is equal to or longer than the imaging period of the image, it is possible to suppress the creation of a plurality of learning data from the same image. Thereby, it is possible to suppress the inclusion of duplicate learning data in the dataset for machine learning.

[0062] When the behavior of the vehicle is large, the image will be such that the lights flicker or appear double. According to the present embodiment, it is possible to suppress labeling an image in which the lights flicker or appear double as the correct value. Thereby, it is possible to suppress the inclusion of incorrect learning data in the dataset for machine learning.

[0063] At a position far from the traffic signal 50, the number of pixels in the region corresponding to the lights in the image is also small, making it difficult to determine the light state. According to the present embodiment, it is possible to suppress labeling an inappropriate image of the lights as the correct value. Thereby, it is possible to suppress the inclusion of incorrect learning data in the dataset for machine learning.

[0064] According to the present embodiment, since the object recognition device 40 can use a learning model machine-learned with an appropriate dataset, the recognition accuracy of the object can be improved. Further, by using the recognition result of the object recognized in this way, the driving control of the vehicle V can be appropriately performed.

[0065] According to the present embodiment, since learning data collected by other vehicles with different driving routes can be used, the dataset can include learning data obtained in various driving environments. Thereby, it is possible to improve the recognition accuracy of the object.

[0066] In addition, in the present embodiment, a traffic signal 50 installed at an intersection is exemplified as the traffic signal. However, the traffic signal 50 to which the automatic labeling device 2 of the present embodiment is applicable may be any traffic signal whose light state changes at a predetermined cycle. For example, the traffic signal 50 may be a pedestrian traffic signal installed at an intersection, a traffic signal for a red or yellow flashing signal, a traffic signal installed on a lane at a road tollgate, a traffic signal installed at a railway level crossing, or the like.

[0067] As described above, the embodiments of the present invention have been described. However, it should not be understood that the discussions and drawings forming a part of this disclosure limit the present invention. Various alternative embodiments, examples, and operation techniques will be apparent to those skilled in the art from this disclosure.

Explanation of Reference Numerals

[0068] 1 Vehicle system 2 Automatic labeling device 5 Camera (imaging device) 6 Map database (map DB) 7 GPS receiver 8 Sensor 10 Image acquisition unit 11 Object position calculation unit 12 Self-position calculation unit 20 Labeling processing unit 21 Period setting unit 22 Coordinate conversion unit 23 Light determination unit 24 Tracking unit 25 Determination result storage unit 26 Correct answer assignment unit 30 Data registration unit 40 Object recognition device 45 Vehicle control device

Claims

1. An automatic labeling method executed by an automatic labeling device that labels the lights of an object based on an image captured by an imaging device mounted on a vehicle, comprising: acquiring the image from the imaging device; determining the state of the lights in the image; inputting the determination result into a buffer area having a capacity capable of storing the determination results for a predetermined number of times; when the latest determination result among the determination results for the predetermined number of times stored in the buffer area is on, and the number of times of the determination results determined to be on is equal to or greater than a threshold value, labeling the lights in the image from which the latest determination result is obtained as a correct value; the process of inputting the determination result into the buffer area is executed at a predetermined sampling period; the sampling period is determined based on at least one of the type of the lights and the position of the vehicle with respect to the object Automatic labeling method.

2. The sampling period is determined based on at least one of the type of the lights, the position of the vehicle with respect to the object, and the speed of the vehicle The automatic labeling method according to claim 1.

3. The object includes a plurality of lights; the process of inputting the determination result into the buffer area is executed for each light The automatic labeling method according to claim 1.

4. acquiring the position of the lights; calculating the position of the vehicle; further comprising specifying the position of the lights in the image by performing coordinate transformation based on the position of the object and the position of the vehicle The automatic labeling method according to claim 1.

5. The object is a traffic signal whose light state switches at a predetermined period The automatic labeling method according to claim 1.

6. The sampling period is equal to or longer than the imaging period of the image by the imaging device The automatic labeling method according to claim 1.

7. When a state quantity indicating the behavior of the vehicle is equal to or greater than a determination value, the lights in the image from which the latest determination result is obtained are not labeled as a correct value The automatic labeling method according to claim 1.

8. When the distance from the vehicle to the object is equal to or greater than a determination value, the lights in the image from which the latest determination result is obtained are not labeled as a correct value The automatic labeling method according to claim 1.

9. Recognize the object using a learning model machine-learned by a dataset for machine learning that includes the learning data with the correct value labeled, Output the recognition result of the object to a device that controls the running of the vehicle The automatic labeling method according to claim 1.

10. The dataset for machine learning includes learning data collected by other vehicles other than the vehicle. The automatic labeling method according to claim 9.

11. When determining the sampling period based on the speed of the vehicle, The higher the speed of the vehicle, the shorter the sampling period. The automatic labeling method according to claim 3.

12. The object is a traffic signal whose light state switches in a predetermined cycle, When determining the sampling period based on the type of the light, When an arrow light or a yellow light is on, the sampling period is shorter than when a blue light is on. The automatic labeling method according to claim 2.

13. The object is a traffic signal whose light state switches in a predetermined cycle, When determining the sampling period based on the position of the vehicle with respect to the object, When the vehicle is within the intersection where the object exists, or when the vehicle is near the stop line of the intersection, the sampling period is shorter than when the vehicle is away from the intersection. The automatic labeling method according to claim 2.

14. An imaging device mounted on a vehicle, A computer that performs labeling on the lights of an object based on an image captured by the imaging device, and has, The computer, Obtains the image from the imaging device, Determines the state of the light in the image, Inputs the determination result into a buffer area having a capacity capable of storing the determination results for a predetermined number of times, Among the determination results for the predetermined number of times stored in the buffer area, when the latest determination result is on and the number of determination results determined to be on is equal to or greater than a threshold value, label the light of the image where the latest determination result was obtained as the correct value. The computer, Determines a sampling period based on at least one of the type of the light and the position of the vehicle with respect to the object, Inputs the determination result into the buffer area at the sampling period. Automatic labeling device.

Images