Autonomous vehicle

Abstract

To provide an autonomous vehicle which suppresses a deterioration in estimation accuracy of a self position.SOLUTION: An autonomous vehicle includes a control part. The control part performs the followings: light emission control of making a light-emitting diode emit light; camara control of making a camera take an image while the light-emitting diode emits the light by the light emission control; self position estimation of estimating a self position by collating map image data with image data acquired by the camera control; and adjustment control of adjusting time from the start of the light emission control to the start of the camera control on the basis of a factor in changing a light quantity increment value relative to elapsed time from the start of the light emission control.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to autonomous vehicles. [Background technology]

[0002] The autonomous vehicle disclosed in Patent Document 1 includes a camera, a light-emitting diode, a storage device, and a control unit. The camera is positioned to capture an image of the road surface. The light-emitting diode irradiates light onto the road surface within the camera's imaging range. The storage device stores map data. The map data is data that links map image data obtained by capturing an image of the road surface in advance with location information. The control unit acquires the image data from the camera. The control unit estimates the vehicle's own location by matching the image data with the map image data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2022-39689 Summary of the Invention [Problem to be solved by the invention]

[0004] The longer the light-emitting diode emits light, the greater the power consumption. For this reason, the light-emitting diode is controlled to emit light intermittently. The light intensity of the light-emitting diode increases as time passes from the start of light-emitting control that causes the light-emitting diode to emit light. Camera control that causes the camera to capture an image is set to start when the light intensity of the light-emitting diode is at its maximum. However, the amount of increase in light intensity over time from the start of light-emitting control may change. In this case, the accuracy of self-location estimation may decrease because camera control is started when the amount of light is insufficient. [Means for solving the problem]

[0005] An autonomous vehicle that solves the above problem comprises a camera positioned to capture an image of the road surface, a light-emitting diode that irradiates light onto the area of ​​the road surface that the camera can capture, a storage device that stores map data that links map image data of the road surface captured in advance with location information, and a control unit, wherein the control unit is configured to perform light emission control that causes the light-emitting diode to emit light, camera control that causes the camera to capture an image while the light-emitting diode is emitting light due to the light-emitting control, self-position estimation that estimates its own position by comparing the image data acquired by the camera control with the map image data, and adjustment control that adjusts the time from the start of the light-emitting control to the start of the camera control based on factors that change the amount of increase in light intensity over time from the start of the light-emitting control.

[0006] According to this, the time from the start of light-emitting control to the start of camera control is adjusted based on factors that change the increase in light intensity over time from the start of light-emitting control. The time from the start of light-emitting control to the start of camera control can be adjusted so that camera control starts at a timing when the light intensity of the light-emitting diode has sufficiently increased. Since camera control is prevented from starting when the amount of light is insufficient, a decrease in the accuracy of self-location estimation can be suppressed.

[0007] For the above-mentioned autonomous vehicle, the adjustment control may be a control that calculates the area of ​​an area from the image data where the brightness value is below a brightness threshold, and if the area is above the area threshold, extends the time from the start of the light emission control to the start of the camera control.

[0008] The autonomous vehicle may be equipped with a temperature sensor that detects the temperature of the light-emitting diode, and the adjustment control may be control that extends the time from the start of the light-emitting control to the start of the camera control when the temperature detected by the temperature sensor is lower than a temperature threshold, compared to when the temperature detected by the temperature sensor is equal to or higher than the temperature threshold. [Effects of the Invention]

[0009] According to the present invention, it is possible to suppress a decrease in the accuracy of self-location estimation. [Brief explanation of the drawings]

[0010] [Figure 1] Figure 1 is a side view of an autonomous vehicle. [Figure 2] Figure 2 is a schematic diagram of an autonomous vehicle. [Figure 3] FIG. 3 is a flowchart showing the control performed by the control unit. [Figure 4] FIG. 4 is a time chart showing the relationship between light emission control and camera control. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of an autonomous vehicle will be described. <Autonomous Vehicles> 1 and 2, autonomous vehicle 10 includes vehicle body 11, drive wheels 21, travel motor driver 22, travel motor 23, steering wheels 31, steering motor driver 32, and steering motor 33. Autonomous vehicle 10 may be a passenger car or an industrial vehicle. The industrial vehicle may be, for example, a forklift, a towing tractor, or an unmanned guided vehicle.

[0012] Travel motor 23 is a motor for rotating drive wheels 21. Travel motor driver 22 drives travel motor 23. Drive of travel motor 23 rotates drive wheels 21, causing autonomous vehicle 10 to travel. Steering motor 33 is a motor for steering steering wheels 31. Steering motor driver 32 drives steering motor 33. Drive of steering motor 33 steers steering wheels 31, causing autonomous vehicle 10 to turn.

[0013] The autonomous vehicle 10 includes a camera 41. The camera 41 is a digital camera. The camera 41 includes an image sensor. The image sensor is, for example, a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. The camera 41 is, for example, an RGB camera, an infrared camera, a grayscale camera, or a visible light camera.

[0014] The camera 41 is positioned to capture an image of the road surface Sr. The camera 41 generates image data that shows the captured image of the road surface Sr. The camera 41 is mounted on the bottom of the vehicle body 11 facing vertically. More specifically, the camera 41 is mounted so that the optical axis of the camera 41 coincides with the vertical direction. Note that the state in which the camera 41 faces vertically allows for errors due to the mounting accuracy of the camera 41, and the camera 41 may be mounted facing in a direction slightly deviated from the vertical direction.

[0015] Autonomous vehicle 10 includes light-emitting diode 51 and LED driver 52 that controls light-emitting diode 51. Light-emitting diode 51 irradiates the road surface Sr within the imaging range of camera 41 with light. Light-emitting diode 51 is provided at the bottom of vehicle body 11 and oriented vertically. LED driver 52, for example, applies and cuts off current to light-emitting diode 51 by passing a current through light-emitting diode 51.

[0016] Autonomous vehicle 10 includes secondary storage device 71. Secondary storage device 71 is, for example, a hard disk drive, a solid state drive, or flash memory. The auxiliary storage device 71 stores map data M1. The map data M1 is obtained by linking map image data of a road surface Sr captured in advance with position information. The range in which the autonomous vehicle 10 travels is determined in advance. The position information includes coordinates and attitude. The coordinates are coordinates of a map coordinate system, which is a coordinate system that represents absolute positions. The map coordinate system may be a Cartesian coordinate system or a geographic coordinate system. The map coordinate system has an X axis and a Y axis. The X axis and the Y axis are orthogonal to each other. The X axis and the Y axis are coordinate systems that represent the horizontal direction. The attitude is information that indicates the inclination of the autonomous vehicle 10 with respect to the coordinate axes of the map coordinate system. In this embodiment, the attitude is information that indicates the inclination with respect to the X axis. The map data M1 is data that indicates the coordinates of the map coordinate system of the range in which the autonomous vehicle 10 travels and the attitude. The auxiliary storage device 71 is an example of a storage device that stores the map data M1.

[0017] The autonomous vehicle 10 includes a control unit 81. The control unit 81 includes a processor 82 and a memory unit 83. The processor 82 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP). The memory unit 83 includes a random access memory (RAM) and a read-only memory (ROM). The memory unit 83 stores program code or instructions configured to cause the processor 82 to execute processing. The memory unit 83, i.e., a computer-readable medium, includes any available medium accessible by a general-purpose or special-purpose computer. The control unit 81 may be configured by a hardware circuit such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). The control unit 81, which is a processing circuit, may include one or more processors operating according to a computer program, one or more hardware circuits such as an ASIC or an FPGA, or a combination thereof.

[0018] Control unit 81 controls autonomous vehicle 10. For example, control unit 81 causes autonomous vehicle 10 to travel by issuing commands to travel motor driver 22 and steering motor driver 32.

[0019] Autonomous vehicle 10 includes battery 90. Battery 90 is a power source for electrical components included in autonomous vehicle 10. The electrical components include camera 41 and light-emitting diode 51. <Control performed by the control unit> The control unit 81 repeatedly performs the following control while the autonomous vehicle 10 is traveling. The following control is performed to estimate the self-position of the autonomous vehicle 10. The control unit 81 causes the autonomous vehicle 10 to travel based on the estimated self-position. For example, the control unit 81 generates a travel route and causes the autonomous vehicle 10 to travel so that the self-position moves along the travel route.

[0020] 3, in step S1, the control unit 81 performs light emission control. The light emission control is control for causing the light emitting diode 51 to emit light. The control unit 81 causes the light emitting diode 51 to emit light by, for example, giving a command to the LED driver 52.

[0021] As shown in Figure 4, when light emission control is started at time T1, light-emitting diode 51 emits light from time T1 until time T5, which is a predetermined light-emitting time T11 after the time T1. The light intensity of light-emitting diode 51 increases over time from the start of light-emitting control. The light intensity increases until time T2, which is a predetermined time after the start of light-emitting control. At time T2, the light intensity of light-emitting diode 51 reaches its maximum. At time T4, control unit 81 issues a command to LED driver 52 to stop light emission of light-emitting diode 51, and the light intensity of light-emitting diode 51 decreases over time.

[0022] 3, next, in step S2, the control unit 81 performs camera control. The camera control is a control for acquiring image data by causing the camera 41 to capture an image while the light-emitting diode 51 is emitting light. More specifically, the camera control is a control for causing the image sensor to receive light by performing exposure.

[0023] 4, camera control is started at time T3, which is later than time T1 when light emission control is started. This is because camera control is preferably started after the light intensity of light-emitting diode 51 has reached its maximum. Camera control continues until time T4, when control unit 81 issues a command to LED driver 52 to stop light emission of light-emitting diode 51. In other words, control unit 81 ends camera control in synchronization with the start of turning off light-emitting diode 51.

[0024] As shown in FIG. 3, next, in step S3, the control unit 81 calculates the area of ​​a region in the image data where the luminance value is equal to or less than the luminance threshold. The area is, for example, the number of pixels in the image data. The control unit 81 determines whether the luminance value of each pixel constituting the image data is equal to or less than the luminance threshold. The luminance value of each pixel is, for example, the Y value when RGB is converted to YCbCr. The control unit 81 calculates the number of pixels constituting the image data where the luminance value is equal to or less than the luminance threshold as the area of ​​the region where the luminance value is equal to or less than the luminance threshold. The luminance threshold is a predetermined value.

[0025] Next, in step S4, the control unit 81 determines whether the area calculated in step S3 is equal to or greater than an area threshold value. The area threshold value is a predetermined value. If the determination result in step S4 is positive, the control unit 81 proceeds to step S5. If the determination result in step S4 is negative, the control unit 81 proceeds to step S6.

[0026] In step S5, the control unit 81 advances the start of light emission control. That is, in the next control cycle, time T1 in the control cycle will be earlier than in the current control cycle. Since time T3 at which camera control starts is maintained, the time from the start of light emission control to the start of camera control will be longer. After completing the processing of step S5, the control unit 81 proceeds to step S6.

[0027] Steps S3 to S5 are adjustment control that adjusts the time from the start of light-emission control to the start of camera control based on factors that change the increase in light intensity over time from the start of light-emission control. The increase in light intensity over time from the start of light-emission control, i.e., the slope of the increase in light intensity over time, varies, for example, due to individual differences between the light-emitting diodes 51 or the temperature of the light-emitting diodes 51. If these factors cause the increase in light intensity over time from the start of light-emission control to change, camera control may be started in a state where the amount of light is insufficient. If camera control is started in a state where the amount of light is insufficient, the determination result of step S4 may be positive due to an increase in the number of pixels whose brightness values ​​are equal to or less than the brightness threshold. The adjustment control of this embodiment is control that lengthens the time from the start of light-emission control to the start of camera control if the area of ​​the region where the brightness values ​​are equal to or less than the brightness threshold is equal to or greater than the area threshold. If the start of camera control in a state where the amount of light is insufficient increases the number of pixels whose brightness values ​​are equal to or less than the brightness threshold, the time from the start of light-emission control to the start of camera control will be longer.

[0028] In step S6, the control unit 81 performs self-location estimation. First, the control unit 81 compares the image data with map image data. The control unit 81 extracts feature points from the image data. The control unit 81 describes the feature amounts of the feature points. The feature amounts are, for example, feature vectors or brightness values. The control unit 81 also extracts feature points and describes the feature amounts using the map image data. The control unit 81 compares the feature points and feature amounts obtained from the image data with the feature points and feature amounts obtained from the map image data, and searches for pairs of feature points with similar feature amounts. The control unit 81 identifies map image data corresponding to the image data based on the feature point pairs. For example, the control unit 81 identifies map image data in which feature point pairs are concentrated as map image data corresponding to the image data. The above-mentioned matching can be performed using a feature amount descriptor. The feature amount descriptor is, for example, ORB (Oriented Fast and Rotated BRIEF), SIFT (Scale-Invariant Feature Transform), or SURF (Speeded Up Robust Features).

[0029] The control unit 81 estimates its own position based on the map image data. The own position includes the coordinates of the autonomous vehicle 10 in a map coordinate system and the attitude of the autonomous vehicle 10. The control unit 81 calculates the relative position between the map image data and the image data, and the relative angle between the map image data and the image data. The relative position between the map image data and the image data is the amount of deviation between the image data and the map image data. The relative angle between the image data and the map image data is the angle of deviation between the image data and the map image data. The image data and the map image data often do not match perfectly. This is because the position and attitude of the autonomous vehicle 10 rarely match perfectly between the time the map image data is acquired and the time the image data is acquired. For this reason, the image data often only matches part of the map image data. If the position of the autonomous vehicle 10 is different between the time the map image data is acquired and the time the image data is acquired, the difference in the position of the autonomous vehicle 10 will cause a deviation between the position of the road surface Sr shown in the map image data and the position of the road surface Sr shown in the image data. This amount of deviation is the relative position between the map image data and the image data. The amount of deviation can be determined from the positional relationship between feature points in the map image data and feature points in the image data. Similarly, the difference in the attitude of autonomous vehicle 10 between the time the map image data was acquired and the time the image data was acquired results in the image data being a rotated version of the map image data. The angle of deviation resulting from this rotation is the relative angle between the image data and the map image data. Control unit 81 estimates its own position based on the position information, relative position, and relative angle associated with the map image data. Control unit 81 shifts the coordinates associated with the map image data by the coordinates corresponding to the relative position. Control unit 81 shifts the attitude associated with the map image data by the relative angle. Control unit 81 regards the coordinates and attitude in the map coordinate system obtained as a result as its own position.

[0030] [Operation of this embodiment] The control unit 81 controls the light emission to cause the light emitting diode 51 to emit light intermittently. Since the power consumption due to the light emission of the light emitting diode 51 can be reduced, the decrease in the remaining capacity of the battery 90 can be suppressed.

[0031] The control unit 81 causes the camera 41 to capture an image during the light-emitting time T11 of the light-emitting diode 51, and estimates the vehicle's own position using the image data acquired by the image capture. Since the vehicle's own position is estimated using the characteristics of the road surface Sr, the light-emitting diode 51 illuminates the imaging range of the camera 41 so that the road surface Sr can be properly captured by the camera 41. In particular, when the camera 41 is provided at the bottom of the vehicle body 11, as in this embodiment, the brightness value of the image data may be insufficient due to the shadow of the vehicle body 11.

[0032] Extraction of feature points and feature quantities from image data is performed based on the brightness values ​​of the image data. Therefore, if the brightness values ​​of the image data are insufficient, it is difficult to extract feature points and feature quantities. As a result, the accuracy of self-location estimation may decrease. The brightness threshold value in step S3 is set, for example, to ensure a brightness value that allows extraction of feature points and feature quantities.

[0033] The image data and map image data are matched based on pairs of feature points. Therefore, if the area of ​​the region where the brightness value is equal to or less than the brightness threshold is insufficient, it may be impossible to match the image data and map image data. As a result, the accuracy of self-location estimation may decrease. The area threshold in step S4 is set, for example, so that the number of feature point pairs is not insufficient.

[0034] If the determination result in step S4 is positive, the control unit 81 lengthens the time from the start of light-emission control to the start of camera control. The light intensity of the light-emitting diode 51 increases as time passes since the start of light-emission control. Therefore, if a short time has passed since the start of light-emission control, the area of ​​the region where the brightness value is equal to or less than the brightness threshold may increase due to insufficient light intensity of the light-emitting diode 51. In steps S3 to S5, if the area of ​​the region where the brightness value is equal to or less than the brightness threshold is equal to or greater than the area threshold, the control unit 81 advances the start of light-emission control. As a result, the light intensity of the light-emitting diode 51 at the time of start of camera control increases compared to when the start of light-emission control is not advanced. For example, as shown by line L in FIG. 4, assume that the increase in light intensity since the start of light-emission control decreases. In this case, camera control is started before the light intensity reaches its maximum. If the determination result in step S4 is positive due to an insufficient brightness, the time T1 at which light-emission control starts is advanced. As a result, in the next control cycle, the time from the start of light-emission control to the start of camera control is lengthened, and the light intensity at time T3 at which camera control starts increases. 3 is repeatedly performed, and each time the determination in step S4 becomes positive, the start of light emission control becomes earlier until the determination result in step S4 becomes negative.

[0035] [Effects of this embodiment] (1) The control unit 81 adjusts the time from the start of light-emission control to the start of camera control based on factors that change the increase in light intensity over time from the start of light-emission control. This makes it possible to adjust the time from the start of light-emission control to the start of camera control so that camera control starts at a timing when the light intensity of the light-emitting diodes 51 has sufficiently increased. Since camera control is prevented from starting when the amount of light is insufficient, it is possible to prevent a decrease in the accuracy of self-location estimation.

[0036] (2) The control unit 81 calculates the area of ​​the region from the image data where the brightness value is equal to or less than the brightness threshold, and if the area is equal to or greater than the area threshold, extends the time from the start of light emission control to the start of camera control. If the brightness value is equal to or less than the brightness threshold, it is difficult to extract feature points and feature amounts from the image data. As the area where feature points and feature points are difficult to extract increases, the accuracy of self-position estimation decreases. By calculating the area of ​​the region from the image data where the brightness value is equal to or less than the brightness threshold, and if the area is equal to or greater than the area threshold, extending the time from the start of light emission control to the start of camera control, adjustments can be made so that the area of ​​the region where the brightness value is equal to or less than the brightness threshold becomes less than the area threshold.

[0037] [Example of change] The embodiment can be modified as follows: The embodiment and the following modifications can be combined with each other to the extent that they are not technically inconsistent.

[0038] As shown in FIG. 2, autonomous vehicle 10 may include temperature sensor 53. Temperature sensor 53 is configured to detect the temperature of light-emitting diode 51. The increase in light intensity over time from the start of light-emitting control decreases as the temperature of light-emitting diode 51 decreases. Controller 81 performs adjustment control to adjust the time from the start of light-emitting control to the start of camera control based on the temperature of light-emitting diode 51. The adjustment control is a control that lengthens the time from the start of light-emitting control to the start of camera control when the temperature detected by temperature sensor 53 is lower than a temperature threshold, compared to when the temperature detected by temperature sensor 53 is equal to or higher than the temperature threshold. For example, two types of timing may be set: a first timing for starting light-emitting control when the temperature detected by temperature sensor 53 is lower than the temperature threshold, and a second timing for starting light-emitting control when the temperature detected by temperature sensor 53 is equal to or higher than the temperature threshold. The second timing is later than the first timing.

[0039] The temperature sensor for detecting the temperature of the light-emitting diode 51 may be a temperature sensor for detecting the outside air temperature. Although the temperature of the light-emitting diode 51 is slightly higher than the outside air temperature, it is correlated with the outside air temperature. Therefore, the outside air temperature can be regarded as the temperature of the light-emitting diode 51.

[0040] The control unit 81 may delay the start of camera control in step S5 to lengthen the time from the start of light emission control to the start of camera control. In this case, the time T1 at which light emission control starts is maintained.

[0041] Control unit 81 may perform adjustment control for a predetermined period of time. For example, adjustment control may be performed for a predetermined period of time after autonomous vehicle 10 is switched from a stopped state to an activated state. The stopped state is a state in which autonomous vehicle 10 is unable to travel. The activated state is a state in which autonomous vehicle 10 is able to travel. The predetermined period of time may be, for example, the time that has elapsed since autonomous vehicle 10 entered the activated state. The predetermined time may be, for example, the number of consecutive times step S4 returns negative.

[0042] The map data M1 may be stored in the storage unit 83, so that the storage unit 83 can be used as a storage device. [Explanation of symbols]

[0043] 10...autonomous vehicle, 41...camera, 51...light-emitting diode, 53...temperature sensor, 81...control unit.

Claims

1. a camera positioned to capture an image of the road surface; a light-emitting diode that irradiates light onto the road surface within the imaging range of the camera; a storage device that stores map data in which map image data of the road surface captured in advance is linked to location information; a control unit, The control unit light emission control for causing the light emitting diode to emit light; camera control that causes the camera to capture an image while the light-emitting diode is emitting light through the light-emission control; a self-location estimation that estimates a self-location by comparing the image data acquired by the camera control with the map image data; and adjustment control that adjusts the time from the start of the light emission control to the start of the camera control based on a factor that changes the amount of increase in light intensity over time from the start of the light emission control.

2. The adjustment control includes: Calculating the area of ​​a region where the brightness value is equal to or less than a brightness threshold value from the image data; The autonomous vehicle according to claim 1 , wherein when the area is equal to or greater than an area threshold, a time period from the start of the light emission control to the start of the camera control is lengthened.

3. a temperature sensor for detecting a temperature of the light-emitting diode; 2. The autonomous vehicle of claim 1, wherein the adjustment control is control that extends the time from the start of the light emission control to the start of the camera control when the temperature detected by the temperature sensor is lower than a temperature threshold, compared to when the temperature detected by the temperature sensor is equal to or higher than the temperature threshold.

Images