Map data creation device and map data creation method

Abstract

To create map data.SOLUTION: A control device acquires map image data from a camera. The control device collates first map image data with second map image data to calculate a difference in pixel positions between the first map image data and the second map image data for the same feature portion included in the first map image data and the second map image data. The second map image data is map image data acquired one time before the first map image data. The control device converts the difference in pixel positions into an actual dimension to calculate a movement amount of a vehicle from the acquisition of the second map image data to the acquisition of the first map image data. The control device associates the first map image data with map coordinates obtained by adding the movement amount to map coordinates associated with the second map image data. The control device associates the first map image data with posture calculated using map coordinates adjacent to the map coordinates associated with the first map image data.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a map data creation device and a map data creation method. [Background technology]

[0002] The autonomous vehicle disclosed in Patent Document 1 includes a camera, a storage device, and a control device. The camera is positioned to capture an image of the road surface. The storage device stores map data. The map data is data in which road surface characteristics are linked to position information. The control device acquires image data from the camera. The control device estimates the vehicle's own position by matching the image data with the map data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-166853 A Summary of the Invention [Problem to be solved by the invention]

[0004] When estimating the vehicle's own position using image data of a road surface, it is necessary to create map data in advance. [Means for solving the problem]

[0005] A map data creation device that solves the above problem is a map data creation device that creates map data by linking map image data obtained by capturing an image of a road surface with an imaging device equipped in a vehicle to position information including the coordinates of the point where the map image data was acquired and the attitude of the vehicle at the time the map image data was acquired, the map data creation device having a computer, which acquires the map image data from the imaging device, and when one of the map image data is first map image data and the map image data acquired one time before the first map image data is second map image data, the computer compares the first map image data with the second map image data to calculate a difference in pixel position between the first map image data and the second map image data for identical characteristic parts included in the first map image data and the second map image data, converts the difference in pixel position into actual dimensions, calculates the amount of movement of the vehicle from the acquisition of the second map image data to the acquisition of the first map image data, links the coordinates obtained by adding the amount of movement to the coordinates linked to the second map image data to the first map image data, and links the attitude calculated using the coordinates adjacent to the coordinates linked to the first map image data to the first map image data.

[0006] The computer can calculate the amount of movement by converting the difference in pixel positions between the first map image data and the second map image data into actual dimensions. This amount of movement is the amount the vehicle has moved from when the first map image data was acquired to when the second map image data was acquired. The computer can obtain the coordinates of the point where the first map image data was acquired by adding this amount of movement to the coordinates of the second map image data. The computer can calculate the attitude of the vehicle at the time when the first map image data was acquired by using coordinates adjacent to the coordinates linked to the first map image data. The computer can create map data by linking position information to map image data as described above.

[0007] In the above-mentioned map data creation device, the computer may determine, as the first map image data, the map image data to which the location information acquired from a positioning device is linked, the map image data to which the location information whose reliability is less than a predetermined threshold is linked, overwrite the coordinates linked to the first map image data and acquired from the positioning device with the coordinates obtained by adding the amount of movement to the coordinates linked to the second map image data, and overwrite the attitude linked to the first map image data and acquired from the positioning device with the attitude calculated using the coordinates adjacent to the coordinates linked to the first map image data.

[0008] With respect to the map data creation device, the computer may determine the map image data to which the location information acquired from the positioning device is linked, the map image data to which the location information is linked, the reliability of which is less than a predetermined threshold value.

[0009] With regard to the above-mentioned map data creation device, when the computer is able to grasp in advance a start point coordinate, which is the coordinate of the start point of a range in which the vehicle travels and in which the reliability is less than the threshold value, and an end point coordinate, which is the coordinate of the end point of the range, the computer may correct the coordinates between the coordinates linked to the map image data corresponding to the start point and the coordinates linked to the map image data corresponding to the end point so that the coordinates linked to the map image data corresponding to the start point and the start point coordinates match, and the coordinates linked to the map image data corresponding to the end point and the end point coordinates match.

[0010] A map data creation method that solves the above problem is a map data creation method that creates map data by linking map image data obtained by capturing an image of a road surface with an imaging device equipped in a vehicle to position information including coordinates of the point where the map image data was acquired and the attitude of the vehicle at the time the map image data was acquired, in which a computer acquires the map image data from the imaging device, and when one of the map image data is designated as first map image data and the map image data acquired one time before the first map image data is designated as second map image data, the computer compares the first map image data with the second map image data to calculate a difference in pixel position between the first map image data and the second map image data for identical characteristic parts included in the first map image data and the second map image data, converts the difference in pixel position into actual dimensions, calculates the amount of movement of the vehicle from the acquisition of the second map image data to the acquisition of the first map image data, links the coordinates obtained by adding the amount of movement to the coordinates linked to the second map image data to the first map image data, and links the attitude calculated using the coordinates adjacent to the coordinates linked to the first map image data to the first map image data.

[0011] The computer can obtain the coordinates of the point where the first map image data was acquired by adding the amount of movement to the coordinates of the second map image data. The computer can calculate the vehicle's attitude at the time when the first map image data was acquired by using coordinates adjacent to the coordinates linked to the first map image data. The computer can create map data by linking position information to map image data as described above. Effect of the Invention

[0012] According to the present invention, map data can be created. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic diagram showing an autonomous driving system. [Diagram 2] FIG. 1 is a block diagram showing an autonomous driving system. [Diagram 3] FIG. 2 is a diagram showing an example of image data. [Figure 4] 10 is a flowchart showing pre-correction map data creation control; [Diagram 5] 3A and 3B are diagrams showing an example of first map image data and second map image data. [Figure 6] 13 is a flowchart showing correction control. [Figure 7] 13 is a diagram showing location information when the reliability of the third location information is equal to or higher than a threshold value. FIG. [Figure 8] 13 is a diagram showing location information when the reliability of the third location information is less than a threshold value. FIG. [Figure 9] FIG. 13 is a diagram illustrating a method of calculating an attitude using map coordinates. [Figure 10] FIG. 13 is a diagram showing a method of correcting map coordinates using start point coordinates and end point coordinates. [Figure 11] FIG. 13 is a diagram illustrating a method of calculating an attitude using map coordinates. [Figure 12] FIG. 13 is a diagram illustrating a method of calculating an attitude using map coordinates. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] An embodiment of a map data creating device and a map data creating method will be described below. As shown in Fig. 1, the autonomous driving system 10 includes a vehicle 20. The vehicle 20 includes a vehicle body 21, driving wheels 22, and steering wheels 25. The vehicle 20 may be a passenger car or an industrial vehicle. Industrial vehicles include forklifts, towing tractors, and automated guided vehicles.

[0015] As shown in Fig. 2, the vehicle 20 includes a running motor driver 23, a running motor 24, a steering motor driver 26, and a steering motor 27. The running motor 24 is a motor for rotating the driving wheels 22. The running motor driver 23 drives the running motor 24. The driving of the running motor 24 rotates the driving wheels 22, causing the vehicle 20 to run. The steering motor 27 is a motor for steering the steering wheels 25. The steering motor driver 26 drives the steering motor 27. The steering of the steering motor 27 causes the steering of the steering wheels 25, causing the vehicle 20 to turn.

[0016] The vehicle 20 includes a camera 41. The camera 41 is an imaging device. The camera 41 is a monocular camera. The camera 41 is a digital camera. The camera 41 includes an imaging element. Examples of the imaging element include a charge coupled device (CCD) image sensor and a complementary metal oxide semiconductor (CMOS) image sensor. Examples of the camera 41 include an RGB camera, an infrared camera, a grayscale camera, and a visible light camera.

[0017] The camera 41 captures images at a predetermined frame rate and generates image data. This image data is digital data of the images captured by the camera 41. 1, the camera 41 is disposed so as to capture an image of the road surface Sr1. It can be said that the camera 41 generates image data showing an image of the road surface Sr1. The camera 41 is provided at the bottom of the vehicle body 21 while facing vertically.

[0018] As shown in Fig. 3, image data IM1 generated by camera 41 is data obtained by capturing an image of a circular range. The image data IM1 may be data obtained by cutting out a part of the circular range. For example, the image data IM1 may be data obtained by capturing an image of a semicircular range. The image data IM1 includes a characteristic portion F1 of the road surface Sr1. The characteristic portion F1 of the road surface Sr1 is, for example, minute irregularities present on the road surface Sr1.

[0019] As shown in Fig. 2, the vehicle 20 includes a lighting device 51. The lighting device 51 is arranged to illuminate the road surface Sr1. More specifically, the lighting device 51 illuminates a range of the road surface Sr1 that is imaged by the camera 41. In this embodiment, the lighting device 51 is provided at the bottom of the vehicle body 21 facing downward. For example, a light-emitting diode can be used as the lighting device 51.

[0020] The vehicle 20 includes a positioning device 61. The positioning device 61 includes a satellite navigation device 62 and an inertial measurement device 65. The satellite navigation device 62 includes a GNSS antenna 63 and a position measurement unit 64. As shown in FIG. 1, the GNSS antenna 63 receives satellite signals transmitted from GNSS (Global Navigation Satellite System) satellites Sa1 to Sa4. The position measurement unit 64 measures the position using the satellite signals. The inertial measurement device 65 includes a gyro sensor and an acceleration sensor.

[0021] The vehicle 20 includes a reflecting member 66. For example, a prism or a retroreflector can be used as the reflecting member 66. The reflecting member 66, the camera 41, and the GNSS antenna 63 are provided at the same position in the horizontal direction.

[0022] As shown in Fig. 2, the vehicle 20 includes a wireless communication device 67. The wireless communication device 67 is a communication device capable of performing communication by any wireless communication method, such as wireless LAN, Zigbee (registered trademark), LPWA (Low Power Wide Area), or a mobile communication system. The wireless communication device 67 is capable of transmitting and receiving wireless signals. The wireless communication device 67 is capable of outputting data obtained by demodulating a received wireless signal. The wireless communication device 67 is capable of modulating input data and transmitting it as a wireless signal.

[0023] The vehicle 20 includes a control device 81. The control device 81 includes a processor 82 and a storage unit 83. Examples of the processor 82 include a central processing unit (CPU), a graphics processing unit (GPU), and a digital signal processor (DSP). The storage unit 83 includes a random access memory (RAM) and a read only memory (ROM). The storage unit 83 stores program codes or instructions configured to cause the processor 82 to execute processes. The storage unit 83, i.e., the computer-readable medium, includes any available medium accessible by a general-purpose or special-purpose computer. The control device 81 may be configured by a hardware circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control device 81, which is a processing circuit, may include one or more processors that operate according to a computer program, one or more hardware circuits such as an ASIC or an FPGA, or a combination thereof. The control device 81 is a computer.

[0024] The vehicle 20 includes an auxiliary storage device 71. The auxiliary storage device 71 stores information that can be read by the control device 81. Examples of the auxiliary storage device 71 include a hard disk drive, a solid state drive, and a flash memory.

[0025] The auxiliary storage device 71 stores map data M1. The map data M1 is obtained by associating map image data of the road surface Sr1 captured in advance with position information. The range in which the vehicle 20 travels is determined in advance. The position information includes coordinates and attitude. The coordinates are coordinates of a map coordinate system that is a coordinate system that represents an absolute position. The map coordinate system may be an orthogonal coordinate system or a geographic coordinate system. The map coordinate system of this embodiment is a two-axis orthogonal coordinate system. The map coordinate system is a coordinate system that represents a position in the horizontal direction. The map coordinate system includes an X axis and a Y axis. The X axis and the Y axis are orthogonal to each other. The attitude is information that indicates the inclination of the vehicle 20 with respect to the coordinate axes of the map coordinate system. The attitude of this embodiment is information that indicates the inclination with respect to the X axis. In the following description, the coordinates of the map coordinate system may be referred to as map coordinates. The map data M1 is data that indicates the map coordinates and attitude of the range in which the vehicle 20 travels.

[0026] The control device 81 performs self-location estimation. The self-location estimation is performed by comparing the image data IM1 with the map image data. The control device 81 extracts feature points from the image data IM1. The control device 81 describes the feature amounts of the feature points. Examples of the feature amounts include feature vectors and brightness values. The control device 81 also uses the map image data to extract the feature points and describe the feature amounts.

[0027] The control device 81 compares the feature points and feature amounts obtained from the image data IM1 with the feature points and feature amounts obtained from the map image data, and searches for pairs of feature points having similar feature amounts. The control device 81 identifies map image data corresponding to the image data IM1 based on the pairs of feature points. For example, the control device 81 identifies map image data in which pairs of feature points are concentrated as map image data corresponding to the image data IM1. The above-mentioned matching can be performed using a feature amount descriptor. Examples of feature amount descriptors include ORB, SIFT, and SURF.

[0028] The control device 81 estimates the vehicle's own position based on the map image data corresponding to the image data IM1. The vehicle's own position includes the coordinates of the vehicle 20 in the map coordinate system and the attitude of the vehicle 20. The control device 81 calculates the relative position between the image data IM1 and the map image data corresponding to the image data IM1, and the relative angle between the image data IM1 and the map image data corresponding to the image data IM1. The relative position between the image data IM1 and the map image data corresponding to the image data IM1 is the amount of deviation between the image data IM1 and the map image data. The relative angle between the image data IM1 and the map image data corresponding to the image data IM1 is the deviation angle between the image data IM1 and the map image data. The image data IM1 and the map image data captured by the camera 41 often do not completely match. This is because the position and attitude of the vehicle 20 rarely completely match at the time when the map image data is acquired and the time when the image data IM1 is acquired. For this reason, the image data IM1 often matches only a part of the map image data. For example, if the position of the vehicle 20 is shifted by half the imaging range of the camera 41 between the time when the map image data was acquired and the time when the image data IM1 was acquired, half of the image data IM1 matches the map image data, and half of the image data IM1 does not match the map image data. This amount of shift is the relative position between the image data IM1 and the map image data corresponding to the image data IM1. Similarly, the image data IM1 becomes the map image data rotated due to the difference between the posture of the vehicle 20 at the time when the map image data was acquired and the time when the image data IM1 was acquired. The shift angle caused by this rotation is the relative angle between the image data IM1 and the map image data corresponding to the image data IM1. The control device 81 estimates the vehicle's own position based on the position information, relative position, and relative angle associated with the map image data corresponding to the image data IM1. The control device 81 shifts the map coordinates associated with the map image data corresponding to the image data IM1 by the coordinates corresponding to the relative position. The control device 81 shifts the posture associated with the map image data corresponding to the image data IM1 by the relative angle. The control device 81 regards the coordinates and posture in the map coordinate system obtained as the vehicle's own position.

[0029] The control device 81 issues commands to the travel motor driver 23 and the steering motor driver 26 based on the vehicle's own position to drive the vehicle 20. The vehicle 20 is an autonomous vehicle that is driven autonomously by the control device 81.

[0030] 2, the autonomous driving system 10 includes a total station 90. The total station 90 is installed at a location where the vehicle 20 travels. The total station 90 measures the three-dimensional position of an object with the total station 90 as the origin. The total station 90 is a positioning device.

[0031] The total station 90 includes a distance measuring unit 91 , a position detecting unit 92 , and a wireless communication device 93 . The distance measurement unit 91 measures the distance to the vehicle 20. The distance measurement unit 91 measures the distance to the reflecting member 66 by, for example, irradiating a laser light and receiving the light reflected by the reflecting member 66.

[0032] The position detection unit 92 calculates the coordinates of the vehicle 20. The position detection unit 92 calculates the coordinates of the reflective member 66 relative to the total station 90 based on the distance to the reflective member 66 measured by the distance measurement unit 91 and the irradiation direction of the laser light at that time. By knowing in advance the coordinates of the total station 90 in the map coordinate system, the position detection unit 92 can calculate the coordinates and attitude of the vehicle 20 in the map coordinate system. That is, the position detection unit 92 can detect position information.

[0033] The wireless communication device 93 is a communication device capable of communicating by any wireless communication method, such as wireless LAN, Zigbee, LPWA, or a mobile communication system. The wireless communication device 93 can transmit and receive wireless signals. The wireless signals include position information detected by the position detection unit 92. The wireless signals transmitted from the wireless communication device can be received by the wireless communication device 67. Therefore, the position information detected by the total station 90 can be transmitted to the vehicle 20 using the wireless communication device 93.

[0034] A map data creation method will be described. In this embodiment, the map data M1 is created by the control device 81. First, the control device 81 creates pre-correction map data. The control device 81 creates pre-correction map data by performing pre-correction map data creation control. The pre-correction map data creation control is performed while the vehicle 20 is traveling. When creating the map data M1, the vehicle 20 may be traveling by human operation or may be traveling under the control of the control device 81.

[0035] As shown in FIG. 4, in step S1, the control device 81 acquires position information from the total station 90 via the wireless communication device 67. Next, in step S2, the control device 81 acquires image data from the camera 41. The image data acquired from the camera 41 and used for the map data M1 is map image data.

[0036] Next, in step S3, the control device 81 links the location information acquired in step S1 to the map image data acquired in step S2. The location information linked to the map image data in step S3 is the location information at the time when the map image data was acquired in step S2.

[0037] The control device 81 repeats the processes of steps S1 to S3. As a result, the control device 81 acquires map image data from the camera 41 multiple times. Each piece of map image data is linked to the position information acquired from the total station 90. When the vehicle 20 has finished traveling in the range in which the map data M1 is created, the pre-correction map data creation control ends. The data in which the map image data of the range in which the vehicle 20 has traveled is linked to the position information acquired from the total station 90 is the pre-correction map data.

[0038] As shown in FIG. 5, in the pre-correction map data creation control, the interval for acquiring map image data is determined so that two map image data acquired successively include an overlapping range A3. The overlapping range A3 is a range of the road surface Sr1 that is included in both of the two map image data acquired successively. The two map image data acquired successively are referred to as first map image data MI1 and second map image data MI2. The second map image data MI2 is map image data acquired one time before the first map image data MI1. The first map image data MI1 is obtained by capturing an image of the first range A1 of the road surface Sr1 by the camera 41. The second map image data MI2 is obtained by capturing an image of the second range A2 of the road surface Sr1 by the camera 41. The range where the first range A1 and the second range A2 overlap each other is the overlapping range A3.

[0039] Next, the control device 81 corrects the pre-correction map data. The pre-correction map data is corrected by the control device 81 performing the correction control. The map data M1 is created by correcting the pre-correction map data. The correction control will be described.

[0040] As shown in FIG. 6, in step S11, the control device 81 determines location information whose reliability is less than a predetermined threshold. The reliability is an index showing the accuracy of the location information. The higher the reliability, the smaller the possibility that the deviation between the location information and the actual map coordinates and attitude corresponding to the location information is. The control device 81 determines whether the reliability is equal to or greater than a threshold for each piece of location information linked to each piece of map image data included in the pre-correction map data. Whether the reliability is equal to or greater than a threshold can be determined based on location information acquired multiple times. As an example, a case will be described in which whether the reliability is equal to or greater than a threshold is determined based on three pieces of location information acquired consecutively.

[0041] As shown in FIG. 7, the three pieces of position information are first position information P1 (x1, y1, θ1), second position information P2 (x2, y2, θ2), and third position information P3 (x3, y3, θ3). The second position information P2 is the position information acquired one time before the third position information P3. The first position information P1 is the position information acquired two times before the third position information P3. x1, x2, and x3 are x coordinates in the map coordinate system. y1, y2, and y3 are y coordinates in the map coordinate system. θ1, θ2, and θ3 are attitudes. The first position information P1 includes first map coordinates (x1, y1) and a first attitude θ1. The second position information P2 includes second map coordinates (x2, y2) and a second attitude θ2. The third position information P3 includes third map coordinates (x3, y3) and a third orientation θ3.

[0042] When the reliability of the location information is high, the map coordinates change linearly. For example, as shown in Figure 7, the x coordinate and the y coordinate are in a proportional relationship. Therefore, if a line Tr is extracted from map coordinates obtained multiple times, the deviation of the map coordinates from the line Tr can be said to represent the reliability of the map coordinates.

[0043] As shown in FIG. 8, it is assumed that the deviation between the third position information P3 and the actual position P10 (x10, y10, θ10) corresponding to the third position information P3 is large. In this case, the deviation between the third map coordinate (x3, y3) and the actual map coordinate (x10, y10) corresponding to the third map coordinate (x3, y3) is large. The deviation of the third map coordinate (x3, y3) with respect to the straight line Tr passing through the first map coordinate (x1, y1) and the second map coordinate (x2, y2) is large. The deviation is the length L1 of the perpendicular line connecting the third map coordinate (x3, y3) and the straight line Tr. The control device 81 determines whether the length L1 is equal to or greater than the coordinate threshold. If the length L1 is less than the coordinate threshold, the reliability of the map coordinate is equal to or greater than the threshold. The coordinate threshold is set, for example, according to the interval at which the position information is acquired. When the vehicle 20 is turning, the map coordinate may not change linearly. Therefore, when position information is acquired while the vehicle 20 is turning and the deviation between the map coordinates and the actual map coordinates is within an acceptable range, the coordinate threshold is set so that the length L1 is less than the coordinate threshold.

[0044] When the reliability of the position information is high, the amount of change in the posture is small. Therefore, it can be said that the difference between postures acquired consecutively represents the reliability of the posture. The difference between postures acquired consecutively is defined as the posture difference. In the example shown in FIG. 7, the first posture θ1, the second posture θ2, and the third posture θ3 are the same value.

[0045] As shown in FIG. 8, it is assumed that the deviation between the third attitude θ3 and the actual attitude θ10 corresponding to the third attitude θ3 is large. In this case, the attitude difference between the third attitude θ3 and the second attitude θ2 becomes large. The control device 81 judges whether or not the attitude difference between the third attitude θ3 and the second attitude θ2 is less than the attitude threshold value. If the attitude difference is less than the attitude threshold value, the reliability of the attitude is equal to or greater than the threshold value. The attitude threshold value, like the coordinate threshold value, is set so that the attitude difference is less than the attitude threshold value when position information is acquired while the vehicle 20 is turning and the deviation between the attitude and the actual attitude is within an allowable range.

[0046] The control device 81 may determine that the reliability of the position information is less than the threshold when the reliability of the map coordinates is less than the threshold and the reliability of the posture is less than the threshold. The control device 81 may determine that the reliability of the position information is less than the threshold when the reliability of the map coordinates is less than the threshold. In this case, the control device 81 may not determine whether the reliability of the posture is less than the threshold. The control device 81 may determine that the reliability of the position information is less than the threshold when the reliability of the posture is less than the threshold. In this case, the control device 81 may not determine whether the reliability of the map coordinates is less than the threshold. It can be said that the control device 81 determines the map image data associated with the position information whose reliability is less than the threshold by the processing of step S11.

[0047] As shown in FIG. 6, in step S12, the control device 81 compares the map image data associated with the position information whose reliability is less than the threshold with the map image data acquired one time before the map image data. This comparison can be performed by the same method as the comparison between image data and map image data performed during self-location estimation. The map image data associated with the position information whose reliability is less than the threshold is set as the first map image data. The map image data acquired one time before the map image data associated with the position information whose reliability is less than the threshold is set as the second map image data. The control device 81 corrects the position information associated with the first map image data. As an example, the first map image data MI1 and the second map image data MI2 shown in FIG. 5 will be used for explanation.

[0048] An overlapping range A3 exists between the first map image data MI1 and the second map image data MI2. The characteristic portion F1 existing in the overlapping range A3 is included in both the first map image data MI1 and the second map image data MI2. The control device 81 can calculate the difference in pixel position between the first map image data MI1 and the second map image data MI2 for the same characteristic portion F1 included in the first map image data MI1 and the second map image data MI2. This difference in pixel position can also be referred to as the movement amount d1 [px] of the vehicle 20 from when the second map image data MI2 is acquired until when the first map image data MI1 is acquired. This movement amount d1 is the movement amount in the coordinate system that represents the position of the map image data. The difference in pixel position for the same characteristic portion F1 can also be referred to as the center pixel of the first map image data MI1 and the center pixel of the second map image data MI2. The first map image data MI1 and the second map image data MI2 may be arranged so that the same feature portion F1 overlaps, and the difference between the central pixel of the first map image data MI1 and the central pixel of the second map image data MI2 may be regarded as the difference in pixel position. Even in this case, it can be said that the difference in pixel position between the first map image data MI1 and the second map image data MI2 is calculated for the same feature portion F1. The difference in pixel position is calculated separately for the difference in the X-axis and the difference in the Y-axis of the coordinate system that represents the position of the map image data.

[0049] Next, in step S13, the control device 81 converts the difference in pixel positions obtained in step S12 into actual dimensions. The control device 81 grasps the pixel scale in advance. The pixel scale is the actual dimension per pixel. Although there are influences due to the undulations of the road surface Sr1, the degree of tire contraction, and the up and down movement of the vehicle 20 accompanying traveling, the distance between the camera 41 and the road surface Sr1 can be considered to be constant. The pixel scale can be grasped in advance. Using this pixel scale, the control device 81 converts the difference in pixel positions into actual dimensions. The actual dimensions obtained by the conversion are the amount of movement of the vehicle 20 from the acquisition of the second map image data MI2 to the acquisition of the first map image data MI1. The actual dimensions can also be said to be dimensions in the map coordinate system.

[0050] Next, in step S14, the control device 81 adds the movement amount calculated in step S13 to the second map image data MI2. The control device 81 links the map coordinates obtained in this way to the first map image data MI1. The map coordinates linked to the second map image data MI2 are (x21, y21). The movement amount obtained in step S13 is dx, dy. x21 is the x coordinate of the map coordinate system. y21 is the y coordinate of the map coordinate system. dx is the movement amount on the X axis of the map coordinate system. dy is the movement amount on the Y axis of the map coordinate system. In this case, the map coordinates (x21+dx, y21+dy) are linked to the first map image data MI1. The map coordinates linked to the first map image data MI1 and acquired by the total station 90 are overwritten with the map coordinates obtained using the movement amount. As a result, the map coordinates linked to the first map image data MI1 are corrected.

[0051] Next, in step S15, the control device 81 performs attitude correction. The target for which the attitude correction is performed is the same as the target for which the map coordinates are corrected. That is, the target for which the attitude correction is performed is the attitude associated with the first map image data MI1.

[0052] The control device 81 calculates the attitude using map coordinates adjacent to the map coordinates linked to the first map image data MI1. The map coordinates adjacent to the map coordinates linked to the first map image data MI1 are the map coordinates linked to the second map image data MI2 and the map coordinates linked to the map image data acquired one time after the first map image data MI1. The map coordinates used to correct the attitude are the map coordinates corrected in step S14.

[0053] As shown in FIG. 9, the map coordinates of the first map image data MI1 are the first map coordinates P12, the map coordinates of the second map image data MI2 are the second map coordinates P11, and the map coordinates associated with the map image data acquired one time after the first map image data MI1 are the third map coordinates P13. The control device 81 calculates the inclination of the vector from the second map coordinates P11 to the third map coordinates P13 with respect to the X-axis of the map coordinate system as the attitude θ. The control device 81 associates this attitude θ with the first map image data MI1. The attitude associated with the first map image data MI1 and acquired by the total station 90 is overwritten with the attitude θ. As a result, the attitude associated with the first map image data MI1 is corrected. Hereinafter, the map coordinates calculated in step S14 and the attitude calculated in step S15 may be referred to as position information calculated by image odometry.

[0054] Although the case where the position information linked to the first map image data MI1 is corrected has been described as an example, the control device 81 corrects all of the first map image data included in the pre-correction map data. As a result, map data M1 is obtained in which the position information of the total station 90 is linked to the map image data when the reliability of the position information of the total station 90 is equal to or higher than a threshold, and the position information calculated by image odometry is linked to the map image data when the reliability of the position information of the total station 90 is less than the threshold. As described above, the control device 81 that creates the map data M1 is the map data creation device.

[0055] The operation of this embodiment will be described. The control device 81 acquires map image data from the camera 41 while the vehicle 20 is moving. The control device 81 can calculate the amount of movement by converting the difference in pixel position between the first map image data MI1 and the second map image data MI2 into actual dimensions. This amount of movement is the amount of movement of the moving object from the acquisition of the first map image data MI1 to the acquisition of the second map image data MI2. The control device 81 can obtain the map coordinates of the point where the first map image data MI1 is acquired by adding this amount of movement to the map coordinates linked to the second map image data MI2. The control device 81 can calculate the attitude of the vehicle 20 at the time when the first map image data MI1 is acquired by using the map coordinates adjacent to the map coordinates linked to the first map image data MI1.

[0056] The effects of this embodiment will be described. (1) The control device 81 can calculate the amount of movement by converting the difference in pixel position between the first map image data MI1 and the second map image data MI2 into actual dimensions. The control device 81 can calculate the map coordinates of the point where the first map image data MI1 was acquired by using this amount of movement and the position information linked to the second map image data MI2. The control device 81 can calculate the attitude of the vehicle 20 at the time when the first map image data MI1 was acquired by using map coordinates adjacent to the map coordinates linked to the first map image data MI1. The control device 81 can create map data M1 by linking the position information to the map image data.

[0057] (2) The control device 81 sets the map image data associated with the position information whose reliability is less than a threshold as the first map image data MI1. The control device 81 overwrites the map coordinates associated with the first map image data MI1 with map coordinates calculated using the amount of movement. The control device 81 overwrites the orientation associated with the first map image data MI1 with an orientation calculated using map coordinates adjacent to the map coordinates associated with the first map image data MI1. This makes it possible to correct the position information acquired by the total station 90 when the reliability of the position information is low. The control device 81 can create map data M1 in which the map image data is associated with position information that is less deviated from the actual position.

[0058] (3) The control device 81 can determine whether the reliability of the location information is less than a threshold value. The control device 81 can extract the first map image data MI1 and correct the location information linked to the first map image data MI1.

[0059] The embodiment can be modified as follows: The embodiment and the following modifications can be combined with each other to the extent that there is no technical contradiction. The control device 81 may create the map data M1 without using the positioning device 61 and the total station 90. That is, the control device 81 may create the map data M1 using only map image data obtained by the camera 41. For example, there are cases where the positioning device 61 cannot be provided on the vehicle 20 due to cost constraints and placement constraints, or where the total station 90 cannot be used. In these cases, the control device 81 creates the map data M1 using only map image data obtained by the camera 41.

[0060] The control device 81 acquires map image data from the camera 41 while the vehicle 20 is moving. When the vehicle 20 finishes traveling in the range in which the map data M1 is created, the control device 81 finishes acquiring the map image data. The control device 81 sets one of the map image data acquired from the camera 41 as the first map image data. The control device 81 calculates the amount of movement of the vehicle 20 from the acquisition of the second map image data to the acquisition of the first map image data, using the first map image data and the second map image data, which is the map image data acquired one time before the first map image data. The calculation of the amount of movement can be performed by a process similar to the process of steps S12 and S13 in the embodiment. The control device 81 links the map coordinates obtained by adding the amount of movement to the map coordinates linked to the second map image data to the first map image data. Note that the map coordinates of the map image data acquired first after the vehicle 20 starts moving may be predetermined, or the map coordinates may be the origin of the map coordinate system. The control device 81 calculates the attitude using the map coordinates adjacent to the map coordinates linked to the first map image data. The attitude can be calculated by the same process as step S15 in the embodiment. The control device 81 links the calculated attitude to the first map image data. The control device 81 sets all map image data except for the first acquired map image data as the first map image data, and links the map coordinates and attitude. In this way, the control device 81 can create the map data M1.

[0061] The control device 81 may create the map data M1 while the vehicle 20 is traveling. In other words, the control device 81 may create the map data M1 without creating pre-correction map data. For example, when the control device 81 acquires map image data from the camera 41 while the vehicle 20 is traveling, the control device 81 links the position information acquired from the total station 90 to the map image data. The control device 81 determines whether the reliability of the position information associated with the map image data is less than a threshold value. If the reliability of the position information is equal to or greater than a threshold value, the control device 81 does not correct the position information. If the reliability of the position information is less than a threshold value, the control device 81 overwrites the position information acquired from the total station 90 with the position information calculated by image odometry. When calculating the attitude by image odometry, the attitude may be calculated using map image data acquired one time after the first map image data. In this case, the control device 81 may correct the attitude in the next control cycle after the correction of the map coordinates.

[0062] The map data creation device may include a control device 81 and an external device different from the control device 81. The external device is a device provided outside the vehicle 20. The hardware configuration of the external device is, for example, the same as that of the control device 81. In this case, the control device 81 creates pre-correction map data. The pre-correction map data created by the control device 81 is stored in the external device through communication between the control device 81 and the external device or by a storage medium. The external device corrects the pre-correction map data by performing correction control. As a result, the external device creates map data M1. The control device 81 and the external device are computers.

[0063] The reliability of the location information varies depending on the surrounding environment. For example, the reliability varies depending on the number of obstacles, the type of obstacle, whether indoors or outdoors, and the presence or absence of moving objects such as people and vehicles. For this reason, there are cases where the map coordinates of the start point and the end point of the range in which the reliability of the location information is less than a threshold value within the range in which the vehicle 20 travels can be known in advance. For example, assume that the reliability of the location information acquired when the vehicle 20 is located indoors is lower than that of the location information acquired when the vehicle 20 is located outdoors. If it is known in advance that the vehicle 20 travels inside a building, the entrance of the building can be set as the start point and the exit of the building as the end point, and it can be known that the reliability of the location information is less than a threshold value in the range between the start point and the end point.

[0064] As shown in FIG. 10, the area between the start point SP1 and the end point LS1 is a range in which the reliability of the position information is less than a threshold. The map coordinates of the start point SP1 are the start point coordinates PS, and the map coordinates of the end point LS1 are the end point coordinates PE. The start point coordinates PS and the end point coordinates PE are known in advance. Assume that the control device 81 acquires map image data from the camera 41 while the vehicle 20 is traveling, and associates the map coordinates with the map image data. These map coordinates are the pre-correction map coordinates P21. The pre-correction map coordinates P21 are map coordinates obtained by image odometry. Since the reliability of the position information is low between the start point SP1 and the end point LS1, errors are accumulated from the start point SP1 to the end point LS1. The deviation between the pre-correction map coordinates P21 and the actual map coordinates becomes larger as the distance from the start point SP1 to the end point LS1 increases. At the start point SP1, the deviation between the pre-correction map coordinates P21 and the actual map coordinates is small. Therefore, the control device 81 can grasp the pre-corrected map coordinate P21 linked to the map image data corresponding to the start point SP1 from the pre-corrected map coordinate P21. In detail, the control device 81 judges the pre-corrected map coordinate P21 that has the smallest difference from the start point coordinate PS among the pre-corrected map coordinates P21 as the pre-corrected map coordinate P21 linked to the map image data corresponding to the start point SP1. The map image data corresponding to the start point SP1 is the map image data captured at the start point SP1. When the vehicle 20 passes the end point LS1, the reliability of the position information becomes equal to or greater than a threshold. When the reliability of the position information becomes equal to or greater than a threshold, the control device 81 judges the map coordinate acquired one time before the position information as the pre-corrected map coordinate P21 linked to the map image data corresponding to the end point LS1. The map image data corresponding to the end point LS1 is the map image data captured at the end point LS1. The control device 81 corrects the pre-corrected map coordinate P21 so that the pre-corrected map coordinate P21 corresponding to the start point SP1 coincides with the start point coordinate PS, and the pre-corrected map coordinate P21 corresponding to the end point LS1 coincides with the end point coordinate PE. In detail, the control device 81 expands or reduces the interval between the pre-corrected map coordinates P21, thereby causing the pre-corrected map coordinate P21 corresponding to the start point SP1 to coincide with the start point coordinate PS, and causing the pre-corrected map coordinate P21 corresponding to the end point LS1 to coincide with the end point coordinate PE. This makes it possible to obtain the corrected map coordinate P22, as shown in FIG. 10.The control device 81 associates the corrected map coordinate P22 with the map image data.

[0065] The pre-correction map coordinate P21 may be a map coordinate obtained by the total station 90 or the positioning device 61. In this case, the control device 81 corrects the map coordinate by the above-mentioned method for a range in which the reliability of the position information is less than a threshold and is known in advance. For position information in which the reliability is less than a threshold and is not known in advance, the control device 81 corrects the map coordinate by image odometry.

[0066] When creating pre-correction map data, the control device 81 may link the position information acquired from the satellite navigation device 62 to the map image data. In this case, the control device 81 may calculate the reliability based on the strength of the satellite signals received from the GNSS satellites Sa1 to Sa4. In this case, the higher the strength of the satellite signals, the higher the reliability. The control device 81 may calculate the reliability based on the number of GNSS satellites Sa1 to Sa4 captured by the satellite navigation device 62. In this case, the higher the number of GNSS satellites Sa1 to Sa4, the higher the reliability. The control device 81 may calculate the reliability based on DOP (Dilution Of Precision). In this case, the lower the DOP, the higher the reliability. The control device 81 may calculate the reliability from a combination of the strength of the satellite signals, the number of GNSS satellites Sa1 to Sa4, and the DOP.

[0067] When creating pre-correction map data, the control device 81 may associate position information calculated using the satellite navigation device 62 and the inertial measurement device 65 with the map image data. Whether the reliability of the location information is less than the threshold value may be determined by a person. For example, if a point where the reliability of the location information is low is known in advance, it can be determined that the reliability of the location information of this point is less than the threshold value. The person operates the control device 81 to specify location information whose reliability is less than the threshold value. The control device 81 sets the map image data to which the specified location information is linked as the first map image data. The control device 81 corrects the location information linked to the first map image data.

[0068] 11, the control device 81 may calculate an attitude θ associated with the first map image data MI1 from the second map coordinate P11 associated with the second map image data MI2 and the first map coordinate P12 associated with the first map image data MI1. The control device 81 calculates the inclination of a vector from the second map coordinate P11 to the first map coordinate P12 with respect to the X-axis of the map coordinate system as the attitude θ. The control device 81 associates this attitude θ with the first map image data MI1.

[0069] 12, the control device 81 may calculate an orientation θ associated with the first map image data MI1 from the first map coordinate P12 and the third map coordinate P13 associated with the first map image data MI1. The control device 81 calculates the inclination of a vector from the first map coordinate P12 to the third map coordinate P13 with respect to the X-axis of the map coordinate system as the orientation θ. The control device 81 associates this orientation θ with the first map image data MI1.

[0070] The imaging device may be a linear image sensor. The storage device in which the map data M1 is stored may be the storage unit 83. [Explanation of symbols]

[0071] d1...amount of movement, F1...characteristic part, PE...end point coordinates, PS...start point coordinates, LS1...end point, M1...map data, MI1...first map image data, MI2...second map image data, SP1...start point, Sr1...road surface, 20...vehicle, 41...camera which is an imaging device, 61...positioning device, 81...control device which is a computer, 90...total station which is a positioning device.

Claims

1. A map data creation device that creates map data by linking map image data obtained by capturing an image of a road surface using an imaging device provided in a vehicle with position information including coordinates of a point where the map image data was acquired and an attitude of the vehicle at the time when the map image data was acquired, The map data creation device includes a computer. The computer includes: acquiring the map image data from the imaging device; when one of the map image data associated with the location information acquired from the positioning device, the location information having a reliability lower than a predetermined threshold value is designated as first map image data, and the map image data acquired one time before the first map image data is designated as second map image data, the first map image data and the second map image data are compared to calculate a difference in pixel position between the first map image data and the second map image data for identical feature parts included in the first map image data and the second map image data, converting the difference in pixel positions into actual dimensions to calculate a movement amount of the vehicle from the acquisition of the second map image data to the acquisition of the first map image data; overwriting the coordinates associated with the first map image data and acquired from the positioning device with the coordinates obtained by adding the amount of movement to the coordinates associated with the second map image data; overwriting the attitude associated with the first map image data and acquired from the positioning device with the attitude calculated using the coordinates adjacent to the coordinates associated with the first map image data; When a start point coordinate, which is the coordinate of the start point of a range in which the vehicle travels and in which the reliability is less than the threshold, and an end point coordinate, which is the coordinate of the end point of the range, are known in advance, the map data creation device corrects the coordinates between the coordinates linked to the map image data corresponding to the start point and the coordinates linked to the map image data corresponding to the end point so that the coordinates linked to the map image data corresponding to the start point and the start point coordinates match, and the coordinates linked to the map image data corresponding to the end point and the end point coordinates match.

2. The computer includes: The map data creation device according to claim 1 , further comprising: determining, from among the map image data to which the position information acquired from the positioning device is linked, the map image data to which the position information is linked, the reliability of which is less than a predetermined threshold value.

3. A map data creation method for creating map data by linking map image data obtained by capturing an image of a road surface using an imaging device provided in a vehicle with position information including coordinates of a point where the map image data was acquired and an attitude of the vehicle at the time when the map image data was acquired, The computer acquiring the map image data from the imaging device; when one of the map image data associated with the location information acquired from the positioning device, the location information having a reliability lower than a predetermined threshold value is designated as first map image data, and the map image data acquired one time before the first map image data is designated as second map image data, the first map image data and the second map image data are compared to calculate a difference in pixel position between the first map image data and the second map image data for identical feature parts included in the first map image data and the second map image data, converting the difference in pixel positions into actual dimensions to calculate a movement amount of the vehicle from the acquisition of the second map image data to the acquisition of the first map image data; overwriting the coordinates associated with the first map image data and acquired from the positioning device with the coordinates obtained by adding the amount of movement to the coordinates associated with the second map image data; overwriting the attitude associated with the first map image data and acquired from the positioning device with the attitude calculated using the coordinates adjacent to the coordinates associated with the first map image data; A map data creation method, in which when a start coordinate, which is the coordinate of the start point of a range in which the vehicle travels and in which the reliability is less than the threshold, and an end coordinate, which is the coordinate of the end point of the range, are known in advance, the coordinate between the coordinate linked to the map image data corresponding to the start point and the coordinate linked to the map image data corresponding to the end point is corrected so that the coordinate linked to the map image data corresponding to the start point matches the start coordinate, and the coordinate linked to the map image data corresponding to the end point matches the end coordinate.

Images