Dynamic map update device, dynamic map update system, and dynamic map update method

Abstract

To obtain a dynamic map update device that can select the optimal detection data according to the attributes of the object and the type of sensor, and update and manage the dynamic map using that data. [Solution] A dynamic map update device comprising: a receiving unit that receives object information from a sensor, which includes the position and attributes of the object generated based on sensor detection data and has the type of sensor added to it; an object information extraction unit that extracts object information for each object from the object information received from multiple sensors by the receiving unit; a reliability calculation unit that calculates reliability based on the attributes of the object and the type of sensor in the object information extracted by the object information extraction unit; and a dynamic map update unit that updates the dynamic map with the position of the object information that has the highest reliability for each object calculated by the reliability calculation unit.

Description

Technical Field

[0001] The present disclosure relates to a dynamic map updating apparatus, a dynamic map updating method, and a dynamic map updating system.

Background Art

[0002] In recent years, the development of autonomous driving technology for automobiles has been actively conducted, and technologies for fully autonomous driving that does not require intervention of a user's driving operation, beyond merely providing driving assistance for users, have attracted attention. The introduction of autonomous driving is expected to solve various social problems such as alleviating driver shortages in the logistics field, improving traffic congestion, and addressing the last-mile problem.

[0003] Systems have been proposed that perform autonomous driving while detecting objects on a driving route in a limited operation area and avoiding contact with objects. Even when an object cannot be detected only by sensors mounted on an autonomous driving vehicle, roadside sensors can detect the object, and a dynamic map, which is formed by superimposing information such as vehicles and pedestrians on a high-precision map, can be generated and updated to assist autonomous driving.

[0004] In order to detect all objects without omission and assist autonomous driving, it is necessary to provide a plurality of sensors to implement object detection without blind spots. For this reason, a technology has been disclosed that compensates for missing regions with no sensor data using data from another sensor (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0006] In the technology disclosed in Patent Document 1, when detecting surrounding objects using a LiDAR (Light Detection and Ranging) that emits its own laser light and receives the reflected wave, detection data from another LiDAR is used to fill in missing areas of the detection data. To perform object detection without blind spots, detection results from different sensors will be obtained for the same object. Depending on the type of sensor, differences will occur in the accuracy and reliability of object detection. The accuracy and reliability of object detection also change depending on the attributes of the target object.

[0007] Patent Document 1 does not address the selection of more desirable detection data in such cases. This disclosure was made to solve the above-mentioned problems. This disclosure aims to provide a dynamic map updating device that can select the optimal detection data according to the attributes of an object and the type of sensor when there is detection data from multiple sensors for the same object, and update the dynamic map with that data. It also aims to provide a dynamic map updating system composed of such an object detection sensor and a dynamic map updating device. Furthermore, it aims to provide a dynamic map updating method for such a dynamic map updating device. [Means for solving the problem]

[0008] The dynamic map update device related to this disclosure is A receiving unit that receives object information from a sensor, which includes the position and attributes of the object generated based on the sensor's detection data, and the type of sensor is added to it. The receiving unit extracts object information for each object from the object information received from multiple sensors. A reliability calculation unit calculates the reliability based on the object attributes and sensor type of the object information extracted by the object information extraction unit, and It includes a dynamic map update unit that updates the dynamic map based on the location of the object information with the highest reliability calculated by the reliability calculation unit for each object.

[0009] The dynamic map update system related to this disclosure is A first object information generation unit generates first object information that includes the position and attributes of an object generated based on detection data from a first sensor, with the type of sensor added; A first sensor having a first object information transmission unit that transmits first object information generated by a first object information generation unit, A second object information generation unit generates second object information, including the object's position, attributes, and sensor type, based on detection data from a second sensor. A second sensor having a second object information transmission unit that transmits second object information generated by a second object information generation unit, and A receiving unit that receives first object information transmitted from a first object information transmitting unit and second object information transmitted from a second object information transmitting unit, An object information extraction unit extracts the first object information and the second object information received by the receiving unit, for each object. A reliability calculation unit calculates the reliability based on the object attributes and sensor type of the object information extracted by the object information extraction unit. The system includes a dynamic map update device which has a dynamic map update unit that updates the dynamic map based on the location of the object information with the highest reliability for each object, as calculated by the reliability calculation unit.

[0010] The dynamic map update method related to this disclosure is: A receiving step of receiving object information from a sensor, which includes the position and attributes of the object generated based on the sensor's detection data, and the type of sensor added to it. The receiving step extracts object information for each object from the object information received from multiple sensors, A confidence calculation step that calculates the confidence level based on the object attributes and sensor type of the object information extracted by the object information extraction step, and The system includes a dynamic map update step that updates the dynamic map based on the location of the object information with the highest confidence level calculated in the confidence level calculation step for each object. [Effects of the Invention]

[0011] According to the dynamic map updating apparatus, dynamic map updating system, and dynamic map updating method according to the present disclosure, when detection data of the same object obtained by a plurality of sensors exists, optimal detection data can be selected according to the attribute of the object and the type of the sensor, and the dynamic map can be updated using said data. Therefore, the dynamic map can be updated using data with the highest accuracy and reliability. As a result, highly accurate and highly reliable dynamic map information for automatic driving assistance can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] [Figure 1] FIG. 1 is a perspective view showing a configuration of a dynamic map updating system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing a configuration of a dynamic map updating system according to the first embodiment. [Figure 3] FIG. 3 is a hardware configuration diagram of a dynamic map updating apparatus according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of object information according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing an example of reliability of object information according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of daytime coefficients for object information according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of sunrise coefficients for object information according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of sunset coefficients for object information according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of nighttime coefficients for object information according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of coefficients for object information in clear and cloudy weather according to the first embodiment. [Figure 11] FIG. 11 is a diagram showing an example of coefficients for object information in light rain and light snow according to the first embodiment. [Figure 12] FIG. 12 is a diagram showing an example of coefficients for object information in heavy rain, heavy snow and fog according to the first embodiment. [Figure 13]FIG. 1 is a diagram showing similarity between object attributes of object information according to the first embodiment. [Figure 14] It is a flowchart showing processing of the dynamic map update apparatus according to the first embodiment. [Figure 15] It is a block diagram showing a configuration of a dynamic map update system according to the second embodiment. [Figure 16] It is a flowchart showing processing of the dynamic map update apparatus according to the second embodiment. MODE FOR CARRYING OUT THE INVENTION

[0013] Hereinafter, embodiments will be described in detail with reference to the drawings. The drawings are schematically illustrated, and configurations are omitted or simplified as appropriate for convenience of description. In the following description, the same reference numerals are given to the same components in the drawings, and the names and functions thereof are also the same. Therefore, detailed description thereof may be omitted to avoid duplication.

[0014] 1. First Embodiment <Dynamic Map Update System> FIG. 1 is a perspective view showing a configuration of a dynamic map update system 900 according to the first embodiment. In FIG. 1, a roadside apparatus 100, a roadside apparatus 300 and the like are provided in the vicinity of an intersection.

[0015] The roadside apparatus 100 is equipped with an image sensor as a first sensor 101, and the roadside apparatus 300 is equipped with LiDAR as a third sensor 301. A vehicle 200, which is an autonomous driving vehicle, is present before the intersection. The vehicle 200 is equipped with LiDAR as a second sensor 201.

[0016] Attempts are made to detect a pedestrian 5 and a vehicle 6 on the left side of the intersection by the first sensor 101, the second sensor 201, and the third sensor 301. Object information is generated based on detection data detected by each sensor. The object information includes the position and attributes of the detected object, and information about the type of sensor is also added.

[0017] The object attributes indicate whether the object is a structure (building, sign, guardrail, etc.), a vehicle (vehicle with three or more wheels), a motorcycle, a person, or something else (fallen object, animal, etc.). Strictly speaking, the first sensor 101 and the third sensor 301 mounted on the roadside device also detect vehicles 200, but this will be omitted from the explanation here.

[0018] The first sensor 101, the second sensor 201, the third sensor 301, and the dynamic map update device 500 are connected by wired or wireless communication. Together, they constitute the dynamic map update system 900.

[0019] <Configuration of sensors and dynamic map update device> Figure 2 is a block diagram showing the configuration of a dynamic map update system according to Embodiment 1. In Figure 2, a first sensor 101 mounted on a roadside device 100 and a second sensor 201 mounted on a vehicle 200 are shown as examples connected to the dynamic map update device 500. The combinations that constitute the dynamic map update system 900 are not limited to these. Sensors mounted on vehicles do not have to be included in the configuration. Also, sensors mounted on roadside devices do not have to be included in the configuration. Typically, sensor data is acquired from sensors mounted on a large number of roadside devices and a large number of vehicles.

[0020] Since the first sensor 101 and the second sensor 201 in Figure 2 have the same configuration, we will explain using the first sensor 101 as a representative example. The first detection unit 102 is the specific sensing device.

[0021] The first object information generation unit 103 analyzes the signal detected by the first detection unit 102 and calculates or determines the position (coordinates), size, speed, direction of movement, and attributes of the detected object. The first object information generation unit 103 further adds information such as the type of sensor, the distance from the sensor to the object, the sensor position (coordinates), the sensor ID, and the detection time to generate object information. The first object information transmission unit 104 then transmits the object information to the receiving unit 501 of the dynamic map update device 500. The same explanation applies to the second sensor 201 (the explanation of the second object information generation unit 203 and the second object information transmission unit 204 is omitted).

[0022] <Types of sensors> Sensors such as image sensors, radio wave sensors, optical sensors, and ultrasonic sensors can be used. Image sensors, as exemplified by surveillance cameras, capture images of objects and calculate the distance to the object from the image data captured within a certain field of view. The image data can also be used to determine the size, direction of movement, speed of movement, and attributes of the object. Visible light cameras and infrared cameras can be used as image sensors.

[0023] Radio wave sensors can include millimeter-wave radar (MMWR) and other types that utilize the 24-79 GHz frequency band. These sensors can detect the position of an object and, using the Doppler effect, the object's velocity.

[0024] Optical sensors such as laser radar and LiDAR can be used. LiDAR emits laser light within a certain field of view and detects point cloud data obtained from the reflection of laser light from objects, thereby determining the position and shape of objects.

[0025] Information processing is performed for each sensor that grasps the external environment, such as image sensors, radio wave sensors, optical sensors, and ultrasonic sensors. This allows the system to process the data acquired by various sensors and transmit only the information about the identified object (e.g., relative position, object attributes, etc.) to the dynamic map update device 500.

[0026] As sensors, image sensors, radio wave sensors, optical sensors, or ultrasonic sensors may be used, but information from multiple sensors may also be used simultaneously. In addition, sensors other than those mentioned above may be used to understand the external environment.

[0027] <Dynamic Map Update> Dynamic maps, also known as dynamic maps, are defined in the "Public-Private ITS Concept and Roadmap" published by the Digital Agency (ITS (Intelligent Transport System) is an advanced road traffic system being developed by the government, industry, and academia under an international organization). Their introduction is also being promoted by companies such as Dynamic Map Platform Co., Ltd., which is funded by various domestic automobile manufacturers. A dynamic map refers to real-time road information that adds dynamic information such as surrounding vehicles and pedestrians to a static, high-precision three-dimensional map. A dynamic map is a database as digital infrastructure consisting of (1) static information such as buildings, signs, and lanes; (2) semi-static information such as traffic regulations and road construction plans; (3) semi-dynamic information such as accident information and congestion information; and (4) dynamic information such as vehicles and pedestrians. The dynamic map update device 500 builds and updates dynamic maps for specific regions and specific users as a local database.

[0028] In the dynamic map update device 500, object information is extracted for each object by the object information extraction unit 503 from the object information received by the receiving unit 501. Here, if the position of an object detected by the first sensor and the position of an object detected by the second sensor are less than or equal to a predetermined determination distance, they are treated as object information relating to the same object. Here, in order to treat multiple pieces of object information as relating to the same object, it may be required that the attributes of the object information be the same or have similar attributes.

[0029] Then, based on the attributes of the object and the type of sensor extracted from the object information, and more preferably based on the distance between the sensor and the object, the reliability calculation unit 504 calculates the reliability. Of the calculated reliability values, the object information with the highest reliability among the reliability values ​​calculated for the same object is determined.

[0030] The dynamic map update unit 506 updates the dynamic map 507 with the most reliable object information. The necessary information from the updated dynamic map 507 is transmitted to the autonomous vehicle via the transmission unit 502.

[0031] <Hardware configuration of the dynamic map update system> Figure 3 is a hardware configuration diagram of the dynamic map update device 500 according to Embodiment 1. The hardware configuration diagram in Figure 3 can also be applied to the first sensor 101, the second sensor 201, and the third sensor 301. Here, we will describe the case where it is applied to the dynamic map update device 500 as a representative example. In this embodiment, the dynamic map update device 500 is an electronic control device that updates dynamic maps. Each function of the dynamic map update device 500 is realized by the processing circuit provided in the dynamic map update device 500. Specifically, the dynamic map update device 500 includes a processing circuit such as a CPU (Central Processing Unit) or other arithmetic processing unit 90 (computer), a storage device 91 that exchanges data with the arithmetic processing unit 90, an input circuit 92 that inputs external signals to the arithmetic processing unit 90, and an output circuit 93 that outputs signals from the arithmetic processing unit 90 to the outside. Each piece of hardware such as the arithmetic processing unit 90, storage device 91, input circuit 92, and output circuit 93 is connected to each other by a wired network such as a bus or a wireless network.

[0032] The arithmetic processing unit 90 may include an ASIC (Application Specific Integrated Circuit), an IC (Integrated Circuit), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), various logic circuits, and various signal processing circuits. Furthermore, multiple arithmetic processing units 90 of the same or different types may be provided, with each unit performing a portion of the processing. The storage device 91 may include a RAM (Random Access Memory) configured to read and write data from the arithmetic processing unit 90, or a ROM (Read-only Memory) configured to read data from the arithmetic processing unit 90. The storage device 91 may also include non-volatile or volatile semiconductor memory such as flash memory, SSD (Solid State Drive), EPROM, EEPROM, magnetic disks, flexible disks, optical disks, compact disks, minidiscs, DVDs, etc. The input circuit 92 is connected to various sensors, switches, and communication lines, and includes an A / D converter, communication circuit, etc., which inputs the output signals and communication information of these sensors and switches to the arithmetic processing unit 90. The output circuit 93 includes a drive circuit, communication circuit, etc., which outputs control signals from the arithmetic processing unit 90. The interfaces of the input circuit 92 and output circuit 93 may be based on specifications such as CAN (Control Area Network) (registered trademark), Ethernet (registered trademark), USB (Universal Serial Bus) (registered trademark), and DVI (Digital Visual Interface) (registered trademark). In addition, communication may be performed by directly connecting the arithmetic processing unit 90 to a communication device, separate from the input circuit 92 and output circuit 93.

[0033] Each function of the dynamic map update device 500 is realized by the arithmetic processing unit 90 executing software (programs) stored in a storage device 91 such as ROM, and cooperating with other hardware of the dynamic map update device 500, such as the storage device 91, input circuit 92, and output circuit 93. Setting data such as thresholds and judgment values ​​used by the dynamic map update device 500 are stored in the storage device 91 such as ROM as part of the software (program). Each function of the dynamic map update device 500 may be composed of software modules, or it may be composed of a combination of software and hardware.

[0034] <Object information> Figure 4 shows an example of object information according to Embodiment 1. Object information is generated by the object information generation unit for each sensor each time an object is detected. When a sensor detects an object, its position (geographic coordinates), size, speed, and direction are confirmed. The sensor then determines the attributes of the object based on its size, shape, speed, and location.

[0035] Figure 4 shows an example of attribute classification, such as 1: Vehicles (vehicles with three or more wheels), 2: People, 3: Two-wheeled vehicles, 4: Others, and 9: Structures. The attribute classification is not limited to what is shown in Figure 4. For example, separate attributes may be established for three-wheeled or four-wheeled saddle-type vehicles.

[0036] "9: Structures" includes buildings, signs, guardrails, lane markings, etc., but these are recorded as static objects on the high-precision 3D map. Therefore, structures detected by sensors can be identified on the dynamic map and are not treated as dynamic objects requiring special attention. "4: Others" are objects that cannot be classified into 1 to 3 or 9. Fallen objects, animals, etc., are possible examples.

[0037] The object information includes details about the type of sensor. Sensor types are classified as follows: 1) Image sensors (visible light or infrared cameras), 2) LiDAR, 3) Radar, and 4) Others. However, the classification of sensor types is not limited to those shown in Figure 4. Depending on the sensor type, object attributes, and distance from the sensor to the object, it becomes possible to determine the reliability of the object's detection position.

[0038] The distance from the sensor may be added to the object information. The reliability of the object's detection position can be determined based on the distance from the sensor to the object. In Figure 4, the distance from the sensor to the object is included as part of the object information; however, if the sensor's position (coordinates) is provided, the distance between it and the object's position can be calculated.

[0039] The object information may include the time of object detection, sensor ID, and sensor position (coordinates). Clearly defining the object's position and the sensor's position allows for confirmation of the sensor's field of view. In particular, if the sensor is an image sensor, the direction and time the sensor is facing can be used to determine if the captured image is backlit. The items in the object information are not limited to those shown in Figure 4. Items in the object information may be deleted or added.

[0040] <Reliability> Figure 5 shows an example of the reliability of object information according to Embodiment 1. In Figure 5, the reliability R is set according to the object's attributes, the type of sensor, and the distance from the sensor to the object. The reliability R is expressed as a percentage and indicates the certainty of the detected object's position. The numerical value of the reliability R in Figure 5 is an example and may be determined by theoretical calculations using a model. Alternatively, an appropriate value may be determined by experimentation.

[0041] In the confidence levels shown in Figure 5, the confidence level R for the image sensor is set to the highest when the object's attribute is a person. LiDAR and radar can measure the distance to an object with great accuracy. However, for objects with a small projection area and complex shapes, such as people, it is easier to identify their attributes from images. It is difficult to measure the shape and determine the attributes of objects with a small projection area and complex shapes, such as people, using LiDAR and radar.

[0042] In contrast, when the object is a vehicle, LiDAR and radar generally prove more reliable than image sensors. This is because objects with a large projection area, such as vehicles, are easier to identify using LiDAR and radar.

[0043] For short distances of 0-7m, the reliability R of the image sensor is highest. This is because the ranging accuracy of both LiDAR and radar decreases at short distances. In contrast, for medium distances of 7-30m and long distances of 30m or more, the reliability R of both LiDAR and radar is higher. In particular, the reliability of radar is higher at long distances. For materials with a large projection area and that easily reflect radio waves, such as vehicles, measurement using radar, which utilizes radio wave reflection, is advantageous.

[0044] When the object's attribute is a motorcycle, the confidence level R between a person and a vehicle is determined by the relationship between the motorcycle's projected area and material. However, when the attribute is something else, the projected area may be smaller than a person's, or the height from the road may be lower. Therefore, the overall confidence level R is lower. In particular, radar may not be able to detect reflected waves, resulting in the lowest confidence level R.

[0045] The above describes a case where the reliability R of image sensor detection increases when the distance from the sensor to the object is short. However, since the reliability R changes depending on the sensor's installation location and the surrounding environment in which the object exists, the reliability R may be adjusted according to the sensor's installation location and the environment in which the object exists.

[0046] Figure 5 illustrates the case where the confidence level R is set according to the object's attributes, the type of sensor, and the distance from the sensor to the object. Here, if the confidence level R is set without using the distance from the sensor to the object, the confidence level R in the distance column (7-30m) in Figure 5 should be used as a representative value. In this way, the confidence level R can be obtained without using the distance from the sensor to the object.

[0047] <Changes in the confidence coefficient over time> Figures 6 to 9 show the change in the coefficient K1 over time. Figure 6 is an example of the daytime coefficient for object information according to Embodiment 1. Figure 7 is an example of the sunrise coefficient for object information. Figure 8 is an example of the sunset coefficient for object information. Figure 9 is an example of the nighttime coefficient for object information.

[0048] The coefficient K1 shown in Figures 6 to 9 is an example; a separate model can be created to determine a theoretical value, or an appropriate value can be determined experimentally to set the coefficient. The confidence level, which varies over time, can be obtained by multiplying the coefficient K1 by the confidence level R defined in Figure 5.

[0049] As shown in Figure 6, during the daytime, the detection results from any sensor are unaffected, so all coefficients K1 are set to 1.0. In contrast, as shown in Figure 9, at night, the detection accuracy of the image sensor decreases. In particular, when using a visible light camera, it becomes difficult to detect objects unless the image is of an area illuminated by streetlights or vehicle headlights. Therefore, the coefficient K1 is set to 0.2.

[0050] Since LiDAR and radar are not affected by ambient light, the coefficient K1 is set to 1.0. Furthermore, when using an infrared camera as the image sensor, object detection can be effectively performed even at night by using an infrared light source in conjunction with the camera.

[0051] As shown in Figure 7, it is assumed that the sun is low in the sky for 3 hours from sunrise. If 3 hours is too long, it may be adjusted in 10-minute increments. If the camera is facing east, backlighting makes it difficult for the image sensor to detect objects. For this reason, the coefficient K1 is set to 0.2.

[0052] If the camera is facing west, it will be illuminated by sunlight from the front, allowing for good object detection. Therefore, the coefficient K1 is set to 1.0. If the camera is facing north or south, the sun is low in the sky, and the contrast between the sunlit and unlit sides of an object becomes intense. This may cause problems with object detection by the image sensor. Therefore, the coefficient K1 is set to 0.5. LiDAR and radar are not affected by ambient light, so the coefficient K1 is set to 1.0.

[0053] As shown in Figure 8, sunset is defined as occurring three hours before the sun is low in the sky. Three hours may be too long and can be adjusted in 10-minute increments. If the camera is facing west, backlighting makes it difficult for the image sensor to detect objects. For this reason, the coefficient K1 is set to 0.2.

[0054] If the camera is facing east, it will be illuminated by sunlight from the front, allowing for good object detection. Therefore, the coefficient K1 is set to 1.0. If the camera is facing north or south, the sun is low in the sky, and the contrast between the sunlit and unlit sides of an object becomes intense. This may cause problems with object detection by the image sensor. Therefore, the coefficient K1 is set to 0.5. LiDAR and radar are not affected by ambient light, so the coefficient K1 is set to 1.0.

[0055] In the above example, the coefficient K1 based on time was set for four categories: daytime, nighttime, sunrise, and sunset. However, it is also possible to set it for more detailed categories. Alternatively, it may be set for only two categories: daytime and nighttime.

[0056] <Changes in the confidence coefficient due to weather> Figures 10 to 12 show the change in the coefficient K2 due to weather conditions. Figure 10 is a diagram showing an example of the coefficient for sunny / cloudy weather in object information according to Embodiment 1. Figure 11 is a diagram showing an example of the coefficient for light rain / light snow in object information. Figure 12 is a diagram showing an example of the coefficient for heavy rain / heavy snow / fog in object information.

[0057] The coefficient K2 shown in Figures 10 to 12 is an example; a separate model can be created to determine a theoretical value, or an appropriate value can be determined experimentally to set the coefficient. The confidence level, which varies with weather conditions, can be obtained by multiplying the coefficient K2 by the confidence level R defined in Figure 5.

[0058] As shown in Figure 10, the detection results from any sensor are unaffected when the weather is sunny or cloudy. Therefore, all coefficients K2 are set to 1.0.

[0059] As shown in Figure 11, the detection accuracy of the image sensor decreases slightly when the weather is light rain or snow. For this reason, the coefficient K2 is set to 0.8. LiDAR and radar are not affected by light rain or snow, so the coefficient K2 is set to 1.0.

[0060] As shown in Figure 12, the detection accuracy of image sensors decreases significantly when the weather is heavy rain, heavy snow, or fog. For this reason, the coefficient K2 is set to 0.2. Similarly, for LiDAR, the laser light is blocked by heavy rain, heavy snow, or fog, and the detection accuracy decreases significantly. For this reason, the coefficient K2 is set to 0.2. For radar, the radio waves are partially affected by heavy rain, heavy snow, or fog, so the coefficient K2 is set to 0.5.

[0061] Figure 13 is a diagram showing the similarity between object attributes of object information according to Embodiment 1. In Figure 13, the main attribute is indicated by ● and similar objects are indicated by ○.

[0062] In other words, the attributes of an object are person, two-wheeled vehicle, vehicle (three wheels or more), and other. The attribute "person" is similar to "two-wheeled vehicle" and "other," the attribute "two-wheeled vehicle" is similar to "person" and "vehicle," the attribute "vehicle" is similar to "two-wheeled vehicle," and the attribute "other" is similar to "person." The similarities and dissimilarities between attributes are defined in Figure 13.

[0063] Here, if the locations of object information detected by multiple sensors are close together and the object attributes are similar, those objects can be considered identical, and their reliability can be compared. Then, the dynamic map can be updated using the object information with the higher reliability. This prevents accidentally updating the dynamic map with information from a different object. Furthermore, even if attribute determination fails, the frequency of dynamic map updates can be improved by using the object information instead of discarding it.

[0064] <Processing by the Dynamic Map Update System> Figure 14 is a flowchart showing the processing of the dynamic map update device 500 according to Embodiment 1. The processing shown in Figure 14 is executed by the arithmetic processing unit of the dynamic map update device 500. This processing may be executed at predetermined intervals (for example, every 1 ms). Alternatively, it may be executed in response to events such as when the dynamic map update device 500 acquires object information from a sensor, rather than at predetermined intervals.

[0065] The process shown in Figure 14 is initiated, and in step S101, object information is acquired from the first sensor 101. In step S102, object information is acquired from the second sensor 201.

[0066] In step S103, object information is extracted for each object from the acquired object information. Here, "for each object" means object information for the same object. To ensure that the objects are the same, it may be determined that the object information is for the same object if the difference in the position of the object information based on detection data from multiple sensors is less than or equal to a predetermined judgment distance.

[0067] Furthermore, objects may be determined to be identical only if the attributes of the object information, based on detection data from multiple sensors, match or are similar. The determination of attribute similarity or dissimilarity shall follow the similarity determination shown in Figure 13. In this case, to make the conditions stricter, objects may be determined to be identical only if their attributes match.

[0068] Furthermore, to confirm that the objects are identical, it may be confirmed that the difference in the object's position within the object information remains below a predetermined judgment distance for a predetermined judgment time or longer. Only if this condition persists for a judgment time or longer will it be determined that the object information pertains to the same object. By doing so, it is possible to prevent incorrect judgments from being made regarding different objects whose positions are temporarily overlapped due to the influence of noise or other factors.

[0069] In step S105, the confidence level is calculated for each of the extracted object information values ​​for the same object. The confidence level R defined in Figure 5 is used. The confidence level may be calculated using only the confidence level R defined in Figure 5. Alternatively, a coefficient K1 that changes with time may be determined by referring to Figures 6 to 9. Alternatively, a coefficient K2 that changes with weather may be determined by referring to Figures 10 to 12. The final confidence level Rt is calculated using confidence level R, confidence level R × coefficient K1, confidence level R × coefficient K2, or confidence level R × coefficient K1 × coefficient K2.

[0070] In step S106, the confidence levels of object information for the same object are compared. In step S107, the object information with the highest confidence level is considered correct, and the dynamic map is updated using that object information.

[0071] In step S108, dynamic map information is provided to the required autonomous vehicles, etc., and the process is then terminated.

[0072] As described above, when detection data from multiple sensors for the same object exists, the optimal detection data can be selected according to the object's attributes and the type of sensor, and the dynamic map can be updated using that data. Therefore, the dynamic map can be updated using the most accurate and reliable data. As a result, a dynamic map update device 500 can be obtained that provides highly accurate and reliable dynamic map information to support autonomous driving.

[0073] Furthermore, a dynamic map update system 900 can be configured with a first sensor 101 mounted on the roadside device 100, a second sensor 201 mounted on the vehicle 200, and a dynamic map update device 500. The dynamic map update system 900 can select the optimal detection data according to the attributes of the object and the type of sensor when detection data from multiple sensors exists, and update the dynamic map with that data.

[0074] Furthermore, a receiving step is performed to receive object information from the sensor, which includes the position and attributes of the object generated based on the sensor's detection data, and the type of sensor is added. The receiving step extracts object information for each object from the object information received from multiple sensors, A confidence calculation step that calculates the confidence level based on the object attributes and sensor type of the object information extracted by the object information extraction step, and A dynamic map update method can be obtained that includes a dynamic map update step which updates the dynamic map using the location of the object information with the highest reliability for each object, as calculated by the reliability calculation step. This dynamic map update method allows the dynamic map to be updated using the most accurate and reliable data. As a result, highly accurate and reliable dynamic map information can be provided to support autonomous driving.

[0075] 2. Embodiment 2 Figure 15 is a block diagram showing the configuration of the dynamic map update system 900 according to Embodiment 2. The reason why the block diagram in Figure 15 differs from the block diagram in Figure 2, which is according to Embodiment 1, is that The only difference is that an object information integration unit 508 is provided between the object information extraction unit 503 and the reliability calculation unit 504 of the dynamic map update device 500.

[0076] Figure 16 is a flowchart showing the processing of the dynamic map update device 500 according to Embodiment 2. The flowchart in Figure 16 differs from that of Figure 14, which is the flowchart according to Embodiment 1, in that a step S104 related to the integration of object information is added between steps S103 and S105. Figures 1, 3 through 14 are also applicable to Embodiment 2.

[0077] The difference between the processing in Embodiment 2 and Embodiment 1 lies in the handling of object information in step S103, when object information is extracted for each object from the acquired object information, specifically when the attributes of the objects are similar or when the attributes of the objects could not be acquired by the sensor.

[0078] In Embodiment 2, when object information is generated based on detection data from multiple sensors, regardless of the similarity of the object's attributes or the presence or absence of attributes, if the difference in the object's position is less than or equal to a predetermined determination distance, it is determined that the object information is for the same object. The object information integration unit 508 then transfers the object attributes from the object information of sensors that were able to acquire the object's attributes, or sensors of a type that are more suitable for acquiring the object's attributes, to the object information of sensors that were unable to acquire the object's attributes or sensors that acquired different attributes.

[0079] Considering the ease of recognizing object attributes, image sensors are the best at recognizing attributes. LiDAR is the next best at recognizing attributes. Radar is at a disadvantage in its ability to grasp the fine shape of objects and has the lowest attribute recognition ability.

[0080] In other words, object information acquired from multiple sensors is integrated. In step S103 of the flowchart in Figure 16, regardless of the object's attributes, object information is extracted for objects whose positional difference is less than or equal to a predetermined determination distance.

[0081] Then, in step S104, the object information integration unit 508 integrates object information for each object. Specifically, it sets the object attributes of the sensor that can acquire the object attributes more accurately as the positive value and rewrites the object attributes of the other object information. After that, the process proceeds to step S105.

[0082] As described above, by rewriting the attributes of objects in the object information, the dynamic map update device 500 can appropriately update the dynamic map. As a result, highly accurate and reliable dynamic map information can be provided to support autonomous driving.

[0083] While this disclosure describes various exemplary embodiments and examples, the various features, aspects, and functions described in one or more embodiments are not limited to the application of a particular embodiment, but are applicable individually or in various combinations to the embodiments. Accordingly, countless variations not illustrated are envisioned within the scope of the art disclosed in this specification. For example, these include modifying, adding or omitting at least one component, or even extracting at least one component and combining it with a component from another embodiment.

[0084] The various aspects of this disclosure are summarized below as an appendix.

[0085] (Note 1) A receiving unit that receives object information from the sensor, which includes the position and attributes of the object generated based on the sensor's detection data, and the type of the sensor is added to the object information. An object information extraction unit extracts object information for each object from the object information received by the receiving unit from a plurality of sensors. A reliability calculation unit calculates a reliability score based on the attributes of the object and the type of sensor of the object information extracted by the object information extraction unit, and A dynamic map update device comprising a dynamic map update unit that updates the dynamic map based on the location of the object information that has the highest reliability calculated by the reliability calculation unit for each object. (Note 2) The dynamic map update device according to Appendix 1, wherein the object information extraction unit determines that the object information is for the same object when the difference in the position of the object based on the detection data of the multiple sensors is less than or equal to a predetermined determination distance. (Note 3) Multiple sensors are part of a dynamic map update device as described in Appendix 1 or 2, deployed around the area in which the autonomous vehicle is traveling. (Note 4) The dynamic map update device described in Appendix 2, which, if the object information extraction unit determines that the object information pertains to the same object, has unknown attributes of the object in one of the object pieces, uses the attributes of the object in another object piece. (Note 5) The aforementioned sensor types include image sensors and LiDAR, The attribute of the object is a dynamic map update device as described in any one of the appendices 1 to 4, including the case of a person and the case of a vehicle. (Note 6) The dynamic map update device according to Appendix 5, wherein the reliability calculation unit calculates the reliability of the object information based on the image sensor detection data to be higher than the reliability of the object information based on the LiDAR detection data when the attribute of the object is a person. (Note 7) The dynamic map update device according to Appendix 5, wherein the reliability calculation unit calculates the reliability of the object information based on the LiDAR detection data to be higher than the reliability of the object information based on the image sensor detection data when the attribute of the object is a vehicle. (Note 8) The reliability calculation unit calculates the reliability according to the distance between the position of the sensor and the position of the object in the dynamic map update device according to any one of the appendices 1 to 7. (Note 9) The object information extraction unit determines that the object information is for the same object if the difference in the position of the object based on the detection data of the multiple sensors is less than or equal to a predetermined determination distance and the attributes of the object are the same. This is a dynamic map update device according to any one of the appendices 1 to 8. (Note 10) The attributes of the object include persons, two-wheeled vehicles, vehicles with three or more wheels, and other cases, wherein the person with the attribute is similar to the two-wheeled vehicle and the other, the two-wheeled vehicle with the attribute is similar to the person and the vehicle with three or more wheels, and the vehicle with three or more wheels with the attribute is similar to the two-wheeled vehicle. The object information extraction unit determines that the object information is for the same object if the difference in the position of the object based on the detection data of a plurality of sensors is less than or equal to a predetermined determination distance and the attributes of the object are the same or similar. This is a dynamic map update device according to any one of the appendices 1 to 8. (Note 11) The object information extraction unit determines that the object information is for the same object if the difference in the position of the object based on the detection data of a plurality of sensors continues to be less than or equal to a predetermined determination distance for a predetermined determination time or longer, as described in any one of the appendices 1 to 10. (Note 12) The reliability calculation unit calculates the reliability based on the attributes of the object and the type of sensor extracted by the object information extraction unit, and the current time, according to any one of the appendices 1 to 11 of the dynamic map update device. (Note 13) The reliability calculation unit calculates the reliability based on the attributes of the object and the type of sensor extracted by the object information extraction unit, and whether it is currently day or night, according to any one of the appendices 1 to 12. (Note 14) The reliability calculation unit calculates the reliability based on the attributes of the object extracted by the object information extraction unit, the type of sensor, the current time, and the direction of the sensor, according to any one of the appendices 1 to 13. (Note 15) The reliability calculation unit calculates the reliability based on the attributes of the object and the type of sensor extracted by the object information extraction unit, as well as the weather, according to any one of the appendices 1 to 14. (Note 16) A first object information generation unit generates first object information that includes the position and attributes of an object generated based on detection data from a first sensor, and adds the type of the sensor to the first object information; A first sensor having a first object information transmission unit that transmits the first object information generated by the first object information generation unit, A second object information generation unit generates second object information, including the object's position, attributes, and sensor type, based on detection data from a second sensor. A second sensor having a second object information transmission unit that transmits the second object information generated by the second object information generation unit, and A receiving unit that receives the first object information transmitted from the first object information transmitting unit and the second object information transmitted from the second object information transmitting unit, An object information extraction unit extracts the first object information and the second object information received by the receiving unit for each object, A reliability calculation unit calculates a reliability score based on the attributes of the object and the type of sensor of the object information extracted by the object information extraction unit, A dynamic map update system comprising a dynamic map update device having a dynamic map update unit that updates the dynamic map based on the location of the object information that has the highest reliability for each object, as calculated by the reliability calculation unit. (Note 17) The first sensor is mounted on a roadside device, The second sensor is the dynamic map update system described in Appendix 16, which is mounted on the vehicle. (Note 18) A receiving step of receiving object information from the sensor, which includes the position and attributes of the object generated based on the sensor's detection data, and the type of the sensor is added to the object information. Object information extraction step, which extracts object information for each object from the object information received from the plurality of sensors in the receiving step, A reliability calculation step that calculates the reliability based on the attributes of the object and the type of sensor of the object information extracted by the object information extraction step, and A dynamic map update method comprising a dynamic map update step, which updates the dynamic map based on the location of the object information that has the highest reliability calculated by the reliability calculation step for each object. [Explanation of Symbols]

[0086] 5 Pedestrian, 6, 200 Vehicle, 100, 300 Roadside device, 101 First sensor, 103 First object information generation unit, 104 First object information transmission unit, 201 Second sensor, 203 Second object information generation unit, 204 Second object information transmission unit, 301 Third sensor, 500 Dynamic map update device, 501 Receiving unit, 503 Object information extraction unit, 508 Object information integration unit, 504 Reliability calculation unit, 506 Dynamic map update unit, 900 Dynamic map update system

Claims

1. A receiving unit that receives object information from the sensor, which includes the position and attributes of the object generated based on the sensor's detection data, and the type of the sensor is added to the object information. An object information extraction unit extracts object information for each object from the object information received by the receiving unit from a plurality of sensors. A reliability calculation unit calculates a reliability score based on the attributes of the object and the type of sensor of the object information extracted by the object information extraction unit, and A dynamic map update device comprising a dynamic map update unit that updates the dynamic map based on the location of the object information that has the highest reliability calculated by the reliability calculation unit for each object.

2. The dynamic map update device according to claim 1, wherein the object information extraction unit determines that the object information is of the same object when the difference in the position of the object based on the detection data of the plurality of sensors is less than or equal to a predetermined determination distance.

3. The dynamic map update device according to claim 1, wherein the plurality of sensors are deployed around the area in which the autonomous vehicle is driving.

4. The dynamic map update device according to claim 2, wherein if the object information extraction unit determines that the object information is for the same object, and the attributes of one of the object pieces of object information are unknown, the attributes of the object from another object piece of object information are used.

5. The type of sensor includes both image sensors and LiDAR sensors. The dynamic map update device according to claim 1, wherein the attributes of the object include the case of a person and the case of a vehicle.

6. The dynamic map update device according to claim 5, wherein the reliability calculation unit calculates the reliability of the object information based on the detection data of the image sensor to be higher than the reliability of the object information based on the detection data of the LiDAR when the attribute of the object is a person.

7. The dynamic map update device according to claim 5, wherein the reliability calculation unit calculates the reliability of the object information based on the LiDAR detection data to be higher than the reliability of the object information based on the image sensor detection data when the attribute of the object is a vehicle.

8. The dynamic map update device according to claim 1, wherein the reliability calculation unit calculates the reliability according to the distance between the position of the sensor and the position of the object.

9. The dynamic map update device according to any one of claims 1 to 8, wherein the object information extraction unit determines that the object information is for the same object when the difference in the position of the object based on the detection data of the plurality of sensors is less than or equal to a predetermined determination distance and the attributes of the object are the same.

10. The attributes of the object include persons, two-wheeled vehicles, vehicles with three or more wheels, and other cases, wherein the person with the attribute is similar to the two-wheeled vehicle and the other, the two-wheeled vehicle with the attribute is similar to the person and the vehicle with three or more wheels, and the vehicle with three or more wheels with the attribute is similar to the two-wheeled vehicle. The dynamic map update device according to any one of claims 1 to 8, wherein the object information extraction unit determines that the object information is for the same object when the difference in the position of the object based on the detection data of the plurality of sensors is less than or equal to a predetermined determination distance and the attributes of the object are the same or similar.

11. The dynamic map update device according to any one of claims 1 to 8, wherein the object information extraction unit determines that the object information is of the same object if the difference in the position of the object based on the detection data of the plurality of sensors is less than or equal to a predetermined determination distance for a predetermined determination time or longer.

12. The dynamic map update device according to any one of claims 1 to 8, wherein the reliability calculation unit calculates the reliability based on the attributes of the object and the type of sensor of the object information extracted by the object information extraction unit, and the current time.

13. The dynamic map update device according to any one of claims 1 to 8, wherein the reliability calculation unit calculates the reliability based on the attributes of the object and the type of sensor of the object information extracted by the object information extraction unit, and whether it is currently day or night.

14. The dynamic map update device according to any one of claims 1 to 8, wherein the reliability calculation unit calculates the reliability based on the attributes of the object extracted by the object information extraction unit, the type of sensor, the current time, and the direction of the sensor.

15. The dynamic map update device according to any one of claims 1 to 8, wherein the reliability calculation unit calculates the reliability based on the attributes of the object and the type of sensor of the object information extracted by the object information extraction unit, and the weather.

16. A first object information generation unit generates first object information that includes the position and attributes of an object generated based on detection data from a first sensor, and adds the type of the sensor to the first object information; A first sensor having a first object information transmission unit that transmits the first object information generated by the first object information generation unit, A second object information generation unit generates second object information, including the object's position, attributes, and sensor type, based on detection data from a second sensor. A second sensor having a second object information transmission unit that transmits the second object information generated by the second object information generation unit, and A receiving unit that receives the first object information transmitted from the first object information transmitting unit and the second object information transmitted from the second object information transmitting unit, An object information extraction unit extracts the first object information and the second object information received by the receiving unit for each object, A reliability calculation unit calculates a reliability score based on the attributes of the object and the type of sensor of the object information extracted by the object information extraction unit, A dynamic map update system comprising a dynamic map update device having a dynamic map update unit that updates the dynamic map based on the location of the object information that has the highest reliability for each object, as calculated by the reliability calculation unit.

17. The first sensor is mounted on a roadside device, The dynamic map update system according to claim 16, wherein the second sensor is mounted on a vehicle.

18. A receiving step of receiving object information from the sensor, which includes the position and attributes of the object generated based on the sensor's detection data, and the type of the sensor is added to the object information. Object information extraction step, which extracts object information for each object from the object information received from the plurality of sensors in the receiving step, A reliability calculation step that calculates the reliability based on the attributes of the object and the type of sensor of the object information extracted by the object information extraction step, and A dynamic map update method comprising a dynamic map update step, which updates the dynamic map based on the location of the object information that has the highest reliability calculated by the reliability calculation step for each object.

Images