Unmanned driving environment perception and navigation system based on digital twinning technology

Abstract

The application relates to an unmanned driving environment perception and navigation system based on a digital twinborn technology and belongs to the technical field of unmanned driving environment perception. The system comprises a headquarters data center, a regional data center and a plurality of environment perception field servers, the environment perception field servers are arranged near road regions, receive the probe rod data of the road regions, encode the probe rods according to road phases and adjacency relations to establish continuous sequence relations, and establish the vehicle information relations of corresponding road sections according to the sequence relations of the probe rods; the regional data center establishes corresponding relations according to the road phases and the adjacency relations of the data of the plurality of environment perception field servers and sends the data to the headquarters data center, and the headquarters data center establishes the adjacency relations of the data of the plurality of regional data centers. Vehicles perceive the surrounding road environment based on the road condition basic data and the road condition advanced data sent by the system, so that environment perception and real-time navigation planning are realized.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving environment perception technology, specifically an autonomous driving environment perception and navigation system based on digital twin technology. Background Technology

[0002] Existing autonomous vehicles utilize onboard multiple sensors (cameras, LiDAR, millimeter-wave radar, and ultrasonic radar) to perceive their surroundings. However, the high cost of LiDAR currently makes it difficult to deploy in economical vehicles. The detection accuracy and range of millimeter-wave radar are directly related to its size and power specifications; limited by the autonomous vehicle's power supply, payload, and size, high-precision, long-range millimeter-wave radar is difficult to deploy in conventional passenger and engineering vehicles. Cameras are relatively inexpensive and technologically mature, but they are susceptible to the influence of their own line of sight and ambient light. Obstacles, darkness, fog, and other special scenarios can affect the accuracy of environmental perception in autonomous vehicles. Ultrasonic radar is inexpensive, but its detection range is short and its accuracy is limited. Therefore, existing autonomous driving environmental perception technologies are constrained by several factors, resulting in bottlenecks in detection accuracy and range, and creating a contradiction between cost and benefit in practical deployment.

[0003] A search revealed a Chinese invention patent: "A Motion Planning Method for Autonomous Driving Based on Digital Twins" (Publication No.: CN113359709A, Publication Date: 2021-09-07). This application's method includes: acquiring environmental data and vehicle posture data surrounding the vehicle; constructing a digital twin driving scenario based on the environmental and vehicle posture data; generating multiple emergency event simulation driving scenarios, and planning multiple corresponding planning results for each scenario; performing real-time matching between the digital twin driving scenario and the multiple emergency event simulation driving scenarios to obtain a matched emergency event simulation driving scenario; and transmitting the planning results corresponding to the matched emergency event simulation driving scenario to the vehicle. By pre-generating multiple emergency event simulation driving scenarios and corresponding motion planning results, and directly matching and synchronizing them to the vehicle, the sensitivity of the autonomous driving system in the face of emergency events is improved, enhancing the safety and reliability of the autonomous driving system. However, this application has a drawback: it is easily affected by inaccurate environmental data acquisition, and improving accuracy involves excessively high equipment costs. Summary of the Invention

[0004] 1. The technical problem that the invention aims to solve

[0005] The purpose of this invention is to solve the problems that existing autonomous vehicles collect environmental data through on-board sensors, which are easily affected by inaccurate environmental data collection, and that improving accuracy is too costly.

[0006] 2. Technical Solution

[0007] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0008] This invention discloses an unmanned driving environment perception and navigation system based on digital twin technology, comprising a headquarters data center, regional data centers, and several environmental perception field servers. The environmental perception field servers are located near road areas and receive data from detector poles in those areas. They encode the detector poles according to road phase and adjacency relationships to establish a continuous sequence relationship, and then establish vehicle information relationships for corresponding road segments based on this sequence relationship. Each environmental perception field server establishes vehicle information relationships for corresponding road segments based on the detector pole sequence relationship. For each road segment, vehicle information is used to establish a basic road condition data table with time and contour coordinates, and a higher-level road condition data table with time, contour coordinates, and vehicle type. This information is then sent to the regional data center. The regional data center establishes a correspondence between the data from multiple environmental perception field servers according to road phase and adjacency relationships and sends the data to the headquarters data center. The headquarters data center then establishes adjacency relationships between the data from the several regional data centers.

[0009] Preferably, the detection pole is installed on both sides of the road where vehicles travel or in the construction area, and the top of the detection pole is equipped with a detection device, which includes a lidar and / or a camera and / or a millimeter-wave radar.

[0010] Preferably, the road is divided into several road segments according to the traffic order, and each road segment is equipped with a detection pole. The detection areas of the detection poles in adjacent road segments have overlapping areas. The camera of the detection pole captures the entire area of ​​its own road segment or the overlapping area with the adjacent road segment.

[0011] Preferably, the environmental perception field server establishes a correlation between multiple detection rods in different areas according to the collection area, and further synthesizes the point cloud data collected by the lidar on several detection rods. Specifically, it stitches the point cloud data maps of multiple areas according to the coordinates of static fixed scene markers, and synthesizes them into a panoramic view cloud data map. Static and dynamic objects are displayed in real time in the panoramic view cloud data map.

[0012] Preferably, the system uses an image recognition algorithm to identify and classify image data to obtain road condition and vehicle model data. It processes the image data by taking screenshots of different times, uses the image recognition algorithm to classify the vehicle model distribution at a certain time, and combines it with the point cloud map at the same time to establish a vehicle fusion correspondence and vehicle model classification according to the coordinates and position distribution order of the reference object, thereby realizing vehicle model classification in the point cloud data.

[0013] Preferably, the system also communicates with the autonomous vehicle in real time. The system sends basic road condition data and advanced road condition data to the autonomous vehicle in motion. The autonomous vehicle has two ways to obtain basic road condition data and advanced road condition data: one is based on its own coordinates and only receives road condition data within its own radius; the other is based on the location of the road segment and submits a viewing request. The traffic digital twin service system sends the corresponding data based on the request.

[0014] Preferably, the computer of the autonomous vehicle receives and processes digital twin logical data, and fuses it with the environmental perception sensor data of the vehicle itself, and finally makes a decision on the specific operation of the vehicle.

[0015] The size of the detection overlap area of ​​the detector pole is set based on the width of the road phase lane, the length of the road that can accommodate the maximum number of vehicles, the single detection time of the detector pole, and the detection cycle, using nd. i Instead of the number of straight lanes, use nl i Instead of the number of left-turn lanes, use nr i Instead of the number of right-turn lanes, the width W of the overlapping area of ​​the road segment is... OS The following conditions must be met

[0016] W OS ≥nd i ·w d +nl i ·w l +nr i ·w r .

[0017] Preferably, when the lane is a two-way road, the overlapping area can be calculated and set according to the overlapping method of a one-way road, that is, a two-way road segment is detected by two sets of detectors in each direction, or a single detector is used to cover the lanes in both directions.

[0018] When using a single probe, then W OS It should meet the following conditions,

[0019] W OS ≥2(nd i ·w d +nl i ·w l +nr i ·wr );

[0020] Let the width of the driving and emergency parking area of ​​the one-way road be Rw. s The driving and emergency parking areas on both sides of the road are Rw dual ;

[0021] When a detector pole covers a section of one-way road, then W OS It should meet the following conditions,

[0022] nd i ·w d +nl i ·w l +nr i ·w r ≤W OS ≤Rw s ;

[0023] When a probe covers a section of a two-way road, then W OS It should meet the following conditions,

[0024] 2(nd i ·w d +nl i ·w l +nr i ·w r )≤W OS ≤Rw dual .

[0025] Preferably, the length L of the overlapping area of ​​the road segment OS It should meet the following conditions,

[0026] RSP j ×(fTP i +SP i )+VLmax v ≤L OS ≤k pa [RSP j ×(fTP i +SP i )+VLmax v ],1≤k pa ≤2;

[0027] Where k pa It is the overlap area length adjustment coefficient, which is the maximum length of the road that can accommodate vehicles, VLmax. v This refers to the maximum vehicle length allowed to travel on this road section, and the single detection time (SP) of the detector pole. i The detection period (fTP) refers to the time consumed to scan the information of vehicles in the coverage area once. iThis refers to the interval at which the detection rod takes a reading, and the maximum speed limit for that section of road is RSP. j .

[0028] 3. Beneficial effects

[0029] Compared with the prior art, the technical solution provided by this invention has the following advantages:

[0030] This invention discloses an unmanned driving environment perception and navigation system based on digital twin technology, comprising a headquarters data center, regional data centers, and several environmental perception field servers. The environmental perception field servers are located near road areas and receive data from detector poles in those areas. They encode the detector poles according to road phase and adjacency relationships to establish a continuous sequence relationship, and then establish vehicle information relationships for corresponding road segments based on this sequence relationship. The environmental perception field servers establish vehicle information relationships for corresponding road segments based on the detector pole sequence relationship. For each road segment, vehicle information is used to establish a basic road condition data table with time and contour coordinates, and a higher-level road condition data table with time, contour coordinates, and vehicle type. This information is then sent to the regional data center. The regional data center establishes a correspondence between the data from multiple environmental perception field servers according to road phase and adjacency relationships and sends the data to the headquarters data center. The headquarters data center then establishes adjacency relationships between the data from the several regional data centers. By installing detector poles on the road to acquire vehicle and road segment data, and establishing a correspondence between the vehicle and other vehicles on the road segment through a dynamic tracking algorithm associated with the detector poles, the system can filter out the vehicle's specific location in the road condition information. Finally, based on the basic and advanced road condition data sent by the traffic digital twin service system, the system can perceive the surrounding road environment, thereby achieving environmental perception and real-time navigation planning. At the same time, vehicles can also send requests to view road condition information in areas other than their own coordinates, thereby planning navigation routes. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the structure of an unmanned driving environment perception and navigation system based on digital twin technology according to the present invention.

[0032] Figure 2 This is a schematic diagram of the deployment of the vehicle driving road and road condition cloud data acquisition device of the present invention;

[0033] Figure 3 This is a schematic diagram of the communication architecture of the vehicle and traffic digital twin service system of the present invention;

[0034] Figure 4 This is a schematic diagram of the operational logic architecture of the autonomous driving navigation system based on digital twin technology of the present invention.

[0035] Explanation of the labels in the diagram:

[0036] 100. Headquarters Data Center; 200. Regional Data Center; 300. Environmental Sensing Field Servers;

[0037] 1. Camera; 2. LiDAR 1; 3. LiDAR 2; 4. Vehicle;

[0038] A. Road data acquisition equipment for detection poles; B. Traffic data twin service system; C. Communication base station; D. Vehicle-mounted module. Detailed Implementation

[0039] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0040] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0042] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0043] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linking," and "socketing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral structure; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0044] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] Example 1

[0046] Reference Figure 1 This embodiment of an unmanned driving environment perception and navigation system based on digital twin technology includes a headquarters data center 100, a regional data center 200, and several environmental perception field servers 300. The environmental perception field servers 300 are located near road areas and receive data from detector poles in those areas. They encode the detector poles according to road phase and adjacency relationships to establish a continuous sequence relationship, and use this sequence relationship to establish vehicle information relationships for corresponding road segments. The environmental perception field servers 300 establish vehicle information relationships for corresponding road segments based on the detector pole sequence relationship. For each road segment, vehicle information is used to establish a basic road condition data information table with time and contour coordinates, and a high-level road condition data information table with time, contour coordinates, and vehicle type. This information is then sent to the regional data center 200. The regional data center 200 establishes a corresponding relationship between the data from multiple environmental perception field servers according to road phase and adjacency relationships and sends the data to the headquarters data center 100. The headquarters data center 100 then establishes adjacency relationships between the data from the several regional data centers.

[0047] The detection pole is installed on both sides of the road where vehicles travel or in the construction area. The top of the detection pole is equipped with a detection device, which includes a lidar and / or a camera and / or a millimeter-wave radar.

[0048] The road is divided into several sections according to the traffic flow order. Each section is equipped with a detection pole, and the detection areas of the detection poles in adjacent sections overlap. The camera of the detection pole captures the entire area of ​​its section or the overlapping area with the adjacent section.

[0049] The environmental perception field server 300 establishes a correlation between multiple detection rods in different areas according to the collection area, and further synthesizes the point cloud data collected by the lidar on several detection rods. Specifically, it stitches together the point cloud data maps of multiple areas according to the coordinates of static fixed scene markers, and synthesizes them into a panoramic view cloud data map. Static and dynamic objects are displayed in real time in the panoramic view cloud data map.

[0050] The system uses image recognition algorithms to identify and classify image data to obtain road condition and vehicle model data. It processes image data by taking screenshots at different times, and the image recognition algorithm classifies the vehicle model distribution at a certain time. Combined with the point cloud map at the same time, it establishes vehicle fusion correspondence and vehicle model classification according to the coordinates and position distribution order of reference objects, thereby realizing vehicle model classification in point cloud data.

[0051] The system also communicates with autonomous vehicles in real time, sending basic road condition data and advanced road condition data to the autonomous vehicles in motion. The autonomous vehicles have two ways to obtain basic road condition data and advanced road condition data: one is based on their own coordinates, only receiving road condition data within their own radius; the other is based on submitting a viewing request based on the location of the road segment, and the traffic digital twin service system sends the corresponding data based on the request.

[0052] The computer of an autonomous vehicle receives and processes digital twin logical data, and fuses it with the vehicle's own environmental perception sensor data to ultimately make decisions on the specific control of the vehicle; the digital twin simulation animation data is provided to the safety driver, co-driver, and passengers of the autonomous vehicle for viewing, or to advanced driver assistance vehicles as a reference.

[0053] The system operation method includes the following steps:

[0054] S100, Obtain data;

[0055] S200, panoramic stitching;

[0056] S300, Identification and Classification;

[0057] S400, data processing yields basic road condition data and advanced road condition data;

[0058] S500 sends basic and advanced road condition data to autonomous vehicles in motion.

[0059] S600, constructing digital twin animation data;

[0060] The S700 autonomous vehicle computer receives and processes digital twin logical data.

[0061] The method of this invention uses detector poles on the road to acquire vehicle and road segment data, and establishes a correspondence between the vehicle and the vehicles on the road segment through a dynamic tracking algorithm associated with the detector poles. From this, the specific location of the vehicle in the road condition information is selected. Finally, based on the basic road condition data and advanced road condition data sent by the traffic digital twin service system, the surrounding road condition environment is perceived, thereby realizing environmental perception and real-time navigation planning. At the same time, the vehicle can also send a request to view the road condition information in areas other than its own coordinates, thereby planning a navigation route.

[0062] The specific data acquisition in step S100 is as follows:

[0063] Detection poles are installed along roadsides or in construction areas. Each pole has a detection device at its top, including a lidar and / or camera and / or millimeter-wave radar. These poles are deployed along roadsides or in construction areas such as coal mines, oil fields, construction sites, and container loading / unloading areas in ports. The effective detection range of the detection device covers a predetermined radius or width / length area of ​​the road or construction area. The size of this radius or width / length area is based on the detection distance and accuracy parameters of the detection device, while also ensuring no blind spots in the detection angle. In this embodiment, a coverage area A is defined. cover It is a circle, i.e., with a radius of r. det The area S of the circular region rdet S rdet =πr det 2 If the coverage area is of length L det Width is W det Rectangle, area S of the rectangular region LWdet S LWdet =L×W; Let the effective detection accuracy of the probe be acc. det Within any area (of any shape) covered by a probe, the effective detection accuracy (acc) is satisfied. det The set of coordinate points is A acc ={(x i ,y j )|i=1,2,…; j=1,2,…}, A acc It can be a region of any geometric shape. From A acc Cut a circular S-shaped piece in the middle rdet Or rectangle S LWdet The area refers to the coverage area A mentioned above. cover Circular S rdet Or rectangle S LWdet The maximum value is A. cover The optimal coordinate region.

[0064] The panoramic stitching in step S200 specifically involves:

[0065] Multiple probes from different areas are linked according to their data collection areas. The point cloud data collected by the lidar on several probes are then synthesized. Specifically, the point cloud data maps of multiple areas are stitched together according to the coordinates of static fixed scene markers to create a panoramic view cloud data map. Both static and dynamic objects are displayed in real time on the panoramic view cloud data map.

[0066] The data fusion algorithm for vehicle type classification and dynamic tracking in step S300 specifically includes:

[0067] Image recognition algorithms are used to identify and classify image data to obtain road condition and vehicle model data. Image data is processed by taking screenshots of different times to capture images. Image recognition algorithms classify the vehicle model distribution at a certain time and combine it with the point cloud map at the same time. According to the coordinates and position distribution order of the reference object, the vehicle fusion correspondence and vehicle model classification are established to realize vehicle model classification in point cloud data.

[0068] Image data is classified using image recognition algorithms (such as convolutional neural networks). Vehicles in the image data are categorized according to specific vehicle types (trucks, sedans, SUVs, pickups, road cleaning vehicles, garbage trucks, hazardous materials tankers, fire trucks, police cars, ambulances; excavators, bulldozers, cargo trucks, loaders). The classified data is called road condition vehicle type data. Image data is processed using time-series screenshots to capture images at different times. The image recognition algorithm classifies the vehicle type distribution at a specific moment and combines it with the point cloud image at the same moment. A vehicle fusion correspondence and vehicle type classification are established based on the coordinates and positional distribution order of reference objects, thus achieving vehicle type classification in the point cloud data. The screenshot time or selected image is determined based on the vehicle type's ability to be identified. For example, images are captured when vehicles are stationary while waiting at traffic lights or moving at low speeds to ensure accurate image recognition.

[0069] When the detection accuracy of lidar or cameras is limited by environmental factors (such as foggy days, uneven road lighting under streetlights, or darkness), millimeter-wave radar data is used to continuously track vehicle coordinates. Lidar data, millimeter-wave radar data, and camera data are combined at the same coordinates at the same time (when lidar, millimeter-wave radar, and camera are all capable of providing accurate detection) to determine the vehicle type. Further vehicle tracking is then performed using lidar or millimeter-wave radar. The data after vehicle type identification is also known as road condition and vehicle type data.

[0070] The specific step S400 is as follows:

[0071] The point cloud data collected by the lidar is processed to extract objects that pose obstacles to road travel, i.e., obstacles. The obstacles are marked according to their overhead outline (or overhead projection outline of the obstacle on the ground) and coordinate position. The obstacle data with marked coordinates is the basic road condition data. The road condition vehicle data obtained in step S300 is fused with the basic road condition data to assign vehicle type information to the vehicle outline in the basic road condition data. This data is the advanced road condition data.

[0072] The specific steps S500 are as follows:

[0073] The regional data center's traffic digital twin service system sends basic road condition data and advanced road condition data to autonomous vehicles in motion. The autonomous vehicles have two ways to obtain basic road condition data and advanced road condition data: one is based on their own coordinates, only receiving road condition data within their own radius; the other is based on submitting a viewing request based on the road segment location, and the traffic digital twin service system sends the corresponding data based on the request.

[0074] The specific step S600 is as follows:

[0075] Construct digital twin animation data by converting the cloud data map of the entire scenic spot into a digital twin simulation animation based on digital twin technology, reflecting the vehicle outline size and model; at the same time, construct digital twin logical data, namely vehicle model, outline size, and relative coordinate position; design a digital twin simulation vehicle library to build a more comprehensive classification of simulation vehicle data.

[0076] The specific step S700 is as follows:

[0077] The computer of an autonomous vehicle receives and processes digital twin logical data, and fuses it with the vehicle's own environmental perception sensor data to ultimately make decisions on the specific control of the vehicle; the digital twin simulation animation data is provided to the safety driver, co-driver, and passengers of the autonomous vehicle for viewing, or to advanced driver assistance vehicles as a reference.

[0078] It also includes a dynamic tracking algorithm for sensor pole association used to acquire road condition cloud data, specifically:

[0079] The road is divided into several segments according to traffic flow. Each segment has a detection pole, and the detection areas of the detection poles in adjacent segments overlap. The camera of each detection pole captures the entire area of ​​its segment or the overlapping area with the adjacent segment. The camera collects vehicle data at coordinate positions within the overlapping area and uses this data for vehicle type identification. Simultaneously, the camera in the overlapping area of ​​the next adjacent segment collects data on the vehicle and calculates its outline and coordinates. Based on the camera data recognition results, the vehicle type is assigned to the coordinate outline. When the vehicle moves to the next segment, the lidar of the next segment continues to track it, constantly recording the vehicle's coordinate outline data and maintaining the previously detected vehicle type data.

[0080] The size of the overlapping detection area of ​​the detector pole is set based on the width of the road phase lane, the maximum vehicle length the road can accommodate, and the single detection time and detection cycle of the detector pole. A phase lane here refers to a lane on the same road that allows vehicles to travel in the same direction, including straight, left, and right turns. The width of a phase lane refers to the sum of the widths of all straight, left, and right turns within the same phase lane (referred to as the "sum width"). The width of the overlapping area should be greater than or equal to the sum width of the road segment.

[0081] Assume a road segment is a one-way road, including: two straight lanes d1 and d2, two left-turn lanes l1 and l2, and one right-turn lane r1. The width of d1 and d2 is w. d The widths of l1 and l2 are both w l The width of r1 is w r Then the width W of the overlapping area of ​​this road segment OS The following conditions must be met, W OS ≥2w d +2w l +w r To further generalize, use nd. i Instead of the number of straight lanes, use nl i Instead of the number of left-turn lanes, use nr i Instead of the number of right-turn lanes, W OS The following conditions must be met, W OS ≥nd i ·w d +nl i ·w l +nr i ·w r .

[0082] When the road is a two-way road (the road is divided in the middle, with opposite directions of travel on both sides), the overlapping area can be calculated and set in the same way as a one-way road, that is, two sets of detectors are used to detect each direction of the two-way road segment, or one detector can be used to cover the lanes in both directions. If one detector is used, then W OS The following conditions must be met, W OS ≥2(nd i ·w d +nl i ·w l +nr i ·w r Let Rw be the width of the driving and emergency parking area on a one-way road. s The driving and emergency parking areas on both sides of the road are Rw dual When a detector pole covers a section of a one-way road, then W OS It should meet the following conditions, nd i ·w d +nl i ·w l +nr i ·w r ≤W OS ≤Rw s When a probe covers a section of a two-way road, then W OS It should meet the following conditions, 2(nd i ·w d +nl i ·w l +nr i ·w r )≤W OS ≤Rw dual The maximum length of the road that can accommodate vehicles, VLmax. v This refers to the maximum vehicle length allowed to travel on this road section, and the single detection time (SP) of the detector pole. i The detection period (fTP) refers to the time consumed to scan the information of vehicles in the coverage area once. i This refers to how often the probe takes a reading.

[0083] The maximum speed limit on this section of road is RSP. j The length L of the overlapping area of ​​this road segment OS RSP should meet the following conditions. j ×(fTP i +SP i )+VLmax v ≤L OS ≤k pa [RSP j ×(fTP i +SP i)+VLmax v ],1≤k pa ≤2,k pa It is the overlap region length adjustment coefficient, k pa The specific value is determined by a regression method that randomly sets a number and observes its actual effect, or by training a machine learning algorithm to obtain the optimal value based on the lowest repetition rate of all vehicles passing through the road segment.

[0084] First, the coordinates of the vehicle requesting traffic information are obtained. Then, the coordinates of all vehicles within a radius around the road segment to which the vehicle belongs are obtained. Finally, the coordinates of the vehicle are merged with the coordinates of all vehicles around the road segment to establish a correspondence between the vehicle and the vehicles in the road segment. From this, the specific location of the vehicle in the traffic information is filtered out. Finally, based on the basic and advanced traffic data sent by the traffic digital twin service system, the surrounding traffic environment is perceived, thereby realizing environmental perception and real-time navigation planning. At the same time, the vehicle can also send a request to view traffic information in areas other than its own coordinates, thereby planning a navigation route.

[0085] Dynamic target tracking refers to establishing the positional relationship of the target and obtaining its trajectory in a continuous data sequence. Based on the target's positional features in the previous frame, it predicts the target's position and contour size in the next frame. Currently known target tracking algorithms based on LiDAR point cloud data are mainly divided into two categories: generative and discriminative. Traditional filtering algorithms belong to generative tracking, while the latest deep learning algorithms belong to discriminative tracking. The following section details various tracking algorithms and the problems they encounter in point cloud target tracking. Generative tracking involves building a target model or extracting target features. It uses the appearance representation of the target model to perform a similarity search in the real-time scene, using the closest appearance representation as the criterion to lock similar targets within a certain error range. Discriminative tracking extracts targets with composite target model features by comparing the differences between the target model and scene information, thus obtaining the target's position in the real-time scene. Currently, generative algorithms mainly include mean shift, Kalman filtering, particle filtering, and kernel correlation filtering. Mean shift is based on probability density distribution, ensuring that the target search always follows the direction of increasing probability gradient, iteratively converging to the peak of the probability distribution. Kalman filtering obtains the observed position of the target through detection or other methods, predicts the current position based on the position information from the previous moment according to the motion equation, and then calculates a weighted average of the observation and prediction results to obtain the final target position information. Particle filtering models the target based on particle distribution and defines a similarity metric to determine the degree of matching with the target. Then, particles are scattered according to a Gaussian distribution, and their similarity is statistically analyzed to determine the possible position information of the target. Kernel correlation filtering designs a filter template and performs correlation operations between the template and the target candidate region; the position of the maximum output response is the position information of the current frame. A typical algorithm for discriminative tracking is deep learning, which uses convolutional neural networks to extract the target from the background and leverages its powerful feature abstraction ability and ability to fit the target's motion process to achieve the tracking task. Mean shift and Kalman filtering algorithms are simple to implement, but parameter tuning is cumbersome and robustness is generally poor. Filtering algorithms have fast tracking speeds, while deep learning methods have high accuracy but require certain hardware resources.

[0086] Based on the aforementioned known discriminant and generative algorithms, this invention also proposes a dynamic tracking algorithm for detector pole association, serving as a specific implementation algorithm for acquiring road condition cloud data through multi-sensor linkage. Specifically:

[0087] Suppose a road is divided into four segments, A, B, C, and D, in sequence. Vehicles travel from A to B, then to C, and finally to D (A→B→C→D). Detector poles P1, P2, P3, and P4 are deployed sequentially on segments A, B, C, and D. Each detector pole is responsible for collecting road condition data for its segment; detector pole A uses a lidar (or a combination of lidar and millimeter-wave radar) to track the outlines and coordinates of vehicles on segment A, and so on. The detection ranges of two detector poles on any two adjacent segments partially overlap. Following the vehicle's travel order (A→B→C→D), the overlapping area is D. AB D BC D CD Overlapping region D AB D BC D CD There must be overlap in the detection of lidar or millimeter-wave radar on two corresponding road segment detectors, and at least one road segment detector must have a camera deployed on it. If there is only the vehicle phase from A to B to C and finally to D (A→B→C→D), then road segment A or D... AB The overlapping area must be covered by the camera on the detection pole in section A. Assume that in section D... AB In the overlapping area, the camera collects vehicle data at coordinate positions. This data is then used for vehicle type identification at those coordinate positions. Vehicle type identification is calculated using the aforementioned discriminant or generative (or a combination of both) algorithm and according to the classification method (truck, sedan, SUV, pickup, road cleaning vehicle, garbage truck, hazardous materials tanker, fire truck, police car, ambulance; excavator, bulldozer, transport vehicle, loader). Simultaneously, the two (or two sets of) lidar sensors P1 and P2 are also operating in D... AB Once a vehicle at the specified coordinates is detected and its outline and coordinates are calculated, the vehicle type is assigned based on the camera's data recognition results. When the vehicle moves towards road segment B, the LiDAR on P2 continues to track it, continuously recording the vehicle's coordinate outline data and maintaining the previously detected vehicle type data. This process is repeated for road segments C and D; that is, the detection poles on associated road segments continuously detect and track vehicles, and the same vehicle is tracked across the entire road segment; a single camera data acquisition provides vehicle type information applicable across all road segments.

[0088] like Figure 1As shown, the environmental perception field server 300 is deployed near the road area and receives data from the detectors in that area. It then establishes a continuous sequence relationship between the detectors based on road phase and adjacency, and uses this sequence relationship to establish vehicle information relationships for corresponding road segments. For each road segment, a basic road condition data table is created using time and contour coordinates, and an advanced road condition data table is created using time, contour coordinates, and vehicle type. The environmental perception field server 300 sends the basic and advanced road condition data tables to the regional data center 200. The regional data center 200 is responsible for establishing corresponding relationships between the data from multiple environmental perception field servers 300 based on road phase and adjacency. Similarly, the headquarters data center 100 establishes adjacency relationships between the data from several regional data centers 200. Simultaneously, the headquarters data center 100 deploys a large data storage array responsible for storing all traffic condition data.

[0089] In addition, the fusion detection system running on the environmental perception field server 300 is responsible for sending road condition data to vehicles in the area. Simultaneously, the traffic digital twin service system running on the regional data center 200 is responsible for sending cross-regional road condition information to the corresponding environmental perception field server 300, while the traffic digital twin service system running on the headquarters data center 100 is responsible for cross-regional (larger area) road condition data correlation. The traffic digital twin service system is also responsible for calculating vehicle dynamic tracking algorithms and vehicle type recognition algorithms.

[0090] Reference Figure 2 , Figure 2 This is a schematic diagram of the deployment of cloud data collection equipment for vehicle driving routes and road conditions. Line segments ad and eh represent the two edges of the road, and the dashed line ij represents the lane lines, dividing the road into two lanes where traffic flows from left to right. Number 1 represents a camera, numbers 2 and 3 represent LiDAR 1 and LiDAR 2 respectively, and number 4 represents a vehicle. LiDAR 1 is responsible for collecting vehicle data in the acge area of ​​the road segment, while LiDAR 2 is responsible for collecting vehicle data in the bdhf area. The bcgf area is the overlapping area collected by LiDAR 1 and LiDAR 2. The two sets of data are fused based on a reference object as the reference coordinates, and real-time vehicle type (including size and outline) and coordinate data for the road segment are established based on a vehicle type database. The vehicle type of vehicle 4 is thus determined. When vehicle 4 travels to the bcgf area, the data from LiDAR 1 and LiDAR 2 are fused based on the vehicle's coordinates. When vehicle 4 enters the cdhg area, its vehicle type data is simultaneously added to the data collection for that road segment, eliminating the need for camera-based identification.

[0091] Reference Figure 3The data acquisition device A (LiDAR data, camera data, millimeter-wave radar) of the road detection pole acquires data and sends it to the digital twin service system B. The digital twin service system B fuses the LiDAR data and camera data, and then converts the fused data into 3D animation data and environmental simplified data. It also fuses the road segment coordinates and vehicle coordinates. The fused data is sent to the vehicle module D through the communication base station C. By default, the vehicle module D obtains the road condition information of its surroundings.

[0092] Reference Figure 4 The process involves several steps: Step S1 is to obtain the vehicle's coordinates; Step S2 is to obtain the coordinates of all vehicles on the road segment; Step S3 is to fuse the vehicle's coordinates with those of all vehicles on the road segment, establishing a correspondence between the vehicle and the vehicles on the road segment; and Step S4 is for the vehicle to perceive its surrounding environment based on data from the digital twin service system. Specifically, the vehicle first obtains the coordinates of the vehicle requesting road condition information, then obtains the coordinates of all vehicles within a radius around the road segment it belongs to, and further fuses these coordinates with those of all surrounding vehicles to establish a correspondence between the vehicle and the vehicles on the road segment. From this, the vehicle's specific location in the road condition information is determined. Finally, based on the basic and advanced road condition data from the digital twin service system, the vehicle perceives the surrounding road environment, thereby achieving environmental perception and real-time navigation planning. Simultaneously, the vehicle can also send a request to view road condition information in areas other than its own coordinates, thus planning a navigation route.

[0093] This invention converts road condition cloud data into simplified environmental data. Simplified environmental data is road condition dynamic data that can be quickly processed by the vehicle's computer system. Road condition dynamic data refers to descriptive data of vehicles, pedestrians, and unidentified objects relative to the vehicle's coordinates. Specifically, the descriptive data refers to the relative coordinates of vehicles, pedestrians, and unidentified objects in the environment surrounding the driving road, as well as the vehicle's size outline and model data.

[0094] This invention fuses the surrounding environment data (road scene objects, vehicles, pedestrians) detected by the autonomous vehicle itself (onboard sensors) with simplified environmental data to generate a merged environmental perception data (hereinafter referred to as merged perception data) divided by coordinates. Then, the merged perception data is fused with a high-precision map to generate a panoramic view of the surrounding environment centered on the vehicle itself. This panoramic view changes dynamically in real time.

[0095] This invention converts multiple panoramic images of the surrounding environment in a time sequence into simulated road condition data, i.e., environmental perception 3D animation, and integrates the vehicle's own position coordinates with the animation coordinates. The environmental perception 3D animation is provided for drivers or passengers to view, serving as an aid to driving and viewing road conditions.

[0096] The above-described embodiments are merely illustrative of certain implementations of the present invention, and are described in a relatively specific and detailed manner. However, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements are all within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. An unmanned driving environment perception and navigation system based on digital twin technology, characterized in that: It includes a headquarters data center (100), a regional data center (200) and several environmental perception field servers (300). The environmental perception field servers (300) are located near the road area and receive the detection rod data of the road area. They encode the detection rods according to the road phase and adjacency relationship to establish a continuous sequence relationship, and establish the vehicle information relationship of the corresponding road segment based on the detection rod sequence relationship. The environmental perception field server (300) establishes the vehicle information relationship of the corresponding road segment based on the sequence relationship of the detector rods. The vehicle information of each road segment establishes a basic road condition data information table based on time and contour coordinates, and establishes an advanced road condition data information table based on time, contour coordinates, and vehicle type. The data is then sent to the regional data center (200). The regional data center (200) establishes a corresponding relationship between the data from multiple environmental perception field servers according to road phase and adjacency relationship and sends the data to the headquarters data center (100). The headquarters data center (100) establishes an adjacency relationship between the data from several regional data centers. The road is divided into several sections according to traffic flow, and each section has one detection pole. The detection areas of adjacent sections overlap. The camera of each detection pole captures the entire area of ​​its section or the overlapping area with adjacent sections. The size of the overlapping detection area is set based on the width of the road phase lanes, the maximum vehicle length the road can accommodate, the single detection time of the detection pole, and the detection cycle. Instead of the number of straight lanes, use Instead of the number of left-turn lanes, use Instead of the number of right-turn lanes, the width of the overlapping area of ​​road segments is... The following conditions must be met ； When the lanes are two-way roads, the overlapping area can be calculated and set according to the overlapping method of one-way roads. That is, a two-way road segment can be detected by two sets of detectors in each direction, or a single detector can be used to cover lanes in both directions. When using a single probe, then It should meet the following conditions, ； Assume the width of the driving and emergency parking area on a one-way road is... The driving and emergency parking areas on both sides of the road are ; When a detector pole covers a section of one-way road, then It should meet the following conditions, ； When a detector pole covers a section of a two-way road, then It should meet the following conditions, ； in, The width of all , The width of all The width is ; The length of the overlapping area of ​​the road segment It should meet the following conditions, ， ； in It is the overlap area length adjustment coefficient, which is the maximum length of the road that can accommodate. This refers to the maximum vehicle length allowed to travel on this road section, and the single detection time of the detector pole. This refers to the time consumed in one detection and completion of a scan of the vehicles in the coverage area; the detection cycle. This refers to the interval at which the detection rod takes a reading, and the maximum speed limit for that section of road. .

2. The unmanned driving environment perception and navigation system based on digital twin technology according to claim 1, characterized in that: The detection pole is installed on both sides of the road where vehicles travel or in the construction area. The top of the detection pole is equipped with a detection device, which includes a lidar and / or a camera and / or a millimeter-wave radar.

3. The unmanned driving environment perception and navigation system based on digital twin technology according to claim 1, characterized in that: The environmental perception field server (300) establishes a correlation between multiple detection rods in different areas according to the collection area, and further synthesizes the point cloud data collected by the lidar on several detection rods. Specifically, it stitches together the point cloud data maps of multiple areas according to the coordinates of static fixed scene markers, and synthesizes them into a panoramic view cloud data map. Static and dynamic objects are displayed in real time in the panoramic view cloud data map.

4. The unmanned driving environment perception and navigation system based on digital twin technology according to claim 1, characterized in that: The system uses image recognition algorithms to identify and classify image data to obtain road condition and vehicle model data. It processes image data by taking screenshots at different times, and the image recognition algorithm classifies the vehicle model distribution at a certain time. Combined with the point cloud map at the same time, it establishes vehicle fusion correspondence and vehicle model classification according to the coordinates and position distribution order of reference objects, thereby realizing vehicle model classification in point cloud data.

5. The unmanned driving environment perception and navigation system based on digital twin technology according to claim 1, characterized in that: The system also communicates with autonomous vehicles in real time, sending basic road condition data and advanced road condition data to the autonomous vehicles in motion. The autonomous vehicles have two ways to obtain basic road condition data and advanced road condition data: one is based on their own coordinates, only receiving road condition data within their own radius; the other is based on submitting a viewing request based on the location of the road segment, and the traffic digital twin service system sends the corresponding data based on the request.

6. The unmanned driving environment perception and navigation system based on digital twin technology according to claim 5, characterized in that: The computer in an autonomous vehicle receives and processes digital twin logical data, and fuses it with the vehicle's own environmental perception sensor data to ultimately decide on the specific control of the vehicle.

Images