Method, device, equipment and storage medium for real-time update of high-precision map for autonomous driving

Abstract

The present disclosure provides a method, apparatus, device, and storage medium for real-time updating of an autonomous driving high-precision map, which relates to the fields of artificial intelligence technologies such as autonomous driving, deep learning, and high-precision maps, and can be applied to intelligent transportation and smart city scenarios. The method includes: obtaining road images and driving trajectories collected by in-vehicle sensors and roadside devices; inputting multi-source road information for discriminating road dynamic events, including road images and driving trajectories, into a dynamic event recognition model pre-deployed on the vehicle side; receiving the recognition result of the road dynamic event output by the dynamic event recognition model; and in response to determining that a target dynamic event occurs on a target road according to the recognition result, updating the operating design domain of the target road based on the target dynamic event. Applying this method can improve the response speed of autonomous driving services to road dynamic events, enhance the autonomous driving effect, and ensure the personal and property safety of passengers.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, specifically to artificial intelligence technical fields such as autonomous driving, deep learning, and high-precision maps, and can be applied to intelligent transportation and smart city scenarios. In particular, it relates to a method, device, electronic device, and computer-readable storage medium for real-time updating of high-precision maps for autonomous driving. Background Art

[0002] According to the statistics of the United Nations Global Road Safety Report, 20 million to 50 million people are injured in traffic accidents globally every year, resulting in an economic burden of approximately $1.85 trillion, causing serious economic losses to society and individuals, and plunging millions of people into poverty.

[0003] Therefore, the most urgent social, economic, and health problem of this era is the road safety problem. In the real world, roads where dynamic events (such as sudden traffic accidents, temporary road maintenance, and other abnormal events) occur will cause a large number of traffic accidents, seriously affecting the driving safety of users.

[0004] Since the life cycle of dynamic events is short and they may occur at any time and place, how to accurately and timely detect dynamic events is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] Embodiments of the present disclosure propose a method, device, electronic device, computer-readable storage medium, and computer program product for real-time updating of high-precision maps for autonomous driving.

[0006] In a first aspect, embodiments of the present disclosure propose a method for real-time updating of high-precision maps for autonomous driving, including: obtaining road images and driving trajectories collected by vehicle-mounted sensors and roadside devices; inputting multi-source road information for discriminating road dynamic events, including road images and driving trajectories, into a dynamic event recognition model pre-deployed on the vehicle side, where the dynamic event recognition model is used to represent the correspondence between multi-source road information and the confidence levels of various types of road dynamic events; receiving the recognition result of the road dynamic event output by the dynamic event recognition model; and in response to determining that a target dynamic event occurs on a target road according to the recognition result, updating the operating design domain of the target road based on the target dynamic event.

[0007] Second aspect, embodiments of the present disclosure propose a real-time update device for an autonomous driving high-precision map, including: an image and trajectory acquisition unit configured to acquire road images and driving trajectories collected by in-vehicle sensors and roadside devices; a multi-source road information input unit configured to input multi-source road information for discriminating road dynamic events, including road images and driving trajectories, into a dynamic event recognition model pre-deployed on the vehicle side, where the dynamic event recognition model is used to represent the correspondence between the multi-source road information and the confidence levels of various types of road dynamic events; a recognition result receiving unit configured to receive the recognition results of road dynamic events output by the dynamic event recognition model; and an operating design domain update unit configured to update the operating design domain of a target road based on a target dynamic event in response to determining that the target road has a target dynamic event according to the recognition results.

[0008] Third aspect, embodiments of the present disclosure provide an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to implement the real-time update method for an autonomous driving high-precision map as described in the first aspect.

[0009] Fourth aspect, embodiments of the present disclosure provide a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to implement the real-time update method for an autonomous driving high-precision map as described in the first aspect when executed.

[0010] Fifth aspect, embodiments of the present disclosure provide a computer program product including a computer program, and when the computer program is executed by a processor, it can implement the steps of the real-time update method for an autonomous driving high-precision map as described in the first aspect.

[0011] In the real-time update solution for an autonomous driving high-precision map provided by the present disclosure, on the vehicle side, road images and driving trajectories are respectively obtained from in-vehicle sensors and roadside devices, and a dynamic event recognition model pre-deployed on the vehicle side is used to identify dynamic events for multi-source road information including road images and driving trajectories, and the operating design domain for providing autonomous driving services is updated based on the target dynamic events existing on the identified target road, so as to improve the response speed of the autonomous driving service to road dynamic events by timely updating the operating design domain, enhance the autonomous driving effect, and ensure the personal and property safety of passengers.

[0012] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0013] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments read in conjunction with the accompanying drawings:

[0014] Figure 1 is an exemplary system architecture to which the present disclosure may be applied;

[0015] Figure 2 is a flowchart of a method for real-time updating of an autonomous driving high-precision map provided by an embodiment of the present disclosure;

[0016] Figure 3 is a flowchart of a method for an input multi-source road information to be recognized by a dynamic event recognition model to obtain a recognition result provided by an embodiment of the present disclosure;

[0017] Figure 4 is a flowchart of a method for determining whether a dynamic event exists on a target road according to a recognition result provided by an embodiment of the present disclosure;

[0018] Figure 5 is a flowchart of a method for initiating an authenticity verification request provided by an embodiment of the present disclosure;

[0019] Figure 6 is a flowchart of a method for updating an operating design domain provided by an embodiment of the present disclosure;

[0020] Figure 7a and Figure 7b and Figure 7c and Figure 7d are respectively schematic diagrams of the trajectories of roads with different abnormalities;

[0021] Figure 8 is a schematic diagram of a method for information extraction of a road information text provided by an embodiment of the present disclosure;

[0022] Figure 9 is a structural block diagram of an apparatus for real-time updating of an autonomous driving high-precision map provided by an embodiment of the present disclosure;

[0023] Figure 10 is a schematic structural diagram of an electronic device suitable for executing the method for real-time updating of an autonomous driving high-precision map provided by an embodiment of the present disclosure. Detailed Description of Specific Embodiments

[0024] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness. It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0025] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0026] Figure 1 An exemplary system architecture 100 is shown that can apply the embodiments of the method, apparatus, electronic device, and computer-readable storage medium for real-time updating of an autonomous driving high-precision map of the present disclosure.

[0027] As Figure 1 shown, the system architecture 100 may include an in-vehicle camera 101, an in-vehicle positioning component 102, roadside equipment 103, and a vehicle 104. Among them, the in-vehicle camera 101 and the in-vehicle positioning component 102 are used to provide the driving trajectory of the road image of the passing road and the surrounding area from the perspective of the vehicle; the roadside equipment 103 fixedly installed on the roadside can also obtain the road image data and the driving trajectory of the passing vehicles in the sensed area with the cameras and sensing devices thereon; since the in-vehicle camera 101 and the in-vehicle positioning component 102 are integrated on the vehicle 104, data transmission can be directly achieved by means of a wired method, and the roadside equipment 103 can send the acquired image data and driving trajectory to the vehicle 104 through a network.

[0028] The vehicle 104 can obtain road images and driving trajectories from the in-vehicle terminal 101, the in-vehicle positioning component 102, and the roadside equipment 103, and can further analyze and perform various processes on the acquired data. Specifically, various applications for implementing various functions can be installed on the in-vehicle camera 101, the in-vehicle positioning component 102, the roadside equipment 103, and the vehicle 104, such as road image acquisition applications, driving trajectory acquisition applications, autonomous driving high-precision map real-time update and processing applications, etc.

[0029] The in-vehicle camera 101 for collecting road images and the in-vehicle positioning component 102 for collecting driving trajectories may be in other forms in other application scenarios. For example, millimeter-wave radars can be used to collect road images, etc. Figure 1It exists only as an example attached drawing. Similarly, for the roadside device 103, as long as it can meet the functions of collecting road images and driving trajectories required by this disclosure, it is sufficient.

[0030] The vehicle 104 can provide various services through various built-in applications. Taking the real-time update of the autonomous driving high-precision map and the real-time update and processing applications of the autonomous driving high-precision map based on the recognition results and performing corresponding subsequent processing services based on multi-source road information including road images and driving trajectories as an example, when the server 105 runs the real-time update and processing applications of the autonomous driving high-precision map, the following effects can be achieved: First, obtain the road images and driving trajectories collected by the in-vehicle camera 101, the in-vehicle positioning component 102, and the roadside device; then, input the multi-source road information for discriminating road dynamic events, including road images and driving trajectories, into the dynamic event recognition model pre-deployed in the in-vehicle terminal or the in-vehicle brain of the vehicle 104, and this dynamic event recognition model is used to represent the corresponding relationship between the multi-source road information and the confidence levels of various types of road dynamic events; next, receive the recognition results of the road dynamic events output by the dynamic event recognition model; finally, in response to determining that a target dynamic event occurs on the target road according to the recognition results, update the operating design domain of the target road based on the target dynamic event.

[0031] The real-time update method of the autonomous driving high-precision map provided in the subsequent embodiments of this disclosure is generally executed by the vehicle 104 or the in-vehicle terminal or the in-vehicle brain representing the computing power of the vehicle. Correspondingly, the real-time update device of the autonomous driving high-precision map is generally also set in the vehicle 104.

[0032] It should be understood that Figure 1 the numbers of the in-vehicle cameras, the in-vehicle positioning components, the roadside devices, and the vehicles in

[0033] Please refer to Figure 2 , Figure 2 which is a flowchart of a real-time update method of the autonomous driving high-precision map provided by an embodiment of this disclosure, and the process 200 includes the following steps:

[0034] Step 201: Obtain the road images and driving trajectories collected by the in-vehicle sensors and the roadside device;

[0035] This step aims to obtain the in-vehicle sensors (such as Figure 1 the in-vehicle camera 101 and the in-vehicle positioning component 102 shown in Figure 1 ) and the roadside device (such as Figure 1The road images and driving trajectories collected by the roadside device 103 shown are obtained by the way of mutual cooperation between the "vehicle (i.e., the vehicle)" and the "road (i.e., the roadside device)", so as to obtain road images and driving trajectories with more perspectives, clearer content and higher accuracy.

[0036] Step 202: Input the multi-source road information for discriminating road dynamic events, including road images and driving trajectories, into the dynamic event recognition model pre-deployed on the vehicle side;

[0037] Among them, the dynamic event recognition model is used to represent the corresponding relationship between the multi-source road information and the confidence levels of various types of road dynamic events. The dynamic event recognition model can be pre-trained on the server side and then the trained model is deployed on the vehicle side, or the trained model is light-weighted (for example, using model distillation technology) and then deployed on the vehicle side; that is, different road dynamic events can be divided into multiple different categories, such as traffic accident categories (such as rear-end collisions, impacts, etc.), vehicle accident categories (such as spontaneous combustion, explosion, the items loaded roll down, violent chemical reactions occur, etc.), crowd gathering categories, road construction and maintenance categories, accidental natural phenomenon categories, etc.

[0038] Based on step 201, this step aims to input the multi-source road information for discriminating road dynamic events, including road images and driving trajectories, as input information into the dynamic event recognition model pre-deployed on the vehicle side, so as to determine through the dynamic event recognition model the confidence levels of the road information in the multi-source road information reflecting different types of road dynamic events. For example, the confidence level that the multi-source road information belongs to the road dynamic event of the traffic accident category is 52%, and the confidence level that it belongs to the road dynamic event of the vehicle accident category is 27%.

[0039] Step 203: Receive the recognition result of the road dynamic event output by the dynamic event recognition model;

[0040] Based on step 202, this step aims to receive the recognition result of the road dynamic event output by the dynamic event recognition model by the above-mentioned execution subject, that is, the recognition result includes the possible dynamic event categories considered (not necessarily including all dynamic event categories, for example, it may not include the dynamic event categories with a confidence level of 0) and the corresponding confidence levels of the corresponding categories.

[0041] Step 204: In response to determining that the target road generates a target dynamic event according to the recognition result, update the operating design domain of the target road based on the target dynamic event.

[0042] Based on the determination that the target road generates a target dynamic event according to the recognition result, this step aims to update the operating design domain of the target road by the above-mentioned execution subject based on the target dynamic event.

[0043] The Operational Design Domain (ODD), which stands for Operational Design Domain in full and is abbreviated as ODD, refers to the operating conditions specifically designed for a particular driving automation system or its functions, including but not limited to environmental, geographical, and time limitations, and / or the presence or absence of certain traffic or road characteristics. The automated driving can only ensure normal operation when all conditions are met. On the contrary, lacking any one of the prerequisite conditions, the system may malfunction and may take actions such as stopping or requiring the driver to take over.

[0044] Simply put, the ODD is to define under which working conditions automated driving is possible. Without these working conditions, automated driving cannot guarantee normal operation. Any automated driving vehicle must have certain limited working conditions. And this working condition can be very broad or very precise, and determines what scenarios the automated driving vehicle can handle. For example, the automated driving system of a vehicle can only be used on highways, where it can automatically maintain the lane, overtake automatically, follow the vehicle in front automatically, give way automatically, pass through the toll station automatically, enter and exit the ramp automatically, etc., but it cannot achieve full automated driving in the city. At the same time, to ensure the integrity of automated driving testing and verification, it is at least necessary to ensure that all aspects of the ODD have been passed by ensuring the safe operation of the system or by ensuring that the system can identify situations beyond the scope of the ODD.

[0045] For the Operational Design Domain on each road, it means that when an automated driving vehicle is driving in the part of the Operational Design Domain on each road, the automated driving function can be enabled to achieve safer automated driving. This step is precisely when it is confirmed according to the recognition result that there is a target dynamic event on the target road, and the Operational Design Domain on the target road is updated according to this target dynamic event, so that the updated Operational Design Domain can timely prevent the vehicle from entering the influence range of the target dynamic event that it should not enter, thereby ensuring driving safety and the personal and property safety of the passengers in the vehicle.

[0046] Specifically, the update of the Operational Design Domain may involve: reducing the coverage area and reducing the level of automated driving services that can be provided.

[0047] The method for real-time update of the high-precision map for automated driving provided by the embodiments of the present disclosure. The vehicle terminal obtains road images and driving trajectories from in-vehicle sensors and roadside devices respectively, and uses a dynamic event recognition model pre-deployed on the vehicle terminal to identify dynamic events for multi-source road information including road images and driving trajectories, and updates the Operational Design Domain for providing automated driving services according to the target dynamic events existing on the identified target road, thereby improving the response speed of the automated driving service to road dynamic events through timely update of the Operational Design Domain, enhancing the automated driving effect, and ensuring the personal and property safety of the passengers.

[0048] In order to improve the types of road information contained in multi-source road information as much as possible, and further improve the comprehensiveness and accuracy of the recognition results output by the dynamic event recognition model, it is also possible to obtain road information texts related to roads issued by authoritative departments (such as traffic management departments, traffic radio information platforms, etc.). Then, step 202 above can be: summarize the road image, driving trajectory, and road information text into multi-source road information, and then input the multi-source road information into the dynamic event recognition model pre-deployed on the vehicle side. Further, in order to adapt to the computing performance of the above-mentioned execution entity, it is also possible to directly receive the text features mined from the road information text by the server side, thereby eliminating the feature extraction operation performed by the above-mentioned execution entity on the road information text.

[0049] To deepen the understanding of how the dynamic event recognition model specifically completes the recognition process of dynamic road events based on the input multi-source road information, the present disclosure also provides a two-stage recognition scheme that adapts to the computing performance of the model deployment side, that is:

[0050] First, use the rough recognition module in the dynamic event recognition model to roughly recognize the image information and trajectory information in the road image and driving trajectory that can roughly reflect the event category to which the corresponding road dynamic event belongs, so as to obtain the target event category;

[0051] Then, use the target category dynamic event recognition module corresponding to the target event category in the dynamic event recognition model to recognize the dynamic road events of the multi-source road information, and obtain the recognition result including the target event category and its confidence level output by the target category dynamic event recognition module.

[0052] That is, the two-stage recognition scheme provided in this embodiment, because the rough recognition module first simply recognizes the event category that is suspected to belong to, only requires a small amount of computing performance for a short operation time. After determining the suspected target event category, only the target category dynamic event recognition module corresponding to the target time category is called to actually recognize the multi-source road information, instead of traversing the recognition modules of all event categories when the event category is unknown, so as to reduce the requirement for computing performance by avoiding a large amount of ineffective operations, and is more suitable for the computing performance of the vehicle side where the dynamic event recognition model is deployed. It is also because the dynamic event recognition model is deployed on the vehicle side that it can directly output highly time-sensitive recognition results based on the road images and driving trajectories collected in real time by vehicle sensors and roadside devices, shortening the time delay caused by the long-distance transmission of information back and forth.

[0053] To further deepen the understanding of how to specifically identify dynamic road events from multi-source road information through the target category dynamic event recognition module, this embodiment also uses Figure 3 to illustrate a specific implementation solution. Its process 300 includes the following steps:

[0054] Step 301: Use the feature mining sub-module in the target category dynamic event recognition module to mine the features of each type of road information that makes up the multi-source road information, and obtain multi-source road features including image features and trajectory features;

[0055] That is, the feature mining sub-module is used to separately mine the features of each type of road information that makes up the multi-source road information. For example, image features and trajectory features are obtained. If the multi-source road information also includes road information text, text features can also be correspondingly mined.

[0056] The mining of road images aims to highlight the parts of the captured road images that are abnormal due to dynamic events, and then form image features for subsequent identification of road dynamic events;

[0057] For the mining of driving trajectories, it mainly mines the abnormal road parts corresponding to the trajectory anomalies in the driving trajectories, and then forms trajectory features for subsequent identification of road dynamic events. Specifically, the abnormal roads include at least one of the following:

[0058] Abnormal U-turn roads, abnormal yaw roads, abnormal speed change roads, abnormal congestion roads.

[0059] That is, any trajectory that can reflect the above-mentioned abnormal roads can be used to extract the obtained trajectory features.

[0060] For the mining of road information text, it mainly accurately extracts the road abnormal information clearly recorded in the text information and forms text features.

[0061] Step 302: Use the feature fusion sub-module in the target category dynamic event recognition module to fuse each type of road feature that makes up the multi-source road features, and obtain multi-source fusion features;

[0062] Based on step 301, this step aims to fuse each type of road feature that makes up the multi-source road features by the above-mentioned execution entity to obtain multi-source fusion features. Further, in addition to using the multi-feature fusion method of uniform fusion, this feature fusion sub-module can also provide corresponding feature fusion weights for different types of road features according to the target event category, and the same type of road features have different feature fusion weights according to the different event categories, so as to correspond to complex road dynamic events as much as possible.

[0063] Step 303: Use the dynamic event discrimination sub-module in the target category dynamic event recognition module to determine the target confidence level of the road dynamic event corresponding to the target event category for the multi-source fusion feature;

[0064] Step 304: Use the recognition result output sub-module in the target category dynamic event recognition module to output the recognition result including the target event category and the target confidence level.

[0065] In Step 303 and Step 304, the dynamic event discrimination sub-module determines the target confidence level of the road dynamic event corresponding to the target event category for the multi-source fusion feature, and the recognition result output sub-module outputs the result.

[0066] In this embodiment, through Steps 301 - 304, the process of the target category dynamic event recognition module for recognizing dynamic road events from multi-source road information is further detailedly decomposed into four sub-processes: feature mining, feature fusion, dynamic event discrimination, and recognition result output. That is, different feature mining methods are used for different types of road-related information, and different feature fusion weights are provided for different types of road features according to the event category during feature fusion to adapt to the belonging event category, which helps to improve the accuracy of the output recognition result.

[0067] To deepen the understanding of how to determine whether there is a dynamic event on the target road based on the recognition result, this embodiment also Figure 4 shows a specific implementation method, and its process 400 includes the following steps:

[0068] Step 401: Determine a confidence list of the target road under at least one type of road dynamic event according to the recognition result;

[0069] The purpose of this step is for the above-mentioned execution entity to determine a confidence list of the target road under at least one type of road dynamic event according to the recognition result, that is, the recognition result includes at least one type of road dynamic event that the target road may belong to (that is, the situation of not including any type is not considered, and in this case, it will be considered that no recognition result is output), and the confidence level of each type of road dynamic event, thus forming this confidence list.

[0070] Step 402: In response to the fact that the confidence list does not contain an actual confidence level greater than the preset confidence level, initiate a verification request for the road dynamic event of the target road to the user within the preset range of the target road;

[0071] This step is based on the fact that the confidence list output in step 401 does not contain an actual confidence greater than the preset confidence, indicating that the confidence of the output categories of these suspected road dynamic events is low. Therefore, in order to prevent misjudgment caused by inaccurate model discrimination, the above-mentioned execution subject initiates a verification request for the road dynamic event of the target road to the user located within the preset range of the target road. The verification request is used to request the user whether a dynamic event has occurred on the target road and further request the user to return the event category to which the dynamic event belongs when a dynamic event has occurred. The inquiry form of the verification request includes: a text pop-up window form (for example, a pop-up window pops up on the map software to inquire about the target road in text form, and the inquiry in text form can be a question sentence: What happened on this section of road or a question sentence: Whether a traffic accident occurred on this section of road, or other similar expressions) or a synthesized voice form (for example, the map software issues a voice inquiry to the target road through a synthesized voice, such as outputting a synthesized voice of the question sentence "What happened on the XX section ahead").

[0072] Step 403: adjusting the recognition result according to the received request feedback;

[0073] Based on step 402, this step aims at adjusting the recognition result according to the received request feedback by the above-mentioned execution subject.

[0074] Furthermore, the above-mentioned execution entity can also send the received request feedback to the background server (for example, the background server of the map application that provides autonomous driving services), so that the server can determine the road dynamic events from a global perspective based on the information returned by each vehicle traveling on the target road (for example, a voting mechanism can be adopted to select the event with the highest number of votes), and then make corresponding updates to the operation design domain of the target road from a global level.

[0075] Step 404: In response to the confidence list containing an actual confidence greater than a preset confidence, determining that a target dynamic event corresponding to the actual confidence exists on the target road.

[0076] This step is based on the fact that the confidence list output in step 401 contains an actual confidence that is greater than a preset confidence, indicating that it contains at least a category of road dynamic events with a higher confidence level. Therefore, the above-mentioned execution entity will directly determine, under a higher confidence level, that there is a target dynamic event corresponding to the actual confidence level on the target road.

[0077] This embodiment provides two different processing branches through steps 401-404, namely, a processing branch for actively initiating a verification request to the user under low confidence conditions corresponding to steps 402-403, and a processing branch for directly determining the existence of a target dynamic event corresponding to the actual confidence level on the target road under high confidence conditions corresponding to step 404.

[0078] It should be noted that the above two processing branches do not exist in a fixed combination. That is, any processing branch can have other solutions as the corresponding other processing branch. This embodiment only exists as a preferred embodiment including the above two processing branches.

[0079] To avoid disturbing driving users or pedestrians with verification requests, this embodiment also Figure 5 shows a method for initiating a verification request, and its process 500 includes the following steps:

[0080] Step 501: Determine all first users within a preset range of the target road;

[0081] Step 502: Determine the request initiation suitability of each first user respectively;

[0082] Among them, the request initiation suitability is used to characterize the impact degree of the first user on passing through the road where they are located in response to the verification request. And there is a negative correlation between the magnitude of the impact degree and the magnitude of the request initiation suitability. That is, the higher the impact degree, the lower the corresponding request initiation suitability, and vice versa. That is to say, if a user will cause a greater impact on the road where they are located in response to the received verification request, it is considered not very suitable to initiate the verification request to this user. That is, it is only inclined to initiate the verification request to users whose impact degree on passing through the road where they are located in response to the verification request is smaller.

[0083] Specifically, the request initiation suitability can be determined based on at least one of the following:

[0084] The travel mode, moving speed, congestion status of the current road where the first user is located, driving behavior, response positivity to historical verification requests, or other factors that can characterize the impact degree.

[0085] Step 503: Determine the first users with an actual request initiation suitability less than the preset suitability threshold as second users;

[0086] Step 504: Initiate a road dynamic verification request for the target road to the second users.

[0087] That is, this embodiment provides a method for calculating the request initiation suitability through steps 501 - 504 to evaluate the impact degree of different users on the road where they are located in the process of responding to the verification request, and then initiate the verification request to the users with a smaller impact, so as to avoid causing unnecessary impacts on other users.

[0088] Based on any of the above embodiments, for the specific implementation manner of the update of the operation design domain involved in step 204, this embodimentFigure 6 A specific implementation solution is provided, and its process 600 includes the following steps:

[0089] Step 601: Determine the original coverage range of the operating design domain of the target road;

[0090] Step 602: Reduce the original coverage range according to the influence range of the target dynamic event.

[0091] That is, this embodiment aims to first determine the original coverage range of the operating design domain of the target road, then determine the influence range of the target dynamic event on the target road, and finally reduce the original coverage range according to this influence range (for example, directly delete the original coverage range that falls within this influence range), so as to complete the update of the operating design domain.

[0092] Among them, when determining the influence range of the target dynamic event on the target road, the influence range of the target dynamic event can be determined according to the event category to which the target dynamic event belongs, so as to match the potentially larger influence range that may exist for different event categories. For example, when a violent chemical reaction occurs to the goods carried by a vehicle in a vehicle accident category, the violent chemical reaction may cause the diffusion and volatilization of toxic or harmful substances, thus involving a larger influence range.

[0093] Furthermore, when it is found that the reduced coverage range is less than the preset range or the reduction time exceeds the preset duration, for the sake of improving the autonomous driving experience, the autonomous driving service level matched by the operating design domain can also be reduced, so as to require the driver to intervene manually as much as possible to improve safety.

[0094] Figure 8 It is a schematic diagram of a method for extracting information from a road information text provided by an embodiment of the present disclosure;

[0095] To deepen the understanding of the entire solution, this embodiment also provides a set of dynamic event recognition systems in combination with a specific scenario:

[0096] This system includes three modules, namely a dynamic data mining module, a dynamic data verification module, and a warning reminder module. Among them, the dynamic data mining module and the dynamic data verification module are the cores of the entire system, and the input of the entire system is road real-time images, trajectories, and Internet text data, and the output is the type, direction, and GPS location of the dynamic data, and safety reminders are provided to navigation users.

[0097] The entire system process is as follows: First, based on road real-time images, trajectories, and Internet text data, use the dynamic data mining module to mine road dynamic data. Secondly, combine user verification to improve the recall and accuracy of dynamic data. Finally, provide safety guarantees to navigation users for the areas covered by the dynamic data.

[0098] I. Dynamic data mining

[0099] The dynamic data mining module first determines whether dynamic data occurs on the road based on real-time road images, trajectories, and Internet text data, using image, trajectory, and text mining techniques.

[0100] 1. Image mining: The occurrence of dynamic elements is related to the occurrence of dynamic data. For example, construction cones are placed on roads under construction, and accident tripods are placed on accident roads. Therefore, based on real-time road images and with the help of a dynamic element image recognition model, it is determined whether dynamic data occurs on the road.

[0101] a) Real-time image acquisition: To solve the problem of low recall and timeliness caused by the image recall method depending on the acquisition plan, a pre-trained dynamic data recognition model is deployed on the mobile terminal. During vehicle driving, the dynamic data image recognition model acquires and recognizes road image data in real time.

[0102] b) Dynamic data recognition: The road image is detected by the dynamic element image recognition model to determine whether it contains dynamic elements, and then dynamic data is mined. The training method of the dynamic data image recognition model is as follows:

[0103] i. Sample extraction: Obtain real-time road images and manually label the types and positions of the dynamic elements contained in the images.

[0104] ii. Model offline training: After obtaining the sample data, use a deep learning detection model, such as the Faster RCNN model, to train the samples to produce a dynamic data recognition model.

[0105] iii. Model online prediction: Obtain real-time road images and use the pre-trained dynamic data recognition model to detect the types and positions of the dynamic elements contained in the images.

[0106] c) Dynamic data positioning: Based on the position of the dynamic element in the image and the GPS coordinates of the trajectory to which it belongs, use multi-view geometry to calculate the GPS coordinates of the dynamic data, and use a trajectory matching algorithm, such as the HMM algorithm, to calculate the road and direction corresponding to the trajectory to which the dynamic data belongs.

[0107] 2. Trajectory mining: There is a relationship between abnormal driving trajectories of users and the occurrence of dynamic data. For example, a large number of user U-turns or yaw trajectories will appear on blocked roads. Therefore, based on real-time road trajectories, roads with abnormal driving trajectories are mined, and multi-dimensional trajectory features are constructed. Using machine learning-related techniques, it is determined whether dynamic data exists on the road.

[0108] d) Real-time trajectory acquisition: Real-time acquire the driving trajectory of the user. Using trajectory matching algorithms, such as the HMM algorithm, match the driving trajectory with the roads in the road network to determine the road where the trajectory is located. Combining the road network and the user's navigation planned route, judge whether there are behaviors such as U-turn, passing through, deviation, and slow driving in the user's driving trajectory.

[0109] e) Abnormal road excavation: Calculate the road features at the minute level in real time. By comparing the feature changes in adjacent time periods, judge whether there are abnormal driving trajectories on the current road. For the road r at time t, compare the trajectory features in the time intervals [t - 2a, t - a) and [t - a, t), where a is 5 minutes. If the trajectory features in adjacent time intervals meet the following conditions, then the road r belongs to an abnormal road at time t.

[0110] i. Abnormal U-turn road excavation: Calculate the number of U-turns on the road in the time periods [t - 2a, t - a) and [t - a, t). If the change in the number of U-turns on the road satisfies formula ①, then the road belongs to an abnormal road.

[0111]

[0112] Among them, known respectively represent the number of U-turns on the road r in the time periods [t - a, t) and [t - 2a, t - a), and the change rate threshold α <u，max> and α <u，min> can take values of 1.5 and 0.5.

[0113] ii. Abnormal deviation road excavation: Calculate the number of deviations on the road in the time periods [t - 2a, t - a) and [t - a, t). If the change in the number of deviations on the road satisfies formula ②, then the road belongs to an abnormal road.

[0114]

[0115] Among them, and respectively represent the number of deviations on the road r in the time periods [t - a, t) and [t - 2a, t - a), and the change rate threshold α <d，max> and α <d，min> can take values of 1.5 or 0.5.

[0116] iii. Abnormal congestion road excavation: Divide the road at 50-meter intervals, and count the average passing speed v of the driving trajectories in each interval l i in the time periods [t - 2a, t - a) and [t - a, t). According to formula ③, calculate the congestion index of each interval If the road congestion index meets formula ④, then the road belongs to an abnormal road.

[0117]

[0118]

[0119] Among them, and respectively represent the congestion indices of l i in the time periods of [t - a, t) and [t - 2a, t - a), and the change rate threshold α <s，max> and α <s，min> can take values of 1.5 or 0.5.

[0120] f) Dynamic data identification: For roads with abnormal driving trajectories of users, construct multi-dimensional trajectory features, and detect whether the road contains dynamic data through a dynamic data trajectory identification model. The model training method is as follows:

[0121] i. Sample extraction: Obtain roads with historical dynamic data, and combine with road images to manually label the types of dynamic data.

[0122] ii. Feature calculation: For the road to be detected, extract road attributes and traffic characteristics, and the extraction methods are as follows:

[0123] 1) Road attribute features: Extract the number of lanes and road grade features of abnormal roads.

[0124] 2) Road traffic characteristics: Calculate the number of U-turns, crossing times, yaw times, congestion coefficients and corresponding change rate characteristics of road r in the time periods of [t - a, t) and [t - 2a, t - a).

[0125] iii. Model offline training: After obtaining the samples and corresponding features, use a machine learning classification model, such as the GBDT (Gradient Boosting Decision Tree) model, to train the samples and output a dynamic data trajectory identification model.

[0126] iv. Model online prediction: Obtain the user's driving trajectory in real time, calculate the road attributes and traffic characteristics, and use the pre-trained dynamic data trajectory identification model to predict whether the road contains dynamic data and the type of dynamic data.

[0127] g) Dynamic data positioning: For roads suspected of having dynamic data, first extract the driving trajectory sequences of user U-turns, yaws, and slow driving, and calculate the abnormal change point p c of a single trajectory sequence, such as Figure 7a , Figure 7b , Figure 7c , Figure 7dAs shown, secondly, a clustering algorithm, such as hierarchical clustering, is used to calculate the clustering center points. Finally, the HMM (Hidden Markov Model) algorithm is used to calculate the GPS positions on the road network mapped from the clustering center points. This GPS position is the position of the dynamic data, and the driving direction at the abnormal behavior change point is the direction of the dynamic data. The calculation method of the abnormal change point is as follows:

[0128] i. For abnormal roads mined from U-turn trajectories: Extract the user's U-turn trajectory. If the trajectory point p i satisfies Formula ⑤ or ⑥, then the trajectory point p i is an abnormal change point of the U-turn trajectory, that is, the user makes a U-turn behavior at the trajectory point p i+1 .

[0129]

[0130]

[0131] Among them, and represent the driving directions of the trajectory points p i and p i+1 . The driving directions of the trajectory points p i and p i+1 are opposite. and represent the roads where the trajectories p i and p i+1 are located. U represents the set of roads parallel to the road network. and respectively represent that the trajectories p i and p i+1 are on the same road or parallel roads.

[0132] ii. For abnormal roads mined from yaw trajectories: Extract the user's yaw trajectory. If the trajectory point p i satisfies Formula ⑦, then the trajectory point p i is an abnormal change point of the yaw trajectory, that is, the user makes a yaw behavior at the trajectory point p i+1 .

[0133]

[0134] Among them, known represents the navigation planned road and the actual driving road passing through the position of the trajectory point p i . represents that the navigation planned road and the actual driving road passing through the position of the trajectory point p i are the same. represents the navigation planned road and the actual driving road passing through the position of the trajectory point p i+1The planned navigation route does not match the actual driving route at the location.

[0135] iii. Abnormal roads for slow - moving trajectory mining: Calculate the user's passing speed. If the road interval l i satisfies formula ⑧, then l i is the changing section of the slow - moving trajectory, and the center point of section l i is the abnormal change point.

[0136]

[0137] Among them, represents the congestion index of road segment l i , the change rate threshold β <s，min> and β <s，max> can take values of 0.5 or 3.

[0138] Text mining: When dynamic data occurs on the road, the traffic management department releases relevant news through the Internet platform. Based on the real - time road news, with the help of event extraction technology, dynamic data is mined.

[0139] h) Real - time news acquisition: Subscribe to the traffic management department's account to obtain road news in real - time. Since the traffic management department will release a large amount of news unrelated to dynamic data, it is necessary to use a deep - learning text classification model, such as the TextCNN (Text Convolutional Neural Network, a convolutional neural network for text) model, to extract news related to dynamic data.

[0140] i) Dynamic data recognition: For road Internet text information, use an event extraction model to identify key information of dynamic data, such as road names, toll station names, time, event types, locations, directions, entrances and exits, etc. The training method of the event extraction model is as follows:

[0141] i. Extraction of sample data: For dynamic data news data, manually label the key entities included in the news and the corresponding SPO relationships between each entity, as Figure 8 shown. In the figure below, different colors of underlines correspond to different types of entities, and arrows represent the mapping relationships existing between different entities.

[0142] ii. Off - line model training: After obtaining the sample data, use ERNIE's Sequence Labeling model to train the samples and produce a dynamic data extraction model.

[0143] iii. Online model prediction: Based on road Internet text information, input word - by - word into the pre - trained dynamic data extraction model to identify the entity and SPO relationship information included in the news, as shown in Table 1 below:

[0144] Table 1 Structured Data of Event Relationships

[0145] Data field Extracted value Highway name Highway X1 Toll station Toll station Y1 Direction Direction Z1 Event type Closed Occurrence time XXXX year, month, day Event reason Heavy fog

[0146] j) Dynamic Data Location: With the location information obtained by means of the event extraction model and combined with the road network structure, determine the location information of the dynamic data.

[0147] i. For ordinary road scenarios: Based on the event extraction model, obtain road and mileage information, and combined with the road network information, obtain the road and GPS locations corresponding to the dynamic data. Taking "The traffic accident at the location of K1137+350 in the Z2 direction of X2 Expressway at 19:06 on January 9, 2022 has been processed and the road has returned to normal traffic" as an example, the expressway name and location extracted by the event extraction model are "X2 Expressway" and "K1137+350" respectively, and they are matched with the road network database to obtain the road and GPS locations of the corresponding road network.

[0148] ii. For highway toll station scenarios: Based on the event extraction model, obtain the expressway name, toll station, and direction, and calculate the GPS location of the dynamic data in the road network. As Figure 8 shown, the expressway name, toll station, and direction extracted by the event extraction model are "X1 Expressway", "Y1 Toll Station", and "Z1 Direction" respectively, and they are matched with the road network database to obtain the road and GPS locations of the corresponding road network.

[0149] 2. Multimodal Fusion Recognition: For suspected dynamic data mined from trajectories, images, and texts, based on the method of transfer learning, extract the trajectory, image, and text feature vectors of the road, and input them into a DNN (Deep Neural Networks) network to output the type and confidence level of the dynamic data.

[0150] a) Feature Calculation:

[0151] i. Trajectory Feature Extraction: In order to learn the spatio-temporal sequence features of the road trajectory, use the speed prediction task to transfer-learn the feature vectors of the road. The specific steps are as follows. First, obtain the spatio-temporal trajectories of the road network roads and calculate the average speed of the road at consecutive time moments. Secondly, use a spatio-temporal sequence model, such as the STGCN model, to obtain a road traffic speed prediction model. Finally, for the road where suspected dynamic data occurs and the speed features at consecutive time moments, extract the traffic vectors of the road.

[0152] ii. Image Feature Extraction: For the road where suspected dynamic data occurs, obtain the real-time image of the road, and use a pre-trained dynamic data image recognition model to extract the image vectors of the road.

[0153] iii. Text feature extraction: For roads where dynamic data is suspected to occur, obtain the real-time Internet text of the road, and use a pre-trained dynamic data extraction model to extract the text vector of the road.

[0154] iv. Feature fusion: After obtaining the trajectory features, image features, and text features of the road, splice the feature vectors and output the final feature vector.

[0155] b) Model training:

[0156] i. Sample extraction: Obtain historical dynamic data that has occurred, combine it with on-site images, and manually label the types of dynamic data.

[0157] ii. Offline model training: After obtaining the samples and the trajectory, image, and text feature vectors of the corresponding roads, input them into the DNN network to train the samples and produce a multi-modal fusion recognition model.

[0158] iii. Online model prediction: For the suspected dynamic data output by the dynamic data mining module, calculate the road trajectory, image, and text features, and perform fusion. Then, use the pre-trained multi-modal fusion recognition model to predict the type and confidence level of the dynamic data.

[0159] II. Verification of dynamic data

[0160] For dynamic data with low confidence, further improve the recall and accuracy of dynamic data by means of user verification. User verification includes two methods: pop-up window verification and voice verification:

[0161] a) Pop-up window verification: Ask the user whether there is dynamic data at the current location through a pop-up window. The user clicks "Yes" or "No" to determine whether there is dynamic data.

[0162] b) Voice verification: Ask the user what has happened at the current location through voice. The user replies with "There is an accident" etc. to determine whether there is dynamic data.

[0163] To reduce the disturbance to users caused by dynamic data verification, when the user drives through the GPS location of dynamic data, score the current user by combining user features, dynamic data features, and road features, and trigger user verification for users exceeding the threshold, where the threshold can be initially set to 0.5. The calculation formula is as follows:

[0164]

[0165] Among them, C is the set of dynamic data to be verified by the user, and E i is the feature of the dynamic data to be verified, such as the type and confidence level of the dynamic data, and U iUser features, such as the response rate of the current user's historical dynamic data queries, the number of dynamic data queries of the current user on the same day, the driving speed of the current user, etc., R i Road network features, such as the number of lanes, road grade, road traffic frequency, etc. of the road to which the dynamic data belongs. f(*) is a scoring function, which can be specified by rules or calculated with the help of a classification model, such as the GBDT model. After obtaining the feedback of multiple users, it is possible to judge whether the dynamic data is valid by voting.

[0166] III. Early warning reminder module

[0167] After obtaining the confident dynamic data, it is necessary to recalculate the optimal navigation route and arrival time, and with the help of navigation electronic products, for example, it can be informed the user in advance by means of page display and voice broadcast. Different dynamic data have different impacts on users. The navigation electronic product combines the dynamic data features, road conditions features, route features, and road features, calculates the scores of different navigation routes, and pushes the optimal navigation route to the user to remind the user to switch, providing personalized safety guarantees for the user. The calculation formula is as follows:

[0168]

[0169] Among them, P is the set of navigation routes, E i is the feature of the dynamic data, such as the type and confidence of the dynamic data, O i is the road condition feature, such as the congestion length of the navigation route, Pi is the navigation route feature, such as the mileage of the navigation route, the user's travel time, the user's driving speed, etc., R i is the road network feature, such as the number of lanes, road grade, etc. of the road to which the dynamic data belongs. f(*) is a scoring function, which can be specified by rules or calculated with the help of a classification model, such as the GBDT model.

[0170] For further reference Figure 9 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an automatic driving high-precision map real-time update device, and this device embodiment corresponds to Figure 2 the method embodiment shown, and this device can be specifically applied to various electronic devices.

[0171] Such as Figure 9As shown in the figure, the real-time update device 900 of the high-precision map for autonomous driving in this embodiment may include: an image and trajectory acquisition unit 901, a multi-source road information input unit 902, an identification result receiving unit 903, and an operating design domain update unit 904. Among them, the image and trajectory acquisition unit 901 is configured to acquire road images and driving trajectories collected by vehicle-mounted sensors and roadside devices; the multi-source road information input unit 902 is configured to input multi-source road information including road images and driving trajectories for discriminating road dynamic events into a dynamic event recognition model pre-deployed on the vehicle side, and the dynamic event recognition model is used to represent the corresponding relationship between the multi-source road information and the confidence levels of various types of road dynamic events; the identification result receiving unit 903 is configured to receive the identification results of road dynamic events output by the dynamic event recognition model; the operating design domain update unit 904 is configured to, in response to determining that a target dynamic event occurs on a target road according to the identification result, update the operating design domain of the target road based on the target dynamic event.

[0172] In this embodiment, in the real-time update device 900 of the high-precision map for autonomous driving: the specific processing of the image and trajectory acquisition unit 901, the multi-source road information input unit 902, the identification result receiving unit 903, and the operating design domain update unit 904 and the technical effects brought by them can respectively refer to Figure 2 the relevant descriptions of steps 201-204 in the corresponding embodiment, which will not be elaborated here.

[0173] In some optional implementation manners of this embodiment, the real-time update device 900 of the high-precision map for autonomous driving may further include:

[0174] a road information text acquisition unit, configured to acquire road information texts related to roads issued by authoritative departments;

[0175] Correspondingly, the multi-source road information input unit 902 may be further configured to:

[0176] aggregate the road images, driving trajectories, and road information texts into multi-source road information;

[0177] input the multi-source road information into a dynamic event recognition model pre-deployed on the vehicle side.

[0178] In some optional implementation manners of this embodiment, the real-time update device 900 of the high-precision map for autonomous driving may further include a dynamic road event recognition unit configured to recognize dynamic road events for the input multi-source road information through the dynamic event recognition model, and the dynamic road event recognition unit may include:

[0179] A rough recognition subunit, configured to use the rough recognition module in the dynamic event recognition model to determine the target event category to which the road dynamic event characterized by the road image and the driving trajectory belongs;

[0180] An event recognition subunit, configured to use the target category dynamic event recognition module corresponding to the target event category in the dynamic event recognition model to recognize dynamic road events from multi-source road information, and obtain an identification result including the target event category and its confidence level output by the target category dynamic event recognition module.

[0181] In some alternative implementation manners of this embodiment, the event recognition subunit may include:

[0182] A feature mining component, configured to use the feature mining sub-module in the target category dynamic event recognition module to perform feature mining on each type of road information constituting the multi-source road information, and obtain multi-source road features including image features and trajectory features;

[0183] A feature fusion component, configured to use the feature fusion sub-module in the target category dynamic event recognition module to fuse each type of road feature constituting the multi-source road features, and obtain multi-source fusion features; wherein, the feature fusion sub-module provides corresponding feature fusion weights for different types of road features according to the target event category to which they belong, and the same type of road features have different feature fusion weights according to the different event categories to which they belong;

[0184] An event discrimination component, configured to use the dynamic event discrimination sub-module in the target category dynamic event recognition module to determine the target confidence level of the road dynamic event of the target event category corresponding to the multi-source fusion features;

[0185] A result output component, configured to use the recognition result output sub-module in the target category dynamic event recognition module to output an identification result including the target event category and the target confidence level.

[0186] In some alternative implementation manners of this embodiment, the feature mining component includes a trajectory feature mining component configured to use the feature mining sub-module to mine trajectory features representing abnormal roads from the driving trajectory, and the abnormal roads may include at least one of the following:

[0187] Abnormal U-turn roads, abnormal yaw roads, abnormal speed change roads, abnormal congestion roads.

[0188] In some alternative implementation manners of this embodiment, the real-time update device 900 for the autonomous driving high-precision map may further include:

[0189] A confidence level list acquisition unit, configured to determine a confidence level list of the target road under at least one type of road dynamic event according to the recognition result;

[0190] A target dynamic event existence determination unit, configured to determine that there is a target dynamic event corresponding to the actual confidence on the target road in response to the actual confidence greater than the preset confidence in the confidence list.

[0191] In some optional implementation manners of this embodiment, the autonomous driving high-precision map real-time update device 900 may further include:

[0192] A verification request initiation unit, configured to initiate a verification request for the road dynamic event of the target road to a user within a preset range of the target road in response to the actual confidence greater than the preset confidence not being included in the confidence list; wherein, the verification request is used to request whether a dynamic event has occurred on the target road by the user and further request the user to return the event category to which the occurred dynamic event belongs when a dynamic event has occurred, and the inquiry form of the verification request includes: a text pop-up window form or a synthesized voice form;

[0193] An identification result adjustment unit, configured to adjust the identification result according to the received request feedback.

[0194] In some optional implementation manners of this embodiment, the verification request initiation unit may be further configured to:

[0195] Determine all first users within a preset range of the target road;

[0196] Respectively determine the request initiation adaptability of each first user; wherein, the request initiation adaptability is used to characterize the influence degree of the first user on passing through the road where the user is located in response to the verification request, and the higher the influence degree, the lower the request initiation adaptability;

[0197] Determine the first user with an actual request initiation adaptability less than the preset adaptability threshold as the second user;

[0198] Initiate a road dynamic verification request for the target road to the second user.

[0199] In some optional implementation manners of this embodiment, the request initiation adaptability may be determined based on at least one of the following:

[0200] The travel mode, moving speed, congestion status of the current road where the first user is located, driving behavior, and reply positivity to historical verification requests of the first user.

[0201] In some optional implementation manners of this embodiment, the operation design domain update unit 904 may include:

[0202] An original coverage range determination subunit, configured to determine the original coverage range of the operation design domain of the target road;

[0203] A coverage reduction subunit, configured to reduce an original coverage according to an influence range of a target dynamic event.

[0204] In some optional implementation manners of this embodiment, the coverage reduction subunit may be further configured to:

[0205] Determine an influence range of a target dynamic event according to an event category to which the target dynamic event belongs;

[0206] Reduce the original coverage according to the influence range.

[0207] In some optional implementation manners of this embodiment, the real-time update device 900 for an autonomous driving high-precision map may further include:

[0208] An autonomous driving service level reduction unit, configured to reduce an autonomous driving service level matched by an operational design domain in response to that a reduced coverage is less than a preset range or a reduction time exceeds a preset duration.

[0209] This embodiment exists as a device embodiment corresponding to the above method embodiment. The real-time update device for an autonomous driving high-precision map provided in this embodiment enables a vehicle side to obtain a road image and a driving trajectory respectively from an in-vehicle sensor and a roadside device, and identify a dynamic event from multi-source road information including the road image and the driving trajectory by means of a dynamic event recognition model pre-deployed on the vehicle side, and update an operational design domain for providing an autonomous driving service according to a target dynamic event existing on a recognized target road, so as to improve a response speed of the autonomous driving service to road dynamic events by timely updating the operational design domain, improve an autonomous driving effect, and ensure the personal and property safety of passengers.

[0210] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to implement the method for real-time updating of an autonomous driving high-precision map described in any of the above embodiments.

[0211] According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the method for real-time updating of an autonomous driving high-precision map described in any of the above embodiments when executed.

[0212] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, and when the computer program is executed by a processor, the computer program is enabled to implement the method for real-time updating of an autonomous driving high-precision map described in any of the above embodiments.

[0213] Figure 10 FIG. shows a schematic block diagram of an exemplary electronic device 1000 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0214] As Figure 10 shown, the device 1000 includes a computing unit 1001 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0215] Multiple components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0216] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 executes the various methods and processes described above, such as the real-time update method for an autonomous driving high-precision map. For example, in some embodiments, the real-time update method for an autonomous driving high-precision map can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the real-time update method for an autonomous driving high-precision map described above can be executed. Alternatively, in other embodiments, the computing unit 1001 can be configured to execute the real-time update method for an autonomous driving high-precision map in any other suitable manner (e.g., by means of firmware).

[0217] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0218] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0219] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0220] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0221] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0222] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is created by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to address the deficiencies of high management difficulty and weak business scalability existing in traditional physical hosts and virtual private server (VPS) services.

[0223] According to the technical solution of the embodiment of the present disclosure, the vehicle terminal obtains a road image and a driving trajectory by respectively acquiring from in-vehicle sensors and roadside devices, and uses a dynamic event recognition model pre-deployed on the vehicle terminal to identify dynamic events for multi-source road information including the road image and the driving trajectory, and updates the operational design domain for providing autonomous driving services according to the identified target dynamic events existing on the target road, so as to improve the response speed of the autonomous driving service to road dynamic events by timely updating the operational design domain, enhance the autonomous driving effect, and ensure the personal and property safety of passengers.

[0224] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present disclosure can be achieved, and no limitation is made herein.

[0225] The above specific embodiments do not constitute a limitation to the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A real-time update method for an autonomous driving high-precision map, comprising: Obtaining road images and driving trajectories collected by in-vehicle sensors and roadside devices; Inputting multi-source road information for discriminating road dynamic events, including the road images and the driving trajectories, into a dynamic event recognition model pre-deployed on the vehicle side, where the dynamic event recognition model is used to represent the correspondence between the multi-source road information and the confidence levels of various types of road dynamic events; Receiving the recognition result of the road dynamic event output by the dynamic event recognition model; Determining a confidence list of the target road under at least one type of road dynamic event according to the recognition result; In response to the fact that the confidence list does not contain an actual confidence level greater than the preset confidence level, determining all first users located within the preset range of the target road; Respectively determining the request initiation fitness of each of the first users; wherein the request initiation fitness is used to represent the influence degree of the first user on the road passing by in response to the verification request, and the magnitude of the influence degree is negatively correlated with the magnitude of the request initiation fitness; determining the first users with an actual request initiation fitness less than the preset fitness threshold as second users; initiating a verification request for the road dynamics of the target road to the second users, where the verification request is used to request whether a dynamic event has occurred on the target road by the user and further request the user to return the event category to which the occurred dynamic event belongs; Adjusting the recognition result according to the received request feedback; In response to determining that the target road generates a target dynamic event according to the recognition result, updating the operating design domain of the target road based on the target dynamic event.

2. The method according to claim 1, further comprising: Obtaining a road information text related to the road issued by an authoritative department; Correspondingly, the inputting the multi-source road information for discriminating road dynamic events, including the road images and the driving trajectories, into a dynamic event recognition model pre-deployed on the vehicle side includes: Summarizing the road images, the driving trajectories and the road information text into the multi-source road information; Inputting the multi-source road information into a dynamic event recognition model pre-deployed on the vehicle side.

3. The method according to claim 1 further comprises: Performing recognition of dynamic road events on the input multi-source road information through the dynamic event recognition model, where the performing recognition of dynamic road events on the input multi-source road information through the dynamic event recognition model includes: Using a rough recognition module in the dynamic event recognition model to determine the target event category to which the road dynamic event characterized by the road images and the driving trajectories belongs; Using a target category dynamic event recognition module corresponding to the target event category in the dynamic event recognition model to perform recognition of dynamic road events on the multi-source road information, and obtaining a recognition result including the target event category and its confidence level output by the target category dynamic event recognition module.

4. The method according to claim 3, wherein, Using the target category dynamic event recognition module corresponding to the target event category in the dynamic event recognition model to identify dynamic road events from the multi-source road information, and obtaining an identification result including the target event category and its confidence level output by the target category dynamic event recognition module, including: Using the feature mining sub-module in the target category dynamic event recognition module to mine features from each type of road information constituting the multi-source road information, and obtaining multi-source road features including image features and trajectory features; Using the feature fusion sub-module in the target category dynamic event recognition module to fuse each type of road feature constituting the multi-source road features, and obtaining multi-source fusion features; wherein, the feature fusion sub-module provides corresponding feature fusion weights for different types of road features according to the target event category to which they belong, and the same type of road features has different feature fusion weights according to the different event categories to which they belong; Using the dynamic event discrimination sub-module in the target category dynamic event recognition module to determine the target confidence level of the road dynamic event of the target event category corresponding to the multi-source fusion features; Using the recognition result output sub-module in the target category dynamic event recognition module to output an identification result including the target event category and the target confidence level.

5. The method according to claim 4, wherein Using the feature mining sub-module to mine trajectory features representing abnormal roads from the driving trajectory, and the abnormal roads include at least one of the following: Abnormal U-turn roads, abnormal deviation roads, abnormal speed change roads, abnormal congestion roads.

6. The method according to claim 1, further comprising: In response to the confidence list including an actual confidence level greater than a preset confidence level, determining that the target road has a target dynamic event corresponding to the actual confidence level.

7. The method according to claim 1, wherein The inquiry form of the verification request includes: a text pop-up window form or a synthesized voice form.

8. The method according to claim 1, wherein The request initiation adaptability is determined based on at least one of the following: The travel mode, moving speed, congestion status of the current road where the first user is located, driving behavior, and response positivity to historical verification requests of the first user.

9. The method according to any one of claims 1-8, wherein Updating the operating design domain of the target road based on the target dynamic event includes: Determining the original coverage range of the operating design domain of the target road; Reducing the original coverage range according to the influence range of the target dynamic event.

10. The method according to claim 9, wherein Reducing the original coverage range according to the influence range of the target dynamic event includes: Determining the influence range of the target dynamic event according to the event category to which the target dynamic event belongs; Reducing the original coverage range according to the influence range.

11. The method according to claim 9, further comprising: In response to the reduced coverage range being less than a preset range or the reduction time exceeding a preset duration, reducing the autonomous driving service level matched by the operating design domain.

12. An autonomous driving high-precision map real-time update device, comprising: An image and trajectory acquisition unit configured to acquire road images and driving trajectories collected by vehicle-mounted sensors and roadside devices; A multi-source road information input unit, configured to input multi-source road information for discriminating road dynamic events, including the road image and the driving trajectory, into a dynamic event recognition model pre-deployed on a vehicle side, where the dynamic event recognition model is used to represent the correspondence between the multi-source road information and the confidence levels of various types of road dynamic events; An identification result receiving unit, configured to receive the identification result of the road dynamic event output by the dynamic event recognition model; A confidence level list obtaining unit, configured to determine a confidence level list of a target road under at least one type of road dynamic event according to the identification result; A verification request initiating unit, configured to determine all first users within a preset range of the target road in response to the fact that the confidence level list does not include an actual confidence level greater than a preset confidence level; Respectively determine the request initiation adaptation degree of each of the first users; wherein the request initiation adaptation degree is used to represent the influence degree of the first user on passing the road where the first user is located in response to the verification request, and the magnitude of the influence degree is negatively correlated with the magnitude of the request initiation adaptation degree; determine the first users with an actual request initiation adaptation degree less than a preset adaptation degree threshold as second users; initiate a verification request for the road dynamics of the target road to the second users; wherein the verification request is used to request the users whether a dynamic event has occurred on the target road and further request the users to return the event category to which the occurred dynamic event belongs; An identification result adjustment unit, configured to adjust the identification result according to the received request feedback; An operation design domain update unit, configured to update the operation design domain of the target road based on the target dynamic event in response to determining that the target road generates a target dynamic event according to the identification result.

13. The device according to claim 12, further comprising: A road information text obtaining unit, configured to obtain a road information text related to a road issued by an authoritative department; Correspondingly, the multi-source road information input unit is further configured to: Summarize the road image, the driving trajectory, and the road information text into the multi-source road information; Input the multi-source road information into a dynamic event recognition model pre-deployed on a vehicle side.

14. The apparatus according to claim 12, further comprising: A dynamic road event recognition unit configured to perform identification of dynamic road events on the input multi-source road information through the dynamic event recognition model, where the dynamic road event recognition unit includes: A rough identification subunit, configured to use a rough identification module in the dynamic event recognition model to determine the target event category to which the road dynamic event represented by the road image and the driving trajectory belongs; An event identification subunit, configured to perform identification of dynamic road events on the multi-source road information by using a target category dynamic event recognition module corresponding to the target event category in the dynamic event recognition model, and obtain an identification result including the target event category and its confidence level output by the target category dynamic event recognition module.

15. The device according to claim 13, wherein, The event identification subunit includes: A feature mining component, configured to use the feature mining sub-module in the target category dynamic event recognition module to perform feature mining on each type of road information in the multi-source road information, and obtain multi-source road features including image features and trajectory features; A feature fusion component, configured to use the feature fusion sub-module in the target category dynamic event recognition module to fuse each type of road feature in the multi-source road features to obtain multi-source fusion features; wherein, the feature fusion sub-module provides corresponding feature fusion weights for different types of road features according to the target event category, and the same type of road features has different feature fusion weights according to different event categories; An event discrimination component, configured to use the dynamic event discrimination sub-module in the target category dynamic event recognition module to determine the target confidence of the road dynamic event of the target event category corresponding to the multi-source fusion features; A result output component, configured to use the recognition result output sub-module in the target category dynamic event recognition module to output a recognition result including the target event category and the target confidence; 16. The device according to claim 15, wherein, The feature mining component includes a trajectory feature mining component configured to use the feature mining sub-module to mine the trajectory features of abnormal roads from the driving trajectories, and the abnormal roads include at least one of the following: Abnormal U-turn roads, abnormal yaw roads, abnormal speed change roads, abnormal congestion roads.

17. The device according to claim 12, further comprising: A target dynamic event existence determination unit, configured to determine that there is a target dynamic event corresponding to the actual confidence in the target road in response to the confidence list including an actual confidence greater than a preset confidence; 18. The device according to claim 12, wherein The inquiry form of the verification request includes: a text pop-up window form or a synthesized voice form.

19. The apparatus according to claim 12, wherein, The request initiation adaptability is determined based on at least one of the following: The travel mode, moving speed, congestion status of the current road, driving behavior, and response positivity to historical verification requests of the first user.

20. The device according to any one of claims 12-19, wherein, The operation design domain update unit includes: An original coverage range determination sub-unit, configured to determine the original coverage range of the operation design domain of the target road; A coverage range reduction sub-unit, configured to reduce the original coverage range according to the influence range of the target dynamic event; 21. The apparatus according to claim 20, wherein, The coverage range reduction sub-unit is further configured to: Determine the influence range of the target dynamic event according to the event category to which the target dynamic event belongs; Reduce the original coverage range according to the influence range.

22. The device according to claim 20, further comprising: An autonomous driving service level reduction unit, configured to reduce the autonomous driving service level matched by the operation design domain in response to the reduced coverage range being less than a preset range or the reduction time exceeding a preset duration; 23. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the real-time update method of the autonomous driving high-precision map according to any one of claims 1-11.

24. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the real-time update method of the autonomous driving high-precision map according to any one of claims 1-11.

Images