Vehicle passing processing method, electronic equipment and vehicle

Abstract

The invention relates to the technical field of vehicle control, in particular to a vehicle passing processing method, electronic equipment and a vehicle. The method comprises the following steps: acquiring environment sensing data and vehicle driving data; performing region division processing on the environment sensing data by using a region extraction model to obtain a region division result; performing traffic strategy prediction processing on the region division result by using a traffic prediction model to obtain a traffic strategy prediction result; and based on the region division result, the passage strategy prediction result and the vehicle driving data, performing fusion analysis processing by using a fusion analysis model, and determining and outputting a passage index of the passage strategy prediction result. According to the method, the traffic difficulty corresponding to the corresponding traffic strategy prediction result can be determined according to the traffic index of the traffic strategy prediction result, a user can be assisted to carry out accurate vehicle driving control, the traffic index can be responded and output in real time, the timely control effect of the vehicle is ensured, and the traffic capacity of the vehicle in various complex environments can be improved.

Description

Vehicle traffic handling methods, electronic equipment, and vehicles Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a method for handling vehicle traffic, electronic equipment, and a vehicle. Background Technology

[0002] Currently, during vehicle traffic, traffic plans are generally determined by analyzing the captured image data.

[0003] However, the captured image data can only reflect the environment in which the vehicle is located, and the data is relatively limited, making the predicted vehicle traffic data inaccurate. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a vehicle traffic processing method, electronic device and vehicle to solve the technical problem that the current method of using single image data for vehicle traffic prediction results in inaccurate predetermined vehicle traffic data.

[0005] To achieve the above objectives, this application provides a vehicle traffic management method, which involves acquiring environmental perception data and vehicle driving data; dividing the environmental perception data into regions using a region extraction model to obtain region division results, wherein the region extraction model is a pre-trained neural network model capable of dividing environmental perception data into regions; predicting traffic strategies using a traffic prediction model using the region division results to obtain traffic strategy prediction results, wherein the traffic prediction model is a pre-trained neural network model capable of predicting traffic strategies based on the region division results output by the region extraction model; and performing fusion analysis based on the region division results, the traffic strategy prediction results, and the vehicle driving data to determine and output a traffic index of the traffic strategy prediction results, wherein the fusion analysis model is a pre-trained neural network model capable of performing trafficability analysis on the traffic strategy prediction results.

[0006] Based on the same inventive concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0007] Based on the same inventive concept, this application also provides a vehicle including the electronic equipment described above.

[0008] As can be seen from the above, the vehicle traffic processing method, electronic device, and vehicle provided in this application acquire environmental perception data of the vehicle's environment and vehicle driving data during the vehicle's journey. Based on the environmental perception data, a region extraction model can be used to divide the environmental area where the vehicle is located into regions, thereby obtaining region division results. These region division results can characterize the environmental conditions of the corresponding region. Then, a traffic prediction model is used to predict traffic strategies based on these region division results, which can effectively predict and analyze the environmental conditions of the region division results and determine the corresponding traffic strategy prediction results. However, the accuracy of these traffic strategy prediction results is not high enough, so the region division results and traffic strategies will be adjusted accordingly. By combining the preliminary prediction results with vehicle driving data and then using a fusion analysis model for accurate fusion analysis, the traffic index of the determined traffic strategy prediction results can be consistent with the environmental conditions corresponding to the regional division results and the driving conditions corresponding to the vehicle driving data. This ensures that the traffic index of the determined traffic strategy prediction results is more accurate. After outputting the traffic index of the determined traffic strategy prediction results, users can know the corresponding traffic difficulty of the traffic strategy prediction results, which can assist users in making accurate vehicle driving control. Moreover, the traffic index of the traffic strategy prediction results can be output in real time, which can ensure the timely control effect of the vehicle and improve the vehicle's ability to pass through various complex environments. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 is a flowchart of a vehicle passage processing method according to an embodiment of this application; Figure 2 is a structural block diagram of a vehicle passage processing device according to an embodiment of this application; Figure 3 is a structural schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0012] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0013] BEVFormer is a pure vision-based autonomous driving perception model based on the Transformer architecture.

[0014] LSS: LiDAR-BEV Feature Learning. The LSS algorithm represents the forward BEV (Bird's Eye View) feature construction method, which directly improves 2D image features to 3D view cones and then projects them onto the BEV space. It is suitable for pure vision or camera-LiDAR fusion scenarios.

[0015] DQN: Double Q-learning is an algorithm that combines deep learning and reinforcement learning. It was proposed by the DeepMind team in 2013 and was the first reinforcement learning model to successfully apply deep neural networks to high-dimensional inputs (such as images). It can learn policies directly from the original pixels and solves the limitations of traditional Q-learning when dealing with continuous or high-dimensional state spaces.

[0016] QR-DQN: Quantile Regression DQN is a network model that extends Q-value learning to quantile regression, directly models the reward distribution, and enhances the network's sensitivity to risk.

[0017] IQN (Implicit Quantile Network) is a network model that approximates the reward distribution using implicit quantization functions, supports flexible risk strategies, and is highly data-efficient.

[0018] TabNet: Attentive Interpretable Tabular Learning, is a neural network model for tabular data, proposed by Google Research in 2019. It achieves interpretable machine learning on tabular data through an attention mechanism.

[0019] XGBoost is a gradient boosting-based decision tree model that excels at tabular data classification and regression tasks. It builds strong predictive models by integrating multiple weak decision trees.

[0020] LightGBM is a gradient boosting framework that uses a leaf-wise growth strategy to optimize efficiency and is suitable for large-scale tabular data.

[0021] IMU: Inertial Measurement Unit.

[0022] GPS: Global Positioning System.

[0023] RTK: Real-time kinematic, is a high-precision positioning technology based on the Global Navigation Satellite System (GNSS). It achieves centimeter-level real-time positioning through carrier phase differential observation between a base station and a rover.

[0024] In related technologies, the driver assistance systems installed on vehicles generally rely on traditional path planning or rule-based judgment methods, lacking a precise understanding of terrain passability and a real-time feedback mechanism. Especially in variable environments such as potholes, water accumulation, gravel, and mud, blind path judgment, strategy delays, or passage failures often occur. This is because passage processing is generally based on images, lacking a unified linkage mechanism for multi-source data, resulting in insufficient accuracy in passage processing evaluation and making it difficult to meet current vehicle passage requirements.

[0025] Based on the above, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0026] The vehicle passage processing method proposed in the embodiments of this application includes a front-wheel drive vehicle, a rear-wheel drive vehicle, or a four-wheel drive vehicle. Preferably, it is a four-wheel drive vehicle, and more preferably an off-road vehicle. The vehicle passage processing method is applied to a vehicle controller, which includes an intelligent driving controller, an assisted driving controller, a vehicle-mounted infotainment system controller, or a power domain controller.

[0027] As shown in Figure 1, the method includes: Step 101, acquiring environmental perception data and vehicle driving data.

[0028] In practice, environmental perception data refers to the environmental data of the vehicle's location collected by the vehicle's environmental perception sensors. This environmental perception data includes: image data collected by an image acquisition device, and / or radar data (e.g., 3D point cloud data) collected by a radar detector.

[0029] Vehicle driving data refers to driving-related parameters collected during vehicle operation. Vehicle driving data includes at least one of the following: vehicle speed, lateral acceleration, longitudinal acceleration, yaw rate, vehicle attitude data (e.g., obtained using an IMU), and position data (e.g., obtained using GPS or RPK).

[0030] The vehicle controller is equipped with a data preprocessing module, which uses a synchronization clock mechanism to unify the timestamps of environmental perception data and vehicle driving data, ensuring that the environmental perception data and vehicle driving data can be accurately aligned at the same time.

[0031] The data preprocessing module also performs color normalization and size scaling on the image data in the environmental perception data, transforming the image data into a standardized format that can be input into the region extraction model. Furthermore, it performs data cleaning on the normalized and scaled image data, removing blurred, occluded, or heavily illuminated images to prevent them from affecting the region extraction model's processing.

[0032] The data preprocessing module uses an external matrix to perform coordinate transformation on the radar data, projects the radar data onto the plane corresponding to the image data, performs pixel-level calibration, and obtains accurate calibrated radar data.

[0033] The data preprocessing module uses a confidence level mechanism to perform dynamic information reliability assessment on vehicle driving data, and removes abnormal vehicle driving data with reliability issues such as GPS drift or IMU oscillation exceeding limits.

[0034] Step 102: The environmental perception data is processed by region extraction model to obtain region division results. The region extraction model is a pre-trained neural network model that can process environmental perception data into regions.

[0035] In specific implementation, the extraction model for this region includes: BEVFormer model and / or LSS model, preferably BEVFormer model.

[0036] After inputting the environmental perception data obtained above into the region extraction model, the region extraction model is used to determine the environmental area where the vehicle is located based on the environmental perception data, and the environmental area is divided into regions. The corresponding regions include at least one of the following: passable area, light obstacle area, impassable area and boundary buffer area.

[0037] The region extraction model outputs a two-dimensional image structure that maps to the actual geographical location and includes the corresponding regions. Each region is labeled with a confidence score.

[0038] Step 103: The region division result is processed by a traffic prediction model to predict the traffic strategy, and the traffic prediction model is a pre-trained neural network model that can extract the region division result output by the model and predict the traffic strategy.

[0039] In practice, the traffic prediction model includes: DQN model, QR-DQN model or IQN model, with DQN model being preferred.

[0040] The region division results output by the above region division model are used to determine multiple traffic strategies. Each traffic strategy is combined with the region division results to predict whether a vehicle can pass, thereby obtaining accurate prediction results for each traffic strategy, which are then output as traffic strategy prediction results.

[0041] Step 104: Based on the region division result, the traffic strategy prediction result, and the vehicle driving data, perform fusion analysis processing using a fusion analysis model to determine and output the traffic index of the traffic strategy prediction result. The fusion analysis model is a pre-trained neural network model capable of performing trafficability analysis on the traffic strategy prediction result.

[0042] In practice, the fusion analysis model includes: TabNet model, XGBoost model or LightGBM model, with TabNet model being preferred.

[0043] The passability index represents the passability of the corresponding passability strategy prediction result. The higher the passability index, the higher the passability and the higher the executability of the corresponding passability strategy prediction result.

[0044] The traffic prediction results obtained by the above traffic prediction model are based only on the regional differentiation results and are not accurate enough. It is necessary to integrate the regional division results, traffic strategy prediction results, and vehicle driving data using a synergistic analysis model. This will allow for a comprehensive analysis of the trafficability of the traffic strategy prediction results, determine the traffic index corresponding to each traffic strategy in the prediction results, and output each traffic strategy and its corresponding traffic index together. This will allow us to know the traffic index corresponding to various traffic strategies for the current vehicle. Based on the traffic index of each traffic strategy, we can provide the driver with risk warnings for the traffic strategy, or determine the vehicle's driving planning parameters based on the traffic index corresponding to each traffic strategy, and then control the vehicle's driving based on these driving planning parameters.

[0045] The above scheme acquires environmental perception data of the vehicle's surroundings and vehicle driving data. Based on the environmental perception data, a region extraction model can be used to divide the vehicle's environment into regions, resulting in region divisions that characterize the environmental conditions of the corresponding areas. These region divisions are then used to predict traffic strategies using a traffic prediction model, allowing for effective prediction and analysis of the environmental conditions within the region divisions and the determination of corresponding traffic strategy predictions. However, the accuracy of these traffic strategy predictions is not yet high enough. Therefore, the region division results, traffic strategy prediction results, and vehicle driving data will be combined... By combining these factors and then using a fusion analysis model for accurate fusion analysis, the traffic index of the determined traffic strategy prediction results can be made consistent with the environmental conditions corresponding to the regional division results and the driving conditions corresponding to the vehicle driving data. This ensures that the traffic index of the determined traffic strategy prediction results is more accurate. After the traffic index of the determined traffic strategy prediction results is output, users can know the corresponding traffic difficulty of the traffic strategy prediction results, which can assist users in making accurate vehicle driving control. Moreover, the traffic index of the traffic strategy prediction results can be output in real time, which can ensure the timely control effect of the vehicle and improve the vehicle's ability to pass through various complex environments.

[0046] In some embodiments, after the environmental perception data is input into the region extraction model in step 102, the processing in the region extraction model is as follows: Step 1021, the image data in the environmental perception data is extracted according to multiple predetermined scales to obtain multi-scale features, and the multi-scale features are aggregated to obtain aggregated image features.

[0047] In practice, the modeling layer of the region extraction model can perform semantic extraction based on the image data in the environmental perception data, and perform spatial distribution processing based on the semantic extraction results to map the image data to a top-down view to form a bird's-eye view.

[0048] The bird's-eye view is processed using the backbone network layer of the region extraction model at multiple predetermined scales to obtain multi-scale features.

[0049] Multi-scale features are aggregated using a region extraction model's multi-head attention mechanism to obtain aggregated image features. These aggregated image features enhance the recognition of unstructured terrain such as potholes, mud, and puddles, and exhibit strong completion capabilities, especially in occluded or poorly defined areas.

[0050] Step 1022: The radar data in the environmental perception data is mapped into a sparse density map. The sparse density map is then spatially aligned with the image data to obtain aligned graphic features.

[0051] In practice, the radar data is the 3D point cloud data detected by the lidar detector. By voxelizing the radar data, it can be mapped into a sparse density map. In order to ensure the spatial correspondence between the sparse density map and the image data, the sparse density map and the image data are spatially aligned, thus obtaining the aligned graphic features.

[0052] Step 1023: The aggregated image features and the aligned graphic features are fused together, and the terrain location features are determined based on the fusion process.

[0053] In practice, the image data of the current frame is fused with the aggregated image features of the image data of the preceding and following frames, along with the corresponding aligned graphic features, to obtain an accurate fusion result. Based on the fusion result, terrain location information is preserved through a location encoding mechanism, resulting in accurate terrain location features. This terrain location feature addresses the problem of incomplete observation in a single frame and also enables semantically preserved spatial transfer.

[0054] Step 1024: Determine at least one region based on the terrain location features, and output at least one region as the region division result.

[0055] In practice, the types of regions determined based on terrain location features include at least one of the following: traversable regions, lightly obstructed regions, impassable regions, and boundary buffer regions. Each region in the at least one division is labeled with its corresponding type, along with the confidence level that the region conforms to the labeled type. At least one region with the assigned label type and confidence level is taken as the region division result and output from the region extraction model.

[0056] The above scheme utilizes a region extraction model to accurately extract multi-scale features from image data. These multi-scale features are then aggregated to obtain aggregated image features, thereby enhancing the ability to identify various terrains. Furthermore, this region extraction model can map radar data into a sparse density map, and then spatially align this sparse density map with the image data. This ensures that the aligned graphic features and aggregated image features are spatially matched, facilitating the fusion of the aggregated image features with the aligned graphic features. The fusion result is more representative of the terrain characteristics, and the determined terrain location features are more accurate. Consequently, the region division based on these terrain location features yields more accurate region division results.

[0057] In some embodiments, step 1024 includes: step 10241, using an attention network layer to perform attention analysis on the terrain location features to determine the terrain features of the location range corresponding to the image data.

[0058] In practice, the region extraction model includes an attention network layer (e.g., DeformableAttention), which can perform attention analysis on terrain location features at different scales and accurately identify terrain features within the location range represented by the image data. This can adapt to the significant differences in complex road surfaces at different scales, improve the accuracy of identifying details such as gravel distribution and pothole density, and significantly improve the accuracy of terrain features.

[0059] Step 10242: Divide the location range into at least one region based on the terrain features, and determine the confidence level for each region.

[0060] In practice, the types of areas to be divided include: passable areas, lightly obstructed areas, impassable areas, and boundary buffer areas.

[0061] Based on the terrain features, it can be determined which of the above types the location belongs to, and nearby locations belonging to the same type are aggregated to form a region of the corresponding type. In this way, at least one region can be obtained, and the confidence level of its belonging to the type can be determined based on the similarity between the terrain features and the terrain features of the corresponding type.

[0062] Step 10243: Generate a semantic map based on the at least one region, and label the corresponding confidence level in each region of the semantic map to obtain a labeled semantic map. Output the labeled semantic map as the region partitioning result.

[0063] In practice, to ensure accurate output of at least one segmented region, a corresponding semantic map (e.g., a structured traffic semantic map) is generated for that region. The number of channels in this semantic map corresponds to the type of the region, and a confidence level is assigned to each region for subsequent calculation of the quality factor of the traffic strategy. This labeled semantic map is a two-dimensional image structure, where each pixel position maps to the actual geographical location. The vehicle coordinate system is then recovered through the inverse mapping of this two-dimensional image structure.

[0064] The above scheme ensures that at least one region is more accurately divided based on terrain features, and generates a corresponding semantic map for each region. The semantic map is labeled with the corresponding confidence level for subsequent calculation of the quality factor of the passage strategy. The labeled semantic map is a two-dimensional image structure, and each pixel position maps to the actual geographical location range. In this way, the vehicle coordinate system can be recovered by inverse mapping of the two-dimensional image structure, thereby achieving physical interpretability.

[0065] As a preferred embodiment, the training process of the region extraction model is as follows: (1) Pre-construct an initial region extraction model.

[0066] (2) Acquire a large amount of image data and radar data collected by the vehicle during driving. The image data is an image sequence ordered by time, and the radar data is also a point cloud feature ordered by time. The image data and radar data are processed with unified timestamps and the corresponding semantic map of the real area is marked to obtain the first training sample. These first training samples are combined to form the first training sample set.

[0067] (3) Input the image data and radar data of each first training sample in the first training sample set into the initial region extraction model. The initial region extraction model has a modeling layer to perform semantic extraction on the image data in the first training sample. Based on the semantic extraction results, perform spatial distribution processing to map the image data to the top view perspective to form a training bird's-eye view.

[0068] (4) The initial region extraction model also has a backbone network layer, which will extract the training bird's-eye view according to multiple predetermined scales to obtain multi-scale training features, and use the multi-head attention mechanism to aggregate the multi-scale training features to obtain training aggregated image features.

[0069] (5) The initial region extraction model has a three-dimensional processing layer, which can process the radar data in the first training sample into a training sparse density map through voxelization, and spatially align the training sparse density map with the image data in the first training sample to obtain the aligned training graphic features.

[0070] (6) The initial region extraction model has a fusion processing layer, which can fuse the training aggregated image features of the image data of the current processing frame with the image data of the previous and subsequent frames, as well as the corresponding aligned training graphic features, to obtain accurate training terrain location features.

[0071] (7) The initial region extraction model has an attention network layer, which can perform attention analysis on the training terrain location features at different scales, accurately identify the terrain features of the location range represented by the image data, and obtain the training terrain features.

[0072] (8) Based on the training terrain features, the location can be determined to be of which type, and nearby locations of the same type are aggregated to form a training region of the corresponding type. In this way, at least one training region can be obtained. Furthermore, the confidence level of the training terrain features and the corresponding type of training terrain features is determined based on their similarity. After generating the corresponding training semantic map from the at least one training region, the corresponding type and the corresponding confidence level are labeled for each training region in the training semantic map. Finally, the labeled training semantic map is output.

[0073] The types of areas that are divided include: passable areas, lightly obstructed areas, impassable areas, and boundary buffer areas.

[0074] (9) Compare the output training semantic map with the semantic map of the real region division, and adjust the parameters of the initial region extraction model according to the comparison error to obtain the adjusted initial region extraction model.

[0075] (10) For each first training sample in the first training sample set, repeat the above process (3) to (9) based on the previously adjusted initial region extraction model until the first training sample set is fully trained. The final adjusted initial region extraction model is then used as the region extraction model.

[0076] In some embodiments, the traffic prediction model includes: a pre-trained traffic strategy determination model and a pre-trained traffic analysis model.

[0077] Among them, the traffic strategy determination model and the trafficability analysis model are the DQN model, the QR-DQN model, or the IQN model, respectively, with the DQN model being preferred.

[0078] Step 103 includes: Step 1031, using the traffic strategy determination model to perform traffic operation analysis on the region division results, determining at least one traffic strategy and the quality factor of each traffic strategy.

[0079] In practice, the region segmentation results are combined with the attitude data from vehicle driving data, and the two are concatenated to form a state vector of a unified dimension. This state vector is then input into the traffic strategy determination model to perform traffic operation analysis, thereby determining various traffic operations (maintaining the current route, changing lanes to the left to avoid the obstacle, changing lanes to the right to detour, or stopping). Based on these traffic operations, at least one traffic strategy is then formulated, and the passability of each traffic strategy is predicted, resulting in a corresponding quality factor. A higher quality factor indicates a higher probability that the traffic strategy will be successful.

[0080] Step 1032: Using the trafficability analysis model, perform traffic capacity analysis on the at least one traffic strategy and the quality factor of each traffic strategy to determine the candidate traffic strategy from the at least one traffic strategy, and determine the traffic capacity value of the candidate traffic strategy. Use the candidate traffic strategy and the traffic capacity value of the candidate traffic strategy as the traffic strategy prediction result.

[0081] In practice, although the traffic strategy determination model outputs traffic strategies and their corresponding quality factors, the accuracy of these quality factors is not high enough. Therefore, at least one traffic strategy and its quality factor need to be input into the trafficability analysis model for further traffic capacity analysis. This allows for the selection of candidate traffic strategies from at least one strategy (if there is only one traffic strategy, that strategy is directly used as a candidate). Furthermore, the quality factors are combined to perform traffic capacity analysis on the candidate traffic strategies to determine accurate traffic capacity values. Finally, the candidate traffic strategies and their traffic capacity values ​​are output as the traffic strategy prediction results.

[0082] The above approach utilizes a traffic strategy determination model to analyze traffic operations in the region division results, thereby identifying at least one traffic strategy and its corresponding quality factor. To further improve prediction accuracy, a trafficability analysis model is used to perform accurate traffic capability analysis on each traffic strategy and its corresponding quality factor, selecting more accurate candidate traffic strategies and determining the traffic capability value corresponding to the candidate strategy. This value is then used as the output of the traffic strategy prediction result, ensuring that relatively accurate traffic strategy prediction results are obtained and providing reliable data support for subsequent fusion analysis.

[0083] In some embodiments, step 1031 includes: step 10311, using the vehicle center as a reference, determining a local observation view of the forward fan-shaped region in the region division result, and stitching the local observation view with the attitude data in the vehicle driving data to form a stitching feature.

[0084] In practice, a sliding window is used to extract the forward fan-shaped region located in front of the vehicle's center from the region segmentation results, obtaining a local observation view. The size of the fan-shaped region can be dynamically adjusted according to the vehicle speed to ensure that the vehicle has consistent forward perception capabilities at different speeds.

[0085] The obtained local observation view is then stitched together with the attitude data from the vehicle driving data to form a stitched feature of a state tensor with a unified dimension.

[0086] Step 10312: Input the spliced ​​features into the traffic strategy determination model. The processing in the traffic strategy determination model is as follows: Step 103121: Extract the state image features from the spliced ​​features, and determine the expected benefit value corresponding to the driving control operation based on the state image features.

[0087] In practical implementation, the traffic strategy determination model includes an operation determination layer that can extract accurate state image features from the stitched features. These state image features accurately represent the vehicle's current position. Then, the recorded trajectory of the vehicle within a test field constructed based on its environment is determined. Based on this trajectory and the obtained state image features, various driving control operations (e.g., maintaining the current route, changing lanes to the left to avoid an obstacle, changing lanes to the right to bypass an obstacle, or stopping) can be determined, thus determining whether each driving control operation can successfully pass through the target area. The expected reward value for each driving control operation is determined based on its success rate. Positive rewards are given for successful passage, and negative rewards are given for unsuccessful passage, thus obtaining the corresponding expected reward value.

[0088] Step 103122: Determine at least one traffic strategy based on the expected revenue value corresponding to the driving control operation, and determine the quality factor of the traffic strategy based on the expected revenue value of the driving control operation corresponding to each traffic strategy.

[0089] In practice, the traffic strategy determination model includes a traffic strategy screening layer, which can combine the expected revenue values ​​of each traffic control operation to select one or more traffic control operations whose expected revenue values ​​are greater than a set revenue threshold from multiple traffic control operations as traffic strategies, and use the corresponding expected revenue values ​​as the quality factor of the traffic strategy.

[0090] Step 103123: Output the at least one access strategy and the quality factor of each access strategy.

[0091] In practice, the travel strategy determination model has an output layer that can accurately output at least one travel strategy and its corresponding quality factor.

[0092] The above scheme can accurately stitch local observation views with vehicle attitude data within a certain period of time. Then, based on the obtained stitching features, state image features that can accurately represent the state of the vehicle's location can be extracted. This ensures that the expected benefit values ​​corresponding to each driving control operation determined based on the state image features are obtained. Then, accurate traffic strategies and their corresponding quality factor outputs are selected for subsequent accurate trafficability analysis.

[0093] As a preferred embodiment, the training process of the access strategy determination model is as follows: (1) Pre-construct an initial access strategy determination model.

[0094] (2) The sample region division result output by the above region division model after processing the first training sample set is obtained in advance. The forward fan-shaped region located in front of the vehicle center in the sample region division result is extracted and processed using a sliding window to obtain a sample local observation view. The sample local observation view is then stitched together with the corresponding vehicle attitude data to obtain the sample stitching feature. The sample stitching feature is used to mark the real traffic strategy as the second training sample. All the second training samples are combined into the second training sample set.

[0095] (3) Input each second training sample from the second training sample set into the initial traffic strategy determination model. Use the operation determination layer in the initial traffic strategy determination model to extract accurate training state image features from the sample splicing features, and record the vehicle's trajectory in the test field constructed based on its environment. Based on the recorded trajectory and the obtained training state image features, determine each driving control operation (e.g., maintain the current route, change lanes to the left to avoid, change lanes to the right to bypass, or stop moving forward). This recorded trajectory can accurately determine whether each driving control operation can pass through the target area in sequence.

[0096] If the target area can be traversed, the corresponding driving control operation will be given a positive reward; if the target area cannot be traversed (for example, encountering an obstacle that causes stagnation, skidding, or the need to detour), the corresponding driving control operation will be given a negative reward.

[0097] The training state image features, driving control operations, and reward values ​​are combined into a triplet and stored in the replay pool. Batch sampling training is then performed based on the replay pool to ensure the stability and convergence of the initial traffic policy determination model during the training process.

[0098] This allows us to obtain the accurate expected training return value for each individual driving control operation.

[0099] (4) The initial traffic strategy determination model has a traffic strategy screening layer, which includes two fully connected layers. One fully connected layer can combine the training expected return value corresponding to each driving control operation to select one or more driving control operations with a training expected return value greater than a set return threshold from multiple driving control operations as training traffic strategies. The other fully connected layer can use the training expected return value corresponding to the training traffic strategy as the training quality factor of the training traffic strategy.

[0100] (5) Determine the training pass strategy and its corresponding training quality factor, and the error between the training pass strategy and the labeled real pass strategy (which corresponds to the highest quality factor), and train and adjust the initial pass strategy determination model to obtain the adjusted initial pass strategy determination model.

[0101] (6) For each second training sample in the second training sample set, repeat the above process (3) to (5) based on the previously adjusted initial access strategy determination model until the training of the second training sample set is completed. The final adjusted initial access strategy determination model is then used as the access strategy determination model.

[0102] In some embodiments, in step 1032, the at least one access strategy and the quality factor of each access strategy are input into the accessibility analysis model. The processing procedure in the accessibility analysis model is as follows: Step 10321, based on the quality factor of each access strategy, determine the initial access strategy corresponding to the maximum quality factor.

[0103] In practice, the trafficability analysis model includes a normalization layer to normalize the quality factor of each traffic strategy. The resulting traffic strategies are then ranked according to their normalized quality factors, and the strategy with the largest normalized quality factor is selected as the initial traffic strategy. This initial traffic strategy is considered the recommended traffic strategy, and its corresponding normalized quality factor represents the vehicle's current traffic confidence.

[0104] Among all the passage strategies, the passage strategies other than the initial passage strategy are the remaining passage strategies.

[0105] Step 10322: Determine whether there is a target quality factor among the remaining quality factors of the passage strategy whose difference from the maximum quality factor is within a predetermined range.

[0106] In practice, the mobility analysis model includes a judgment analysis layer, which is used to determine whether there is a target quality factor among the remaining mobility strategies whose difference from the maximum quality factor is within a predetermined range after the initial mobility strategy is selected. If it exists, it proves that there is a target quality factor among the remaining mobility strategies that is close to the maximum quality factor; otherwise, it proves that there is no target quality factor close to the maximum quality factor.

[0107] Step 10323: In response to the existence of a target quality factor among the quality factors of the remaining passage strategies, where the difference between the target quality factor and the maximum quality factor is within a predetermined range, the target remaining passage strategy corresponding to the target quality factor is selected, and the target remaining passage strategy and the initial passage strategy are used as the candidate passage strategies. The passage capacity value of the candidate passage strategies is then determined as the passage strategy prediction result.

[0108] In practice, the mobility analysis model has a screening layer. When a target quality factor exists, the remaining mobility strategy corresponding to that target quality factor will be used as the target remaining mobility strategy and together with the initial mobility strategy as candidate mobility strategies.

[0109] The filtering layer is also used to verify the passage of each candidate passage strategy. If a candidate passage strategy exhibits abnormal passage behavior (e.g., skidding, deceleration, yaw, or abnormal turning), the quality factor (e.g., target quality factor or maximum quality factor) corresponding to that candidate passage strategy needs to be adjusted downwards, and the adjusted quality factor is used as the passage capability value of that candidate passage strategy. If a candidate passage strategy does not exhibit abnormal passage behavior, its corresponding quality factor (e.g., target quality factor or maximum quality factor) remains unchanged.

[0110] After all candidate travel strategies have been validated, their corresponding final quality factors are used as the travel capability values ​​of the candidate travel strategies. These travel capability values ​​are then added to the travel strategy prediction results.

[0111] Alternatively, in step 10324, in response to the absence of a target quality factor among the remaining quality factors of the travel strategy that has a difference within a predetermined range from the maximum quality factor, the initial travel strategy is taken as the candidate travel strategy, and the travel capacity value of the candidate travel strategy is determined as the travel strategy prediction result.

[0112] In practice, the mobility analysis model includes a screening layer. This screening layer is used to determine the target quality factor that does not have a difference from the maximum quality factor within a predetermined range. The initial mobility strategy is then directly used as the candidate mobility strategy, and the maximum quality factor of the initial mobility strategy is used as the mobility capability value. The mobility capability value of the candidate mobility strategy is then added to the mobility strategy prediction result.

[0113] The final filtering layer outputs the predicted traffic strategies obtained through the traffic analysis model's output layer.

[0114] The above approach allows for the use of traffic analysis models to analyze traffic flow in conjunction with quality factors, thereby identifying candidate traffic flow strategies with a certain degree of accessibility. Furthermore, it enables the determination of accurate traffic capacity values ​​for each candidate strategy in conjunction with quality factors, thus improving the accuracy of traffic flow strategy prediction results.

[0115] As a preferred embodiment, the training process of the traffic analysis model includes: (1) inputting each second training sample in the second training sample set into the traffic strategy determination model, obtaining at least one sample traffic strategy and a quality factor corresponding to each sample traffic strategy; and marking the corresponding real traffic strategy prediction results as third training samples using the at least one sample traffic strategy and the quality factor corresponding to each sample traffic strategy; and combining these third training samples to obtain a third training sample set.

[0116] (2) Pre-construct an initial traffic analysis model.

[0117] (3) Input the third training sample into the initial traffic analysis model in sequence. Use the normalization processing layer in the initial traffic analysis model to normalize the quality factor of each traffic strategy in the third training sample. Sort the traffic strategies of the samples according to the normalized quality factors and select the traffic strategy of the sample with the largest normalized quality factor as the initial sample traffic strategy.

[0118] (4) The initial passability analysis model is provided with a judgment analysis layer, which is used to determine whether there is a target quality factor whose difference with the maximum quality factor is within a predetermined range from the remaining sample passability strategies after the initial sample passability strategy is selected.

[0119] (5) The initial mobility analysis model includes a screening layer. When a target quality factor exists, the remaining sample mobility strategy corresponding to that target quality factor is used as the target remaining sample mobility strategy, and this strategy, along with the initial sample mobility strategy, is used as a candidate sample mobility strategy. Each candidate sample mobility strategy is validated. If an abnormal mobility behavior occurs in a candidate sample mobility strategy, the quality factor corresponding to that strategy is adjusted downwards, and the adjusted quality factor is used as the mobility value of that candidate sample mobility strategy. If no abnormal mobility behavior occurs in a candidate sample mobility strategy, its corresponding quality factor remains unchanged. After all candidate sample mobility strategies have been validated, their final quality factors are used as the mobility values ​​of the candidate sample mobility strategies. The mobility values ​​of the candidate sample mobility strategies are then added to the training mobility strategy prediction results.

[0120] Alternatively, if the screening layer determines that there is no target quality factor whose difference from the maximum quality factor is within a predetermined range, it can directly use the initial sample passage strategy as the candidate sample passage strategy, use the maximum quality factor of the initial sample passage strategy as the passage capability value, and add the passage capability value of the candidate sample passage strategy to the training passage strategy prediction result.

[0121] The filtering layer outputs the training traffic policy prediction results obtained through the training layer into the output layer of the initial traffic analysis model.

[0122] (6) Determine whether the training traffic strategy prediction result matches the actual traffic strategy prediction result. If they do not match, adjust the parameters of the initial traffic analysis model and then train it according to the above steps (3) to (5) so that the training traffic strategy prediction result output by the adjusted initial traffic analysis model matches the actual traffic strategy prediction result. If they match, train the next third training sample according to the above steps (3) to (5).

[0123] (7) Once all the third training sample sets have been trained, or the accuracy of the adjusted initial traffic analysis model has reached a predetermined value, the training is considered complete, and the final adjusted initial traffic analysis model is used as the traffic analysis model.

[0124] In some embodiments, step 104 includes: step 1041, performing fusion processing based on the region division result, the traffic strategy prediction result, and the vehicle driving data to obtain a fused feature vector.

[0125] In practice, in order to input the fusion analysis model, the obtained regional division results, traffic strategy prediction results and vehicle driving data need to be fused to obtain a unified standard fusion feature vector.

[0126] Step 1042: Input the fused feature vector into the fusion analysis model. The processing procedure in the fusion analysis model is as follows: Step 10421: Based on the fused feature vector, use the first decision layer in the fusion analysis model to determine the environmental conditions and obtain the current environmental conditions of the vehicle.

[0127] In practice, the first decision layer can determine the environmental conditions of the fused feature vector to obtain the accurate current environmental conditions of the vehicle. The current environmental conditions are used to characterize the environmental conditions at the vehicle's location, and include at least one of the following: muddy conditions, rainy conditions, snowy road conditions, stable conditions, uphill conditions, downhill conditions, and gravel road conditions.

[0128] Step 10422: Using the second decision layer in the fusion analysis model, perform condition adaptability analysis on the fusion feature vector based on the current environmental conditions, and determine the adaptability index of the traffic strategy prediction results in the fusion feature vector.

[0129] In practice, this second decision-making layer determines the corresponding condition adaptability analysis strategy based on the current environmental conditions. It then performs condition adaptability analysis on the fused feature vector according to this strategy, determining the adaptability index corresponding to the predicted travel strategy within the fused feature vector. This adaptability index characterizes the degree of matching between the predicted travel strategy and the condition adaptability analysis strategy; a higher adaptability index indicates a higher degree of matching, and a greater probability that the travel strategy will successfully pass through.

[0130] Step 10423: Using the third decision layer in the fusion analysis model, determine the passage index based on the adaptability index of the passage strategy prediction results, and output the passage index of the passage strategy prediction results.

[0131] In practice, the third decision-making layer can normalize the adaptability index to obtain an accurate traffic index. It outputs the traffic index of each traffic strategy in the traffic strategy prediction results. The higher the traffic index, the higher the passability of the corresponding traffic strategy. In this way, risk analysis can be carried out based on the traffic index, and accurate target traffic strategies can be selected to control vehicle driving.

[0132] The first, second, and third decision layers all have sparse attention mechanisms that can perform hierarchical fusion of features of different dimensions. Each of the first, second, and third decision layers has a feature selection module to dynamically weight the results (such as the current environmental conditions of the first decision layer, the adaptability index of the second decision layer, and the travel index of the third decision layer).

[0133] The above scheme can combine the regional division results, vehicle driving data and traffic strategy prediction results using a fusion analysis model to accurately determine the current environmental conditions of the vehicles. Then, based on the current environmental conditions, an accurate condition adaptability analysis can be performed on the traffic strategy prediction results to ensure the accuracy of the adaptability index of the determined traffic strategy prediction results. In order to facilitate analysis, the adaptability index is converted into a traffic index output so as to conduct risk analysis based on the traffic index and select accurate target traffic strategies to control vehicle driving.

[0134] In some embodiments, step 1041 includes: step 10411, determining the dynamic driving characteristics of the vehicle based on the vehicle driving data.

[0135] In practice, dynamic driving characteristics are parameter data that characterize the dynamic state of a vehicle, including at least one of the following: lateral acceleration, rate of change of vehicle speed, yaw rate, vehicle speed, and pitch angle.

[0136] Step 10412: Determine the undulation of the terrain where the vehicle is located based on the image data or radar data in the environmental perception data.

[0137] In practice, the location of terrain undulations is determined based on image data, and the change in undulation height corresponding to that location is determined based on radar data. The change in undulation height is then used as the amount of undulation in the terrain where the vehicle is located.

[0138] Step 10413: Obtain the prediction results of the traffic strategy for the historical time period and the corresponding historical traffic index, and determine the average value of the historical traffic index.

[0139] In practice, to ensure accuracy, the prediction results of traffic strategies and the corresponding historical traffic indices for historical time periods before the current moment (e.g., the previous 3 minutes) will be determined, and the average index of the traffic indices corresponding to each traffic strategy within the historical time period will be calculated.

[0140] Step 10414: The region division result, the traffic strategy prediction result, the dynamic driving characteristics, the undulation amount, and the index average are fused according to a predetermined vector scale to obtain the fused feature vector.

[0141] In practice, this fused feature vector has at least five dimensions: region segmentation results, traffic strategy prediction results, dynamic driving characteristics, undulations, and the average index. This allows for the use of a fusion analysis model to combine data from these five dimensions and obtain an accurate traffic index based on the traffic strategy prediction results.

[0142] The above scheme can integrate at least five dimensions, including the regional division results, traffic strategy prediction results, dynamic driving characteristics, fluctuations, and index average, to ensure that the resulting fused feature vector has the characteristics of these five dimensions. This makes the traffic index obtained after specific analysis by the fusion analysis model more accurate.

[0143] As a preferred embodiment, the training process of the fusion analysis model is as follows: (1) The sample region division result output by the region division model after processing the first training sample set again, the training traffic strategy prediction result obtained by the traffic strategy determination model and the trafficability analysis model after processing the sample region division result, the sample dynamic driving characteristics, sample fluctuation amount and sample index average value synchronized with the first training sample are fused to obtain the sample fusion feature vector, and the real traffic index of the corresponding training traffic strategy prediction result is marked as the fourth training sample, and these fourth training samples are combined to obtain the fourth training sample set.

[0144] Among them, the dynamic driving characteristics of the sample vehicles are determined based on the driving data of the sample vehicles.

[0145] Sample undulation: Based on image data or radar data in the sample environment perception data, determine the sample undulation of the terrain where the vehicle is located.

[0146] Average sample index: Obtain the prediction results of the traffic strategy and the corresponding historical traffic index for the historical time period before the collection of the first sample data, and determine the average sample index of the historical traffic index.

[0147] (2) Pre-construct the initial fusion analysis model.

[0148] (3) Input the fourth training sample into the initial fusion analysis model in sequence, and use the first decision layer to determine the environmental conditions of the sample fusion feature vector in the fourth training sample to obtain the accurate sample environmental conditions of the vehicle's location. The current environmental conditions are used to characterize the environmental conditions of the vehicle's location, and the current environmental conditions include at least one of the following: muddy conditions, rainy conditions, snowy road conditions, stable conditions, uphill conditions, downhill conditions, and gravel road conditions.

[0149] (4) Using the second decision layer in the initial fusion analysis model, determine the corresponding working condition adaptation analysis strategy based on the sample environment working condition, perform working condition adaptation analysis on the sample fusion feature vector according to the corresponding working condition adaptation analysis strategy, and determine the training adaptation index corresponding to the training pass strategy prediction result in the sample fusion feature vector.

[0150] (5) Using the third decision layer in the initial fusion analysis model, the training adaptability index is normalized to obtain the accurate training pass index. The training pass index of each pass strategy in the pass strategy prediction result is sent to the output layer of the initial fusion analysis model for output.

[0151] (6) Determine whether the training access index matches the labeled real access index. If they do not match, adjust the parameters of the initial fusion analysis model and then train it according to the above steps (3) to (5) so that the training access index output by the adjusted initial fusion analysis model matches the labeled real access index. If they match, train the next fourth training sample according to the above steps (3) to (5).

[0152] (7) Once all fourth training sample sets have been trained, or the accuracy of the adjusted initial fusion analysis model has reached a predetermined value, the training is considered complete, and the final adjusted initial fusion analysis model is used as the fusion analysis model.

[0153] The trained region extraction model, traffic strategy determination model, trafficability analysis model, and fusion analysis model are all added to the vehicle controller. This allows the vehicle controller to use these models to predict traffic strategies and determine the traffic indices corresponding to each strategy. This ensures timely vehicle control based on the traffic indices corresponding to the traffic strategies and improves the vehicle's ability to navigate in various complex environments.

[0154] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0155] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0156] Based on the same inventive concept, and corresponding to any of the above embodiments, this application also provides a vehicle passage processing device.

[0157] Referring to Figure 2, the device includes: an acquisition module 201 configured to acquire environmental perception data and vehicle driving data; a region division module 202 configured to perform region division processing on the environmental perception data using a region extraction model to obtain a region division result, wherein the region extraction model is a pre-trained neural network model capable of performing region division processing on the environmental perception data; a traffic strategy prediction module 203 configured to perform traffic strategy prediction processing on the region division result using a traffic prediction model to obtain a traffic strategy prediction result, wherein the traffic prediction model is a pre-trained neural network model capable of predicting traffic strategies based on the region division result output by the region extraction model; and a fusion analysis module 204 configured to perform fusion analysis processing on the region division result, the traffic strategy prediction result, and the vehicle driving data using a fusion analysis model to determine and output the traffic index of the traffic strategy prediction result, wherein the fusion analysis model is a pre-trained neural network model capable of performing trafficability analysis on the traffic strategy prediction result.

[0158] In some embodiments, the region segmentation module 202 is specifically configured to: input the environmental perception data into the region extraction model, and then perform the following processing in the region extraction model: extract the image data in the environmental perception data according to multiple predetermined scales to obtain multi-scale features, and aggregate the multi-scale features to obtain aggregated image features; perform mapping processing on the radar data in the environmental perception data to map it into a sparse density map, and spatially align the sparse density map with the image data to obtain aligned graphic features; fuse the aggregated image features with the aligned graphic features, and determine the terrain location features based on the fusion processing result; determine at least one region based on the terrain location features, and output at least one region as the region segmentation result.

[0159] In some embodiments, the region segmentation module 202 is further configured to: perform attention analysis on the terrain location features using an attention network layer to determine the terrain features of the location range corresponding to the image data; divide the location range into at least one region based on the terrain features, and determine the confidence level for each region; generate a semantic map based on the at least one region, and label the corresponding confidence level for each region in the semantic map to obtain a labeled semantic map, and output the labeled semantic map as the region segmentation result.

[0160] In some embodiments, the traffic prediction model includes: a pre-trained traffic strategy determination model and a pre-trained trafficability analysis model; the traffic strategy prediction module 203 is specifically configured to: analyze the traffic operation using the traffic strategy determination model based on the region division results to determine at least one traffic strategy and a quality factor for each traffic strategy; analyze the traffic capability of the at least one traffic strategy and the quality factor of each traffic strategy using the trafficability analysis model to determine candidate traffic strategies from the at least one traffic strategy and determine the traffic capability value of the candidate traffic strategy; and use the candidate traffic strategy and the traffic capability value of the candidate traffic strategy as the traffic strategy prediction result.

[0161] In some embodiments, the traffic strategy prediction module 203 is further configured to: determine a local observation view of the forward fan-shaped region in the region division result based on the vehicle center; stitch the local observation view with the attitude data in the vehicle driving data to form a stitched feature; input the stitched feature into the traffic strategy determination model; the processing in the traffic strategy determination model is as follows: extract state image features from the stitched feature; determine the expected reward value corresponding to the driving control operation based on the state image features; determine at least one traffic strategy based on the expected reward value corresponding to the driving control operation; and determine the quality factor of the traffic strategy based on the expected reward value of the driving control operation corresponding to each traffic strategy; and output the at least one traffic strategy and the quality factor of each traffic strategy.

[0162] In some embodiments, the traffic strategy prediction module 203 is further configured to: input the at least one traffic strategy and the quality factor of each traffic strategy into the trafficability analysis model, wherein the processing procedure in the trafficability analysis model is as follows: based on the quality factor of each traffic strategy, determine the initial traffic strategy corresponding to the maximum quality factor; determine whether there exists a target quality factor among the remaining traffic strategy quality factors whose difference from the maximum quality factor is within a predetermined range; in response to the existence of a target quality factor among the remaining traffic strategy quality factors whose difference from the maximum quality factor is within a predetermined range, select the target remaining traffic strategy corresponding to the target quality factor, and use the target remaining traffic strategy and the initial traffic strategy as the candidate traffic strategy, and determine the traffic capacity value of the candidate traffic strategy as the traffic strategy prediction result; or, in response to the existence of a target quality factor among the remaining traffic strategy quality factors whose difference from the maximum quality factor is within a predetermined range, select the initial traffic strategy as the candidate traffic strategy, and determine the traffic capacity value of the candidate traffic strategy as the traffic strategy prediction result.

[0163] In some embodiments, the fusion analysis module 204 is specifically configured to: perform fusion processing based on the region division result, the traffic strategy prediction result, and the vehicle driving data to obtain a fusion feature vector; input the fusion feature vector into the fusion analysis model, wherein the processing in the fusion analysis model is as follows: based on the fusion feature vector, use the first decision layer in the fusion analysis model to determine the environmental conditions and obtain the current environmental conditions of the vehicle; use the second decision layer in the fusion analysis model to perform condition adaptability analysis on the fusion feature vector based on the current environmental conditions to determine the adaptability index of the traffic strategy prediction result in the fusion feature vector; use the third decision layer in the fusion analysis model to determine the traffic index based on the adaptability index of the traffic strategy prediction result, and output the traffic index of the traffic strategy prediction result.

[0164] In some embodiments, the fusion analysis module 204 is further configured to: determine the dynamic driving characteristics of the vehicle based on the vehicle driving data; determine the undulation of the terrain where the vehicle is located based on the image data or radar data in the environmental perception data; obtain the prediction results of traffic strategies and the corresponding historical traffic indices for historical time periods, and determine the index average of the historical traffic indices; and fuse the region division results, the traffic strategy prediction results, the dynamic driving characteristics, the undulation, and the index average according to a predetermined vector scale to obtain the fused feature vector.

[0165] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0166] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0167] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described in any of the above embodiments.

[0168] Figure 3 shows a more specific hardware structure diagram of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0169] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0170] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0171] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0172] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0173] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0174] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0175] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0176] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the methods described in any of the above embodiments.

[0177] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0178] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0179] Based on the same concept, corresponding to any of the above embodiments, this application also provides a computer program product, including computer program instructions, which, when run on a computer, cause the computer to perform the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0180] Based on the same inventive concept, this application also provides a vehicle including the device or electronic device described in the above embodiments. The beneficial effects of embodiments having corresponding devices or electronic devices will not be elaborated further here.

[0181] It is understood that before using the technical solutions of the various embodiments in this application, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0182] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations described in this application.

[0183] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0184] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0185] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0186] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0187] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0188] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for handling vehicle traffic, characterized in that, include: Acquire environmental perception data and vehicle driving data; The environmental perception data is processed into regions using a region extraction model to obtain region division results. The region extraction model is a pre-trained neural network model capable of processing environmental perception data into regions. The region division results are then processed into traffic strategy prediction results using a traffic prediction model. The traffic prediction model is a pre-trained neural network model capable of predicting traffic strategies based on the region division results output by the region extraction model. Based on the region division results, the traffic strategy prediction results, and the vehicle driving data, a fusion analysis model is used to perform fusion analysis to determine and output the traffic index of the traffic strategy prediction results. The fusion analysis model is a pre-trained neural network model capable of performing trafficability analysis on the traffic strategy prediction results.

2. The method according to claim 1, characterized in that, The step of processing the environmental perception data into regions using a region extraction model to obtain region division results includes: inputting the environmental perception data into the region extraction model, and the processing in the region extraction model is as follows: extracting image data from the environmental perception data according to multiple predetermined scales to obtain multi-scale features, and aggregating the multi-scale features to obtain aggregated image features; mapping radar data from the environmental perception data to a sparse density map, spatially aligning the sparse density map with the image data to obtain aligned graphic features; fusing the aggregated image features with the aligned graphic features, and determining terrain location features based on the fusion result; determining at least one region based on the terrain location features, and outputting at least one region as the region division result.

3. The method according to claim 2, characterized in that, The step of determining at least one region based on the terrain location features and outputting at least one region as the region segmentation result includes: performing attention analysis on the terrain location features using an attention network layer to determine the terrain features of the location range corresponding to the image data; dividing the location range into at least one region based on the terrain features and determining the confidence level for each region; generating a semantic map based on the at least one region, and marking the corresponding confidence level for each region in the semantic map to obtain a marked semantic map; and outputting the marked semantic map as the region segmentation result.

4. The method according to claim 1, characterized in that, The traffic prediction model includes: a pre-trained traffic strategy determination model and a pre-trained trafficability analysis model. The process of using the traffic prediction model to predict traffic strategies based on the region division results to obtain traffic strategy prediction results includes: analyzing traffic operation based on the traffic strategy determination model to determine at least one traffic strategy and a quality factor for each traffic strategy; analyzing the traffic capability of the at least one traffic strategy and the quality factor of each traffic strategy using the trafficability analysis model to determine candidate traffic strategies from the at least one traffic strategy and determine the traffic capability value of the candidate traffic strategy; and using the candidate traffic strategy and the traffic capability value of the candidate traffic strategy as the traffic strategy prediction result.

5. The method according to claim 4, characterized in that, The step of analyzing traffic operation using the traffic strategy determination model based on the region division results to determine at least one traffic strategy and a quality factor for each traffic strategy includes: determining a local observation view of the forward fan-shaped region in the region division results with the vehicle center as the reference; stitching the local observation view with the attitude data in the vehicle driving data to form a stitched feature; inputting the stitched feature into the traffic strategy determination model; the processing in the traffic strategy determination model includes: extracting state image features from the stitched feature; determining the expected reward value corresponding to the driving control operation based on the state image features; determining at least one traffic strategy based on the expected reward value corresponding to the driving control operation; and determining the quality factor of the traffic strategy based on the expected reward value of the driving control operation corresponding to each traffic strategy; and outputting the at least one traffic strategy and the quality factor of each traffic strategy.

6. The method according to claim 4, characterized in that, The step of performing traffic capacity analysis using the trafficability analysis model on the at least one traffic strategy and the quality factor of each traffic strategy, determining candidate traffic strategies from the at least one traffic strategy, determining the traffic capacity value of the candidate traffic strategy, and using the candidate traffic strategy and the traffic capacity value of the candidate traffic strategy as the traffic strategy prediction result includes: inputting the at least one traffic strategy and the quality factor of each traffic strategy into the trafficability analysis model; the processing procedure in the trafficability analysis model is as follows: determining the initial traffic strategy corresponding to the largest quality factor based on the quality factor of each traffic strategy; determining whether there exists a traffic strategy among the remaining traffic strategies whose quality factors match the largest quality factor. A target quality factor whose difference is within a predetermined range; in response to the remaining quality factors of the travel strategy, if there is a target quality factor whose difference with the maximum quality factor is within a predetermined range, the target remaining travel strategy corresponding to the target quality factor is selected, and the target remaining travel strategy and the initial travel strategy are selected as the candidate travel strategies, and the travel capacity value of the candidate travel strategy is determined as the travel strategy prediction result; or, in response to the remaining quality factors of the travel strategy, if there is no target quality factor whose difference with the maximum quality factor is within a predetermined range, the initial travel strategy is selected as the candidate travel strategy, and the travel capacity value of the candidate travel strategy is determined as the travel strategy prediction result.

7. The method according to claim 1, characterized in that, The step of performing fusion analysis based on the region division results, the traffic strategy prediction results, and the vehicle driving data, and determining and outputting the traffic index of the traffic strategy prediction results, includes: performing fusion processing on the region division results, the traffic strategy prediction results, and the vehicle driving data to obtain a fusion feature vector; inputting the fusion feature vector into the fusion analysis model, where the processing steps are as follows: determining the current environmental conditions of the vehicle using the first decision layer of the fusion analysis model based on the fusion feature vector; performing condition adaptability analysis on the fusion feature vector based on the current environmental conditions using the second decision layer of the fusion analysis model to determine the adaptability index of the traffic strategy prediction results in the fusion feature vector; determining the traffic index based on the adaptability index of the traffic strategy prediction results using the third decision layer of the fusion analysis model, and outputting the traffic index of the traffic strategy prediction results.

8. The method according to claim 7, characterized in that, The step of fusing the region division results and the traffic strategy prediction results to obtain a fused feature vector includes: determining the dynamic driving characteristics of the vehicle based on the vehicle driving data; determining the undulation of the terrain where the vehicle is located based on image data or radar data in the environmental perception data; obtaining the traffic strategy prediction results and corresponding historical traffic indices for historical time periods, and determining the index average of the historical traffic indices; and fusing the region division results, the traffic strategy prediction results, the dynamic driving characteristics, the undulation, and the index average according to a predetermined vector scale to obtain the fused feature vector.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 8.

10. A vehicle, characterized in that, Includes the electronic device as described in claim 9.

Images