Automatic driving perception method and device based on dynamic visual angle selection, electronic equipment and storage medium

Abstract

The invention relates to the technical field of automatic driving decisions, in particular to an automatic driving perception method and device based on dynamic visual angle selection, electronic equipment and a storage medium. According to the method, the key view angle is intelligently selected for processing through a dynamic routing decision-making mechanism, full-view-angle calculation redundancy is avoided, the system calculation load and power consumption are directly and greatly reduced, and the processing efficiency is improved; meanwhile, the mechanism adjusts an expert network activation strategy in real time based on multi-modal information fusion, so that the complex scene adaptability and perception accuracy are enhanced, and the problem of unsmooth control caused by view angle switching oscillation is improved; in addition, by introducing a multi-level security bottom-taking strategy formed by dual real-time verification and periodic global perception, the risk of wrong decision making is effectively reduced, and the reliability and security of system operation are comprehensively improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving decision-making technology, and in particular to an autonomous driving perception method, device, electronic device, and storage medium based on dynamic perspective selection. Background Art

[0002] Currently, there are three main solutions for autonomous driving perception technology: the first is a multi-sensor full fusion solution (such as Tesla's FSD and Waymo systems), which achieves panoramic perception by synchronously processing all camera data and fusing them in the BEV space, but the computational load is huge; the second is a rule-driven perspective selection solution (such as traditional ADAS blind spot monitoring), which triggers specific perspectives based on simple signals (such as turn signals). Although this improves efficiency, the decision-making is rigid and difficult to adapt to complex scenarios; the third is a static expert model solution (such as DriveMOE), which draws on the MoE concept to pre-set multiple expert networks, but the expert selection lacks real-time dynamic adjustment capabilities and cannot deeply integrate spatiotemporal context and navigation intent.

[0003] Existing autonomous driving perception technology has three major flaws: First, there is a serious efficiency bottleneck. The full-view fusion solution generates up to 70% computational redundancy due to the need to process all sensor data in parallel, resulting in end-to-end delays exceeding 50ms and system power consumption exceeding 100W, which cannot meet the real-time requirements of high-speed scenarios; second, the perception quality is insufficient. The rule-driven solution has poor adaptability and the static expert system cannot dynamically respond to changes in complex scenarios. At the same time, information overload will overwhelm key features. In addition, the lack of temporal continuity constraints leads to perspective switching oscillations. Summary of the Invention In view of this, the purpose of the present invention is to provide an autonomous driving perception method, device and electronic device storage medium based on dynamic perspective selection, so as to reduce the amount of calculation, improve the efficiency of autonomous driving strategy generation, perception accuracy and safety reliability.

[0004] In a first aspect, an embodiment of the present invention provides an autonomous driving perception method based on dynamic view selection, the method comprising: Determine the scene fusion characteristics of the driving environment based on the perception state information, navigation information and historical environmental feature information of the current time step during the vehicle's driving process; Dynamic routing decisions are made based on scene fusion features to generate activation probability distributions corresponding to multiple visual expert networks; Activate the first visual expert network with the highest activation probability in the probability distribution, and generate an initial autonomous driving strategy based on the image data of the corresponding perspective of the first visual expert network; Determine the target visual expert network based on the reliability score of the initial autonomous driving strategy; Activate the target visual expert network and generate the target autonomous driving strategy based on the image data of the perspective corresponding to the target visual expert network.

[0005] In conjunction with the first aspect, the step of determining the scene fusion characteristics of the driving environment based on the perception state information, navigation information, and historical environmental feature information at the current time step during the vehicle's driving process includes: Feature extraction is performed on the perception state information, navigation information, and historical environmental feature information of the current time step during vehicle driving to obtain the corresponding forward-view image feature embedding, navigation target vector, and historical bird's-eye view feature sequence; The forward-view image feature embedding is used as the query vector, and the navigation target vector and the historical bird's-eye view feature sequence are concatenated or fused as the key vector and value vector; Cross-attention calculation is performed based on the query vector, key vector and value vector, and scene fusion features are output.

[0006] In combination with the first aspect, the steps of making dynamic routing decisions based on scene fusion features and generating activation probability distributions corresponding to multiple visual expert networks include: Input the scene fusion features into the routing decision engine; The scene fusion features are processed by the probabilistic prediction head of the routing decision engine; wherein the probabilistic prediction head includes at least one fully connected layer for mapping the scene fusion features into a high-dimensional feature representation; The high-dimensional feature representation is probabilistically normalized to generate activation probability distributions of multiple visual expert networks.

[0007] In combination with the first aspect, the reliability score includes a confidence score and a continuity score; after the steps of activating the first visual expert network with the highest activation probability in the probability distribution and generating an initial autonomous driving strategy based on image data corresponding to the perspective of the first visual expert network, the method further includes: Input the initial autonomous driving strategy into the confidence evaluation network and output the confidence score of the initial autonomous driving strategy; The activation probability distribution of the current time step and the activation probability distribution of multiple consecutive historical time steps are input into the recurrent neural network, and a continuity score representing the consistency between the initial autonomous driving strategy and the historical autonomous driving strategy is output.

[0008] Combined with the first aspect, the reliability score includes a confidence score; combined with the reliability score of the initial autonomous driving policy, the steps for determining the target visual expert network include: Determine whether the confidence score of the initial autonomous driving strategy is greater than the confidence threshold; If not, the preset safe driving strategy is triggered and all visual expert networks are used as the target visual expert network.

[0009] In combination with the first aspect, the reliability score also includes a continuity score; after the step of determining whether the confidence score of the initial autonomous driving strategy is greater than the confidence threshold, the following steps are further included: If so, determining whether the continuity score is greater than a preset continuity score threshold; If so, the first visual expert network is used as the target visual expert network, and the initial autonomous driving policy is used as the target autonomous driving policy; If not, the second visual expert network corresponding to the autonomous driving strategy of the previous time step is used as the target visual expert network.

[0010] In combination with the first aspect, the method further includes: The entire visual expert network is periodically activated to combine image data from all perspectives to generate the target autonomous driving strategy.

[0011] In a second aspect, an embodiment of the present application further provides an autonomous driving perception device based on dynamic perspective selection, the device comprising: A fusion module is used to determine the scene fusion characteristics of the driving environment based on the perception state information, navigation information and historical environmental feature information of the current time step during the vehicle's driving process; A probability generation module is used to make dynamic routing decisions based on scene fusion features and generate activation probability distributions corresponding to multiple visual expert networks; An initial strategy generation module is used to activate the first visual expert network with the highest activation probability in the probability distribution, and generate an initial autonomous driving strategy based on the image data of the corresponding perspective of the first visual expert network; A determination module is used to determine the target visual expert network by combining the confidence score and continuity score of the initial autonomous driving strategy; The target strategy generation module is used to activate the target visual expert network and generate the target autonomous driving strategy based on the image data of the perspective corresponding to the target visual expert network.

[0012] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the above method.

[0013] In a fourth aspect, the present application provides a storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the above method is executed.

[0014] The embodiments of the present invention bring the following beneficial effects: The present application provides an autonomous driving perception method, device, electronic device, and storage medium based on dynamic perspective selection, the method comprising: determining the scene fusion characteristics of the driving environment based on the perception state information, navigation information, and historical environmental feature information of the current time step during vehicle driving; making dynamic routing decisions based on the scene fusion characteristics, and generating an activation probability distribution corresponding to multiple visual expert networks; activating the first visual expert network with the highest activation probability in the probability distribution, and generating an initial autonomous driving strategy based on the image data of the corresponding perspective of the first visual expert network; determining the target visual expert network in combination with the reliability score of the initial autonomous driving strategy; activating the target visual expert network, and generating the target autonomous driving strategy based on the image data of the perspective corresponding to the target visual expert network.

[0015] This application uses a dynamic routing decision mechanism to intelligently select key perspectives for processing, avoiding full-perspective computational redundancy, thereby directly and significantly reducing the system's computational load and power consumption and improving processing efficiency. At the same time, the mechanism adjusts the expert network activation strategy in real time based on multimodal information fusion, enhancing adaptability to complex scenarios and perception accuracy, and improving control irregularities caused by perspective switching oscillations. In addition, by introducing a multi-level safety backup strategy consisting of double real-time verification and periodic global perception, the risk of erroneous decisions is effectively reduced, and the reliability and security of system operation are comprehensively improved.

[0016] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the present invention. The purposes and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic diagram of the process of the autonomous driving perception method based on dynamic view selection provided in Example 1 of the present invention; Figure 2 A schematic diagram of a flow chart of an autonomous driving perception method based on dynamic viewpoint selection provided in Example 2 of the present invention; Figure 3 A schematic diagram illustrating the principle of data fusion to obtain scene fusion features according to an embodiment of the present invention; Figure 4 A schematic diagram of the information flow for driving decision-making in an autonomous driving perception method based on dynamic viewpoint selection provided by an embodiment of the present invention; Figure 5 A schematic diagram of a flow chart of an autonomous driving perception method based on dynamic viewpoint selection provided in Example 3 of the present invention; Figure 6 A schematic diagram illustrating the principle of an autonomous driving perception method based on dynamic viewpoint selection provided by an embodiment of the present invention; Figure 7 A schematic diagram of a flow chart of an autonomous driving perception method based on dynamic view selection provided in Example 4 of the present invention; Figure 8 A schematic diagram of a flow chart of an autonomous driving perception method based on dynamic viewpoint selection provided in Example 5 of the present invention; Figure 9 A schematic diagram of a flow chart of an autonomous driving perception method based on dynamic viewpoint selection provided in Example 7 of the present invention; Figure 10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0020] Reference numerals: 10-fusion module, 20-probability generation module, 30-initial strategy generation module, 40-determination module, 50-target strategy generation module; 130 - processor, 131 - memory, 132 - bus, 133 - communication interface. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0022] To facilitate understanding of this embodiment, the application scenarios and design concepts of the embodiment of this application are briefly introduced below.

[0023] The existing technology suffers from problems such as low computational efficiency due to computational redundancy in full-view processing, poor scene adaptability and lack of temporal continuity due to static rules or fixed expert models, and insufficient system reliability due to the lack of effective safety redundancy mechanisms. These problems ultimately limit the efficiency, accuracy, and safety of autonomous driving perception systems.

[0024] Based on this, the embodiments of the present application provide an autonomous driving perception method, device, electronic device, and storage medium based on dynamic perspective selection to solve the problems of existing autonomous driving perception systems such as large computational redundancy, poor scene adaptability, and insufficient safety, and significantly improve system efficiency, perception accuracy, and safety and reliability.

[0025] Example 1 This application provides an autonomous driving perception method based on dynamic perspective selection, combined with Figure 1 As shown, the method includes: S110 , determining scene fusion features of the driving environment based on the perception state information, navigation information, and historical environment feature information of the current time step during the vehicle's driving process.

[0026] S120 , performing dynamic routing decisions based on the scene fusion features, and generating activation probability distributions corresponding to multiple visual expert networks.

[0027] S130: Activate a first visual expert network with the highest activation probability in the activation probability distribution, and generate an initial autonomous driving strategy based on image data of a corresponding perspective of the first visual expert network.

[0028] S140: Determine a target visual expert network based on the reliability score of the initial autonomous driving strategy.

[0029] S150, activating the target visual expert network, and generating a target autonomous driving strategy based on image data of a perspective corresponding to the target visual expert network.

[0030] The present application provides real-time fusion of navigation information, perception status information and historical environmental feature information, dynamically allocates perception resources according to scene changes, and dynamically selects the expert network with the most relevant perspective for processing, so as to focus on processing key perspective data, avoid the redundancy of full-perspective calculations, and improve the perception accuracy and safety in complex scenarios, thereby achieving coordinated optimization of the efficiency and reliability of the autonomous driving system.

[0031] In step S110, the "current time step" refers to the current processing cycle of the system's perception and decision-making (e.g., an image frame cycle, typically 10ms-100ms). At each time step, the system collects and processes a batch of multimodal sensor data. The collected data can include image data collected by the vehicle's front camera, temperature data collected by the temperature sensor, voiceprint data collected by the sound sensor, audio data, and other data that can be collected during the vehicle's driving process. "Perception status information" refers to the raw data and its low-level features collected in real time by on-board sensors and after preliminary processing, mainly including but not limited to: visual data (images or video streams collected by front, side, and rear cameras), thermal data (environmental and component temperature data collected by temperature sensors), acoustic data (environmental soundprints and audio data collected by microphone arrays, such as horn and emergency vehicle siren recognition), positioning and inertial data (vehicle position, speed, acceleration, and attitude information collected by GPS and IMU (inertial measurement unit)), etc.

[0032] "Navigation information" refers to high-level semantic and intention information from the vehicle navigation system, mainly including: path planning information (such as the sequence of waypoints (Waypoints) in the next few seconds, lane-level driving routes), traffic participant prediction information (such as the predicted trajectories and behavioral intentions of surrounding vehicles and pedestrians); traffic rules and events (such as the current road speed limit, traffic light status, congestion events, construction zones, etc.).

[0033] "Historical environmental feature information" refers to the system's representation and memory of the environmental status of multiple consecutive time steps in the past, which is usually manifested as temporal BEV (bird's-eye view) features and historical decision-making states. Among them, the temporal BEV (bird's-eye view) features refer to the BEV feature tensor sequence aligned after motion compensation of several frames in the past, which is used to encode the motion trends of dynamic targets around the vehicle and the spatiotemporal evolution of the scene; the historical decision-making state refers to the expert network activated in the previous time steps, the routing decision confidence, etc., which is used to ensure the temporal continuity of the decision.

[0034] In step S110 , by fusing the three types of dynamically changing information, a comprehensive scene fusion feature rich in spatiotemporal context is constructed to provide a basis for subsequent dynamic routing decisions.

[0035] Subsequently, step S120 performs dynamic routing based on the scene fusion features obtained by fusion in step S110, and provides the system with multiple candidate perception processing solutions based on the intelligent scheduling mechanism based on scene understanding. This is the basis for efficient resource allocation, avoids invalid calculations, and reduces the system computing load from the decision source.

[0036] Subsequently, step S130 selects the visual expert network with the highest probability (such as the "forward view + right view" expert) for activation, and only loads its corresponding view sensor data (such as the right camera image) to perform calculations such as target detection and drivable area segmentation, and generates a preliminary driving strategy (such as "recommend changing lanes to the right"). This achieves "on-demand perception" and greatly reduces the amount of data processing; it significantly reduces computing latency (down to an average of 18ms) and power consumption, while ensuring perception accuracy in typical scenarios.

[0037] Subsequently, step S140 performs a multi-dimensional reliability assessment on the initial autonomous driving strategy, and based on the reliability assessment results, determines in step S150 that the target visual expert network that is finally activated processes the data from its expertise perspective, generates a highly reliable target autonomous driving strategy (such as "confirm safety, execute right lane change"), and outputs it to the control layer for execution. Through the reliability assessment, a key safety redundancy layer is constructed to ensure the robustness of the dynamic routing system; the error routing rate is reduced to below 0.3%, effectively preventing system risks caused by erroneous decisions and improving the functional safety level. In addition, the target autonomous driving strategy that is finally output undergoes a double-verification (performance + safety) optimization strategy, which not only fully utilizes the efficiency of dynamic perspective selection, but also ensures the reliability of the final decision, ultimately achieving the effect of improving efficiency while ensuring driving safety.

[0038] Example 2 The present application also provides an automatic driving perception method based on dynamic perspective selection, combined with Figure 2 As shown, the method includes: S210 , feature extraction is performed on the perception state information, navigation information, and historical environmental feature information of the current time step during vehicle driving, to obtain corresponding forward-view image feature embedding, navigation target vector, and historical bird's-eye view feature sequence.

[0039] S220 , embedding the front view image feature as a query vector, and using the features obtained by concatenating or fusing the navigation target vector with the historical bird's-eye view feature sequence as a key vector and a value vector.

[0040] S230, performing cross-attention calculation based on the query vector, the key vector, and the value vector, and outputting the scene fusion feature.

[0041] S240 , performing dynamic routing decisions based on the scene fusion features, and generating activation probability distributions corresponding to multiple visual expert networks.

[0042] S250: Activate a first visual expert network with the highest activation probability in the activation probability distribution, and generate an initial autonomous driving strategy based on image data of a corresponding perspective of the first visual expert network.

[0043] S260: Determine a target visual expert network based on the reliability score of the initial autonomous driving strategy.

[0044] S270: Activate the target visual expert network and generate a target autonomous driving strategy based on image data of a perspective corresponding to the target visual expert network.

[0045] The autonomous driving perception method based on dynamic perspective selection provided in this application differs from that in Example 1 in steps S210-S230. It is understood that the perception state information, navigation information, and historical environmental feature information all contain multiple types of information. In this embodiment, step S210 performs feature extraction on the aforementioned information. Specifically, the image data captured by the forward-looking camera in the perception state information is extracted and encoded to obtain a forward-looking image feature embedding; the path point sequence and semantic information in the navigation information are extracted and encoded to obtain a navigation target vector; and the historical bird's-eye view features in the historical environmental feature information are extracted and encoded to obtain a historical bird's-eye view feature sequence.

[0046] Step S220 embeds the front view image feature as a query vector, concatenates or fuses the navigation target vector with the historical bird's-eye view feature sequence to form a unified context feature representation, and uses this as the key vector and value vector. Figure 3 shown.

[0047] Step S230 performs cross-attention calculation based on the query vector, key vector and value vector. Through the scaled dot product attention mechanism, the similarity weight of the query vector and the key vector is calculated, and the value vector is weighted summed. Finally, the CrossAttention fusion mechanism is used to output the scene fusion feature that integrates current perception, future intention and historical environment.

[0048] The specific fusion process is as follows Figure 3As shown, the "forward-view feature embedding e_t" is the forward-view image feature, representing what the vehicle is currently seeing; the "navigation target embedding dst_emb" is the navigation target vector, representing the vehicle's intended future destination; and the "historical BEV context bev_ctx" is the sequence of historical bird's-eye view features, representing the vehicle's environmental memory of what it has just experienced. Subsequently, the "forward-view feature embedding e_t" is passed through the Query projection layer to obtain the Q vector (the query vector), which is equivalent to asking the question, "Given what I currently see, what should I focus on?" The "navigation target embedding dst_emb" and the "historical BEV context bev_ctx" are concatenated to form a unified contextual knowledge base. This concatenation is then passed through the Key projection and Value projection layers to generate the K vector (the key vector) and V vector (the value vector). The K vector is used to match the Q vector, while the V vector contains the actual information to be aggregated. Subsequently, the projected Q, K, and V are used for attention calculation (Cross-Attention calculation), allowing the current forward-looking feature (Q) to query (Q&K matching) the contextual knowledge base (K) formed by the fusion of history and intent, and extract (V) the most relevant information from it based on the degree of matching, and finally generate an output combined feature (i.e., scene fusion feature).

[0049] It is understandable that before outputting the scene fusion feature, the output of the Attention layer is added to the original input (usually Q) and normalized to ensure that the information flow is not degraded. This feature is not a simple splicing, but an intelligent and dynamic way to combine the current perception, historical environment and future intention. Figure 3 The two are organically integrated into one.

[0050] Combine Figure 4The figure below vividly illustrates the human driving cognitive decision-making process simulated in steps S210-S230. Specifically, "What should I do?" (forward vision as the Query) corresponds to the Q vector; "Surrounding state (BEV as the Key)" corresponds to a portion of the K vector. The driver uses peripheral vision and rearview mirrors to perceive the dynamics and historical behavior of surrounding vehicles and lanes (bev_ctx). This information constitutes the environmental context for decision-making; "Goal guidance (navigation as the Value)" corresponds to a portion of the V vector. The driver's mental destination and route plan (dst_emb) provide the goal and intention for decision-making; and "Comprehensive decision" corresponds to the Cross-Attention calculation and output of environmental fusion features. For example, by combining these three pieces of information: "I am looking forward (Query), but according to navigation, I know I will soon turn right (Value), so I need to pay special attention to the historical trajectory of the vehicle behind me to the right (Key)"—then a comprehensive judgment is formed: "It is now safe to turn right." It can be seen that the embodiment of the present application standardizes heterogeneous multi-source information (current image, historical environment, navigation intent) in the form of Q, K, and V, and intelligently integrates it through a set of learnable parameters (projection layer, attention weight), replicating the human cognitive logic of "looking ahead (Query), referencing the environment (Key), focusing on the goal (Value), and making a comprehensive decision (Output). Utilizing the Cross-Attention mechanism, it simulates the decision-making process of human drivers from both technical and functional levels. The resulting scene fusion feature is an intelligent representation rich in semantic and contextual information. It clearly shows "what information is important at the current moment based on history and intent," providing a solid and accurate basis for subsequent dynamic routing decisions.

[0051] Example 3 This application also provides an autonomous driving perception method based on dynamic perspective selection, combined with Figure 5 As shown, the method includes: S310 , determining scene fusion features of the driving environment based on the perception state information, navigation information, and historical environment feature information of the current time step during the vehicle's driving process.

[0052] S320: Input the fusion feature into the routing decision engine.

[0053] S330, processing the scene fusion feature through the probability prediction head of the routing decision engine; wherein the probability prediction head includes at least one fully connected layer for mapping the scene fusion feature into a high-dimensional feature representation. S340: Perform probability normalization processing on the high-dimensional feature representation to generate activation probability distributions of the multiple visual expert networks.

[0054] S350: Activate a first visual expert network with the highest activation probability in the activation probability distribution, and generate an initial autonomous driving strategy based on image data of a corresponding perspective of the first visual expert network.

[0055] S360: Determine a target visual expert network based on the reliability score of the initial autonomous driving strategy.

[0056] S370: Activate the target visual expert network and generate a target autonomous driving strategy based on image data of a perspective corresponding to the target visual expert network.

[0057] Compared with Example 1, the present application differs in steps S320-S340. In this embodiment, after obtaining the environmental fusion features, they are input into the routing decision engine. The routing decision engine itself is a lightweight neural network module responsible for making real-time decisions on "which expert is most suitable for handling the current scene." In this embodiment, the routing decision engine includes a probabilistic prediction head, which is typically composed of one or more fully connected layers (i.e., linear layers, which can be combined with activation functions such as ReLU). Its function is to map the input comprehensive features into a higher-dimensional, more abstract feature space. After inputting the scene fusion features into the routing decision engine in step S320, the fully connected layers in the probabilistic prediction head in step S330 project the environmental fusion features into a high-dimensional space to learn a more complex decision boundary, thereby more accurately distinguishing which visual expert network should be called for different scenarios. The "high-dimensional feature representation" contains information about the matching degree between the current scene and the areas of expertise of each visual expert network, preparing for the next step of generating a probability distribution. Finally, step S340 performs probabilistic normalization on the high-dimensional vector obtained in step S330, compressing the real vector into a probability distribution. This transforms it into an activation probability distribution whose elements sum to 1, representing the probability of activating each visual expert network in the current scene. For example, steps S320-S340 yield a probability distribution of [0.05, 0.8, 0.1… 0.02].

[0058] Table 1 shows the information of each visual expert network.

[0059]

[0060] There are n visual expert networks in total, as shown in Table 1. Figure 6As shown, a multi-view camera serves as the basic sensor, providing 360° raw image data around the vehicle. After alignment and forward-view feature extraction, it serves as the "query" in the attention mechanism. The navigation system provides high-level intent information (e.g., "turn right at the next intersection"). A sequence model (e.g., LSTM or Transformer) is used to encode a sequence of future waypoints into a fixed-dimensional vector representing the vehicle's "future driving intention," resulting in a navigation target encoding. Simultaneously, motion-compensated alignment is performed on bird's-eye view features from past frames stored in a historical BEV database, forming a spatiotemporal contextual memory of the vehicle's surroundings. This sequence of historical bird's-eye view features represents the motion trends of dynamic targets (e.g., vehicles and pedestrians). The navigation target encoding and the historical bird's-eye view feature sequences are concatenated or fused together in step S310 to generate "Key" and "Value" vectors. The attention mechanism then fuses the temporal features extracted from the historical BEV database, the intent features provided by the navigation target encoding, and the forward-view features to generate a rich scene fusion feature. This feature comprehensively expresses "where I am," "what I've experienced," "where I'm going," and "what I see now." The fusion features are then input into the "dynamic routing decision module", which contains a routing decision engine (usually composed of a fully connected layer and Softmax). By analyzing the current scene fusion features, the activation probability distribution of each visual expert network (expert 1, expert 2...expert N) in the "Vision MOE expert pool" is calculated. Among them, each visual expert network is only responsible for processing a specific combination of perspectives (such as expert 1 only processes image data collected by the front and right cameras), thereby greatly reducing the computing load and power consumption. In addition, in step 360, the reliability of the initial autonomous driving strategy generated in step S350 is judged. If the reliability does not meet the requirements, it triggers the following steps: Figure 6 The safety monitoring module in the

[15] is used to perform full-view forced activation or minimum risk strategy to determine the target visual expert network and then generate the target autonomous driving strategy.

[0061] Example 4 This application also provides an autonomous driving perception method based on dynamic perspective selection, and the reliability score includes a confidence score and a continuity score. Figure 7 As shown, the method includes: S410 , determining scene fusion features of the driving environment based on the perception state information, navigation information, and historical environment feature information of the current time step during the vehicle driving process.

[0062] S420 , performing dynamic routing decisions based on the scene fusion features, and generating activation probability distributions corresponding to multiple visual expert networks.

[0063] S430: Activate a first visual expert network with the highest activation probability in the activation probability distribution, and generate an initial autonomous driving strategy based on image data of a corresponding perspective of the first visual expert network.

[0064] S440: Input the initial autonomous driving strategy into a confidence assessment network, and output the confidence score of the initial autonomous driving strategy.

[0065] S450: Input the activation probability distribution of the current time step and the activation probability distribution of multiple consecutive historical time steps into a recurrent neural network, and output a continuity score that represents the consistency between the initial autonomous driving strategy and the historical autonomous driving strategy.

[0066] S460: Determine a target visual expert network based on the reliability score of the initial autonomous driving strategy.

[0067] S470: Activate the target visual expert network and generate a target autonomous driving strategy based on image data of a perspective corresponding to the target visual expert network.

[0068] The method provided in this application differs from Example 1 in that, in steps S440-S450, a dual reliability scoring mechanism (confidence scoring and continuity scoring) is introduced to comprehensively verify the initial decision, thereby improving the security and robustness of the system.

[0069] Steps S410-S430, similar to steps S110-S130, fuse current perception, future intent, and historical context to generate an environmental fusion feature. Dynamic routing decisions are then made to determine the activation probability distribution for each visual expert network. The first visual expert network with the highest activation probability in the probability distribution is selected and activated. Perception calculations are then performed based solely on the specific perspective image data (e.g., "front view + right view") for which the expert is responsible, generating an initial autonomous driving strategy (e.g., "recommend lane change right"). Subsequently, step S440 of this embodiment inputs the initial autonomous driving strategy into a dedicated confidence assessment network (typically a lightweight binary classification network). This network analyzes the strategy's reliability and outputs a confidence score to assess the quality of the decision (i.e., the initial autonomous driving strategy). A low confidence score indicates that the perception result may be unreliable (e.g., fuzzy object detection or overly complex scene), serving as a key signal for triggering the safety fallback mechanism. Next, step S450 feeds the activation probability distribution of the current time step and the sequence of activation probability distributions for multiple consecutive historical time steps into a recurrent neural network (such as an LSTM). This RNN learns the temporal pattern of decisions and outputs a continuity score to assess the temporal consistency of the current decision with previous decisions. A low continuity score indicates a dramatic jump in decision making (e.g., selecting "forward-looking left expert" in the previous frame and "forward-looking right expert" in the current frame). This could indicate decision oscillation or routing errors, impacting driving comfort and safety.

[0070] Then, in step S460, the confidence score and continuity score are comprehensively considered to ultimately determine the target visual expert network to be activated. If both scores are above a threshold, the initial autonomous driving strategy is approved, and the first visual expert network is determined as the target visual expert network. If either score is below a threshold, the safety strategy is triggered to switch to a more reliable visual expert network (such as a more conservative visual expert network or a full-view visual expert network). Furthermore, in step S470, the finalized target visual expert network is activated, and based on the perspective image data it processes, the final target autonomous driving strategy is generated, outputting a highly reliable final perception result that has undergone dual safety verification, achieving a balance between safety and efficiency.

[0071] In this way, the change from single verification to dual verification of "quality + timing" builds a more solid safety line of defense, which can effectively capture errors that cannot be discovered by single verification (such as anomalies that seem to have high confidence but violate historical decision-making patterns). Through continuous scoring verification, the frequent switching of expert networks caused by jitter in perception results is reduced, making vehicle behavior smoother and more stable, improving the driving experience, and making the system more tolerant to complex scenarios, edge cases and sensor noise.

[0072] Example 5 The embodiment of the application also provides an automatic driving perception method based on dynamic view selection. In the embodiment, the reliability score comprises a confidence score. Figure 8 As shown in the figure, the method comprises the following steps. S510, determining scene fusion features of a driving environment according to perception state information of a current time step in a driving process of a vehicle, navigation information and historical environment feature information.

[0073] S520, performing dynamic routing decision based on the scene fusion features to generate an activation probability distribution corresponding to a plurality of visual expert networks.

[0074] S530, activating a first visual expert network with the highest activation probability in the activation probability distribution, and generating an initial automatic driving strategy based on image data of a corresponding view angle of the first visual expert network.

[0075] S540, judging whether a confidence score of the initial automatic driving strategy is greater than a confidence threshold value.

[0076] If not, step S550 is performed; if yes, step S551 is performed.

[0077] S550, triggering a preset safe driving strategy, and taking all the visual expert networks as the target visual expert networks.

[0078] S560, activating the target visual expert network, and generating a target automatic driving strategy based on image data of a corresponding view angle of the target visual expert network.

[0079] Compared with the embodiment 1, the difference of the application lies in steps S540-S550. After the scene fusion features are determined based on a plurality of data, dynamic routing decision is performed, the first visual expert network is selected and activated to generate the initial automatic driving strategy, the confidence score of the initial automatic driving strategy is compared with the preset confidence threshold value. If not (i.e. the confidence score is lower than the threshold value), it indicates that the initial automatic driving strategy itself is unreliable. At this time, the most conservative safe driving strategy is triggered, i.e. all the visual expert networks in the system are taken as the target visual expert networks, which means that the "full view angle expert" processing all view angles is forcibly activated, so as to provide the most comprehensive and reliable environment perception when the system is most uncertain about its own decision, and form the final safety line.

[0080] In combination with the first aspect, the reliability score also comprises a continuity score. After step S540, the following steps are further included. S551, judging whether the continuity score is greater than a preset continuity score threshold value.

[0081] If yes, step S552 is performed; if not, step S553 is performed.

[0082] S552: Use the first visual expert network as the target visual expert network, and use the initial autonomous driving strategy as the target autonomous driving strategy.

[0083] S553: ​​Using the second visual expert network corresponding to the autonomous driving strategy of the previous time step as the target visual expert network.

[0084] After evaluating the confidence score of the initial autonomous driving strategy in step S540 and determining whether the score is greater than a preset confidence threshold, if the confidence is high, proceed to the second-level continuity check to form a second safety checkpoint to ensure the temporal smoothness of the decision. If the continuity is high, execute step S552 to approve the initial autonomous driving strategy; if the continuity is low, indicating that although the current decision is credible, there is an abrupt jump from the historical decision-making pattern, then the smoothness backup safety strategy is triggered, that is, the second visual expert network corresponding to the autonomous driving strategy of the previous time step is used as the target visual expert network, and it falls back to the stable state of the previous time step, effectively avoiding frequent switching of the expert network caused by perception noise or misjudgment in this time step, and ensuring the smoothness of vehicle behavior. In this way, the present application constructs a three-level decision-making system from high to low. The first level (highest risk response) corresponds to the situation where the confidence score is lower than the confidence threshold. At this time, step S550 is executed to start the full view; the second level (medium risk response) corresponds to the situation where the confidence score is high but the continuity score is low. At this time, step S553 is executed to fall back to the autonomous driving strategy of the previous time step to ensure smooth vehicle operation; the third level (normal response) corresponds to the situation where both the confidence score and the continuity score are high. At this time, step S552 is executed to adopt the initial autonomous driving strategy generated by the dynamic routing of this time step as the target autonomous driving strategy.

[0085] This application not only ensures the accuracy of perception results through confidence checks, but also ensures the stability of decision sequences through continuity checks, fundamentally solving the decision jitter problem that may be caused by dynamic systems and improving the driving experience.

[0086] It is understandable that the confidence score threshold and the continuity score threshold can be set according to actual needs. In this embodiment, the confidence score threshold is preset to 0.8 and the continuity score threshold is preset to 0.7. This is only an example and is not limited to this.

[0087] Example 6 The present application also provides an autonomous driving perception method based on dynamic perspective selection, the method further comprising: S610: Periodically activate all of the visual expert networks to generate the target autonomous driving strategy by combining image data from all perspectives.

[0088] In this application, by setting a fixed safety clock cycle (for example, every 5 time-step images or every 0.5 seconds), ignoring the output results of the current dynamic routing decision engine, all visual expert networks in the system are forcibly activated, integrating and processing image data from all perspectives, and generating the target autonomous driving strategy based on global information to solve the "blindness" risk that may exist in pure dynamic routing, that is, the long-term focus on a local perspective may not lead to the perception of potential dangers that slowly evolve or suddenly appear in other perspectives.

[0089] By periodically performing a comprehensive, comprehensive scan of the surrounding environment, new or slow-moving targets (such as vehicles suddenly entering or emergency vehicles approaching from a distance) that have not been observed for several consecutive time steps can be effectively detected. This also provides a regular "ground truth," helping to reset the accumulated errors in the historical BEV database and indirectly calibrate the results of dynamic routing decisions, preventing the system from continuously operating in the wrong routing direction. This allows the system to run an efficient dynamic routing process and intelligently select target visual expert networks in parallel with the methods provided in Examples 1-5, saving computing power. When the security clock triggers, the current dynamic routing logic is interrupted and S610 is forced to execute for periodic policy calibration. After the cycle completes, the system resumes normal dynamic routing decisions, adding a powerful yet simple security layer. By sacrificing a minimal amount of peak computing efficiency, the system significantly improves overall reliability and security, achieving a perfect combination of efficient dynamic perception and periodic global assurance.

[0090] Example 7 This application provides an autonomous driving perception device based on dynamic perspective selection, combined with Figure 9 As shown, the device includes: a fusion module 10, a probability generation module 20, an initial strategy generation module 30, a determination module 40 and a target strategy generation module 50.

[0091] The fusion module 10 is used to determine the scene fusion characteristics of the driving environment based on the perception state information, navigation information and historical environmental feature information of the current time step during the vehicle's driving process.

[0092] The probability generation module 20 is used to make dynamic routing decisions based on the scene fusion features and generate activation probability distributions corresponding to multiple visual expert networks.

[0093] The initial strategy generation module 30 is used to activate the first visual expert network with the highest activation probability in the activation probability distribution, and generate an initial autonomous driving strategy based on the image data of the corresponding perspective of the first visual expert network.

[0094] The determination module 40 is configured to determine a target visual expert network based on the confidence score and the continuity score of the initial autonomous driving strategy.

[0095] The target strategy generation module 50 is used to activate the target visual expert network and generate a target autonomous driving strategy based on the image data of the perspective corresponding to the target visual expert network.

[0096] In combination with the second aspect, the steps of the fusion module 10 include: a feature extraction module, a splicing or fusion module and a calculation module.

[0097] The feature extraction module is used to extract the perception state information, navigation information and historical environmental feature information of the current time step during vehicle driving, and obtain the corresponding front view image feature embedding, navigation target vector and historical bird's-eye view feature sequence respectively.

[0098] A splicing or fusion module is used to embed the front view image feature as a query vector, and use the features obtained by splicing or fusing the navigation target vector with the historical bird's-eye view feature sequence as a key vector and a value vector.

[0099] A calculation module is used to calculate cross-attention calculation based on the query vector, the key vector and the value vector, and output the scene fusion feature.

[0100] In combination with the second aspect, the probability generation module 20 includes: an input module, a processing module and a normalization module.

[0101] The input module is used to input the scene fusion features into the routing decision engine.

[0102] The processing module is used to process the scene fusion features through the probability prediction head of the routing decision engine; wherein the probability prediction head includes at least one fully connected layer for mapping the scene fusion features into a high-dimensional feature representation. The normalization module is used to perform probability normalization processing on the high-dimensional feature representation to generate activation probability distributions of the multiple visual expert networks.

[0103] In combination with the second aspect, the reliability score includes a confidence score and a continuity score; after the initial strategy generation module 30, it also includes: a confidence evaluation module and a continuity scoring module.

[0104] The confidence assessment module is used to input the initial autonomous driving strategy into the confidence assessment network and output the confidence score of the initial autonomous driving strategy.

[0105] The continuity evaluation module is used to input the activation probability distribution of the current time step and the activation probability distribution of multiple consecutive historical time steps into the recurrent neural network, and output the continuity score that represents the consistency between the initial autonomous driving strategy and the historical autonomous driving strategy.

[0106] In combination with the second aspect, the reliability score includes a confidence score; the determination module 40 includes: a first judgment module and a first trigger module.

[0107] The first judgment module is used to determine whether the confidence score of the initial autonomous driving strategy is greater than a confidence threshold.

[0108] The first trigger module is used to trigger a preset safe driving strategy when the confidence score of the initial autonomous driving strategy is less than or equal to a confidence threshold, and use all the visual expert networks as the target visual expert networks.

[0109] In combination with the second aspect, the reliability score also includes a continuity score; and after the first judgment module, it also includes: a second judgment module, a second trigger module and a third trigger module.

[0110] The second judgment module is used to judge whether the continuity score is greater than a preset continuity score threshold when the confidence score of the initial autonomous driving strategy is greater than the confidence threshold.

[0111] The second triggering module is configured to use the first visual expert network as the target visual expert network and the initial autonomous driving strategy as the target autonomous driving strategy when the continuity score is greater than a preset continuity score threshold.

[0112] The third triggering module is configured to use the second visual expert network corresponding to the autonomous driving strategy of the previous time step as the target visual expert network when the continuity score is less than or equal to a preset continuity score threshold.

[0113] In combination with the second aspect, the device further includes: a periodic activation module.

[0114] The periodic activation module is used to periodically activate the entire visual expert network to generate the target autonomous driving strategy by combining image data from all perspectives.

[0115] In a third aspect, the present application provides an electronic device, Figure 10 As shown, the electronic device includes a memory 131 and a processor 130. The memory 131 is used to store computer programs, and the processor 130 runs the computer programs to enable the electronic device to perform the above method.

[0116] Further, combined with Figure 10The electronic device shown further includes a bus 132 and a communication interface 133 , and the processor 130 , the communication interface 133 and the memory 131 are connected via the bus 132 .

[0117] The memory 131 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is achieved through at least one communication interface 133 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 132 may be an ISA bus, a PCI bus, or an EISA bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0118] The processor 130 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 130 or by software instructions. The processor 130 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 131, and processor 130 reads information in memory 131 and, in conjunction with its hardware, completes the steps of the method of the aforementioned embodiment.

[0119] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the above-mentioned method is executed.

[0120] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0121] In addition, in the description of the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0122] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0123] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0124] Finally, it should be noted that the above embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An autonomous driving perception method based on dynamic view selection, characterized in that: The method comprises: Determine the scene fusion characteristics of the driving environment based on the perception state information, navigation information and historical environmental feature information of the current time step during the vehicle's driving process; Performing dynamic routing decisions based on the scene fusion features to generate activation probability distributions corresponding to multiple visual expert networks; activating a first visual expert network with a highest activation probability in the activation probability distribution, and generating an initial autonomous driving strategy based on image data of a corresponding perspective of the first visual expert network; Determining a target visual expert network based on the reliability score of the initial autonomous driving strategy; The target visual expert network is activated, and a target autonomous driving strategy is generated based on image data of a perspective corresponding to the target visual expert network.

2. The method according to claim 1, characterized in that The step of determining the scene fusion features of the driving environment based on the perception state information, navigation information, and historical environmental feature information at the current time step during the vehicle's driving process includes: Feature extraction is performed on the perception state information, navigation information, and historical environmental feature information of the current time step during vehicle driving to obtain the corresponding forward-view image feature embedding, navigation target vector, and historical bird's-eye view feature sequence; Embedding the front view image feature as a query vector, and concatenating or fusing the navigation target vector with the historical bird's-eye view feature sequence as a key vector and a value vector; A cross-attention calculation is performed based on the query vector, the key vector, and the value vector, and the scene fusion feature is output.

3. The method according to claim 1, characterized in that The step of making dynamic routing decisions based on the scene fusion features and generating activation probability distributions corresponding to multiple visual expert networks includes: Inputting the scene fusion features into a routing decision engine; Processing the scene fusion features through a probabilistic prediction head of the routing decision engine; wherein the probabilistic prediction head includes at least one fully connected layer for mapping the scene fusion features into a high-dimensional feature representation; Probabilistic normalization is performed on the high-dimensional feature representation to generate activation probability distributions of the multiple visual expert networks.

4. The method according to claim 1, wherein The reliability score includes a confidence score and a continuity score; after the steps of activating a first visual expert network having the highest activation probability in the activation probability distribution and generating an initial autonomous driving strategy based on image data corresponding to a perspective of the first visual expert network, the method further includes: Inputting the initial autonomous driving strategy into a confidence assessment network, and outputting the confidence score of the initial autonomous driving strategy; The activation probability distribution of the current time step and the activation probability distribution of multiple consecutive historical time steps are input into a recurrent neural network, and the continuity score representing the consistency between the initial autonomous driving strategy and the historical autonomous driving strategy is output.

5. The method according to claim 1, wherein The reliability score includes a confidence score; and the step of determining a target visual expert network in combination with the reliability score of the initial autonomous driving strategy includes: Determining whether a confidence score of the initial autonomous driving strategy is greater than a confidence threshold; If not, the preset safe driving strategy is triggered, and all the visual expert networks are used as the target visual expert networks.

6. The method according to claim 5, characterized in that The reliability score also includes a continuity score; after the step of determining whether the confidence score of the initial autonomous driving strategy is greater than a confidence threshold, the method further includes: If so, determining whether the continuity score is greater than a preset continuity score threshold; If so, using the first visual expert network as the target visual expert network and the initial autonomous driving strategy as the target autonomous driving strategy; If not, the second visual expert network corresponding to the autonomous driving strategy of the previous time step is used as the target visual expert network.

7. The method according to claim 1, characterized in that The method further comprises: All of the visual expert networks are periodically activated to generate the target autonomous driving strategy by combining image data from all perspectives.

8. An autonomous driving perception device based on dynamic perspective selection, characterized in that: The device comprises: A fusion module is used to determine the scene fusion characteristics of the driving environment based on the perception state information, navigation information and historical environmental feature information of the current time step during the vehicle's driving process; a probability generation module, configured to make dynamic routing decisions based on the scene fusion features and generate activation probability distributions corresponding to multiple visual expert networks; an initial strategy generation module, configured to activate a first visual expert network having the highest activation probability in the activation probability distribution, and generate an initial autonomous driving strategy based on image data of a corresponding perspective of the first visual expert network; a determination module, configured to determine a target visual expert network based on the confidence score and the continuity score of the initial autonomous driving strategy; A target strategy generation module is used to activate the target visual expert network and generate a target autonomous driving strategy based on image data of the perspective corresponding to the target visual expert network.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images