A Visualization Display Method and System for the Running Trajectory of an Autonomous Vehicle

Abstract

The embodiment of the present invention discloses a method and system for visually displaying the running trajectory of an autonomous driving vehicle, which belongs to the field of data visualization technology. The embodiment of the present invention proposes a comprehensive technical solution for visualizing the running trajectory of an intelligent electric snowmobile. By integrating a variety of sensor monitoring information, including the on-board environmental perception sensor monitoring information of the target intelligent electric snowmobile and the first laser radar point cloud sensor monitoring information and the associated mixed-mode sensor monitoring information of the associated intelligent electric snowmobile, and then using multiple subnets in the running trajectory visualization processing network to deeply process this information, and finally selecting a suitable visualization conversion strategy based on the obtained fully connected quantitative knowledge of the running trajectory, the problems of incomplete information, insufficient information processing and lack of adaptability of visualization strategies existing in traditional technologies are effectively solved.

Description

Technical Field

[0001] The embodiments of the present invention belong to the field of data visualization technology, and specifically relate to a method and system for visualizing the running trajectory of an autonomous driving vehicle. Background Art

[0002] In the field of operation management and monitoring of smart electric snowmobiles, accurately displaying the operation trajectory of the snowmobile is of vital importance to ensuring its safe driving, optimizing path planning, and improving overall operational efficiency.

[0003] Traditional trajectory display technologies often have numerous limitations. Relying solely on a single type of sensor data, such as onboard cameras or lidar data, fails to fully capture the environmental information surrounding the snowmobile. In information processing, there is a lack of effective methods to integrate key information from different sensor data types. Previous technologies may simply overlay or process different sensor data separately, failing to fully explore the inherent connections between the data. Regarding visualization, traditional technologies lack the ability to adaptively select visualization strategies based on actual conditions. They typically employ fixed visualization methods, failing to provide the most appropriate trajectory display for different driving scenarios (such as varying snowy terrain and traffic flows) and task requirements (such as real-time monitoring and post-event analysis). This can result in poor visualization in some cases, failing to effectively support snowmobile operation management and monitoring. Summary of the Invention

[0004] The embodiments of the present invention provide a method and system for visually displaying the running trajectory of an autonomous driving vehicle, which can solve or partially solve the technical problems involved in the above-mentioned background technology.

[0005] An embodiment of the present invention provides a method for visually displaying the running trajectory of an autonomous driving vehicle, which is applied to a visualization display system. The method includes: obtaining a target mixed-mode sensor monitoring information set of a target intelligent electric snowmobile to be visualized for running trajectory display and an associated mixed-mode sensor monitoring information set of an associated intelligent electric snowmobile of the target intelligent electric snowmobile, wherein the target mixed-mode sensor monitoring information set includes: on-board environment perception sensor monitoring information and first laser radar point cloud sensor monitoring information; obtaining on-board environment perception quantitative knowledge of the on-board environment perception sensor monitoring information through the on-board environment perception mining subnet of the running trajectory visualization processing network, and obtaining the first laser radar point cloud sensor monitoring information through the laser radar point cloud mining subnet of the running trajectory visualization processing network. The laser radar point cloud quantitative knowledge of the information; the sensor monitoring interaction knowledge features corresponding to the on-board environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set are obtained through the knowledge feature interaction subnet of the running trajectory visualization processing network; the on-board environment perception quantitative knowledge, the laser radar point cloud quantitative knowledge and the sensor monitoring interaction knowledge features are fully connected through the fully connected subnet of the running trajectory visualization processing network to obtain the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile; based on the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile, the target visualization conversion strategy is obtained from the visualization conversion strategy pool; the target visualization conversion strategy is used to realize the visualization display of the running trajectory of the target intelligent electric snowmobile.

[0006] An embodiment of the present invention provides a visualization display system, comprising at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the above method.

[0007] An embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the above method are implemented.

[0008] To address the above-mentioned technical issues, the present invention proposes a comprehensive technical solution for visualizing the trajectory of an intelligent electric snowmobile. This solution integrates multiple sensor monitoring information, including the onboard environmental perception sensor monitoring information of the target intelligent electric snowmobile, the first lidar point cloud sensor monitoring information, and the associated mixed-mode sensor monitoring information of the associated intelligent electric snowmobile. This information is then deeply processed using multiple subnets in the trajectory visualization processing network. Finally, an appropriate visualization conversion strategy is selected based on the fully connected quantitative knowledge of the trajectory obtained. This effectively addresses the problems of incomplete information, insufficient information processing, and lack of adaptability of visualization strategies that exist in traditional technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a flowchart of a method for visually displaying the running trajectory of an autonomous driving vehicle provided by an embodiment of the present invention.

[0010] Figure 2 A schematic diagram of the structure of a visual display system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0011] Figure 1 A method for visually displaying the running trajectory of an autonomous driving vehicle is shown, which is applied to a visual display system. The method includes the following steps 110 to 150.

[0012] The embodiment of the present invention is applied around the technology of visual display of the running trajectory of an intelligent electric snowmobile. By obtaining the sensor monitoring information set of the target intelligent electric snowmobile and its associated intelligent electric snowmobiles, the information is processed using multiple subnets in the running trajectory visualization processing network, and finally the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile is obtained, and then the target visualization conversion strategy is obtained from the visualization conversion strategy pool to realize the visualization display of the running trajectory.

[0013] Step 110: Obtain a target mixed-mode sensor monitoring information set of a target intelligent electric snowmobile whose running trajectory is to be visualized and an associated mixed-mode sensor monitoring information set of associated intelligent electric snowmobiles of the target intelligent electric snowmobile.

[0014] Among them, the target mixed-mode sensor monitoring information set includes: vehicle-mounted environment perception sensor monitoring information and first laser radar point cloud sensor monitoring information.

[0015] This step first requires obtaining the target mixed-mode sensor monitoring information set for the target intelligent electric snowmobile. This information set includes onboard environmental perception sensor monitoring information and first lidar point cloud sensor monitoring information. Onboard environmental perception sensor monitoring information can include various types of sensor data, such as image data captured by cameras, which is used to identify objects around the snowmobile (such as other vehicles, pedestrians, obstacles, etc.) and road conditions (such as the smoothness of the snow and the presence of gullies). For example, the image data in the onboard environmental perception sensor monitoring information has a resolution of 1920*1080 pixels, which provides a relatively clear image of the surrounding environment for subsequent analysis.

[0016] The first lidar point cloud sensor monitoring information includes multiple lidar scans compiled sequentially according to the scanning cycle. LiDAR constructs point cloud data of the surrounding environment by emitting laser beams and receiving reflected light. For example, with a lidar scanning frequency of 10Hz, each scan can obtain 3D coordinate information for thousands of points in the surrounding environment. This point cloud data can accurately reflect the terrain around the snowmobile, as well as the distance and shape of obstacles.

[0017] At the same time, the associated mixed-mode sensor monitoring information set of the target intelligent electric snowmobile must also be obtained. For example, if the number of associated intelligent electric snowmobiles is X = 3 (X is a positive integer), there are three associated intelligent electric snowmobiles. The associated mixed-mode sensor monitoring information corresponding to each associated snowmobile includes its own onboard environmental perception sensor monitoring information and LiDAR point cloud sensor monitoring information. This information is crucial for comprehensively analyzing the target snowmobile's trajectory.

[0018] Step 120: Obtain the vehicle-mounted environment perception quantitative knowledge of the vehicle-mounted environment perception sensor monitoring information by running the vehicle-mounted environment perception mining subnet of the trajectory visualization processing network, and obtain the lidar point cloud quantitative knowledge of the first lidar point cloud sensor monitoring information by running the lidar point cloud mining subnet of the trajectory visualization processing network.

[0019] In an embodiment of the present invention, the running trajectory visualization processing network includes a vector transfer subnet, and the vehicle environment perception mining subnet includes a bidirectional long short-term memory subnet.

[0020] Based on this, the vehicle environment perception mining subnet of the running trajectory visualization processing network obtains the vehicle environment perception quantitative knowledge of the vehicle environment perception sensor monitoring information, including: using the vehicle environment migration features obtained by performing vector space migration on the vehicle environment perception sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network as the input of the vehicle environment perception mining subnet; performing knowledge quantification on the vehicle environment migration features through the bidirectional long short-term memory subnet of the vehicle environment perception mining subnet to obtain the vehicle environment perception quantitative knowledge of the vehicle environment perception sensor monitoring information.

[0021] Furthermore, the laser radar point cloud quantitative knowledge of the first laser radar point cloud sensor monitoring information is obtained through the laser radar point cloud mining subnet of the running trajectory visualization processing network, including: using the multiple laser radar point cloud migration features of the first laser radar point cloud sensor monitoring information obtained through the vector migration subnet of the running trajectory visualization processing network as the input of the laser radar point cloud mining subnet of the running trajectory visualization processing network, the first laser radar point cloud sensor monitoring information includes multiple laser radar scanning information sorted in sequence according to the scanning cycle; processing the multiple laser radar point cloud migration features through the laser radar point cloud mining subnet of the running trajectory visualization processing network to obtain the laser radar point cloud quantitative knowledge of the first laser radar point cloud sensor monitoring information.

[0022] Among them, the lidar point cloud mining subnet of the operation trajectory visualization processing network obtains the lidar point cloud quantitative knowledge of the first lidar point cloud sensing monitoring information based on multiple lidar scanning information whose scanning cycle meets the step size requirement; or, the lidar point cloud mining subnet of the operation trajectory visualization processing network obtains multiple three-dimensional scene drawing element features corresponding to the multiple lidar point cloud migration features through the residual processing subnet of the lidar point cloud mining subnet, and processes the multiple three-dimensional scene drawing element features through the feature downsampling branch of the lidar point cloud mining subnet to obtain the lidar point cloud quantitative knowledge of the first lidar point cloud sensing monitoring information.

[0023] Step 120, on the one hand, focuses on acquiring quantitative knowledge about vehicle-based environmental perception. The key technical approach lies in vector space migration, and the trajectory visualization processing network includes a vector migration subnet. First, the vehicle-based environmental perception sensor monitoring information is input into the vector migration subnet for vector space migration. For example, a neural network-based vector space migration algorithm is employed. For example, the number of neurons in the input layer of this algorithm is set to 1024 based on the characteristic dimensions of the vehicle-based environmental perception sensor monitoring information. The vehicle-based environmental migration features are obtained through processing in the multi-layer neural network structure of the vector migration subnet (e.g., comprising three hidden layers, with 512, 256, and 128 neurons in each hidden layer, respectively).

[0024] Furthermore, considering subsequent knowledge quantification, the vehicle environment perception mining subnetwork includes a bidirectional long short-term memory (Bi-LSTM) subnetwork. The obtained vehicle environment transfer features serve as the input to the vehicle environment perception mining subnetwork. By processing the input vehicle environment transfer features, the Bi-LSTM subnetwork can effectively capture long-term dependencies in sequential data. For example, for image sequence data (e.g., a sequence length of 50 frames) in the vehicle environment transfer features, each LSTM unit in the Bi-LSTM subnetwork can selectively memorize and forget features in each frame of image data based on its internal gating mechanism (including input gate, forget gate, and output gate). After this processing, quantitative vehicle environment perception knowledge of the vehicle environment perception sensor monitoring information is obtained.

[0025] Another aspect of step 120 involves acquiring quantitative knowledge of the LiDAR point cloud. First, the LiDAR point cloud migration features are acquired. Similarly, the first LiDAR point cloud sensor monitoring information is processed by the vector migration subnet of the trajectory visualization processing network. Because the first LiDAR point cloud sensor monitoring information includes multiple LiDAR scan information sequentially sorted according to scan cycles (for example, each scan cycle is 0.1 seconds, and a total of 10 scan cycles are collected), the vector migration subnet processes these LiDAR scan information to obtain multiple LiDAR point cloud migration features. For example, a vector migration algorithm based on a convolutional neural network (CNN) is used. The CNN has a convolution kernel size of 3*3 and a stride of 1. Through multiple layers of convolutional layers and pooling layers (such as a maximum pooling layer with a pooling window size of 2*2), the point cloud migration features corresponding to each LiDAR scan information are obtained.

[0026] Quantitative knowledge is then obtained through corresponding processing. The LiDAR point cloud mining subnet of the trajectory visualization processing network obtains LiDAR point cloud quantitative knowledge of the first LiDAR point cloud sensing monitoring information based on multiple LiDAR scanning information whose scanning cycles meet the step size requirement (for example, a step size of 2 scanning cycles). Alternatively, the LiDAR point cloud mining subnet obtains multiple 3D scene drawing element features corresponding to multiple LiDAR point cloud migration features through its residual processing subnet, and then processes these 3D scene drawing element features through the feature downsampling branch to obtain LiDAR point cloud quantitative knowledge. For example, the residual processing subnet adopts a ResNet architecture, processing LiDAR point cloud migration features through residual blocks (for example, each residual block contains two 3*3 convolutional layers and a shortcut connection) to obtain 3D scene drawing element features. The feature downsampling branch uses average pooling to downsample the 3D scene drawing element features (for example, the downsampling ratio is 1 / 2), thereby obtaining LiDAR point cloud quantitative knowledge.

[0027] As can be understood, step 120 is a critical step in the intelligent electric snowmobile trajectory visualization technology. It primarily involves acquiring quantitative knowledge of the onboard environmental perception sensor monitoring information and quantitative knowledge of the lidar point cloud monitoring information from the first lidar point cloud sensor monitoring information through different subnets in the trajectory visualization processing network. Acquiring these two types of quantitative knowledge lays the foundation for the subsequent accurate display of the snowmobile trajectory.

[0028] The above content of step 120 is further described below.

[0029] Acquisition of quantitative knowledge of vehicle-mounted environment perception

[0030] (1) Vector Migration Subnet and Vehicle Environment Migration Characteristics

[0031] 1) The role of the vector migration subnet

[0032] The vector transfer subnetwork within the trajectory visualization processing network plays an important initial role in acquiring quantitative knowledge about vehicle environmental perception. The vector transfer subnetwork is designed to perform vector space migration of vehicle environmental perception sensor monitoring information. This information contains a rich variety of information, such as image data captured by cameras and data such as ambient temperature and humidity acquired by other sensors. (For example, camera images comprise the majority of the data, with image size of 1280 x 720 pixels and stored in RGB three-channel format.)

[0033] The vector transfer subnet uses a specific algorithm to process this information. One possible algorithm is a vector space transfer algorithm based on a multi-layer perceptron (MLP). For example, this MLP has three hidden layers, with 512 neurons in the first hidden layer, 256 in the second, and 128 in the third. The number of neurons in the input layer is determined by the characteristic dimensions of the vehicle's environmental perception, sensing, and monitoring information. For example, if the image data is expanded into a one-dimensional vector and the characteristic dimensions of other sensor data are added, the total is 1024, so the number of input layer neurons is 1024. Through this multi-layered neural network, the vector transfer subnet can map the vehicle's environmental perception, sensing, and monitoring information from the original feature space to a new vector space, thereby obtaining the vehicle's environmental transfer features.

[0034] 2) The significance of vehicle environment migration characteristics

[0035] The vehicle environment migration feature is the result of vector space migration, and it has properties that are more conducive to subsequent processing. For example, the original vehicle environment perception sensor monitoring information may have complex correlations between features and uneven data distribution. Through processing by the vector migration subnet, the vehicle environment migration feature can better reflect the relationship between different features in the new vector space, and the data distribution may be more suitable for the requirements of certain subsequent processing algorithms. This feature abstracts and optimizes the original information to a certain extent, enabling the subsequent vehicle environment perception mining subnet to more effectively quantify knowledge.

[0036] (2) Bidirectional Long Short-Term Memory Subnetwork and Knowledge Quantization

[0037] 1) Introduction to Bidirectional Long Short-Term Memory (Bi-LSTM)

[0038] The bidirectional long short-term memory subnetwork within the vehicle-based environment perception and mining subnetwork is a specialized recurrent neural network (RNN) structure specifically designed for processing sequential data. In this embodiment of the present invention, vehicle-based environment transition features serve as its input. These features may include image sequence data (e.g., multiple consecutive frames of image data, such as one frame every 5 seconds for a total of 30 frames) or other time series components of sensor data, which can be processed using the characteristics of a Bi-LSTM.

[0039] The Bi-LSTM consists of a forward LSTM and a backward LSTM. The LSTM unit has three internal gating structures: the input gate, the forget gate, and the output gate. The input gate determines what new information is allowed into the cell state, the forget gate determines what information in the cell state is forgotten, and the output gate controls what information in the cell state is output. The forward LSTM processes data in chronological order, from the beginning to the end of the sequence; the backward LSTM processes data from the end to the beginning of the sequence. This bidirectional processing approach enables the Bi-LSTM to better capture long-term dependencies in sequential data.

[0040] 2) Knowledge quantification process

[0041] After the vehicle environment migration features are input into the Bi-LSTM subnet, each LSTM unit in the Bi-LSTM subnet processes the sequence data in the input features frame by frame. Taking image sequence data as an example, for each frame of image data, the LSTM unit selectively memorizes and forgets the features in the image data according to its internal gating mechanism. For example, when processing a frame of image, the input gate determines whether to allow new feature information into the cell state based on the difference between the current frame image features and the previous frame image features (for example, by calculating the Euclidean distance between feature vectors to measure the difference, with a threshold of 0.1. If the distance is greater than the threshold, it indicates a significant change, and the input gate allows more new information to enter the cell state).

[0042] The forget gate determines whether to forget some of the cell state information based on the existing information in the cell state and the overall characteristics of the current frame image (for example, by calculating the image's average brightness, contrast, and other features). After bidirectional processing by the forward LSTM and backward LSTM, the output is the vehicle environment perception quantitative knowledge, which quantifies the knowledge of the vehicle environment transfer characteristics. This quantitative knowledge can represent the key content of the vehicle environment perception sensor monitoring information in a more compact and representative form. For example, it can quantify information such as object motion trends and environmental state changes in image sequences.

[0043] 3. Acquisition of Quantitative Knowledge of LiDAR Point Cloud

[0044] (1) Acquisition of LiDAR Point Cloud Migration Features

[0045] 1) Processing of LiDAR point clouds by the vector migration subnet

[0046] Similarly, the vector transfer subnet of the trajectory visualization processing network also processes the first lidar point cloud sensor monitoring information. The first lidar point cloud sensor monitoring information includes multiple lidar scans, sorted sequentially according to the scanning cycle. For example, if the lidar scan frequency is 20Hz, meaning a scan is performed every 0.05 seconds, 20 lidar scans can be obtained within a specific time period (e.g., 1 second).

[0047] The vector transfer subnet uses a specific algorithm to process these LiDAR scans to obtain multiple LiDAR point cloud transfer features. One possible algorithm is based on a convolutional neural network (CNN). The convolution kernel size of a CNN can be set to 3*3 with a stride of 1. The input LiDAR scan is first processed through multiple convolutional layers. For example, there may be three convolutional layers, with the first layer having 32 output channels, the second layer having 64 output channels, and the third layer having 128 output channels. There may also be pooling layers between the convolutional layers, such as a max pooling layer with a pooling window size of 2*2. After these multiple convolutional and pooling operations, point cloud transfer features corresponding to each LiDAR scan are obtained. These point cloud transfer features are more compact and abstract in their representation than the original LiDAR scans, and can better reflect key information in the LiDAR point cloud data, such as the spatial distribution characteristics and density variations of the point cloud.

[0048] (2) Processing of LiDAR Point Cloud Mining Subnet

[0049] 1) Quantitative knowledge acquisition based on scanning cycle

[0050] The LiDAR point cloud mining subnet of the trajectory visualization processing network can obtain the LiDAR point cloud quantitative knowledge of the first LiDAR point cloud sensing monitoring information based on multiple LiDAR scan information whose scanning cycles meet the step size requirements. For example, if the step size requirement is to calculate the quantitative knowledge once every three scanning cycles, the LiDAR point cloud mining subnet will then perform specific processing on the point cloud migration features corresponding to every three consecutive LiDAR scan information. This processing may involve a fusion operation on these three point cloud migration features, such as an exemplary weighted summation (the weights can be determined based on factors such as the quality of the point cloud and the scanning angle, such as performing a weighted summation of the three point cloud migration features in a ratio of 1:2:1) to obtain a comprehensive feature representation. This feature representation is part of the LiDAR point cloud quantitative knowledge based on the scanning cycle.

[0051] By performing this processing on multiple LiDAR scan information groups that meet the step size requirements, we can gradually construct complete quantitative knowledge of the LiDAR point cloud. This scan cycle-based quantitative knowledge acquisition method fully utilizes the time series characteristics of LiDAR point cloud data and reflects the changing patterns of point cloud data under different scan cycles.

[0052] 2) Acquire quantitative knowledge through residual processing subnet and feature downsampling branch

[0053] In addition, the LiDAR point cloud mining subnet can also obtain multiple 3D scene rendering feature features corresponding to multiple LiDAR point cloud migration features through its residual processing subnet. The residual processing subnet can adopt a residual network structure similar to the ResNet architecture. For example, the residual block in the residual processing subnet contains two 3*3 convolutional layers and a shortcut connection. For each LiDAR point cloud migration feature, after processing by the residual block, the corresponding 3D scene rendering feature can be obtained. These 3D scene rendering feature features can reflect the relationship between the LiDAR point cloud data and the surrounding 3D scene, such as the shape, position, and relative relationship of the object represented by the point cloud to other objects.

[0054] These 3D scene rendering feature features are then processed through the feature downsampling branch of the LiDAR point cloud mining subnetwork to obtain quantitative knowledge of the LiDAR point cloud. The feature downsampling branch can use average pooling for downsampling. For example, if the downsampling ratio is 1 / 2, and the size of the 3D scene rendering feature feature is 64*64*3 (representing the dimensions in the width, height, and depth directions, as well as the number of feature channels), it will become 32*32*3 after downsampling. This downsampling operation can further reduce the amount of data while retaining key feature information, ultimately obtaining quantitative knowledge of the LiDAR point cloud. This method of acquiring quantitative knowledge through the residual processing subnetwork and feature downsampling branch can mine the features of LiDAR point cloud data from different perspectives, and improve processing efficiency and reduce data complexity through downsampling.

[0055] It can be seen that in step 120 of the embodiment of the present invention, the vehicle-mounted environmental perception sensor monitoring information and the first laser radar point cloud sensor monitoring information are processed by different subnets in the running trajectory visualization processing network to obtain vehicle-mounted environmental perception quantitative knowledge and laser radar point cloud quantitative knowledge respectively. The acquisition process of these quantitative knowledge involves the application of multiple neural network structures and algorithms, such as the MLP or CNN algorithm in the vector transfer subnet, the Bi-LSTM subnet in the vehicle-mounted environmental perception mining subnet, and the residual processing subnet and feature downsampling branch in the laser radar point cloud mining subnet. These quantitative knowledge will provide an important data basis for the subsequent visualization of the running trajectory of the intelligent electric snowmobile, so that the subsequent processing can more accurately reflect the running environment and trajectory characteristics of the snowmobile.

[0056] Step 130 : Obtain sensor monitoring interaction knowledge features corresponding to the vehicle-mounted environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set by running the knowledge feature interaction subnet of the trajectory visualization processing network.

[0057] In an embodiment of the present invention, the associated mixed-mode sensor monitoring information set includes X pieces of associated mixed-mode sensor monitoring information corresponding to X associated intelligent electric snowmobiles, where X is a positive integer; the running trajectory visualization processing network includes a vector transfer subnet, and the knowledge feature interaction subnet includes X feature enhancement branches, a residual processing subnet, and a feature downsampling branch.

[0058] Based on this, the sensor monitoring interaction knowledge features corresponding to the vehicle environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set are obtained through the knowledge feature interaction subnet of the running trajectory visualization processing network. The method includes: using the vehicle environment migration features obtained by performing vector space migration on the vehicle environment perception sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network, and X associated migration features obtained by performing vector space migration on the X associated mixed-mode sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network, as inputs to the knowledge feature interaction subnet; processing the vehicle environment migration features and the X associated migration features through X feature enhancement branches of the knowledge feature interaction subnet to obtain X associated vehicle environment enhancement features; obtaining associated residual labeled features corresponding to the X associated vehicle environment enhancement features through the residual processing subnet of the knowledge feature interaction subnet; and processing the associated residual labeled features through the feature downsampling branch of the knowledge feature interaction subnet to obtain the sensor monitoring interaction knowledge features.

[0059] In a more detailed step, the processing of the vehicle environment migration feature and the X associated migration features by the X feature enhancement branches of the knowledge feature interaction subnet to obtain X associated vehicle environment enhancement features includes: obtaining a visual layer overlay feature between the vehicle environment migration feature and a u-th associated migration feature among the X associated migration features, where u is a positive integer not greater than X; integrating the vehicle environment migration feature, the u-th associated migration feature, and the visual layer overlay feature, and inputting the result of the integration into a weighted processing module of a u-th feature enhancement branch among the X feature enhancement branches to obtain an output of the weighted processing module of the u-th feature enhancement branch; and using the output of the weighted processing module of the u-th feature enhancement branch as an input to a normalization module of the u-th feature enhancement branch, which outputs the u-th associated vehicle environment enhancement feature.

[0060] In the invention of the technology for visualizing the trajectory of an intelligent electric snowmobile, step 130 is a key step. Its core lies in obtaining sensor-monitoring interaction knowledge features corresponding to the onboard environmental perception sensor monitoring information and the associated mixed-mode sensor monitoring information set through the knowledge feature interaction subnet of the trajectory visualization processing network. This step aims to integrate the onboard environmental perception information of the target intelligent electric snowmobile itself and the information of other associated intelligent electric snowmobiles (number X). Through a complex processing flow, it mines more comprehensive and in-depth interaction knowledge features, providing a rich information foundation for the subsequent accurate display of the target snowmobile's trajectory.

[0061] 1. Input feature preparation

[0062] (1) Processing of vector migration subnets

[0063] 1) Acquisition of vehicle environment migration characteristics

[0064] The vector transfer subnet in the trajectory visualization processing network first performs vector space transfer on the onboard environmental perception sensor monitoring information to obtain onboard environmental transfer features. This onboard environmental perception sensor monitoring information contains a wide range of information about the target smart electric snowmobile's surroundings, such as camera image data (e.g., image resolution of 1920*1080 pixels, RGB color mode), temperature sensor data, humidity sensor data, and so on. The vector transfer subnet may employ a deep learning-based algorithm, such as a deep neural network (DNN), to perform vector space transfer.

[0065] For example, this DNN has multiple hidden layers. The number of neurons in the input layer is determined by the feature dimensions of the vehicle's environmental perception, sensor, and monitoring information. This might be achieved by expanding the image data into a one-dimensional vector and then adding the feature dimensions of other sensor data, for a total of 2048 feature dimensions, for example. After nonlinear transformations in the hidden layers (e.g., with 1024, 512, or 256 hidden layer neurons), the output is a vehicle environment transfer feature. This vehicle environment transfer feature effectively extracts and optimizes the representation of the original vehicle environmental perception, sensor, and monitoring information in a new vector space.

[0066] 2) Acquisition of associated migration features

[0067] Similarly, the vector transfer subnet performs vector space transfer on X pieces of associated mixed-mode sensor monitoring information, generating X associated transfer features. Each piece of associated mixed-mode sensor monitoring information is similar to the sensor monitoring information of the target smart electric snowmobile, also containing environmental perception information and LiDAR point cloud information. For example, the image data in the associated mixed-mode sensor monitoring information is 1280*720 pixels, and other sensor data includes information such as temperature and wind speed.

[0068] The vector transfer subnet performs vector space transfer using a similar DNN structure (although network parameters, such as the number of hidden layer neurons and learning rate, may be adjusted based on data characteristics). Each piece of associated mixed-mode sensor monitoring information is processed by the vector transfer subnet to generate corresponding associated transfer features. These associated transfer features share the same purpose as the vehicle environment transfer features: transforming the original information into a feature space for more efficient processing in the knowledge feature interaction subnet.

[0069] 2. Obtaining enhanced features of the associated vehicle environment through feature enhancement branches

[0070] (1) Obtaining Visible Layer Overlay Features

[0071] 1) The concept of visual layer overlay features

[0072] For each associated transfer feature (taking the u-th associated transfer feature as an example, where u is a positive integer not greater than X), the visual layer overlay feature between the vehicle environment transfer feature and the u-th associated transfer feature is obtained. This process involves constructing a graph-based relationship representation. The visual layer overlay feature is designed to capture the complex relationship between the vehicle environment transfer feature and the associated transfer feature.

[0073] For example, the individual feature elements in the vehicle environment migration feature and the u-th associated migration feature can be considered as nodes of the graph. Then, the edge connection is determined based on the correlation between the feature elements. There are many ways to calculate the correlation, such as calculating the cosine similarity between feature vectors. For example, a cosine similarity threshold is set to 0.6. If the cosine similarity between two feature elements is greater than this threshold, an edge is established between them. The graph structure constructed in this way can represent the intrinsic connection between the vehicle environment migration feature and the u-th associated migration feature. The information contained in this graph structure is the visual layer overlay feature.

[0074] 2) Feature integration and weighted processing

[0075] Feature Integration: Once the visible layer overlay features are obtained, the vehicle environment transfer features, the u-th associated transfer features, and the visible layer overlay features are integrated. This integration can be an exemplary feature vector concatenation, where the three feature vectors are connected in a certain order to form a longer feature vector. For example, if the vehicle environment transfer feature vector has a dimension of 256, the u-th associated transfer feature vector has a dimension of 256, and the visible layer overlay feature vector has a dimension of 128, then the resulting representation vector after integration has a dimension of 640.

[0076] Weighted Processing: The integrated result is input into the weighted processing module of the uth feature enhancement branch among the X feature enhancement branches. The purpose of the weighted processing module is to assign different weights to different features based on their importance. An adaptive weighting algorithm can be used here, for example, to determine weights based on the feature's information entropy. Information entropy is a measure of the amount of information contained in a feature. The lower the information entropy, the higher the certainty of the feature and the more important it may be.

[0077] For example, the initial weights of the vehicle environment transfer feature, the u-th associated transfer feature, and the visible layer overlay feature in the integrated representation are 0.3, 0.3, and 0.4, respectively. The weighting module adjusts these weights based on their respective information entropies (calculated from a certain number of sample data). For example, if the vehicle environment transfer feature is found to have low information entropy, its weight might be increased to 0.4, and the weights of other features might be adjusted accordingly, such as adjusting the weight of the u-th associated transfer feature to 0.3 and the weight of the visible layer overlay feature to 0.3. After weighting, the output of the weighting module is obtained.

[0078] 3) Normalization

[0079] The output of the weighted processing module of the u-th feature enhancement branch is used as the input to the normalization module of the u-th feature enhancement branch. The normalization module uses the Batch Normalization algorithm, a commonly used neural network normalization technique. Batch Normalization normalizes the input data to a mean of 0 and a variance of 1, thereby accelerating the neural network training process and improving the model's generalization ability.

[0080] After batch normalization, the normalization module of the u-th feature enhancement branch outputs the u-th associated vehicle environment enhancement feature. This associated vehicle environment enhancement feature integrates the vehicle environment transfer feature, the u-th associated transfer feature, and the visual layer overlay features between them. After weighting and normalization, it can better reflect the interactive relationship between the target smart electric snowmobile and the u-th associated smart electric snowmobile in terms of environmental perception. By performing this processing on each associated transfer feature (u ranges from 1 to X), X associated vehicle environment enhancement features can be obtained.

[0081] 3. Obtaining the associated residual annotation features and sensor monitoring interaction knowledge features

[0082] (1) Acquisition of associated residual annotation features

[0083] 1) The role of the residual processing subnet

[0084] The residual processing subnet in the knowledge-feature interaction subnet is used to obtain the associated residual annotation features corresponding to X associated vehicle environment enhancement features. This residual processing subnet adopts the Residual Network (ResNet) architecture. The core of ResNet is the residual block. For example, the residual block structure here is: input -> convolutional layer (convolution kernel size is 3*3, number of output channels is 64) -> batch normalization -> ReLU activation function -> convolutional layer (convolution kernel size is 3*3, number of output channels is 64) -> batch normalization -> input + convolutional layer output.

[0085] Each associated vehicle environment enhancement feature is input into the residual block of the residual processing subnet for processing. The convolutional layers in the residual block effectively extract features, batch normalization is used for data normalization, and the ReLU activation function introduces nonlinear characteristics. Through the processing of the residual block, the associated residual annotation features corresponding to each associated vehicle environment enhancement feature are obtained. These associated residual annotation features, to a certain extent, represent the remaining information or difference information of the associated vehicle environment enhancement feature after residual processing, which is important for mining deeper interactive knowledge features.

[0086] 2. Acquisition of sensor monitoring interaction knowledge features

[0087] 1) Processing of feature downsampling branches

[0088] The feature downsampling branch of the knowledge feature interaction subnetwork processes the associated residual annotation features to obtain sensor monitoring interaction knowledge features. This feature downsampling branch can use various downsampling methods, such as max-pooling. For example, the pooling window size for max-pooling is 2*2.

[0089] When the associated residual annotation feature is input into the feature downsampling branch, the max pooling operation selects the maximum value within a local region (a 2*2 window) of the feature map as the output. For example, if the feature map size of the associated residual annotation feature is 16*16*64 (representing width, height, and number of channels), after max pooling downsampling, the feature map size becomes 8*8*64. This downsampling operation can reduce the amount of data while retaining key feature information, ultimately resulting in a sensor monitoring interaction knowledge feature. This sensor monitoring interaction knowledge feature integrates the interactive information of the on-board environmental perception sensor monitoring information and the associated mixed-mode sensor monitoring information set. After complex processing by multiple subnetworks, it can provide a more comprehensive and representative information foundation for the subsequent visualization of the intelligent electric snowmobile's running trajectory.

[0090] Therefore, in step 130 of the embodiment of the present invention, a series of complex and orderly processing is performed on the vehicle-mounted environment migration features of the target intelligent electric snowmobile and the associated migration features of the associated intelligent electric snowmobiles through the knowledge feature interaction subnet of the trajectory visualization processing network. Starting with obtaining the visual layer overlay features, the associated vehicle-mounted environment enhancement features are obtained through feature integration, weighting processing, and normalization processing. The associated residual annotation features are then obtained through the residual processing subnet. Finally, the sensor monitoring interaction knowledge features are obtained using the feature downsampling branch. This process involves multiple algorithms and network structures in graph theory and deep learning, such as constructing visual layer overlay features based on cosine similarity, a weighted processing module using an adaptive weighting algorithm, a residual processing subnet based on a ResNet structure, and a feature downsampling branch using maximum pooling. The comprehensive application of these processing steps and technical means enables the final sensor monitoring interaction knowledge features to fully integrate the relevant information of the target vehicle and the associated vehicles, providing an important interactive information foundation for the entire intelligent electric snowmobile trajectory visualization display technology.

[0091] Step 140: Perform full-connection processing on the vehicle environment perception quantitative knowledge, the lidar point cloud quantitative knowledge, and the sensor monitoring interaction knowledge features through the fully-connected subnet of the running trajectory visualization processing network to obtain the fully-connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile.

[0092] In step 140, the fully connected subnetwork of the trajectory visualization processing network performs fully connected processing on the quantitative knowledge of vehicle-mounted environmental perception, the quantitative knowledge of lidar point clouds, and the sensor-monitoring interaction knowledge features. For example, the dimension of the quantitative knowledge of vehicle-mounted environmental perception is 128, the dimension of the quantitative knowledge of lidar point clouds is 64, and the dimension of the sensor-monitoring interaction knowledge features is 32. The structure of the fully connected subnetwork may include multiple fully connected layers (for example, two fully connected layers, with the first fully connected layer having 256 neurons and the second fully connected layer having 128 neurons). Neurons in the fully connected subnetwork are connected via a weight matrix. The initial value of the weight matrix can be randomly initialized (e.g., randomly selected from a normal distribution with a mean of 0 and a standard deviation of 0.01). The weight values are then continuously adjusted during training using a backpropagation algorithm, ultimately obtaining the fully connected quantitative knowledge of the target intelligent electric snowmobile's trajectory.

[0093] In detail, the further development of the fully connected quantitative knowledge of the running trajectory obtained based on the fully connected subnetwork processing is introduced as follows.

[0094] (1) Introduction to the fully connected subnet

[0095] 1) The position and role of the fully connected subnet in the network structure

[0096] In the trajectory visualization processing network, the fully connected subnetwork plays a key role in integrating different types of quantitative knowledge. It receives as input the quantitative knowledge of the vehicle's environmental perception, the quantitative knowledge of the LiDAR point cloud, and the sensor-monitoring interaction features obtained from the previous steps. The fully connected subnetwork is a classic neural network structure in which each neuron is fully connected to all neurons in the previous layer. This structure enables the network to learn the complex relationships between input features, thereby providing a comprehensive quantitative representation of the target intelligent electric snowmobile's trajectory.

[0097] 2) Example of basic structural parameters of a fully connected subnet

[0098] For example, the dimension of the quantitative knowledge of vehicle-mounted environmental perception is 128, the dimension of the quantitative knowledge of lidar point clouds is 64, and the dimension of sensor-monitoring interaction knowledge features is 32. The structure of the fully connected subnetwork can be configured to include multiple fully connected layers. For example, the number of neurons in the first fully connected layer is 256, and the number of neurons in the second fully connected layer is 128. The number of neurons in the input layer is the sum of the dimensions of the quantitative knowledge of vehicle-mounted environmental perception, the quantitative knowledge of lidar point clouds, and the sensor-monitoring interaction knowledge features, that is, 128 + 64 + 32 = 224.

[0099] (2) Full connection processing

[0100] 1) Weight initialization

[0101] Before the fully connected subnet is fully connected, the weights of the fully connected layer need to be initialized. A common initialization method is random initialization. For example, the weights can be initialized by randomly drawing values from a normal distribution with a mean of 0 and a standard deviation of 0.01. The bias term can also be initialized to 0. This initialization provides a starting point for training the fully connected subnet, allowing the network to gradually adjust the weights to adapt to the characteristics of the data during subsequent training.

[0102] 2) Forward propagation process

[0103] First fully connected layer processing: When the quantitative knowledge of vehicle environmental perception, the quantitative knowledge of LiDAR point clouds, and the sensor-monitoring interaction features are input to the first fully connected layer of the fully connected subnet, each input neuron is connected to the 256 neurons in the first fully connected layer via a weight matrix. For example, if the input vector is (x = [x_1, x_2, cdots, x_{224}]), the weight matrix is (W_1) (dimension is (224*256)), and the bias vector is (b_1) (dimension is 256), then the output (y_1) of the first fully connected layer can be calculated using the formula (y_1 = f(W_1x + b_1)), where (f) is the activation function, for example, the Rectified Linear Unit (ReLU) activation function, i.e., (f(z) = max(0,z)). The ReLU activation function increases the nonlinearity of the network, enabling the fully connected subnet to learn more complex functional relationships.

[0104] Second fully connected layer processing (optional): If the fully connected subnet includes a second fully connected layer, the output of the first fully connected layer (y_1) will serve as the input of the second fully connected layer. The weight matrix of the second fully connected layer is set to (W_2) (dimension is (256*128)), and the bias vector is (b_2) (dimension is 128). The output of the second fully connected layer (y_2) is calculated using the formula (y_2=f(W_2y_1+b_2)). Through this multi-layer fully connected layer processing, the fully connected subnet can gradually integrate and abstract different types of input quantitative knowledge, ultimately obtaining fully connected quantitative knowledge of the target intelligent electric snowmobile's trajectory.

[0105] 3) The significance of fully connected processing

[0106] By fully connecting the vehicle's environmental perception quantitative knowledge, the LiDAR point cloud quantitative knowledge, and the sensor-monitoring interaction knowledge features, the fully connected subnetwork can fuse these diverse knowledge sources and dimensions into a unified representation space. This fusion considers the interrelationships between the vehicle's environmental perception information (such as camera-recognized objects and environmental conditions), the LiDAR point cloud information (such as the 3D structure of surrounding terrain and obstacles), and the interaction information with other connected vehicles (sensor-monitoring interaction knowledge features). For example, a feature in the vehicle's environmental perception quantitative knowledge may be associated with specific structural information in the LiDAR point cloud quantitative knowledge. The fully connected subnetwork learns this association and reflects it in the fully connected quantitative knowledge of the trajectory. This quantitative knowledge can more comprehensively and accurately describe various factors related to the target intelligent electric snowmobile's trajectory, providing a rich information foundation for the subsequent selection of visualization transformation strategies.

[0107] Step 150: Based on the fully connected quantitative knowledge of the target intelligent electric snowmobile's running trajectory, a target visualization conversion strategy is obtained from a visualization conversion strategy pool; the target visualization conversion strategy is used to achieve a visualization display of the target intelligent electric snowmobile's running trajectory.

[0108] In step 150, a target visualization conversion strategy is obtained from a visualization conversion strategy pool based on the fully connected quantitative knowledge of the target intelligent electric snowmobile's trajectory. The visualization conversion strategy pool contains multiple predefined visualization conversion strategies. For example, one strategy may be based on a 2D planar map display, suitable for an exemplary trajectory display. It primarily displays the snowmobile's route on a planar map, with different colored lines representing different driving states (e.g., green for normal driving and red for abnormal driving). Another strategy may be based on 3D scene reconstruction and trajectory display. This strategy utilizes quantitative knowledge from a LiDAR point cloud to construct a 3D scene around the snowmobile and display the trajectory within the 3D scene. The trajectory is presented as a three-dimensional curve and the positional relationship of the snowmobile with respect to surrounding obstacles (e.g., snowdrifts, trees, etc.) is also displayed. Based on the features of the fully connected quantitative knowledge of the trajectory (e.g., trajectory complexity, surrounding environmental danger level, etc.), an appropriate target visualization conversion strategy is selected to achieve a visual display of the target intelligent electric snowmobile's trajectory.

[0109] In detail, the above-mentioned target visualization conversion strategy based on fully connected quantitative knowledge selection is explained as follows.

[0110] (1) Visualization Conversion Strategy Pool

[0111] 1) Composition of the Strategy Pool

[0112] The visualization transition strategy pool is a collection of predefined visualization transition strategies. These strategies are designed to visualize the target smart electric snowmobile's trajectory in various ways. For example, one strategy might be based on a 2D map. With this strategy, the trajectory is plotted on a two-dimensional map, with lines of different colors representing different driving states, such as green for normal driving and red for abnormal driving. Key geographic information can also be marked on the map, such as terrain features like hills and gullies in the snow, as well as fixed facilities (such as rest stops at a ski resort).

[0113] Another strategy could be based on 3D scene reconstruction and trajectory display. This strategy leverages quantitative knowledge from LiDAR point clouds to construct a 3D scene around the snowmobile, within which the trajectory is displayed. The trajectory is presented as a three-dimensional curve, while also showing the snowmobile's position relative to surrounding obstacles (such as snowdrifts and trees). Furthermore, a multi-perspective display strategy could be used to display the snowmobile's trajectory from multiple perspectives (such as top-down, side, and front views), providing a more comprehensive view of the snowmobile's movement in different directions.

[0114] 2) Strategy parameters and characteristics

[0115] Each visualization conversion strategy has its own parameters and characteristics. For a 2D map display strategy, its parameters may include the map scale, line width, and color coding rules. The scale determines the proportional relationship between the distance on the map and the actual distance. For example, a scale of 1:1000 means that 1 centimeter on the map represents 10 meters in actual distance. The line width can be adjusted based on the speed or importance of the snowmobile. Faster or more important tracks can have wider lines. The color coding rules specify the colors corresponding to different driving states.

[0116] Parameters for 3D scene reconstruction and trajectory display strategies may include 3D scene construction accuracy, the color and thickness of trajectory curves, and the level of detail in obstacle display. The accuracy of 3D scene construction depends on the quality and processing of the quantitative knowledge from the LiDAR point cloud. Higher accuracy results in a more realistic reproduction of the environment surrounding the snowmobile. The color and thickness of the trajectory curves can also be used to distinguish different driving states or trajectory types. The level of detail in obstacle display can be adjusted based on user needs. For example, when detailed analysis of the relationship between the snowmobile and a specific obstacle is required, the level of detail in the obstacle display can be increased.

[0117] 2. Strategy Selection Based on Fully Connected Quantitative Knowledge

[0118] 1) The relationship between features and strategy selection in fully connected quantitative knowledge

[0119] The fully connected quantitative knowledge of the target intelligent electric snowmobile's trajectory contains rich information, which will determine which target visualization conversion strategy to select from the visualization conversion strategy pool. For example, if certain features in the fully connected quantitative knowledge indicate that the snowmobile's trajectory is mainly on a plane and the surrounding environment is relatively exemplary, without much three-dimensional structural information to display, then a strategy based on 2D plane map display may be more suitable. These features can be reflected by the values of certain dimensions in the fully connected quantitative knowledge, such as the dimension representing vertical changes has a smaller value, while the dimension representing horizontal travel is dominant.

[0120] Conversely, if the fully connected quantitative knowledge contains a wealth of information related to the surrounding three-dimensional environment structure, such as the quantitative knowledge of LiDAR point clouds, which retains rich information such as terrain undulations and obstacle heights after processing through the fully connected subnetwork, then a strategy based on 3D scene reconstruction and trajectory display may be a better choice. Furthermore, if the fully connected quantitative knowledge also includes features related to interactions with other associated vehicles, such as in a scenario where multiple vehicles are driving together and the relative position of a snowmobile with other vehicles needs to be displayed from multiple perspectives, a strategy based on multi-perspective display may be chosen.

[0121] 2) Example of the specific process of strategy selection

[0122] For example, the fully connected quantitative knowledge of the running trajectory can be represented as a vector (q=[q_1, q_2, cdots, q_{128}]) (here we take 128-dimensional fully connected quantitative knowledge as an example). Some rules can be defined to select a visualization transformation strategy based on the element values in the vector (q). For example, calculate the average value (bar{q}) of the elements in the vector (q) that represent vertical motion features (for example, the 11 elements from (q_{10}) to (q_{20})). If (bar{q}<0.1) (here 0.1 is a pre-set threshold that can be adjusted according to actual conditions), the 2D plane map display strategy is given priority.

[0123] If (bar{q}geq0.1) and the average value of the elements in vector (q) representing interaction features with surrounding three-dimensional objects (for example, the 11 elements from (q_{30}) to (q_{40})) is greater than another threshold (such as 0.2), then the strategy based on 3D scene reconstruction and trajectory display is selected. If neither of these conditions is met, but the variance of the elements in vector (q) representing interaction features with other vehicles (for example, the 11 elements from (q_{50}) to (q_{60})) is greater than a certain value (such as 0.3), then the strategy based on multi-view display is selected. Through such specific rules, the target visualization transformation strategy can be accurately selected from the visualization transformation strategy pool based on the different features of the fully connected quantitative knowledge of the running trajectory, thereby achieving a visualization display of the running trajectory of the target intelligent electric snowmobile.

[0124] (3) The significance of target visualization conversion strategy

[0125] 1) Targeted visualization of running tracks

[0126] The selected target visualization conversion strategy is customized for the features of the fully connected quantitative knowledge of the target intelligent electric snowmobile's trajectory, and can display the snowmobile's trajectory in the most appropriate way. For example, the 2D plane map display strategy is concise and clear for users who only need to understand the snowmobile's route on a flat surface and its basic driving status (normal or abnormal). The 3D scene reconstruction and trajectory display strategy, on the other hand, provides more intuitive and detailed information for users who need to deeply analyze the relationship between the snowmobile and the surrounding complex three-dimensional environment (such as driving on snowy terrain).

[0127] 2) Improve the effectiveness and comprehensibility of visual presentations

[0128] The selection of target visualization conversion strategies helps improve the effectiveness and comprehensibility of visualization presentations. By choosing the appropriate strategy, key information from the fully connected quantitative knowledge of the trajectory can be presented in a user-friendly manner. For example, under the multi-view display strategy, different perspectives allow users to observe the driving conditions of a snowmobile from multiple perspectives, better understanding the snowmobile's position and behavior throughout the driving scene, thereby providing more valuable visualization support for the monitoring, management, and research of intelligent electric snowmobiles.

[0129] As can be seen, steps 140 and 150 play a crucial role in the entire intelligent electric snowmobile trajectory visualization technology. Step 140 fuses multiple quantitative knowledge points through a fully connected subnetwork to obtain fully connected quantitative knowledge of the trajectory. Based on this fully connected quantitative knowledge, step 150 selects the most appropriate target visualization transformation strategy from a pool of visualization transformation strategies, ultimately achieving efficient, accurate, and targeted trajectory visualization.

[0130] In summary, the present invention implements a rigorous series of steps and complex network subnet processing, starting with acquiring multiple sensor monitoring information, gradually conducting quantitative knowledge mining, acquiring interactive knowledge features, and calculating fully connected quantitative knowledge, ultimately determining an appropriate visualization conversion strategy to achieve effective visualization of the trajectory of an intelligent electric snowmobile. This technology has important application value in fields such as intelligent transportation and autonomous snowmobile monitoring.

[0131] To facilitate understanding of the above technical solution, the following further introduces and explains it through actual application examples.

[0132] Target intelligent electric snowmobile 3D running trajectory visualization example

[0133] 1. Example of target mixed-mode sensor monitoring information set

[0134] Vehicle-mounted environmental perception sensor monitoring information

[0135] 1) Camera image data

[0136] The image captured by the camera is in RGB format with a resolution of 1920 x 1080 pixels. The image shows a snowy scene, with the snowmobile positioned slightly left of center. Directly ahead is a winding snowy trail flanked by undulating snowdrifts. In the distance, a large snow-capped mountain can be seen, its outline clearly defined against the blue sky.

[0137] Image analysis techniques can be used to determine the edge of the snow track. For example, the coordinates of the left edge of the snow track in the image range from (500, 500) to (1500, 800), and the right edge ranges from (700, 500) to (1700, 800). (The coordinates here are based on the upper left corner of the image, with the positive x-axis pointing rightward and the positive y-axis pointing downward.) The approximate position and shape of the snowdrifts in the image are also detected. For example, the coordinates of the larger snowdrift on the left side of the image range from (300, 600) to (600, 900).

[0138] 2) Other sensor data

[0139] The temperature sensor indicates the current ambient temperature is -15°C, and the wind direction sensor indicates the wind is northwest at a speed of 5 m / s. This environmental data helps us understand the driving environment of the snowmobile. For example, wind direction and speed can affect the snowmobile's handling performance.

[0140] The first laser radar point cloud sensing monitoring information

[0141] 1) LiDAR scanning information

[0142] The lidar scans at a 15Hz frequency, capturing point cloud data within a 20-meter radius during each scan. At one point, the lidar detected a small gully 10 meters in front of the snowmobile. The point cloud data for the gully ranges from (8, -2, -0.5) to (12, 2, -0.3) in the lidar coordinate system. (The coordinates are expressed in meters, with the lidar's location as the origin, the x-axis pointing in the direction of the snowmobile's forward motion, the y-axis pointing horizontally, and the z-axis pointing upward.)

[0143] There's a high snowdrift 5 meters to the left of the snowmobile, with some of its point cloud coordinates ranging from (-5, 3, 1) to (-5, 5, 2). Additionally, some tree branches above the snowy path are detected, with their point cloud coordinates scattered across the snow. For example, some branches have coordinates ranging from (2, 1, 3) to (4, 3, 4).

[0144] 2. Target Visualization Conversion Strategy Example - Strategy Based on 3D Scene Reconstruction and Trajectory Display

[0145] 1) 3D scene construction

[0146] A 3D coordinate system is established with the snowmobile's initial position as the origin. The x-axis represents the snowmobile's forward direction, the y-axis represents the horizontal direction, and the z-axis represents the vertical direction. A 3D scene is constructed based on the LiDAR point cloud data. The scene's spatial scope is set to a rectangular space centered on the snowmobile, with a length of 30 meters in front, behind, left, and right, and a height of 10 meters.

[0147] For objects like snowdrifts, gullies, and tree branches detected by the LiDAR, corresponding 3D models are constructed based on their point cloud coordinates. Snowdrifts are represented using irregular polyhedron models, gullies using concave models with a certain depth, and tree branches using a combination of slender cylinders.

[0148] 2) Trajectory drawing

[0149] The snowmobile's trajectory is represented by a blue three-dimensional curve. The thickness of the curve is dynamically adjusted based on the snowmobile's speed. For example, at speeds between 0 and 5 m / s, the curve thickness is 1 mm; at speeds between 5 and 10 m / s, the curve thickness is 2 mm.

[0150] 3) Environmental element labeling

[0151] In the 3D scene, the snow-capped mountains detected in the camera image are represented as large triangular facets, and their position and scale in the 3D scene are estimated based on their position and size in the image. The snow trails are represented as white ribbon-like plane models, with their width determined based on the actual width and scene scale. The trail's direction is determined by converting the trail edge coordinates in the image into a 3D coordinate system.

[0152] Wind direction and speed can be represented by an arrow model. The direction of the arrow indicates the wind direction, and the length and thickness of the arrow are determined according to the wind speed. For example, when the wind speed is 5 m / s, the length of the arrow is 1 m and the thickness is 0.1 m.

[0153] 3. Visualization of the Conversion Process

[0154] (1) Coordinate transformation and feature mapping

[0155] 1) Conversion from image coordinates to 3D coordinates

[0156] The coordinates of the trail edge and snowdrifts in the camera image need to be converted to a 3D coordinate system. First, the conversion is performed based on the camera's intrinsic parameters (such as focal length) and extrinsic parameters (such as the camera's position and attitude). For example, the coordinates of the left edge of the trail (500, 500) in the image can be converted to the coordinates (10, -5, 0) meters in the 3D coordinate system. (This conversion process involves complex computer vision algorithms, such as perspective transformation.)

[0157] The coordinates of the snow pile are also converted. For example, the coordinate range of the large snow pile on the left side of the screen is converted from (300, 600) to (600, 900) to (-8, -3, 1) to (-5, 0, 2) meters in the 3D coordinate system.

[0158] 2) Conversion of LiDAR coordinates to 3D coordinates

[0159] The coordinates of objects detected by the lidar are already in its native coordinate system and need to be converted to a 3D coordinate system with the snowmobile's initial position as the origin. For the coordinates of the gully ahead, which range from (8, -2, -0.5) to (12, 2, -0.3), these coordinates are converted to the 3D scene coordinate system (8, -2, -0.5) to (12, 2, -0.3)) through translation and rotation (based on the lidar's installation position and attitude relative to the snowmobile). (For example, the relative relationship between the lidar and snowmobile coordinate systems is shown below; the coordinates remain essentially unchanged.)

[0160] The coordinate range of the snow pile on the left ((-5, 3, 1) to (-5, 5, 2)) is converted to ((-5, 3, 1) to (-5, 5, 2)) meters in the 3D scene coordinate system.

[0161] 3) Feature mapping to visualization elements

[0162] The features corresponding to the converted coordinates are mapped into 3D visualization elements. For example, the converted coordinate points of the snow trail edge are connected to construct a white ribbon plane model to represent the snow trail; the coordinates of the snow pile are constructed into a polyhedron model; the coordinate points of the snowmobile's running track are connected to form a blue 3D curve; and wind direction and speed information are mapped into an arrow model.

[0163] (2) Recording coordinate features or image coding features

[0164] 1) Coordinate feature record

[0165] Record the sequence of coordinate points of the snowmobile's trajectory in the 3D coordinate system. For example, the sequence of coordinate points of the snowmobile's trajectory over a period of time is [(0, 0, 0), (1, 0, 0), (2, 0, 0.1), (3, 0, 0.2)] (the coordinates here are in meters). Also record the coordinates of other environmental elements and obstacles in the 3D coordinate system, such as the coordinate range of a gully ((8, -2, -0.5) to (12, 2, -0.3)) meters, and the coordinates of a snowdrift ((-5, 3, 1) to (-5, 5, 2)) meters. These coordinate features can fully describe the trajectory of the target intelligent electric snowmobile in the 3D scene and the relative positional relationships of the surrounding environmental elements.

[0166] 2) Image coding features (relatively complex)

[0167] A hierarchical encoding approach is used. First, the 3D scene is divided into multiple cubic units, for example, with a side length of 1 meter. Each unit is encoded based on the elements it contains (such as snowmobile tracks, snow piles, gullies, etc.).

[0168] Define a coding dictionary. For example, a snowmobile track is coded as 1, a snowdrift is coded as 2, a gully is coded as 3, a snowy mountain is coded as 4, a branch is coded as 5, and so on. For each cube unit, if it contains a certain element, record the encoding value of that element. For example, in a 3*3*3 cube unit area centered at the origin, the coding matrix can be expressed as (partial example): [

[0170] begin{bmatrix}

[0171] 0&0&0

[0172] 0&1&0

[0173] 0&0&0

[0174] end{bmatrix} ]

[0176] At a higher level, the position coordinates of each cube unit are recorded. For example, the coordinates of the cube unit corresponding to the above encoding matrix are (0, 0, 0). Through this layered encoding method, various elements in the 3D scene and their positional relationships can be fully recorded, thereby representing the characteristics of the 3D trajectory image of the target intelligent electric snowmobile.

[0177] In a preferred example, the visualization conversion strategy pool includes multiple visualization conversion strategies, each visualization conversion strategy generates a corresponding visual element mapping feature through a spatiotemporal graph neural network, and the visual element mapping feature of the target visualization conversion strategy obtained from the visualization conversion strategy pool is adapted to the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile; and the spatiotemporal graph neural network and the running trajectory visualization processing network are obtained through collaborative debugging, and the error variable determined by the debugging error indicator used in the collaborative debugging link includes: the fully connected quantitative knowledge of the running trajectory of the intelligent electric snowmobile sample obtained through the running trajectory visualization processing network during debugging, the visual element mapping feature of the visualization conversion strategy sample obtained through the spatiotemporal graph neural network during debugging, and the debugging guidance information determined based on the quality evaluation of the visualization conversion strategy sample based on the intelligent electric snowmobile sample.

[0178] On this basis, the method further comprises:

[0179] (1) Obtaining a three-dimensional mixed-mode visual annotation information set of a target visualization conversion strategy, wherein the three-dimensional mixed-mode visual annotation information set includes: trajectory visual positioning annotation information and visual element annotation information at multiple attention levels;

[0180] In detail, obtaining visual element annotation information of multiple attention levels of the target visualization conversion strategy includes: performing task sampling on the 3D visual conversion task information included in the target visualization conversion strategy, and determining the visual element annotation information of the 3D attention level according to the 3D visual conversion task event obtained by the task sampling; performing vehicle posture conversion task mining on the 3D visual conversion task event obtained by the task sampling to obtain the vehicle posture conversion task event, and determining the visual element annotation information of the vehicle posture attention level based on the vehicle posture conversion task event and the visualized vehicle status data of the target visualization conversion strategy; determining the visual element annotation information of multiple attention levels of the target visualization conversion strategy based on the visual element annotation information of the 3D attention level and the visual element annotation information of the vehicle posture attention level;

[0181] (2) generating attention annotation features at multiple attention levels based on the annotation information of each visual element and the visual positioning annotation information of the trajectory, and performing full connection processing on the attention annotation features at the multiple attention levels to obtain cross-attention annotation features;

[0182] (3) obtaining a cross-attention residual annotation feature corresponding to the cross-attention annotation feature through the residual processing subnet of the spatiotemporal graph neural network;

[0183] (4) Feature mapping is performed on the cross-attention residual annotation features through the bidirectional long short-term memory subnetwork of the spatiotemporal graph neural network to obtain visual element mapping features of the target visualization conversion strategy; the target visualization conversion strategy is any one of the multiple visualization conversion strategies.

[0184] The above embodiment summarizes the technical solution for generating visual element mapping features of the target visualization conversion strategy, and the following describes the technical solution in detail.

[0185] 1. Visualization Conversion Strategy Pool and Visual Element Mapping Feature Adaptation

[0186] (1) Composition of the Visual Conversion Strategy Pool

[0187] The visualization transformation strategy pool contains multiple visualization transformation strategies. Each strategy generates corresponding visual element mapping features through a spatiotemporal graph neural network. These mapping features are generated to adapt the visualization transformation strategy to the fully connected quantitative knowledge of the target smart electric snowmobile's trajectory. For example, the visualization transformation strategy pool contains three strategies: Strategy A (suitable for 2D trajectory display in exemplary terrain), Strategy B (suitable for 3D trajectory display in complex terrain and focusing on environmental interaction), and Strategy C (suitable for trajectory comparison display in multi-vehicle scenarios).

[0188] (2) Collaborative debugging process

[0189] 1) Concepts related to debugging error indicators

[0190] The spatiotemporal graph neural network and the trajectory visualization processing network optimize each other's performance through collaborative debugging. In this collaborative debugging phase, the error variables determined by the debugging error indicators used include multiple parts.

[0191] First, the fully connected quantitative knowledge of the trajectory of the intelligent electric snowmobile example is obtained through the trajectory visualization processing network during debugging. For example, during one debugging process, the fully connected quantitative knowledge of the trajectory of an intelligent electric snowmobile example in a specific snowy driving scenario is a 128-dimensional vector, such as [0.1, 0.2, -0.3, …, 0.05] (the values here are only examples). This vector contains information fused from the quantitative knowledge of the vehicle's environmental perception, the quantitative knowledge of the lidar point cloud, and the interactive knowledge of sensor monitoring.

[0192] Next, the visual element mapping features of the visualization transformation strategy example are obtained through the debugging spatiotemporal graph neural network. Taking strategy B as an example, its visual element mapping features may be in the form of a matrix, such as a 32*32 matrix. The elements in the matrix represent the mapping relationship between different visual elements (such as the spatial position of the trajectory and its relationship with environmental elements) in the spatiotemporal graph neural network.

[0193] Finally, we evaluate the quality of the visualization conversion strategy example based on the smart electric snowmobile example to determine debugging guidance information. This quality evaluation can be done using various methods, such as calculating the similarity between the trajectory after visualization conversion and the actual trajectory (this can be measured by calculating the mean squared error between the trajectory coordinate points. For example, if the actual trajectory coordinate points are [(0, 0, 0), (1, 0, 0), (2, 0, 0.1)] and the trajectory coordinate points after visualization conversion are [(0, 0, 0), (1.1, 0, 0), (2.2, 0, 0.15)], the mean squared error is the average of the sum of the squares of the differences between the two sets of coordinate points). If the mean squared error exceeds a certain threshold (such as 0.1), the quality is considered poor, and the parameters of the spatiotemporal graph neural network or the trajectory visualization processing network need to be adjusted.

[0194] 2) The significance of collaborative debugging

[0195] This collaborative debugging mechanism helps ensure that the visualization conversion strategy accurately transforms the fully connected quantitative knowledge of the target intelligent electric snowmobile's trajectory into a suitable visualization. By continuously adjusting the parameters of the two networks, the visual feature mapping characteristics are better aligned with the fully connected quantitative knowledge of the trajectory, thereby improving the accuracy and effectiveness of the visualization.

[0196] 2. Obtaining the 3D mixed-model visual annotation information set of the target visualization conversion strategy

[0197] (1) Track visual positioning annotation information

[0198] Track visual positioning annotation information is primarily used to determine the positioning of the snowmobile's trajectory in three-dimensional space within the target visualization transformation strategy. For example, in a 3D scene, the starting point of the track is (0, 0, 0) meters, and the ending point is (10, 5, 2) meters. (The coordinates here are based on a specific 3D coordinate system, with the x-axis representing the forward direction, the y-axis representing the horizontal direction, and the z-axis representing the vertical direction.) This coordinate information can be obtained in various ways, such as by combining track point information from LiDAR point cloud data with a positioning algorithm.

[0199] 2. Visual element annotation information at multiple attention levels

[0200] 1) Acquisition of annotation information based on 3D visual conversion task information

[0201] First, sample the 3D visual transformation tasks included in the target visualization transformation strategy. For example, for Strategy B (suitable for 3D trajectory display in complex terrain and focusing on environmental interaction), the 3D visual transformation tasks may include building a 3D scene, drawing trajectories, and annotating environmental elements (such as snowdrifts and trees). When sampling tasks, select tasks based on a certain probability (e.g., a 0.33 probability of each task being sampled).

[0202] For example, the two tasks of constructing a 3D scene and drawing a trajectory are sampled, and the visual element annotation information at the 3D attention level is determined based on the 3D visual conversion task events obtained by task sampling. For the task of constructing a 3D scene, the visual element annotation information may include the range of the scene (such as 20 meters in front, back, left, and right, and 10 meters in the top and bottom, centered on the snowmobile), the resolution of the scene (such as each cubic meter is divided into 100 small cube units for building a scene model), etc. For the task of drawing a trajectory, the visual element annotation information may include the color of the trajectory (such as blue), thickness (such as 1 mm at 0-5 m / s and 2 mm at 5-10 m / s according to speed), etc.

[0203] 2) Acquisition of annotation information based on vehicle-mounted posture conversion task mining

[0204] The 3D visual transformation task events obtained from task sampling are used to mine vehicle-mounted posture transformation tasks. For example, in the 3D scene construction task, vehicle-mounted posture transformation task events may involve the impact of the snowmobile's posture (such as tilt angle and azimuth angle) at different positions on the 3D scene construction. For example, at a certain location, the snowmobile's tilt angle is 5 degrees (obtained by the vehicle-mounted posture sensor). This tilt angle affects the coordinate transformation of the lidar point cloud data in the 3D scene.

[0205] After receiving the vehicle posture transition task event, the visual element annotation information for the vehicle posture attention layer is determined based on the vehicle posture transition task event and the visual vehicle status data of the target visualization transition strategy (such as the snowmobile's speed and steering angle). For example, when the snowmobile's speed is 3 meters per second and the steering angle is 10 degrees, the visual element annotation information for the vehicle posture attention layer may include displaying the snowmobile's posture in a specific manner in the 3D scene (such as by marking the lean direction and steering direction on the snowmobile model).

[0206] 3) Determine the annotation information of visual elements at multiple attention levels

[0207] Based on the visual element annotation information at the 3D attention level and the visual element annotation information at the vehicle posture attention level, the visual element annotation information at multiple attention levels of the target visualization conversion strategy is determined. For example, the range and resolution of the 3D scene construction is combined with the tilt angle and steering direction of the vehicle posture to form a complete visual element annotation information set, which can be used for subsequent attention annotation feature generation.

[0208] 3. Generate Cross-Attention Annotation Features

[0209] (1) Generating attention annotation features at each attention level

[0210] Attention annotation features at multiple attention levels are generated based on the annotation information of each visual element and the visual positioning annotation information of the trajectory. For example, for the coordinates of the starting point and end point in the visual positioning annotation information of the trajectory, and the scene range in the visual element annotation information at the 3D attention level, a distance metric-based algorithm (such as the Euclidean distance algorithm) is used to calculate the relationship between the trajectory and the scene range, and this relationship is quantified into a vector. This vector is part of the attention annotation feature at the 3D attention level. For the vehicle posture attention level, a similar algorithm is used to generate attention annotation features based on the relative relationship between the vehicle posture and the trajectory (such as the influence of the posture angle on the trajectory direction).

[0211] (2) Fully connected processing to obtain cross-attention annotation features

[0212] The attention annotation features from multiple attention levels are fully connected. For example, the attention annotation feature from the 3D attention level is a 16-dimensional vector, and the attention annotation feature from the vehicle posture attention level is a 12-dimensional vector. This fully connected processing can be implemented using a fully connected layer, where the weight matrix can be initialized to random values (e.g., randomly drawn from a normal distribution with mean 0 and standard deviation 0.01). After calculations in the fully connected layer, a cross-attention annotation feature is obtained. This feature integrates information from different attention levels and can more comprehensively describe the various element relationships under the target visualization transformation strategy.

[0213] 4. Obtaining Visual Element Mapping Features through Spatiotemporal Graph Neural Network

[0214] (1) Obtaining cross-attention residual annotation features through the residual processing subnet

[0215] 1) Residual processing subnet structure and principle

[0216] The residual processing subnet of the spatiotemporal graph neural network is used to process cross-attention annotation features. The residual processing subnet can adopt a residual block structure similar to the ResNet architecture. For example, a residual block contains two 3*3 convolutional layers and a shortcut connection.

[0217] When the cross-attention annotation features are input to the residual processing subnet, they first pass through the first convolutional layer, where the convolution kernel slides over the feature map to perform a convolution operation and extract features. For example, the convolution kernel stride is 1, and the padding method is uniform padding, which maintains the size of the feature map. Batch normalization then normalizes the data to a distribution with mean 0 and variance 1, and then a ReLU activation function introduces nonlinearity. The second convolutional layer then performs further feature extraction. Finally, a shortcut connection is used to add the input and convolutional layer outputs to obtain the cross-attention residual annotation features.

[0218] 2) The significance of cross-attention residual annotation features

[0219] This feature represents the remaining information or difference information after the cross-attention annotation feature has been processed. It can mine deeper feature relationships and provide more meaningful input for subsequent feature mapping.

[0220] (2) Feature Mapping via Bidirectional Long Short-Term Memory Subnetwork

[0221] 1) Characteristics of Bi-LSTM

[0222] The bidirectional long short-term memory subnetwork of the spatiotemporal graph neural network performs feature mapping on the cross-attention residual annotation features. The Bi-LSTM consists of a forward LSTM and a backward LSTM. The LSTM unit has an input gate, a forget gate, and an output gate. The input gate determines what new information can enter the cell state, the forget gate determines what information in the cell state needs to be forgotten, and the output gate controls what information in the cell state can be output.

[0223] For sequential data such as features with cross-attention residual annotations (for example, if the features are arranged in a certain order to form a time series), the forward LSTM processes the data from the beginning to the end of the sequence in chronological order, while the backward LSTM processes the data from the end to the beginning of the sequence. This bidirectional processing approach enables the Bi-LSTM to better capture long-term dependencies in sequential data.

[0224] 2) Obtain visual element mapping features

[0225] After processing by the Bi-LSTM subnetwork, the visual element mapping features of the target visualization conversion strategy are obtained. This visual element mapping feature accurately reflects the adaptive relationship between the target visualization conversion strategy and the fully connected quantitative knowledge of the target intelligent electric snowmobile's trajectory, thus ensuring high-quality trajectory visualization.

[0226] This design, firstly, by collaboratively debugging the spatiotemporal graph neural network and the trajectory visualization processing network, ensures the adaptation of the visual element mapping features to the fully connected quantitative knowledge of the trajectory, thereby improving the accuracy of the visualization conversion. Secondly, the process of obtaining the three-dimensional mixed-mode visual annotation information set and generating the cross-attention annotation features comprehensively considers multiple factors such as trajectory positioning and visual elements at different attention levels, so that the visualization display can more comprehensively reflect the operating status of the snowmobile and its relationship with the environment. Furthermore, the residual processing subnet and bidirectional long short-term memory subnet in the spatiotemporal graph neural network are used to obtain the visual element mapping features, which can mine deep feature relationships, improve the quality and effectiveness of the visualization display, and provide a comprehensive, accurate and efficient technical solution for the visualization of the trajectory of intelligent electric snowmobiles.

[0227] On the basis of the above content, the method further includes:

[0228] (1) The visualization conversion strategy management server transfers the visualization conversion strategy generated on the edge side and the visualization vehicle status data of the visualization conversion strategy to the visualization conversion strategy pool;

[0229] (2) performing a policy feasibility evaluation by instructing the policy feasibility evaluation server, wherein the policy feasibility evaluation includes any one or two of the following: instructing the policy feasibility evaluation server to perform a visual conversion feasibility evaluation on the visualization conversion strategy, instructing the policy feasibility evaluation server to perform a visual conversion feasibility evaluation on the visualization vehicle status data of the visualization conversion strategy, and instructing the policy optimization server to optimize the visualization conversion strategy.

[0230] Among them, the strategy feasibility assessment server obtains the visualization conversion strategy and / or the visualization vehicle status data of the visualization conversion strategy through the visualization conversion strategy management server; the strategy optimization server obtains the visualization conversion strategy and / or the visualization vehicle status data of the visualization conversion strategy through the visualization conversion strategy management server.

[0231] In the next step, the instruction strategy optimization server optimizes the visualization conversion strategy, including: generating visual element mapping features of each reported visualization conversion strategy through a spatiotemporal graph neural network; performing adaptability analysis based on the visual element mapping features of each reported visualization conversion strategy to obtain adaptation analysis information, wherein the adaptation analysis information represents the correlation between any two visualization conversion strategies in each reported visualization conversion strategy; and merging and optimizing the reported visualization conversion strategies based on the correlation between any two visualization conversion strategies in each reported visualization conversion strategy.

[0232] Among them, when generating the visual element mapping features of each reported visualization conversion strategy through the spatiotemporal graph neural network, the input information of the spatiotemporal graph neural network includes: the trajectory visualization connection features of each reported visualization conversion strategy and the 3D visual conversion task events obtained by task sampling of each reported visualization conversion strategy, and the trajectory visualization connection features include the visualization vehicle status data identified from the visualization conversion strategy pool and the vehicle posture conversion task events identified from the 3D visual conversion task events obtained from the task sampling.

[0233] It is understood that the above technical solution involves design ideas related to visual conversion strategy management, evaluation, and optimization. The following is an exemplary introduction to these design ideas.

[0234] 1. Decentralization of Visual Conversion Strategy

[0235] (1) The role of the visual conversion policy management server

[0236] The visualization conversion strategy management server plays a key role in the entire process. It is responsible for delegating visualization conversion strategies generated by the edge side and their associated visualization vehicle status data to the visualization conversion strategy pool. For example, the edge side may generate multiple visualization conversion strategies based on different snowmobile driving environments and requirements. For example, the edge side may generate two visualization conversion strategies, Strategy A and Strategy B. Strategy A is suitable for relatively flat snowy terrain, focusing on displaying the relationship between the snowmobile and surrounding exemplary obstacles (such as a small snowdrift). Its visualization vehicle status data may include information such as the snowmobile's speed range of 0-5 m / s and minimal steering angle variation. Strategy B is suitable for snowy environments with a certain slope, focusing on displaying the relationship between the snowmobile's driving trajectory and the terrain slope. Its visualization vehicle status data includes information such as the snowmobile's speed range of 3-8 m / s and frequent tilt angles. The visualization conversion strategy management server accurately delegates these strategies and their associated visualization vehicle status data to the visualization conversion strategy pool, providing a foundation for subsequent evaluation and optimization.

[0237] 2. Strategic Feasibility Assessment

[0238] (1) Operation of the Strategy Feasibility Assessment Server

[0239] 1) Visual conversion feasibility assessment

[0240] Instruct the policy feasibility evaluation server to perform a visual conversion feasibility evaluation on the visualization conversion policy. This evaluation process may involve multiple aspects. For example, for policy A in the visualization conversion policy pool, evaluate whether it can accurately convert the fully connected quantitative knowledge of the snowmobile's trajectory into a visual representation. A sample data-based evaluation method can be used to select a certain number (e.g., 100) of snowmobile trajectory samples. These samples have known fully connected quantitative knowledge of the trajectory and actual trajectory conditions.

[0241] For each sample, use strategy A to perform a visual transformation, and then calculate the degree of match between the transformed visual trajectory and the actual trajectory. The matching degree can be calculated using various algorithms, such as calculating the mean absolute error (MAE) between trajectory coordinate points. For example, if the coordinate points of the actual trajectory are [(0, 0, 0), (1, 0, 0), (2, 0, 0.1)], and the coordinate points of the trajectory after the visual transformation using strategy A are [(0, 0, 0), (1.1, 0, 0), (2.2, 0, 0.15)], then the mean absolute error is:

[0242] (frac{|0-0|+|1-1.1|+|2-2.2|+|0-0|+|0-0|+|0.1-0.15|}{6}). If the mean absolute error exceeds a certain threshold (such as 0.1), strategy A is considered to have low feasibility in visual transformation; otherwise, it is considered to have high feasibility.

[0243] 2) Feasibility evaluation of visual conversion of visualized vehicle status data

[0244] The strategy feasibility assessment server is instructed to perform a feasibility assessment on the visual transformation of the visualized vehicle state data of the visualization transformation strategy. Taking the visualized vehicle state data of Strategy B as an example, which contains information such as the speed range and bank angle of the snowmobile, the feasibility of this data in the visualization transformation process is assessed. For example, the visualization of the speed range is checked to see if it is reasonable and accurately reflects the snowmobile's driving state in the visualization. This assessment can be performed by comparing the motion characteristics of the snowmobile model in the visualization at different speeds (such as the visual perception of movement speed) with the actual speed. For the visualization of bank angles, the bank angle of the snowmobile in the 3D visualization is checked to see if it matches the actual bank angle data. This can be done by calculating the error between the bank angle in the visualization and the actual bank angle (e.g., using cosine similarity to measure the angle error). If the error exceeds a certain limit (e.g., 0.2), the visual transformation of the visualized vehicle state data is considered problematic and the feasibility is low.

[0245] 3) Strategy optimization server operation

[0246] When a problem is found in the policy feasibility assessment, the policy optimization server is instructed to optimize the visualization conversion policy. For example, when policy A performs poorly in the visualization conversion feasibility assessment, the policy optimization server intervenes.

[0247] 3. Optimization of Visual Conversion Strategy

[0248] 1. Generate visual element mapping features

[0249] 1) Input information of spatiotemporal graph neural network

[0250] When generating the visual element mapping features of each reported visualization conversion strategy (such as strategy A and strategy B) through the spatiotemporal graph neural network, the input information of the spatiotemporal graph neural network includes the trajectory visualization connection features of each reported visualization conversion strategy and the 3D visual conversion task events obtained by task sampling for each reported visualization conversion strategy.

[0251] The trajectory visualization connection feature includes the visual vehicle state data identified from the visualization transition strategy pool and the vehicle posture transition task events identified from the 3D visual transition task events obtained from task sampling. Taking strategy A as an example, if the speed of the visual vehicle state data identified from the visualization transition strategy pool is 3 meters per second, this speed information is included as part of the trajectory visualization connection feature. If the sampled 3D visual transition task event is to construct a 3D scene around a snowmobile, the identified vehicle posture transition task event may be that the snowmobile's steering angle at the current speed is 5 degrees. This steering angle information is also included in the trajectory visualization connection feature.

[0252] 2) The generation process of visual element mapping features

[0253] The spatiotemporal graph neural network processes this input information to generate visual feature mapping features. For example, the spatiotemporal graph neural network adopts a multi-layer structure. The first layer is a convolutional layer with a convolution kernel size of 3*3, a stride of 1, and 16 channels. The input trajectory visualization connection features and 3D visual transformation task events are processed by the convolutional layer to extract preliminary features. Downsampling is then performed through a pooling layer (such as a max pooling layer with a pooling window size of 2*2) to reduce the data volume while retaining key features. Further feature fusion and abstraction are then performed through a fully connected layer (for example, a fully connected layer with 128 neurons), ultimately resulting in the visual feature mapping features. For Strategy A and Strategy B, separate visual feature mapping features are obtained. For example, the visual feature mapping features of Strategy A may be a 64-dimensional vector, while the visual feature mapping features of Strategy B may be an 80-dimensional vector.

[0254] (2) Adaptability Analysis and Merger Optimization

[0255] 1) Adaptability analysis

[0256] Compatibility analysis is performed based on the visual element mapping features of each reported visualization conversion strategy to generate adaptation analysis information. This adaptation analysis information represents the correlation between any two visualization conversion strategies within the reported strategies. For example, the correlation between the visual element mapping features of strategies A and B is calculated. The cosine similarity algorithm can be used to calculate the similarity between two vectors (i.e., the visual element mapping feature vectors of strategies A and B). For example, if the visual element mapping feature vector of strategy A is (a=[0.1, 0.2, -0.3, cdots, 0.05]) and the visual element mapping feature vector of strategy B is (b=[0.2, 0.3, -0.2, cdots, 0.1]), the cosine similarity between them is (frac{acdotb}{|a|*|b|}). The correlation between the two is judged based on this similarity value. If the similarity is high (such as greater than 0.8), it means the correlation is strong; if the similarity is low (such as less than 0.3), it means the correlation is weak.

[0257] 2) Merge optimization

[0258] The reported visualization conversion strategies are merged and optimized based on the correlation between any two of them. If the correlation between strategies A and B is strong, it means they are similar in some aspects and can be merged and optimized. For example, the 3D scene construction parts of strategies A and B are merged, and the advantages of each are taken. If strategy A is more accurate in constructing the 3D model of the exemplary obstacle, while strategy B is better at showing the relationship between the snowmobile and the terrain, then the advantages of these two parts are integrated in the merged and optimized strategy to obtain a more complete visualization conversion strategy, thereby improving the effect and efficiency of visualization conversion.

[0259] In this way, first, by delegating policies and data through the visualization conversion policy management server, an effective connection between the edge side and the visualization conversion policy pool is achieved, ensuring the effective integration of data and policies. Secondly, the policy feasibility evaluation server's evaluation of the visualization conversion policy and its visualization vehicle status data can comprehensively and accurately identify potential problems and improve the quality of visualization conversion. Furthermore, the policy optimization server performs adaptability analysis and merge optimization based on the visual element mapping characteristics of the spatiotemporal graph neural network. It can fully utilize the advantages of different policies, reduce redundancy, and improve the effectiveness and adaptability of visualization conversion policies, thereby providing higher quality and more efficient policy support for the visualization of the trajectory of the smart electric snowmobile.

[0260] In some scalable examples, the running trajectory visualization processing network includes a vector migration subnet, which is used to perform vector space migration on original mixed-mode sensor monitoring information, wherein the original mixed-mode sensor monitoring information includes any one or more of: vehicle-mounted environmental perception sensor monitoring information, first laser radar point cloud sensor monitoring information, and mixed-mode sensor monitoring information in an associated mixed-mode sensor monitoring information set. When the original mixed-mode sensor monitoring information belongs to mixed-mode sensor monitoring information with a mutually exclusive relationship, the original mixed-mode sensor monitoring information is vector-space migrated through the vector migration subnet; when the original mixed-mode sensor monitoring information belongs to mixed-mode sensor monitoring information with an affiliation relationship, the original mixed-mode sensor monitoring information is vector-space migrated through the vector migration subnet, and the migration features obtained after the vector space migration are subjected to feature pyramid conversion, and the size of the migration features obtained after the feature pyramid conversion is a preset size.

[0261] In this embodiment, the vector transfer subnet in the trajectory visualization processing network is used to perform vector space transfer on the original mixed-mode sensor monitoring information. When the original mixed-mode sensor monitoring information is mutually exclusive, vector space transfer is performed directly through the vector transfer subnet. For example, camera image data and lidar point cloud data in vehicle-mounted environmental perception sensor monitoring information can be considered mutually exclusive (the camera provides visual images, while the lidar provides point cloud information, and the two have fundamentally different information acquisition methods and content). For camera image data (e.g., image resolution of 1920*1080 pixels, stored in three RGB channels, resulting in large data volume and complex features), the vector transfer subnet uses a specific algorithm (e.g., an algorithm based on a multi-layer perceptron (MLP)) to perform vector space transfer. For example, the MLP has three hidden layers with 1024, 512, and 256 neurons, respectively. The feature vector (the expanded high-dimensional vector) of the image data is input into the MLP. Through linear transformations and activation functions (e.g., ReLU activation functions) at each layer, the image data is transferred from the original feature space to a new vector space to extract features that are more conducive to subsequent processing.

[0262] When the original mixed-mode sensor monitoring information belongs to a related mixed-mode sensor monitoring information, vector space migration is similarly performed using the vector migration subnet. The resulting migration features are then transformed into a feature pyramid. For example, sensor data of the same type from different related vehicles in a related mixed-mode sensor monitoring information set (e.g., lidar point cloud data from multiple related vehicles) can be considered to represent a related relationship. Taking the lidar point cloud data from a specific related vehicle as an example, the migration features are obtained using the vector migration subnet (using an algorithm similar to the MLP-based algorithm described above or another suitable algorithm). For example, the migration features are a 512-dimensional vector.

[0263] The migration features are then transformed into a feature pyramid. The size of the migration features obtained after the feature pyramid transformation is a preset size. For example, if the preset size is 128 dimensions, the feature pyramid transformation process may use a hierarchical feature extraction and compression algorithm. For example, the 512-dimensional migration features are first divided into multiple sub-feature sets according to a certain rule (such as based on feature importance or spatial distribution). Each sub-feature set is then subjected to dimensionality reduction (principal component analysis (PCA) can be used to retain a certain proportion of principal components, such as 80%). Finally, the processed sub-feature sets are recombined into 128-dimensional migration features to meet the preset size requirement. This processing method can reduce data dimensionality while maintaining key information, improving the efficiency of subsequent processing.

[0264] As can be seen, for mutually exclusive mixed-mode sensor monitoring information, vector space migration within the vector migration subnetwork helps extract their respective effective features. For example, camera images and lidar point cloud data, after processing, can be better used for subsequent trajectory visualization-related processing. For mixed-mode sensor monitoring information with affiliation, vector space migration is combined with feature pyramid transformation to reduce data dimensionality while retaining key information, outputting migrated features at a preset size. This not only improves data processing efficiency but also enables subsequent operations based on these features (such as processing by other subnetworks in the trajectory visualization processing network) to be performed more efficiently and accurately, contributing to the overall improvement in the performance of the intelligent electric snowmobile trajectory visualization display.

[0265] In an expandable embodiment, the associated mixed-mode sensor monitoring information set includes X pieces of associated mixed-mode sensor monitoring information corresponding to X associated intelligent electric snowmobiles. Obtaining the associated mixed-mode sensor monitoring information set of an associated intelligent electric snowmobile of the target intelligent electric snowmobile includes: obtaining second laser radar point cloud sensor monitoring information of any associated intelligent electric snowmobile among the X associated intelligent electric snowmobiles, and vehicle-borne cross-sensing monitoring information between the any associated intelligent electric snowmobile and the target intelligent electric snowmobile, the vehicle-borne cross-sensing monitoring information including path intersection area features and signal intersection area features; and determining the second laser radar point cloud sensor monitoring information and the vehicle-borne cross-sensing monitoring information as the associated mixed-mode sensor monitoring information corresponding to the any associated intelligent electric snowmobile.

[0266] Specifically, when constructing the associated mixed-mode sensor monitoring information set, for each of the X associated smart electric snowmobiles, the second lidar point cloud sensor monitoring information is first obtained. Lidar point cloud data can provide three-dimensional structural information about the surrounding environment. For example, the lidar scans at a certain scanning frequency (e.g., 10 Hz), and each scan acquires point cloud data within a certain range (e.g., 20 meters). This point cloud data represents the location of surrounding objects in the form of three-dimensional coordinates. By analyzing large amounts of point cloud data, detailed information about the terrain and obstacles surrounding the associated smart electric snowmobile can be obtained.

[0267] At the same time, vehicle-mounted cross-sensing monitoring information between any associated smart electric snowmobile and the target smart electric snowmobile is obtained, which includes path intersection area characteristics and signal intersection area characteristics.

[0268] The path intersection feature is used to determine whether the paths of the two vehicles intersect. For example, using the trajectory planning data of the two vehicles, if the planned trajectory of the target smart electric snowmobile is from the coordinate point (0, 0, 0) to (10, 5, 0) (the coordinates here are based on a global coordinate system, with the x-axis representing the forward direction, the y-axis representing the lateral direction, and the z-axis representing the vertical direction), and the planned trajectory of the associated smart electric snowmobile is from the coordinate point (5, -3, 0) to (15, 2, 0), by calculating the intersection point of the two trajectories (using the method for calculating the intersection of lines in analytic geometry), if an intersection point exists, the path intersection feature is determined.

[0269] The signal intersection area feature considers the regional characteristics of signal interaction between two vehicles. For example, when wireless communication (such as Wi-Fi communication, with a frequency band of 2.4 GHz) is used between the two vehicles, the effective signal coverage range is determined based on signal propagation models (such as the Friis transmission formula, which calculates parameters such as transmit power, receive power, and antenna gain). If the signal coverage areas of the two vehicles overlap, this overlap is considered a signal intersection area feature.

[0270] The acquired second LiDAR point cloud sensor monitoring information and the onboard cross-sensor monitoring information are determined as the associated mixed-mode sensor monitoring information corresponding to any associated smart electric snowmobile. This allows for the construction of associated mixed-mode sensor monitoring information for each associated smart electric snowmobile, including its own LiDAR point cloud information and cross-sensor information with the target smart electric snowmobile. This information is then used for subsequent operations such as trajectory visualization.

[0271] This allows for comprehensive construction of correlated mixed-mode sensor monitoring information. By acquiring correlated LiDAR point cloud information from the intelligent electric snowmobile, the perception of the surrounding environment can be enriched. The path intersection area features within the onboard intersection sensor monitoring information are used to accurately determine the likelihood of two vehicles encountering or interacting through trajectory calculation, which is crucial for practical trajectory planning and collision avoidance. The signal intersection area features, based on signal propagation models, determine signal interaction areas, facilitating communication analysis and coordinating interactions between the two vehicles. This combined information provides a more comprehensive and accurate data foundation for intelligent electric snowmobile trajectory visualization and other applications.

[0272] Applying the above-mentioned technical solution of the embodiment of the present invention, by first acquiring a mixed-mode sensor monitoring information set for the target intelligent electric snowmobile and its associated intelligent electric snowmobiles, including multi-source information such as onboard environmental perception sensor monitoring information and LiDAR point cloud sensor monitoring information, it can comprehensively capture the surrounding environmental conditions and the snowmobile's own status during operation. This integration of multi-source information lays a solid data foundation for subsequent accurate trajectory visualization.

[0273] The trajectory visualization processing network plays a key role in information processing. The vehicle-mounted environmental perception mining subnet and the lidar point cloud mining subnet mine corresponding sensor monitoring information, respectively, to obtain quantitative knowledge about the vehicle-mounted environmental perception and lidar point cloud. This helps transform complex raw sensor data into more representative and analyzable quantitative knowledge, thereby extracting key environmental and target features. For example, the quantitative knowledge about the vehicle-mounted environmental perception can accurately reflect information such as object types and road conditions around the snowmobile, while the quantitative knowledge about the lidar point cloud can precisely describe three-dimensional spatial information such as the terrain and obstacle locations around the snowmobile.

[0274] The knowledge feature interaction subnetwork further extracts sensor-monitoring interaction knowledge features. This process comprehensively considers the target vehicle's onboard environmental perception information and the mixed-mode sensor monitoring information of associated vehicles, capturing the interactions between vehicles and between vehicles and their environment. For example, in scenarios involving multi-vehicle coordinated driving or snowmobiles experiencing complex interactions with their surroundings (such as when navigating narrow passages or approaching other vehicles), this interactive knowledge feature can more comprehensively reflect the overall situation, improving the accuracy and practicality of trajectory visualization.

[0275] The fully connected subnetwork performs fully connected processing on multiple types of quantitative knowledge to generate fully connected quantitative knowledge of the trajectory. This processing method integrates knowledge from different sources and types, fully exploring the inherent connections between them. The resulting fully connected quantitative knowledge of the trajectory comprehensively reflects various factors related to the snowmobile's trajectory, including environmental factors, the vehicle's own state, and interactions between vehicles.

[0276] Finally, based on the fully connected quantitative knowledge of the trajectory, the target visualization conversion strategy is obtained from the visualization conversion strategy pool. The most appropriate visualization method can be selected to display the snowmobile's trajectory according to the actual situation. This ensures the targeted and effective visualization. For example, it can provide the most appropriate visualization for different driving scenarios (such as open snowfields and complex mountain snowfields) and different task requirements (such as safety monitoring and path planning), thus providing strong support for the operation management, monitoring, and decision-making of intelligent electric snowmobiles.

[0277] Furthermore, Figure 2 Schematic diagram of a visual display system 200 provided in an embodiment of the present invention. Figure 2 The visual display system 200 shown includes a processor 210. The processor 210 can call and run a computer program from a memory to implement the method in the embodiment of the present invention.

[0278] Alternatively, as Figure 2As shown, the visual display system 200 may further include a memory 230. The processor 210 may call and run a computer program from the memory 230 to implement the method in the embodiment of the present invention.

[0279] The memory 230 may be a separate device independent of the processor 210, or may be integrated into the processor 210. Figure 2 As shown, the visualization system 200 may also include a transceiver 220. The processor 210 may control the transceiver 220 to interact with other devices. Specifically, it may send information or data to other devices or receive information or data sent by other devices. Optionally, the visualization system 200 may implement the corresponding processes of the storage engine, components within the storage engine (such as a processing module), or devices equipped with the storage engine in the various methods of the embodiments of the present invention. For the sake of brevity, these processes are not further described here. It should be understood that the processor in the embodiments of the present invention may be an integrated circuit chip with signal processing capabilities. It should be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, any suitable type of memory. Based on the above, a readable storage medium is provided, storing a program or instructions that, when executed by a processor, implements the steps of the above-described method.

[0280] The above describes an embodiment of the present invention in conjunction with the accompanying drawings, but the embodiment of the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the embodiment of the present invention, ordinary technicians in this field can also make many forms without departing from the purpose of the embodiment of the present invention and the scope of protection of the embodiment of the present invention, all of which are protected by the embodiment of the present invention.

Claims

1. A method for visualizing the trajectory of an autonomous driving vehicle, characterized in that: include: Obtaining a target mixed-mode sensor monitoring information set of a target intelligent electric snowmobile to be subjected to a visual display of its running trajectory and an associated mixed-mode sensor monitoring information set of an associated intelligent electric snowmobile of the target intelligent electric snowmobile, wherein the target mixed-mode sensor monitoring information set includes: on-board environment perception sensor monitoring information and first laser radar point cloud sensor monitoring information; Obtaining vehicle environment perception quantitative knowledge of the vehicle environment perception sensor monitoring information by running the vehicle environment perception mining subnet of the trajectory visualization processing network, and obtaining lidar point cloud quantitative knowledge of the first lidar point cloud sensor monitoring information by running the lidar point cloud mining subnet of the trajectory visualization processing network; Obtaining sensor monitoring interaction knowledge features corresponding to the vehicle environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set by running the knowledge feature interaction subnet of the trajectory visualization processing network; The fully connected subnet of the running trajectory visualization processing network performs full-connection processing on the vehicle environment perception quantitative knowledge, the lidar point cloud quantitative knowledge, and the sensor monitoring interaction knowledge features to obtain the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile; Based on the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile, a target visualization conversion strategy is obtained from a visualization conversion strategy pool; the target visualization conversion strategy is used to realize the visualization display of the running trajectory of the target intelligent electric snowmobile; The visualization conversion strategy pool includes multiple visualization conversion strategies, each of which generates corresponding visual element mapping features through a spatiotemporal graph neural network, and the visual element mapping features of the target visualization conversion strategy obtained from the visualization conversion strategy pool are adapted to the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile; The spatiotemporal graph neural network and the running trajectory visualization processing network are obtained through collaborative debugging. The error variables determined by the debugging error indicators used in the collaborative debugging link include: the fully connected quantitative knowledge of the running trajectory of the intelligent electric snowmobile sample obtained by the running trajectory visualization processing network during debugging, the visual element mapping characteristics of the visualization conversion strategy sample obtained by the spatiotemporal graph neural network during debugging, and the debugging guidance information determined based on the quality evaluation of the visualization conversion strategy sample based on the intelligent electric snowmobile sample.

2. The method according to claim 1, wherein The method further comprises: Obtaining a three-dimensional mixed-mode visual annotation information set of a target visualization conversion strategy, wherein the three-dimensional mixed-mode visual annotation information set includes: trajectory visual positioning annotation information and visual element annotation information of multiple attention levels; Generating attention annotation features at multiple attention levels based on the annotation information of each visual element and the trajectory visual positioning annotation information, and performing full connection processing on the attention annotation features at the multiple attention levels to obtain cross-attention annotation features; Obtaining a cross-attention residual annotation feature corresponding to the cross-attention annotation feature through the residual processing subnet of the spatiotemporal graph neural network; The cross-attention residual annotation features are feature mapped by the bidirectional long short-term memory subnetwork of the spatiotemporal graph neural network to obtain the visual element mapping features of the target visualization conversion strategy; the target visualization conversion strategy is any one of the multiple visualization conversion strategies.

3. The method according to claim 2, wherein Obtain visual element annotation information at multiple attention levels of the target visualization conversion strategy, including: Performing task sampling on the 3D visual conversion task information included in the target visualization conversion strategy, and determining visual element annotation information at the 3D attention level according to the 3D visual conversion task events obtained from the task sampling; Performing vehicle posture conversion task mining on the 3D visual conversion task events obtained by the task sampling to obtain vehicle posture conversion task events, and determining visual element annotation information at the vehicle posture attention level based on the vehicle posture conversion task events and the visualized vehicle state data of the target visualization conversion strategy; Based on the visual element annotation information at the 3D attention level and the visual element annotation information at the vehicle posture attention level, the visual element annotation information at multiple attention levels of the target visualization conversion strategy is determined.

4. The method according to any one of claims 1 to 3, wherein The method further comprises: The visualization conversion policy management server transfers the visualization conversion policy generated on the edge side and the visualization vehicle status data of the visualization conversion policy to the visualization conversion policy pool; Performing a policy feasibility evaluation by instructing a policy feasibility evaluation server, wherein the performed policy feasibility evaluation includes any one or two of the following: instructing the policy feasibility evaluation server to perform a visual conversion feasibility evaluation on the visualization conversion strategy, instructing the policy feasibility evaluation server to perform a visual conversion feasibility evaluation on the visualized vehicle status data of the visualization conversion strategy, and instructing the policy optimization server to optimize the visualization conversion strategy; Among them, the strategy feasibility assessment server obtains the visualization conversion strategy and / or the visualization vehicle status data of the visualization conversion strategy through the visualization conversion strategy management server; the strategy optimization server obtains the visualization conversion strategy and / or the visualization vehicle status data of the visualization conversion strategy through the visualization conversion strategy management server.

5. The method according to claim 4, wherein The instructing strategy optimization server to optimize the visualization conversion strategy includes: Generate visual element mapping features of each reported visualization transformation strategy through spatiotemporal graph neural network; Performing adaptability analysis based on visual element mapping features of each reported visualization conversion strategy to obtain adaptability analysis information, wherein the adaptability analysis information indicates the correlation between any two visualization conversion strategies among the reported visualization conversion strategies; Merge and optimize the reported visualization conversion strategies based on the correlation between any two visualization conversion strategies among the reported visualization conversion strategies; Among them, when generating the visual element mapping features of each reported visualization conversion strategy through the spatiotemporal graph neural network, the input information of the spatiotemporal graph neural network includes: the trajectory visualization connection features of each reported visualization conversion strategy and the 3D visual conversion task events obtained by task sampling of each reported visualization conversion strategy, and the trajectory visualization connection features include the visualization vehicle status data identified from the visualization conversion strategy pool and the vehicle posture conversion task events identified from the 3D visual conversion task events obtained from the task sampling.

6. The method according to claim 1, wherein The running trajectory visualization processing network includes a vector transfer subnet, the vehicle environment perception mining subnet includes a bidirectional long short-term memory subnet, and the vehicle environment perception quantitative knowledge of the vehicle environment perception sensor monitoring information obtained by the vehicle environment perception mining subnet of the running trajectory visualization processing network includes: Using the vehicle environment perception mining subnet as input to the vehicle environment perception mining subnet, the vehicle environment perception sensing monitoring information is subjected to vector space migration by the vector migration subnet of the running trajectory visualization processing network; The vehicle environment migration characteristics are quantified through the bidirectional long short-term memory subnet of the vehicle environment perception mining subnet to obtain the vehicle environment perception quantified knowledge of the vehicle environment perception sensor monitoring information.

7. The method according to claim 1, wherein The running trajectory visualization processing network includes a vector migration subnet, and the lidar point cloud quantified knowledge of the first lidar point cloud sensing monitoring information obtained through the lidar point cloud mining subnet of the running trajectory visualization processing network includes: Using multiple lidar point cloud migration features of the first lidar point cloud sensor monitoring information obtained by the vector migration subnet of the running trajectory visualization processing network as input to the lidar point cloud mining subnet of the running trajectory visualization processing network, where the first lidar point cloud sensor monitoring information includes multiple lidar scanning information obtained by sequentially sorting according to scanning cycles; Processing the migration features of multiple lidar point clouds by running the lidar point cloud mining subnet of the trajectory visualization processing network to obtain lidar point cloud quantitative knowledge of the first lidar point cloud sensing monitoring information; The laser radar point cloud mining subnet of the running trajectory visualization processing network obtains the laser radar point cloud quantitative knowledge of the first laser radar point cloud sensing monitoring information based on multiple laser radar scanning information whose scanning cycles meet the step size requirements; Alternatively, the lidar point cloud mining subnet of the running trajectory visualization processing network obtains multiple three-dimensional scene drawing element features corresponding to the multiple lidar point cloud migration features through the residual processing subnet of the lidar point cloud mining subnet, and processes the multiple three-dimensional scene drawing element features through the feature downsampling branch of the lidar point cloud mining subnet to obtain the lidar point cloud quantitative knowledge of the first lidar point cloud sensing monitoring information.

8. The method according to claim 1, wherein The associated mixed-mode sensor monitoring information set includes X associated mixed-mode sensor monitoring information corresponding to X associated intelligent electric snowmobiles, where X is a positive integer; the running trajectory visualization processing network includes a vector migration subnet, and the knowledge feature interaction subnet includes X feature enhancement branches, a residual processing subnet, and a feature downsampling branch; The sensor monitoring interaction knowledge features corresponding to the vehicle-mounted environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set are obtained by running the knowledge feature interaction subnet of the trajectory visualization processing network, including: Using the vehicle environment migration features obtained by performing vector space migration on the vehicle environment perception sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network, and the X associated migration features obtained by performing vector space migration on the X associated mixed-mode sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network, as inputs to the knowledge feature interaction subnet; Processing the vehicle environment migration feature and the X associated migration features through the X feature enhancement branches of the knowledge feature interaction subnet to obtain X associated vehicle environment enhancement features; Obtaining associated residual annotation features corresponding to the X associated vehicle environment enhancement features through the residual processing subnet of the knowledge feature interaction subnet; Processing the associated residual annotation features through the feature downsampling branch of the knowledge feature interaction subnet to obtain the sensor monitoring interaction knowledge features; The processing of the vehicle environment migration feature and the X associated migration features through the X feature enhancement branches of the knowledge feature interaction subnet to obtain X associated vehicle environment enhancement features includes: Obtaining a visual layer overlay feature between the vehicle environment migration feature and a u-th associated migration feature among the X associated migration features, where u is a positive integer not greater than X; Integrating the vehicle environment migration feature, the u-th associated migration feature, and the visual layer overlay feature, and inputting the result of the integration into the weighted processing module of the u-th feature enhancement branch among the X feature enhancement branches to obtain an output of the weighted processing module of the u-th feature enhancement branch; The output of the weighted processing module of the u-th feature enhancement branch is used as the input of the normalization module of the u-th feature enhancement branch, and the normalization module of the u-th feature enhancement branch outputs the u-th associated vehicle environment enhancement feature.

9. A visual display system, characterized in that: The method comprises at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method according to any one of claims 1 to 8.

Images