Visual display method and system for running track of automatic driving vehicle

Abstract

The embodiment of the invention discloses a visual display method and system for the running track of an automatic driving vehicle, and belongs to the technical field of data visualization. The embodiment of the invention provides a comprehensive technical scheme for visually displaying the running track of the intelligent electric snowmobile. The method comprises the following steps: integrating various sensing monitoring information, including vehicle-mounted environment perception sensing monitoring information of a target intelligent electric snowmobile, first laser radar point cloud sensing monitoring information and associated mixed-mode sensing monitoring information associated with the intelligent electric snowmobile; and then deep processing is carried out on the information by using a plurality of subnets in the running track visualization processing network, and finally a proper visualization conversion strategy is selected based on the obtained running track full-connection quantification knowledge. The problems that in the prior art, information is not comprehensive, information processing is not sufficient, and a visualization strategy lacks adaptability are effectively solved.

Description

Technical Field

[0001] The embodiments of the present invention belong to the field of data visualization technology, and specifically relate to a method and system for visualizing the running trajectory of an autonomous driving vehicle. Background Art

[0002] In the field of operation management and monitoring of smart electric snowmobiles, accurately displaying the operation trajectory of the snowmobile is of vital importance to ensuring its safe driving, optimizing path planning, and improving overall operational efficiency.

[0003] Traditional trajectory display technology often has many limitations. In terms of information acquisition, relying only on a single type of sensor data, such as using only on-board cameras or lidar data, it is impossible to fully obtain environmental information around the snowmobile. In the information processing link, there is a lack of effective methods to integrate key information from different types of sensor data. Previous technologies may simply superimpose or process different sensor data separately, failing to fully explore the intrinsic connections between the data. In terms of visualization, traditional technologies lack the ability to adaptively select visualization strategies based on actual conditions. Usually, a fixed visualization method is used, which cannot provide the most appropriate trajectory display for different driving scenarios (such as different snow terrains, different traffic flows, etc.) and task requirements (such as real-time monitoring, post-analysis, etc.). This may lead to poor visualization effects in some cases, and cannot provide effective support for the operation management and monitoring of snowmobiles. Summary of the invention

[0004] The embodiments of the present invention provide a method and system for visually displaying the running trajectory of an autonomous driving vehicle, which can solve or partially solve the technical problems involved in the above-mentioned background technology.

[0005] An embodiment of the present invention provides a method for visualizing the running trajectory of an autonomous driving vehicle, which is applied to a visualization display system. The method includes: obtaining a target mixed-mode sensor monitoring information set of a target intelligent electric snowmobile to be visualized for running trajectory display and an associated mixed-mode sensor monitoring information set of an associated intelligent electric snowmobile of the target intelligent electric snowmobile, wherein the target mixed-mode sensor monitoring information set includes: vehicle-mounted environment perception sensor monitoring information and first laser radar point cloud sensor monitoring information; obtaining vehicle-mounted environment perception quantitative knowledge of the vehicle-mounted environment perception sensor monitoring information through a vehicle-mounted environment perception mining subnet of a running trajectory visualization processing network, and obtaining the first laser radar point cloud sensor monitoring information through a laser radar point cloud mining subnet of the running trajectory visualization processing network. The method comprises the following steps: obtaining the laser radar point cloud quantitative knowledge of the target intelligent electric snowmobile; obtaining the sensor monitoring interactive knowledge features corresponding to the on-board environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set through the knowledge feature interaction subnet of the running trajectory visualization processing network; performing full-connection processing on the on-board environment perception quantitative knowledge, the laser radar point cloud quantitative knowledge and the sensor monitoring interactive knowledge features through the fully-connected subnet of the running trajectory visualization processing network to obtain the fully-connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile; obtaining the target visualization conversion strategy from the visualization conversion strategy pool based on the fully-connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile; and the target visualization conversion strategy is used to realize the visualization display of the running trajectory of the target intelligent electric snowmobile.

[0006] An embodiment of the present invention provides a visualization display system, comprising at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the above method.

[0007] An embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the above method are implemented.

[0008] In view of the above technical problems, the embodiment of the present invention proposes a comprehensive intelligent electric snowmobile running track visualization display technology solution. By integrating multiple sensor monitoring information, including the on-board environment perception sensor monitoring information of the target intelligent electric snowmobile and the first laser radar point cloud sensor monitoring information and the associated mixed-mode sensor monitoring information of the associated intelligent electric snowmobile, and then using multiple subnets in the running track visualization processing network to deeply process these information, and finally selecting a suitable visualization conversion strategy based on the obtained running track fully connected quantitative knowledge, the problems of incomplete information, insufficient information processing and lack of adaptability of visualization strategy existing in traditional technologies are effectively solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flowchart of a method for visualizing the running trajectory of an autonomous driving vehicle provided in an embodiment of the present invention.

[0010] Figure 2 A schematic diagram of the structure of a visual display system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0011] Figure 1 A method for visualizing the running trajectory of an autonomous driving vehicle is shown, which is applied to a visualization display system. The method includes the following steps 110 to 150.

[0012] The embodiment of the present invention is applied around the visualization display technology of the running trajectory of the intelligent electric snowmobile. By obtaining the sensor monitoring information set of the target intelligent electric snowmobile and its associated intelligent electric snowmobiles, the information is processed by using multiple subnets in the running trajectory visualization processing network, and finally the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile is obtained, and then the target visualization conversion strategy is obtained from the visualization conversion strategy pool to realize the visualization display of the running trajectory.

[0013] Step 110: Obtain a target mixed-mode sensor monitoring information set of a target intelligent electric snowmobile whose running trajectory is to be visualized and an associated mixed-mode sensor monitoring information set of an associated intelligent electric snowmobile of the target intelligent electric snowmobile.

[0014] Among them, the target mixed-mode sensor monitoring information set includes: vehicle-mounted environment perception sensor monitoring information and first laser radar point cloud sensor monitoring information.

[0015] In this step, the target mixed-mode sensor monitoring information set of the target intelligent electric snowmobile must first be obtained. This information set contains the on-board environmental perception sensor monitoring information and the first lidar point cloud sensor monitoring information. The on-board environmental perception sensor monitoring information can cover various types of sensor data, such as image data collected by the camera, which is used to identify objects around the snowmobile (such as other vehicles, pedestrians, obstacles, etc.) and road conditions (such as the flatness of the snow, whether there are gullies, etc.). For example, the image data resolution in the on-board environmental perception sensor monitoring information is 1920*1080 pixels, which can provide a clearer image of the surrounding environment for subsequent analysis.

[0016] The first laser radar point cloud sensing monitoring information includes multiple laser radar scanning information obtained in sequence according to the scanning cycle. The laser radar constructs point cloud data of the surrounding environment by emitting laser beams and receiving reflected light. For example, the scanning frequency of the laser radar is 10Hz, and each scan can obtain the three-dimensional coordinate information of thousands of points in the surrounding environment. These point cloud data can accurately reflect the terrain undulations around the snowmobile, the distance and shape of obstacles, and other information.

[0017] At the same time, it is also necessary to obtain the associated mixed-mode sensor monitoring information set of the target intelligent electric snowmobile. For example, the number of associated intelligent electric snowmobiles X=3 (X is a positive integer), that is, there are 3 associated intelligent electric snowmobiles. The associated mixed-mode sensor monitoring information corresponding to each associated snowmobile includes the respective vehicle-mounted environment perception sensor monitoring information and laser radar point cloud sensor monitoring information, etc. This information is of great significance for the comprehensive analysis of the running trajectory of the target snowmobile.

[0018] Step 120, obtain the vehicle environment perception quantitative knowledge of the vehicle environment perception sensor monitoring information by running the vehicle environment perception mining subnet of the trajectory visualization processing network, and obtain the lidar point cloud quantitative knowledge of the first lidar point cloud sensor monitoring information by running the lidar point cloud mining subnet of the trajectory visualization processing network.

[0019] In an embodiment of the present invention, the running trajectory visualization processing network includes a vector migration subnet, and the vehicle environment perception mining subnet includes a bidirectional long short-term memory subnet.

[0020] Based on this, the vehicle environment perception mining subnet of the running trajectory visualization processing network obtains the vehicle environment perception quantitative knowledge of the vehicle environment perception sensor monitoring information, including: using the vehicle environment migration characteristics obtained by performing vector space migration on the vehicle environment perception sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network as the input of the vehicle environment perception mining subnet; performing knowledge quantization on the vehicle environment migration characteristics through the bidirectional long short-term memory subnet of the vehicle environment perception mining subnet to obtain the vehicle environment perception quantitative knowledge of the vehicle environment perception sensor monitoring information.

[0021] Furthermore, the laser radar point cloud quantitative knowledge of the first laser radar point cloud sensor monitoring information obtained through the laser radar point cloud mining subnet of the running trajectory visualization processing network includes: using the multiple laser radar point cloud migration features of the first laser radar point cloud sensor monitoring information obtained through the vector migration subnet of the running trajectory visualization processing network as the input of the laser radar point cloud mining subnet of the running trajectory visualization processing network, the first laser radar point cloud sensor monitoring information includes multiple laser radar scanning information sorted in sequence according to the scanning cycle; processing the multiple laser radar point cloud migration features through the laser radar point cloud mining subnet of the running trajectory visualization processing network to obtain the laser radar point cloud quantitative knowledge of the first laser radar point cloud sensor monitoring information.

[0022] Among them, the laser radar point cloud mining subnet of the operation trajectory visualization processing network obtains the laser radar point cloud quantitative knowledge of the first laser radar point cloud sensor monitoring information based on multiple laser radar scanning information whose scanning cycle meets the step size requirement; or, the laser radar point cloud mining subnet of the operation trajectory visualization processing network obtains multiple three-dimensional scene drawing element features corresponding to the multiple laser radar point cloud migration features through the residual processing subnet of the laser radar point cloud mining subnet, and processes the multiple three-dimensional scene drawing element features through the feature downsampling branch of the laser radar point cloud mining subnet to obtain the laser radar point cloud quantitative knowledge of the first laser radar point cloud sensor monitoring information.

[0023] On the one hand, step 120 focuses on the acquisition of quantitative knowledge of vehicle-mounted environmental perception. The key technical idea lies in vector space migration, and the operation trajectory visualization processing network includes a vector migration subnet. First, the vehicle-mounted environmental perception sensor monitoring information is input into the vector migration subnet for vector space migration. For example, a vector space migration algorithm based on a neural network is adopted. For example, the number of neurons in the input layer of the algorithm is set to 1024 according to the characteristic dimension of the vehicle-mounted environmental perception sensor monitoring information. After being processed by the multi-layer neural network structure in the vector migration subnet (such as containing 3 hidden layers, and the number of neurons in each hidden layer is 512, 256, and 128 respectively), the vehicle-mounted environmental migration characteristics are obtained.

[0024] Furthermore, considering the subsequent knowledge quantization, the vehicle environment perception mining subnet includes a bidirectional long short-term memory subnet (Bi-LSTM). The obtained vehicle environment migration features are used as the input of the vehicle environment perception mining subnet. The Bi-LSTM subnet can effectively capture the long-term dependencies in the sequence data by processing the input vehicle environment migration features. For example, for the image sequence data in the vehicle environment migration features (for example, the sequence length is 50 frames), each LSTM unit in the Bi-LSTM subnet can selectively memorize and forget the features in each frame of image data according to its internal gating mechanism (including input gate, forget gate and output gate). After such processing, the vehicle environment perception quantitative knowledge of the vehicle environment perception sensor monitoring information is obtained.

[0025] Another aspect of step 120 involves the acquisition of quantitative knowledge of the laser radar point cloud. First, the laser radar point cloud migration features are obtained, and the first laser radar point cloud sensor monitoring information is also processed by the vector migration subnet of the running trajectory visualization processing network. Since the first laser radar point cloud sensor monitoring information includes multiple laser radar scanning information obtained by sorting out in sequence according to the scanning cycle (for example, each scanning cycle is 0.1 seconds, and a total of 10 scanning cycle information is collected), the vector migration subnet will process these laser radar scanning information to obtain multiple laser radar point cloud migration features. For example, a vector migration algorithm based on a convolutional neural network (CNN) is used, the convolution kernel size of CNN is 3*3, the step size is 1, and the point cloud migration features corresponding to each laser radar scanning information are obtained through the processing of multiple convolutional layers and pooling layers (such as a maximum pooling layer, with a pooling window size of 2*2).

[0026] Then, quantitative knowledge is obtained through corresponding processing, and the lidar point cloud mining subnet of the running trajectory visualization processing network obtains the lidar point cloud quantitative knowledge of the first lidar point cloud sensing monitoring information based on multiple lidar scanning information whose scanning cycles meet the step length requirements (for example, the step length is 2 scanning cycles). Alternatively, the lidar point cloud mining subnet obtains multiple three-dimensional scene drawing element features corresponding to multiple lidar point cloud migration features through its residual processing subnet, and then processes these three-dimensional scene drawing element features through the feature downsampling branch to obtain the lidar point cloud quantitative knowledge. For example, the residual processing subnet adopts the ResNet architecture, and processes the lidar point cloud migration features through residual blocks (for example, each residual block contains two 3*3 convolutional layers and a shortcut connection) to obtain the three-dimensional scene drawing element features. The feature downsampling branch uses average pooling to downsample the three-dimensional scene drawing element features (for example, the downsampling ratio is 1 / 2), thereby obtaining the lidar point cloud quantitative knowledge.

[0027] It can be understood that in the intelligent electric snowmobile running track visualization display technology, step 120 is a key link, which mainly involves obtaining the vehicle-mounted environment perception quantitative knowledge of the vehicle-mounted environment perception sensor monitoring information and the lidar point cloud quantitative knowledge of the first lidar point cloud sensor monitoring information through different subnets in the running track visualization processing network. The acquisition of these two quantitative knowledge lays the foundation for the subsequent accurate display of the snowmobile running track.

[0028] The above content of step 120 is further described below.

[0029] Acquisition of Quantitative Knowledge of Vehicle Environment Perception 1. Vector Migration Subnet and Vehicle Environment Migration Characteristics 1) The role of the vector migration subnet The vector migration subnet in the trajectory visualization processing network plays an important starting point in the process of acquiring quantitative knowledge of vehicle-mounted environmental perception. The vector migration subnet is designed to perform vector space migration on the vehicle-mounted environmental perception sensor monitoring information. The vehicle-mounted environmental perception sensor monitoring information contains a rich variety of information, such as image data captured by the camera, and data about the surrounding environment temperature, humidity, etc. obtained by other sensors (for example, camera image data accounts for the majority, the image size is 1280*720 pixels, and it is stored in RGB three-channel format).

[0030] The vector transfer subnet uses a specific algorithm to process this information. One possible algorithm is a vector space transfer algorithm based on a multi-layer perceptron (MLP). For example, the MLP has three hidden layers, with 512 neurons in the first hidden layer, 256 neurons in the second hidden layer, and 128 neurons in the third hidden layer. The number of neurons in the input layer is determined according to the characteristic dimension of the vehicle environment perception sensor monitoring information. For example, if the image data is expanded into a one-dimensional vector, plus the characteristic dimension of other sensor data, the total is 1024 dimensions, then the number of neurons in the input layer is 1024. Through this multi-layer neural network, the vector transfer subnet can map the vehicle environment perception sensor monitoring information from the original feature space to a new vector space to obtain the vehicle environment migration feature.

[0031] 2) The significance of vehicle environment migration characteristics The vehicle environment migration feature is the result of vector space migration, and it has characteristics that are more conducive to subsequent processing. For example, the original vehicle environment perception sensor monitoring information may have complex correlations between features and uneven data distribution. Through the processing of the vector migration subnet, the vehicle environment migration feature can better reflect the relationship between different features in the new vector space, and the data distribution may be more in line with the requirements of certain subsequent processing algorithms. This feature abstracts and optimizes the original information to a certain extent, so that the subsequent vehicle environment perception mining subnet can quantify knowledge more effectively.

[0032] 2. Bidirectional LSTM Subnetwork and Knowledge Quantization 1) Introduction to Bi-LSTM The bidirectional long short-term memory subnet in the vehicle environment perception mining subnet is a special recurrent neural network (RNN) structure specifically used for processing sequence data. In the embodiment of the present invention, the vehicle environment migration feature is used as its input, and the image sequence data (such as continuous multiple frames of image data, such as one frame collected every 5 seconds, a total of 30 frames) or other sensor data time series part that may be contained in the vehicle environment migration feature can be processed using the characteristics of Bi-LSTM.

[0033] Bi-LSTM consists of forward LSTM and backward LSTM. There are three gate structures inside the LSTM unit, namely the input gate, forget gate and output gate. The input gate determines which new information can enter the cell state, the forget gate determines which information in the cell state needs to be forgotten, and the output gate controls which information in the cell state can be output. For the forward LSTM, it processes data from the beginning to the end of the sequence in chronological order; while the backward LSTM processes data from the end to the beginning of the sequence. This bidirectional processing method enables Bi-LSTM to better capture long-term dependencies in sequence data.

[0034] 2) Knowledge quantification process When the vehicle environment migration features are input into the Bi-LSTM subnet, each LSTM unit in the Bi-LSTM subnet processes the sequence data in the input features frame by frame. Taking image sequence data as an example, for the features in each frame of image data, the LSTM unit selectively memorizes and forgets according to its internal gating mechanism. For example, when processing a certain frame of image, the input gate decides whether to allow new feature information to enter the cell state based on the difference between the features of the current frame image and the features of the previous frame image (for example, by calculating the Euclidean distance between feature vectors to measure the difference, setting the threshold to 0.1, if the distance is greater than the threshold, it means there is a large change, and the input gate allows more new information to enter the cell state).

[0035] The forget gate decides whether to forget some of the information in the cell state based on the existing information in the cell state and the overall characteristics of the current frame image (for example, by calculating the average brightness, contrast and other characteristics of the image for comprehensive judgment). After bidirectional processing by the forward LSTM and the backward LSTM, the output result is the vehicle environment perception quantitative knowledge after the knowledge quantification of the vehicle environment migration characteristics. This quantitative knowledge can represent the key content of the vehicle environment perception sensor monitoring information in a more compact and representative form. For example, it can quantify the information such as the movement trend of objects in the image sequence and the change of environmental status.

[0036] 3. Acquisition of Quantitative Knowledge of LiDAR Point Cloud 1. Acquisition of LiDAR Point Cloud Migration Features 1) Processing of LiDAR point cloud by vector migration subnet Similarly, the vector migration subnet of the running trajectory visualization processing network also processes the first laser radar point cloud sensor monitoring information. The first laser radar point cloud sensor monitoring information includes multiple laser radar scanning information obtained by sorting out the scanning cycle. For example, the scanning frequency of the laser radar is 20Hz, that is, a scan is performed every 0.05 seconds, and 20 laser radar scanning information can be obtained in a specific time period (for example, 1 second).

[0037] The vector transfer subnet uses a specific algorithm to process these lidar scan information to obtain multiple lidar point cloud transfer features. One possible algorithm is based on a convolutional neural network (CNN). The convolution kernel size of CNN can be set to 3*3 with a step size of 1. The input lidar scan information is first processed by multiple convolution layers, for example, there are 3 convolution layers, the output channel number of the first convolution layer is 32, the second is 64, and the third is 128. There may also be pooling layers between the convolution layers, such as the maximum pooling layer with a pooling window size of 2*2. After such multiple convolution and pooling operations, the point cloud transfer features corresponding to each lidar scan information are obtained. Compared with the original lidar scan information, these point cloud transfer features are more compact and abstract in feature representation, and can better reflect the key information in the lidar point cloud data, such as the spatial distribution characteristics of the point cloud, the density changes of the point cloud, etc.

[0038] 2. Processing of LiDAR Point Cloud Mining Subnet 1) Quantitative knowledge acquisition based on scanning cycle The LiDAR point cloud mining subnet of the running trajectory visualization processing network can obtain the LiDAR point cloud quantitative knowledge of the first LiDAR point cloud sensor monitoring information based on multiple LiDAR scanning information that meet the step length requirements of the scanning cycle. For example, the step length requirement is to calculate the quantitative knowledge once every 3 scanning cycles. Then, the LiDAR point cloud mining subnet will perform specific processing on the point cloud migration features corresponding to every 3 consecutive LiDAR scanning information. This processing may involve a fusion operation on these 3 point cloud migration features, such as an exemplary weighted summation (the weight can be determined based on factors such as the quality of the point cloud, the scanning angle, etc., such as weighted summation of the 3 point cloud migration features in a ratio of 1:2:1), to obtain a comprehensive feature representation, which is part of the LiDAR point cloud quantitative knowledge based on the scanning cycle.

[0039] By performing such processing on multiple LiDAR scanning information groups that meet the step length requirements, complete LiDAR point cloud quantitative knowledge can be gradually constructed. This scanning cycle-based quantitative knowledge acquisition method can make full use of the characteristics of LiDAR point cloud data in time series and reflect the change rules of point cloud data under different scanning cycles.

[0040] 2) Acquire quantitative knowledge through residual processing subnet and feature downsampling branch In addition, the LiDAR point cloud mining subnet can also obtain multiple 3D scene drawing element features corresponding to multiple LiDAR point cloud migration features through its residual processing subnet. The residual processing subnet can adopt a residual network structure similar to the ResNet architecture. For example, the residual block in the residual processing subnet contains two 3*3 convolutional layers and a shortcut connection. For each LiDAR point cloud migration feature, after processing by the residual block, the corresponding 3D scene drawing element feature can be obtained. These 3D scene drawing element features can reflect the relationship between the LiDAR point cloud data and the surrounding 3D scene, such as the shape, position and relative relationship of the object represented by the point cloud with other objects.

[0041] Then, the features of these 3D scene drawing elements are processed through the feature downsampling branch of the LiDAR point cloud mining subnet to obtain the quantitative knowledge of the LiDAR point cloud. The feature downsampling branch can be downsampled by average pooling. For example, if the downsampling ratio is 1 / 2, if the size of the 3D scene drawing element feature is 64*64*3 (indicating the size in the width, height and depth directions and the number of feature channels), it becomes 32*32*3 after downsampling. Through this downsampling operation, the amount of data can be further reduced while retaining key feature information, and finally the quantitative knowledge of the LiDAR point cloud is obtained. This method of obtaining quantitative knowledge through the residual processing subnet and the feature downsampling branch can mine the features of LiDAR point cloud data from different angles, and improve processing efficiency and reduce data complexity through downsampling.

[0042] It can be seen that in step 120 of the embodiment of the present invention, the vehicle-mounted environment perception sensor monitoring information and the first laser radar point cloud sensor monitoring information are processed by different subnets in the running trajectory visualization processing network to obtain the vehicle-mounted environment perception quantitative knowledge and the laser radar point cloud quantitative knowledge respectively. The acquisition process of these quantitative knowledge involves the application of a variety of neural network structures and algorithms, such as the MLP or CNN algorithm in the vector migration subnet, the Bi-LSTM subnet in the vehicle-mounted environment perception mining subnet, and the residual processing subnet and feature downsampling branch in the laser radar point cloud mining subnet. These quantitative knowledge will provide an important data basis for the subsequent visualization of the running trajectory of the intelligent electric snowmobile, so that the subsequent processing can more accurately reflect the running environment and trajectory characteristics of the snowmobile.

[0043] Step 130: Obtain sensor monitoring interaction knowledge features corresponding to the vehicle environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set by running the knowledge feature interaction subnet of the trajectory visualization processing network.

[0044] In an embodiment of the present invention, the associated mixed-mode sensor monitoring information set includes X associated mixed-mode sensor monitoring information corresponding to X associated intelligent electric snowmobiles, where X is a positive integer; the running trajectory visualization processing network includes a vector migration subnet, and the knowledge feature interaction subnet includes X feature enhancement branches, a residual processing subnet, and a feature downsampling branch.

[0045] Based on this, the sensor monitoring interaction knowledge features corresponding to the vehicle environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set are obtained through the knowledge feature interaction subnet of the running trajectory visualization processing network, including: using the vehicle environment migration features obtained by performing vector space migration on the vehicle environment perception sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network, and the X associated migration features obtained by performing vector space migration on the X associated mixed-mode sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network, as inputs of the knowledge feature interaction subnet; processing the vehicle environment migration features and the X associated migration features through the X feature enhancement branches of the knowledge feature interaction subnet to obtain X associated vehicle environment enhancement features; obtaining the associated residual annotation features corresponding to the X associated vehicle environment enhancement features through the residual processing subnet of the knowledge feature interaction subnet; and processing the associated residual annotation features through the feature downsampling branch of the knowledge feature interaction subnet to obtain the sensor monitoring interaction knowledge features.

[0046] In a more detailed step, the vehicle environment migration feature and the X associated migration features are processed through the X feature enhancement branches of the knowledge feature interaction subnet to obtain X associated vehicle environment enhancement features, including: obtaining a visible layer overlay feature between the vehicle environment migration feature and the u-th associated migration feature among the X associated migration features, where u is a positive integer not greater than X; integrating the vehicle environment migration feature, the u-th associated migration feature and the visible layer overlay feature, and inputting the result obtained after the integration into a weighted processing module of the u-th feature enhancement branch among the X feature enhancement branches to obtain an output of the weighted processing module of the u-th feature enhancement branch; using the output of the weighted processing module of the u-th feature enhancement branch as an input of a normalization module of the u-th feature enhancement branch, and the normalization module of the u-th feature enhancement branch outputs the u-th associated vehicle environment enhancement feature.

[0047] In the invention of the technology for visualizing the running track of the intelligent electric snowmobile, step 130 is a key link, the core of which is to obtain the sensor monitoring interactive knowledge features corresponding to the vehicle-mounted environmental perception sensor monitoring information and the associated mixed-mode sensor monitoring information set through the knowledge feature interactive subnet of the running track visualization processing network. This step is intended to integrate the vehicle-mounted environmental perception information of the target intelligent electric snowmobile itself and the information of other intelligent electric snowmobiles (the number is X) associated with it, and to mine more comprehensive and in-depth interactive knowledge features through a complex processing flow, so as to provide a rich information basis for the subsequent accurate display of the running track of the target snowmobile.

[0048] 1. Input feature preparation 1. Processing of vector migration subnet 1) Acquisition of vehicle environment migration characteristics The vector migration subnet in the running trajectory visualization processing network first performs vector space migration on the vehicle-mounted environmental perception sensor monitoring information to obtain the vehicle-mounted environmental migration features. The vehicle-mounted environmental perception sensor monitoring information contains a lot of information about the surrounding environment of the target intelligent electric snowmobile, such as image data collected by the camera (for example, the image resolution is 1920*1080 pixels and the color mode is RGB), temperature sensor data, humidity sensor data, etc. The vector migration subnet may use a deep learning-based algorithm, such as a deep neural network (DNN) to perform vector space migration.

[0049] For example, this DNN has multiple hidden layers, and the number of neurons in the input layer is determined according to the feature dimension of the vehicle environment perception sensor monitoring information. It may be obtained by expanding the image data into a one-dimensional vector and adding the feature dimensions of other sensor data, for example, a total of 2048 feature dimensions. After nonlinear transformation of the hidden layer (such as the number of neurons in the hidden layer is 1024, 512, 256, etc.), the vehicle environment migration feature is output. This vehicle environment migration feature effectively extracts and optimizes the original vehicle environment perception sensor monitoring information in a new vector space.

[0050] 2) Acquisition of associated migration features Similarly, the vector migration subnet performs vector space migration on X associated mixed-mode sensor monitoring information to obtain X associated migration features. Each associated mixed-mode sensor monitoring information is similar to the sensor monitoring information of the target smart electric snowmobile, and also contains environmental perception information and lidar point cloud information. Take one of the associated smart electric snowmobiles as an example. For example, the image data size in its associated mixed-mode sensor monitoring information is 1280*720 pixels, and other sensor data includes temperature, wind speed and other information.

[0051] When the vector migration subnet performs vector space migration, a similar DNN structure is used (but network parameters such as the number of hidden layer neurons and learning rate may be adjusted according to data characteristics). For each associated mixed-mode sensor monitoring information, the corresponding associated migration feature is obtained after being processed by the vector migration subnet. These associated migration features have the same purpose as the vehicle environment migration features, that is, to transform the feature space of the original information so that it can be more effectively processed in the knowledge feature interaction subnet later.

[0052] 2. Obtaining enhanced features of the associated vehicle environment through feature enhancement branches 1. Obtaining Visible Layer Overlay Features 1) Concept of visual layer overlay features For each associated migration feature (taking the u-th associated migration feature as an example, where u is a positive integer not greater than X), the visual layer overlay feature between the vehicle environment migration feature and the u-th associated migration feature is obtained. This process involves constructing a relationship representation based on graph theory. The visual layer overlay feature is designed to capture the complex relationship between the vehicle environment migration feature and the associated migration feature.

[0053] For example, each feature element in the vehicle environment migration feature and the u-th associated migration feature can be regarded as a node of the graph. Then, the connection of the edge is determined based on the correlation between the feature elements. There are many ways to calculate the correlation, such as calculating the cosine similarity between feature vectors. For example, a cosine similarity threshold is set to 0.6. If the cosine similarity between two feature elements is greater than this threshold, an edge is established between them. The graph structure constructed in this way can represent the intrinsic connection between the vehicle environment migration feature and the u-th associated migration feature. The information contained in this graph structure is the visible layer overlay feature.

[0054] 2) Feature integration and weighted processing Feature integration: Once the visible layer overlay feature is obtained, the vehicle environment migration feature, the u-th associated migration feature and the visible layer overlay feature are integrated. This integration can be an exemplary feature vector concatenation, that is, connecting the three feature vectors in a certain order into a longer feature vector. For example, if the vehicle environment migration feature vector dimension is 256 dimensions, the u-th associated migration feature vector dimension is 256 dimensions, and the visible layer overlay feature vector dimension is 128 dimensions, then the integrated result represents a vector dimension of 640 dimensions.

[0055] Weighted processing: The integrated result is input into the weighted processing module of the uth feature enhancement branch among the X feature enhancement branches. The purpose of the weighted processing module is to assign different weights according to the importance of different features. An adaptive weighting algorithm can be used here, for example, to determine the weight according to the information entropy of the feature. Information entropy is an indicator to measure the amount of information contained in a feature. The lower the information entropy, the higher the certainty of the feature and the more important it may be.

[0056] For example, the initial weights of the vehicle environment migration feature, the u-th associated migration feature, and the visible layer overlay feature in the integrated result representation are 0.3, 0.3, and 0.4, respectively. The weighted processing module makes adjustments based on their respective information entropies (calculated on a certain number of sample data). For example, if the calculation finds that the information entropy of the vehicle environment migration feature is low, its weight may be increased to 0.4, and the weights of other features may be adjusted accordingly, such as adjusting the weight of the u-th associated migration feature to 0.3 and the weight of the visible layer overlay feature to 0.3. After weighted processing, the output of the weighted processing module is obtained.

[0057] 3) Normalization The output of the weighted processing module of the u-th feature enhancement branch is used as the input of the normalization module of the u-th feature enhancement branch. The normalization module uses the batch normalization algorithm, which is a commonly used neural network normalization technology. Batch normalization normalizes the input data so that the mean of the data is 0 and the variance is 1, thereby accelerating the training process of the neural network and improving the generalization ability of the model.

[0058] After batch normalization, the normalization module of the u-th feature enhancement branch outputs the u-th associated vehicle environment enhancement feature. This associated vehicle environment enhancement feature integrates the information of the vehicle environment migration feature, the u-th associated migration feature, and the visual layer overlay feature between them, and after weighted and normalized processing, it can better reflect the interactive relationship between the target smart electric snowmobile and the u-th associated smart electric snowmobile in environmental perception. By performing such processing on each associated migration feature (u from 1 to X), X associated vehicle environment enhancement features can be obtained.

[0059] 3. Obtaining the associated residual annotation features and sensor monitoring interactive knowledge features 1. Acquisition of associated residual annotation features 1) The role of the residual processing subnet The residual processing subnet in the knowledge feature interaction subnet is used to obtain the associated residual annotation features corresponding to the X associated vehicle environment enhancement features. The residual processing subnet adopts the idea of ​​the residual network (ResNet) structure. The core of ResNet is the residual block. For example, the residual block structure here is: input->convolution layer (convolution kernel size is 3*3, the number of output channels is 64)->batch normalization->ReLU activation function->convolution layer (convolution kernel size is 3*3, the number of output channels is 64)->batch normalization->input+convolution layer output.

[0060] For each associated vehicle environment enhancement feature, it is input into the residual block of the residual processing subnet for processing. The convolution layer in the residual block can effectively extract features, batch normalization is used for data normalization, and the ReLU activation function introduces nonlinear characteristics. Through the processing of the residual block, the associated residual annotation features corresponding to each associated vehicle environment enhancement feature can be obtained. These associated residual annotation features represent the remaining information or difference information of the associated vehicle environment enhancement features after residual processing to a certain extent, which is of great significance for mining deeper interactive knowledge features.

[0061] 2. Acquisition of sensor monitoring interaction knowledge features 1) Processing of feature downsampling branches The feature downsampling branch of the knowledge feature interaction subnet processes the associated residual annotation features to obtain the sensor monitoring interaction knowledge features. The feature downsampling branch can use a variety of downsampling methods, such as max-pooling. For example, the pooling window size of max-pooling is 2*2.

[0062] When the associated residual annotation feature is input to the feature downsampling branch, the maximum pooling operation selects the maximum value in the local area (2*2 window) of the feature map as the output. For example, if the feature map size of the associated residual annotation feature is 16*16*64 (indicating width, height, and number of channels), after the maximum pooling downsampling, the feature map size becomes 8*8*64. Through this downsampling operation, the amount of data can be reduced while retaining key feature information, and finally the sensor monitoring interactive knowledge feature is obtained. This sensor monitoring interactive knowledge feature integrates the interactive information of the vehicle-mounted environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set. After complex processing of multiple subnets, it can provide a more comprehensive and representative information basis for the subsequent visualization of the operation trajectory of the intelligent electric snowmobile.

[0063] Therefore, in step 130 of the embodiment of the present invention, a series of complex and orderly processing is performed on the vehicle environment migration features of the target intelligent electric snowmobile and the associated migration features of the associated intelligent electric snowmobile through the knowledge feature interaction subnet of the running trajectory visualization processing network. Starting from obtaining the visible layer overlay features, the associated vehicle environment reinforcement features are obtained through feature integration, weighted processing, and normalization processing, and then the associated residual annotation features are obtained through the residual processing subnet, and finally the sensor monitoring interactive knowledge features are obtained using the feature downsampling branch. This process involves a variety of algorithms and network structures in graph theory and deep learning, such as constructing visible layer overlay features based on cosine similarity, a weighted processing module using an adaptive weighting algorithm, a residual processing subnet based on a ResNet structure, and a feature downsampling branch of maximum pooling. The comprehensive application of these processing steps and technical means enables the sensor monitoring interactive knowledge features finally obtained to fully integrate the relevant information of the target vehicle and the associated vehicle, providing an important interactive information basis for the entire intelligent electric snowmobile running trajectory visualization display technology.

[0064] Step 140: Perform full-connection processing on the vehicle environment perception quantitative knowledge, the lidar point cloud quantitative knowledge, and the sensor monitoring interaction knowledge features through the fully-connected subnet of the running trajectory visualization processing network to obtain the fully-connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile.

[0065] In step 140, the vehicle environment perception quantitative knowledge, the laser radar point cloud quantitative knowledge, and the sensor monitoring interactive knowledge features are fully connected through the fully connected subnet of the running trajectory visualization processing network. For example, the dimension of the vehicle environment perception quantitative knowledge is 128 dimensions, the dimension of the laser radar point cloud quantitative knowledge is 64 dimensions, and the dimension of the sensor monitoring interactive knowledge features is 32 dimensions. The structure of the fully connected subnet may include multiple fully connected layers (for example, including two fully connected layers, the number of neurons in the first fully connected layer is 256, and the number of neurons in the second fully connected layer is 128). The neurons in the fully connected subnet are connected through a weight matrix. The initial value of the weight matrix can be randomly initialized (such as randomly taking values ​​from a normal distribution with a mean of 0 and a standard deviation of 0.01), and then the weight value is continuously adjusted during the training process through the back propagation algorithm, and finally the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile is obtained.

[0066] In detail, the further development of the fully connected quantitative knowledge of the running trajectory obtained based on the fully connected subnet processing is introduced as follows.

[0067] 1.1 Introduction to the fully connected subnet 1) The position and role of the fully connected subnet in the network structure In the running trajectory visualization processing network, the fully connected subnet plays a key role in integrating different types of quantitative knowledge. It receives the quantitative knowledge of vehicle environment perception, the quantitative knowledge of lidar point cloud, and the sensor monitoring interaction knowledge features obtained from the previous steps as input. The fully connected subnet is a classic neural network structure in which each neuron is fully connected to all neurons in the previous layer. This structure enables the network to learn the complex relationship between the input features, thereby making a comprehensive quantitative representation of the running trajectory of the target smart electric snowmobile.

[0068] 2) Example of basic structural parameters of a fully connected subnet For example, the dimension of the vehicle environment perception quantitative knowledge is 128 dimensions, the dimension of the lidar point cloud quantitative knowledge is 64 dimensions, and the dimension of the sensor monitoring interaction knowledge feature is 32 dimensions. The structure of the fully connected subnet can be set to include multiple fully connected layers. For example, the number of neurons in the first fully connected layer is 256, and the number of neurons in the second fully connected layer is 128. The number of neurons in the input layer is the sum of the dimensions of the vehicle environment perception quantitative knowledge, the lidar point cloud quantitative knowledge, and the sensor monitoring interaction knowledge feature, that is, 128+64+32=224.

[0069] (II) Full connection processing 1) Weight initialization Before the fully connected subnet is fully connected, the weights of the fully connected layer need to be initialized. A common initialization method is random initialization. For example, the weights can be initialized by randomly drawing values ​​from a normal distribution with a mean of 0 and a standard deviation of 0.01. For the bias term, it can be initialized to 0. This initialization method provides a starting point for the training of the fully connected subnet, allowing the network to gradually adjust the weights to adapt to the characteristics of the data during subsequent training.

[0070] 2) Forward propagation process Processing of the first fully connected layer: When the quantized knowledge of vehicle environment perception, the quantized knowledge of LiDAR point cloud, and the sensor monitoring interaction knowledge features are input into the first fully connected layer of the fully connected subnet, each input neuron is connected to the 256 neurons of the first fully connected layer through a weight matrix. For example, if the input vector is (x=[x_1, x_2, cdots, x_{224}]), the weight matrix is ​​(W_1) (dimension is (224*256)), and the bias vector is (b_1) (dimension is 256), then the output (y_1) of the first fully connected layer can be calculated by the formula (y_1=f(W_1x+b_1)), where (f) is the activation function, for example, the ReLU (RectifiedLinearUnit) activation function can be used, that is, (f(z)=max(0,z)). The ReLU activation function can increase the nonlinear characteristics of the network, allowing the fully connected subnet to learn more complex functional relationships.

[0071] Processing of the second fully connected layer (optional): If the fully connected subnet contains a second fully connected layer, the output (y_1) of the first fully connected layer will be used as the input of the second fully connected layer. The weight matrix of the second fully connected layer is set to (W_2) (dimension is (256*128)), and the bias vector is (b_2) (dimension is 128). Then the output (y_2) of the second fully connected layer is calculated by the formula (y_2=f(W_2y_1+b_2)). Through such multi-layer fully connected layer processing, the fully connected subnet can gradually integrate and abstract different types of input quantitative knowledge, and finally obtain the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile.

[0072] 3) The significance of fully connected processing By fully connecting the vehicle environment perception quantitative knowledge, the lidar point cloud quantitative knowledge, and the sensor monitoring interaction knowledge features through the fully connected subnet, these knowledge from different sources and dimensions can be integrated into a unified representation space. This fusion takes into account the relationship between the vehicle environment perception information (such as objects recognized by the camera, environmental conditions, etc.), the lidar point cloud information (such as the surrounding terrain, the three-dimensional structure of obstacles, etc.), and the interaction information with other associated vehicles (sensor monitoring interaction knowledge features). For example, a feature in the vehicle environment perception quantitative knowledge may be associated with specific structural information in the lidar point cloud quantitative knowledge. The fully connected subnet can learn this association and reflect it in the fully connected quantitative knowledge of the running trajectory. This quantitative knowledge can more comprehensively and accurately describe the various factors related to the running trajectory of the target intelligent electric snowmobile, providing a rich information basis for the subsequent selection of visualization conversion strategies.

[0073] Step 150: Based on the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile, a target visualization conversion strategy is obtained from a visualization conversion strategy pool; the target visualization conversion strategy is used to realize the visualization display of the running trajectory of the target intelligent electric snowmobile.

[0074] In step 150, based on the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile, the target visualization conversion strategy is obtained from the visualization conversion strategy pool. The visualization conversion strategy pool contains a variety of predefined visualization conversion strategies. For example, one strategy can be a strategy based on 2D plane map display, which is suitable for exemplary running trajectory display, mainly showing the driving route of the snowmobile on the plane map, with different colored lines representing different driving states (such as green for normal driving and red for abnormal driving); another strategy can be a strategy based on 3D scene reconstruction and trajectory display, which can use the quantitative knowledge of laser radar point cloud to construct a 3D scene around the snowmobile, and display the running trajectory in the 3D scene. The trajectory is presented in the form of a three-dimensional curve, and the positional relationship of the snowmobile relative to the surrounding obstacles (such as snow piles, trees, etc.) can be displayed. According to the features in the fully connected quantitative knowledge of the running trajectory (such as indicators such as the complexity of the trajectory and the degree of danger of the surrounding environment), a suitable target visualization conversion strategy is selected to realize the visualization display of the running trajectory of the target intelligent electric snowmobile.

[0075] In detail, the above-mentioned strategy for selecting target visualization conversion based on fully connected quantized knowledge is described as follows.

[0076] 1. Visualization of conversion strategy pool 1) Composition of the strategy pool The visualization conversion strategy pool is a collection of multiple predefined visualization conversion strategies. These strategies are designed to visualize the running trajectory of the target smart electric snowmobile in different ways. For example, one of the strategies can be a strategy based on 2D plane map display. Under this strategy, the running trajectory will be drawn on a two-dimensional plane map, and different driving states may be represented by lines of different colors, such as green lines for normal driving and red lines for abnormal driving. Some key geographic information can also be marked on the map, such as terrain features such as hills and gullies in the snow, as well as some fixed facilities (such as rest stops at ski resorts, etc.).

[0077] Another strategy can be based on 3D scene reconstruction and trajectory display. This strategy uses the quantitative knowledge of the LiDAR point cloud to construct a 3D scene around the snowmobile and display the running trajectory in this 3D scene. The trajectory is presented in the form of a three-dimensional curve, and the positional relationship of the snowmobile relative to surrounding obstacles (such as snow piles, trees, etc.) can be displayed. In addition, there can also be a strategy based on multi-perspective display to display the running trajectory of the snowmobile from multiple different perspectives (such as top view, side view, front view, etc.) to more comprehensively observe the driving conditions of the snowmobile in different directions.

[0078] 2) Parameters and characteristics of the strategy Each visualization transformation strategy has its own parameters and characteristics. For a 2D flat map display strategy, its parameters may include the scale of the map, the width of the lines, and the color coding rules. The scale determines the proportional relationship between the distance on the map and the actual distance. For example, a scale of 1:1000 means that 1 cm on the map represents 10 meters in actual distance. The line width can be adjusted according to the speed or importance of the snowmobile. The lines of fast or important tracks can be set wider. The color coding rules specify the colors corresponding to different driving states.

[0079] For 3D scene reconstruction and trajectory display strategies, the parameters may involve the accuracy of 3D scene construction, the color and thickness of the trajectory curve, the detail level of obstacle display, etc. The accuracy of 3D scene construction depends on the quality and processing method of the quantitative knowledge of the lidar point cloud. Higher accuracy can more realistically restore the environment around the snowmobile. The color and thickness of the trajectory curve can also be used to distinguish different driving states or trajectory types, and the detail level of obstacle display can be adjusted according to user needs. For example, when a detailed analysis of the relationship between the snowmobile and a specific obstacle is required, the detail level of the obstacle display can be increased.

[0080] 2. Strategy selection based on fully connected quantitative knowledge 1) The relationship between features and strategy selection in fully connected quantitative knowledge The fully connected quantitative knowledge of the target intelligent electric snowmobile's running trajectory contains rich information, which will determine which target visualization transformation strategy to select from the visualization transformation strategy pool. For example, if certain features in the fully connected quantitative knowledge indicate that the snowmobile's running trajectory is mainly on a plane, and the surrounding environment is relatively exemplary, and there is not much three-dimensional structure information to be displayed, then a strategy based on 2D plane map display may be more suitable. These features can be reflected by the values ​​of certain dimensions in the fully connected quantitative knowledge, such as the dimension value representing the vertical change is small, while the dimension value representing the horizontal direction of travel is dominant.

[0081] On the contrary, if the fully connected quantitative knowledge contains a lot of information related to the surrounding three-dimensional environment structure, such as the quantitative knowledge of the lidar point cloud retains rich information such as terrain undulations and obstacle heights after being processed by the fully connected subnet, then the strategy based on 3D scene reconstruction and trajectory display may be a better choice. In addition, if the fully connected quantitative knowledge also contains features related to the interaction information with other associated vehicles, for example, in a scene where multiple vehicles are driving in coordination, the relative position relationship between the snowmobile and other vehicles needs to be displayed from multiple perspectives. At this time, the strategy based on multi-perspective display may be selected.

[0082] 2) Example of the specific process of strategy selection For example, the fully connected quantized knowledge of the running trajectory can be represented as a vector (q=[q_1, q_2, cdots, q_{128}]) (here we take the 128-dimensional fully connected quantized knowledge as an example). Some rules can be defined to select the visualization transformation strategy based on the element values ​​in the vector (q). For example, the average value (bar{q}) of the elements in the vector (q) that represent the vertical motion characteristics (for example, the 11 elements from (q_{10}) to (q_{20})) is calculated. If (bar{q}<0.1) (here 0.1 is a pre-set threshold that can be adjusted according to actual conditions), the 2D plane map display strategy is given priority.

[0083] If (bar{q}geq0.1), and the average value of the elements in the vector (q) that represent the interaction features with the surrounding three-dimensional objects (for example, the 11 elements from (q_{30}) to (q_{40})) is greater than another threshold value (such as 0.2), then the strategy based on 3D scene reconstruction and trajectory display is selected. If the above two conditions are not met, but the variance of the elements in the vector (q) that represent the interaction features with other vehicles (for example, the 11 elements from (q_{50}) to (q_{60})) is greater than a certain value (such as 0.3), then the strategy based on multi-view display is selected. Through such specific rules, the target visualization conversion strategy can be accurately selected from the visualization conversion strategy pool according to the different features in the fully connected quantitative knowledge of the running trajectory, thereby realizing the visualization display of the running trajectory of the target intelligent electric snowmobile.

[0084] 3. The significance of target visualization conversion strategy 1) Targeted visualization of running tracks The selected target visualization conversion strategy is customized for the features in the fully connected quantitative knowledge of the target smart electric snowmobile's running trajectory, and can display the snowmobile's running trajectory in the most suitable way. For example, under the 2D plane map display strategy, for users who only need to understand the snowmobile's driving route on the plane and the basic driving status (normal or abnormal) by example, this display method is concise and clear. Under the 3D scene reconstruction and trajectory display strategy, for users who need to deeply analyze the relationship between the snowmobile and the surrounding complex three-dimensional environment (such as driving in the snow on complex terrain), more intuitive and detailed information can be provided.

[0085] 2) Improve the effectiveness and comprehensibility of visual presentations The selection of the target visualization conversion strategy helps to improve the effectiveness and comprehensibility of the visualization display. By selecting the appropriate strategy, the key information in the fully connected quantitative knowledge of the running trajectory can be presented in a way that is easy for users to understand. For example, under the multi-view display strategy, the display of different perspectives allows users to observe the driving conditions of the snowmobile from multiple aspects and better understand the position and behavior of the snowmobile in the entire driving scene, thereby providing more valuable visualization support for the monitoring, management and research of smart electric snowmobiles.

[0086] It can be seen that step 140 and step 150 play a vital role in the entire intelligent electric snowmobile running track visualization display technology. Step 140 integrates multiple quantitative knowledge through the fully connected subnet to obtain the running track fully connected quantitative knowledge, and step 150 selects the most suitable target visualization conversion strategy from the visualization conversion strategy pool based on this fully connected quantitative knowledge, and finally realizes efficient, accurate and targeted running track visualization display.

[0087] In summary, the embodiment of the present invention applies a series of rigorous steps and complex network subnet processing, starting from obtaining multiple sensor monitoring information, gradually mining quantitative knowledge, obtaining interactive knowledge features, calculating fully connected quantitative knowledge, and finally determining a suitable visualization conversion strategy, thereby achieving effective visualization of the running trajectory of the intelligent electric snowmobile. This technology has important application value in the fields of intelligent transportation, snowmobile automatic driving monitoring, etc.

[0088] To facilitate understanding of the above technical solution, a practical application example is provided below to further introduce and illustrate it.

[0089] Target smart electric snowmobile 3D running track visualization example 1. Example of target mixed-mode sensor monitoring information set Vehicle environment perception sensor monitoring information 1) Camera image data The image captured by the camera is in RGB format with a resolution of 1920*1080 pixels. The image shows a snowy scene, with the snowmobile located slightly to the left of the center of the picture, and a winding snow track in front of it, with undulating snow piles on both sides of the snow track. In the distance of the snow track, a large snow mountain can be seen, and the outline of the snow mountain is very clear against the blue sky.

[0090] Through image analysis technology, the edge information of the snow track can be obtained. For example, the coordinate range of the left edge of the snow track in the image is from (500, 500) to (1500, 800), and the right edge is from (700, 500) to (1700, 800) (the coordinates here are based on the upper left corner of the image as the origin, the right is the positive direction of the x-axis, and the downward is the positive direction of the y-axis). At the same time, the approximate position and shape of the snow pile in the image are detected. For example, the coordinate range of the larger snow pile on the left side of the picture is approximately (300, 600) to (600, 900).

[0091] 2) Other sensor data The temperature sensor shows that the current ambient temperature is -15°C, and the wind direction sensor shows that the wind direction is northwest and the wind speed is 5 meters per second. These environmental data help us understand the driving environment of the snowmobile. For example, the wind direction and wind speed may affect the handling performance of the snowmobile.

[0092] The first laser radar point cloud sensing monitoring information 1) LiDAR scanning information The laser radar scan frequency is 15Hz, and each scanning cycle acquires point cloud data within a 20-meter range. At a certain moment, the laser radar detected a small gully 10 meters in front of the snowmobile. The coordinate range of the point cloud data of the gully in the laser radar coordinate system is ((8, -2, -0.5) to (12, 2, -0.3)) (the coordinates here are based on the laser radar location as the origin, the positive direction of the x-axis is the direction of the snowmobile's forward movement, the y-axis is the horizontal direction perpendicular to the forward direction, and the z-axis is the vertical upward direction, and the unit is meter).

[0093] There is a high snow pile 5 meters to the left of the snowmobile, and some of its point cloud coordinates range from ((-5, 3, 1) to (-5, 5, 2)). In addition, some branches above the snow road are detected, and their point cloud coordinates are more scattered, for example, some of their coordinates are ((2, 1, 3) to (4, 3, 4)).

[0094] 2. Target Visualization Conversion Strategy Example - Strategy Based on 3D Scene Reconstruction and Trajectory Display 1) 3D scene construction A 3D coordinate system is established with the initial position of the snowmobile as the origin. The x-axis represents the direction of the snowmobile, the y-axis represents the horizontal direction, and the z-axis represents the vertical direction. A 3D scene is constructed based on the LiDAR point cloud data, and the spatial range of the scene is set as a rectangular space with the snowmobile as the center, 30 meters in front, back, left, right, and 10 meters in the top and bottom.

[0095] For objects such as snow piles, gullies, and branches detected by the LiDAR, corresponding 3D models are constructed based on their point cloud coordinates. Snow piles are represented by irregular polyhedron models, gullies are represented by concave models with a certain depth, and branches are represented by slender cylindrical combination models.

[0096] 2) Trajectory drawing The trajectory of the snowmobile is represented by a blue 3D curve. The thickness of the curve is dynamically adjusted according to the speed of the snowmobile. For example, when the speed is 0-5 m / s, the thickness of the curve is 1 mm; when the speed is 5-10 m / s, the thickness of the curve is 2 mm.

[0097] 3) Environmental element labeling In the 3D scene, the snow mountain detected in the camera image is represented by a larger triangular patch model, and its position and scale in the 3D scene are estimated based on its position and size in the image. The snow trail is represented by a white ribbon plane model, and its width is determined according to the actual width and scene scale. The direction of the snow trail is determined by converting the edge coordinates of the snow trail in the image into the 3D coordinate system.

[0098] Wind direction and wind speed can be represented by an arrow model. The direction of the arrow indicates the wind direction, and the length and thickness of the arrow are determined according to the wind speed. For example, when the wind speed is 5 m / s, the length of the arrow is 1 m and the thickness is 0.1 m.

[0099] 3. Visualization of the Conversion Process 1. Coordinate transformation and feature mapping 1) Conversion from image coordinates to 3D coordinates The coordinates of the edge of the snow trail and the snow pile in the camera image need to be converted to a 3D coordinate system. First, the conversion is performed based on the camera's internal parameters (such as focal length and other parameters) and external parameters (such as the camera's position and posture). For example, by calculation, the coordinates of the left edge of the snow trail in the image (500, 500) are converted to the coordinates in the 3D coordinate system (10, -5, 0) meters (the conversion process here involves complex computer vision algorithms, such as perspective transformation, etc.).

[0100] The coordinates of the snow pile are also converted. For example, the coordinate range of the large snow pile on the left side of the screen is converted from (300, 600) to (600, 900) to (-8, -3, 1) to (-5, 0, 2) meters in the 3D coordinate system.

[0101] 2) Conversion of LiDAR coordinates to 3D coordinates The coordinates of the objects detected by the LiDAR are already in their own coordinate system, and need to be converted to a 3D coordinate system with the initial position of the snowmobile as the origin. The coordinate range of the gully in front is from ((8, -2, -0.5) to (12, 2, -0.3)), which is converted to the coordinates in the 3D scene coordinate system ((8, -2, -0.5) to (12, 2, -0.3)) by coordinate translation and rotation (according to the installation position and posture of the LiDAR relative to the snowmobile) (for example, the relative relationship between the LiDAR and the snowmobile coordinate system is shown, where the coordinates remain basically unchanged).

[0102] For the coordinate range of the snow pile on the left ((-5, 3, 1) to (-5, 5, 2)), it is converted to ((-5, 3, 1) to (-5, 5, 2)) meters in the 3D scene coordinate system.

[0103] 3) Feature mapping to visualization elements The features corresponding to the converted coordinates are mapped into 3D visualization elements. For example, the coordinate points of the edge of the snow track after the coordinate conversion are connected to construct a white strip plane model to represent the snow track; the coordinates of the snow pile are used to construct a polyhedron model; the coordinate points of the snowmobile's running track are connected into a blue three-dimensional curve; the wind direction and wind speed information are mapped into an arrow model, etc.

[0104] 2. Recording coordinate features or image coding features 1) Coordinate feature record Record the coordinate point sequence of the snowmobile's running track in the 3D coordinate system. For example, the coordinate point sequence of the snowmobile's track over a period of time is [(0, 0, 0), (1, 0, 0), (2, 0, 0.1), (3, 0, 0.2)] (the coordinates here are in meters). At the same time, record the coordinates of other environmental elements and obstacles in the 3D coordinate system, such as the coordinate range of the gully ((8, -2, -0.5) to (12, 2, -0.3)) meters, the coordinates of the snow pile ((-5, 3, 1) to (-5, 5, 2)) meters, etc. These coordinate features can fully describe the running track of the target smart electric snowmobile in the 3D scene and the relative position relationship of the surrounding environmental elements.

[0105] 2) Image coding features (relatively complex) A hierarchical encoding method is adopted. First, the 3D scene is divided into multiple cubic units, such as a cubic unit with a side length of 1 meter. For each unit, the elements contained in it (such as snowmobile tracks, snow piles, gullies, etc.) are encoded.

[0106] Define a coding dictionary, for example, snowmobile track points are coded as 1, snow piles are coded as 2, gullies are coded as 3, snow mountains are coded as 4, branches are coded as 5, etc. For each cubic unit, if it contains a certain element, record the coding value of the element. For example, in a 3*3*3 cubic unit area centered at the origin, the coding matrix can be expressed as (partial example): [ begin{bmatrix} 0&0&0 0&1&0 0&0&0 end{bmatrix} ] At a higher level, the position coordinates of each cube unit are recorded. For example, the coordinates of the cube unit corresponding to the above coding matrix are (0, 0, 0). Through this hierarchical coding method, various elements in the 3D scene and their positional relationships can be fully recorded, thereby representing the characteristics of the 3D running track image of the target smart electric snowmobile.

[0107] In a preferred example, the visualization conversion strategy pool includes multiple visualization conversion strategies, each visualization conversion strategy generates a corresponding visual element mapping feature through a spatiotemporal graph neural network, and the visual element mapping feature of the target visualization conversion strategy obtained from the visualization conversion strategy pool is adapted to the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile; and the spatiotemporal graph neural network and the running trajectory visualization processing network are obtained through collaborative debugging, and the error variables determined by the debugging error indicators used in the collaborative debugging link include: the fully connected quantitative knowledge of the running trajectory of the intelligent electric snowmobile sample obtained by the running trajectory visualization processing network during debugging, the visual element mapping feature of the visualization conversion strategy sample obtained by the spatiotemporal graph neural network during debugging, and the debugging guidance information determined based on the quality evaluation of the visualization conversion strategy sample based on the intelligent electric snowmobile sample.

[0108] On this basis, the method further comprises: (1) obtaining a three-dimensional mixed-mode visual annotation information set of a target visualization conversion strategy, wherein the three-dimensional mixed-mode visual annotation information set includes: trajectory visual positioning annotation information and visual element annotation information at multiple attention levels; In detail, the visual element annotation information of multiple attention levels of the target visualization conversion strategy is obtained, including: performing task sampling on the 3D visual conversion task information included in the target visualization conversion strategy, and determining the visual element annotation information of the 3D attention level according to the 3D visual conversion task event obtained by the task sampling; performing vehicle posture conversion task mining on the 3D visual conversion task event obtained by the task sampling to obtain the vehicle posture conversion task event, and determining the visual element annotation information of the vehicle posture attention level based on the vehicle posture conversion task event and the visualized vehicle status data of the target visualization conversion strategy; based on the visual element annotation information of the 3D attention level and the visual element annotation information of the vehicle posture attention level, determining the visual element annotation information of multiple attention levels of the target visualization conversion strategy; (2) generating attention annotation features at multiple attention levels based on the annotation information of each visual element and the trajectory visual positioning annotation information, and performing full connection processing on the attention annotation features at the multiple attention levels to obtain cross-attention annotation features; (3) obtaining a cross-attention residual annotation feature corresponding to the cross-attention annotation feature through the residual processing subnet of the spatiotemporal graph neural network; (4) Feature mapping is performed on the cross-attention residual annotation features through the bidirectional long short-term memory subnetwork of the spatiotemporal graph neural network to obtain visual element mapping features of the target visualization conversion strategy; the target visualization conversion strategy is any one of the multiple visualization conversion strategies.

[0109] The above embodiment summarizes the technical solution for generating visual element mapping features of the target visualization conversion strategy, and the technical solution is described in detail below.

[0110] 1. Visualization conversion strategy pool and visual element mapping feature adaptation 1. Composition of the Visual Conversion Strategy Pool The visualization conversion strategy pool contains multiple visualization conversion strategies. Each strategy generates corresponding visual element mapping features through a spatiotemporal graph neural network. The generation of this mapping feature is to adapt the visualization conversion strategy to the fully connected quantitative knowledge of the target smart electric snowmobile's running trajectory. For example, there are three strategies in the visualization conversion strategy pool, namely Strategy A (suitable for 2D trajectory display under exemplary terrain), Strategy B (suitable for 3D trajectory display under complex terrain and focusing on environmental interaction) and Strategy C (suitable for trajectory comparison display in multi-vehicle scenarios).

[0111] 2. Collaborative debugging process 1) Concepts related to debugging error indicators The spatiotemporal graph neural network and the running trajectory visualization processing network optimize each other's performance through collaborative debugging. In this collaborative debugging link, the error variable determined by the debugging error indicator used includes multiple parts.

[0112] First, the fully connected quantitative knowledge of the running trajectory of the smart electric snowmobile example is obtained through the running trajectory visualization processing network in debugging. For example, during a debugging process, there is a smart electric snowmobile example in a specific snow driving scene, and its running trajectory fully connected quantitative knowledge is a 128-dimensional vector, such as [0.1, 0.2, -0.3, ..., 0.05] (the values ​​here are only examples). This vector contains information fused from the quantitative knowledge of vehicle-mounted environment perception, the quantitative knowledge of lidar point cloud, and the sensor monitoring interaction knowledge features.

[0113] Secondly, the visual element mapping features of the visualization conversion strategy samples are obtained through the debugging spatiotemporal graph neural network. Taking strategy B as an example, its visual element mapping features may be in the form of a matrix, such as a 32*32 matrix, and the elements in the matrix represent the mapping relationship between different visual elements (such as the spatial position of the trajectory, the relationship with environmental elements, etc.) in the spatiotemporal graph neural network.

[0114] Finally, the quality evaluation of the visualization conversion strategy sample based on the smart electric snowmobile sample determines the debugging guidance information. This quality evaluation can be done in a variety of ways, such as calculating the similarity between the trajectory after visualization conversion and the actual trajectory (which can be measured by calculating the mean square error between the trajectory coordinate points, for example, the actual trajectory coordinate points are [(0, 0, 0), (1, 0, 0), (2, 0, 0.1)], and the trajectory coordinate points after visualization conversion are [(0, 0, 0), (1.1, 0, 0), (2.2, 0, 0.15)], then the mean square error is the average of the sum of the squares of the difference values ​​of the two sets of coordinate points). If the mean square error exceeds a certain threshold (such as 0.1), the quality is considered poor, and the parameters of the spatiotemporal graph neural network or the running trajectory visualization processing network need to be adjusted.

[0115] 2) The significance of collaborative debugging This collaborative debugging mechanism helps ensure that the visualization conversion strategy can accurately transform the fully connected quantitative knowledge of the target smart electric snowmobile's running trajectory into a suitable visualization. By continuously adjusting the parameters of the two networks, the visual element mapping features can better match the fully connected quantitative knowledge of the running trajectory, thereby improving the accuracy and effectiveness of the visualization.

[0116] 2. Obtaining the 3D mixed model visual annotation information set of the target visualization conversion strategy 1. Visual positioning and annotation information of tracks The visual positioning annotation information of the track is mainly used to determine the positioning information of the snowmobile running track in three-dimensional space under the target visualization conversion strategy. For example, in a 3D scene, the coordinates of the starting point of the track are (0, 0, 0) meters, and the coordinates of the end point are (10, 5, 2) meters (the coordinates here are based on a specific 3D coordinate system, the x-axis represents the forward direction, the y-axis represents the lateral direction, and the z-axis represents the vertical direction). These coordinate information can be obtained in a variety of ways, such as combining the track point information in the lidar point cloud data with the positioning algorithm.

[0117] 2. Visual element annotation information at multiple attention levels 1) Acquisition of annotation information based on 3D visual transformation task information First, the 3D visual conversion task information included in the target visualization conversion strategy is sampled. For example, for strategy B (suitable for 3D trajectory display under complex terrain and focusing on environmental interaction), its 3D visual conversion task information may include tasks such as building 3D scenes, drawing trajectories, and marking environmental elements (such as snow piles, trees, etc.). When sampling tasks, tasks can be selected according to a certain probability (such as each task has a probability of 0.33 being sampled).

[0118] For example, the two tasks of building a 3D scene and drawing a track are sampled, and the visual element annotation information at the 3D attention level is determined based on the 3D visual conversion task events obtained by task sampling. For the task of building a 3D scene, the visual element annotation information may include the scope of the scene (such as 20 meters in front, back, left, right, and 10 meters in the top and bottom, with the snowmobile as the center), the resolution of the scene (such as each cubic meter is divided into 100 small cube units for building a scene model), etc. For the task of drawing a track, the visual element annotation information may include the color of the track (such as blue), thickness (such as 1 mm at 0-5 m / s and 2 mm at 5-10 m / s according to the speed change), etc.

[0119] 2) Acquisition of annotation information based on vehicle posture conversion task mining The 3D visual transformation task events obtained by task sampling are mined for vehicle-mounted posture transformation tasks. For example, in the task of building a 3D scene, the vehicle-mounted posture transformation task event may involve the influence of the posture of the snowmobile at different positions (such as tilt angle, direction angle, etc.) on the construction of the 3D scene. For example, at a certain position, the tilt angle of the snowmobile is 5 degrees (obtained by the vehicle-mounted posture sensor), and this tilt angle will affect the coordinate transformation of the lidar point cloud data in the 3D scene.

[0120] After obtaining the vehicle posture conversion task event, the visual element annotation information at the vehicle posture attention level is determined based on the vehicle posture conversion task event and the visualized vehicle status data of the target visualization conversion strategy (such as the speed and steering angle of the snowmobile). For example, when the speed of the snowmobile is 3 meters per second and the steering angle is 10 degrees, the visual element annotation information at the vehicle posture attention level may include displaying the posture of the snowmobile in a specific way in the 3D scene (such as by marking the tilt direction and steering direction on the snowmobile model).

[0121] 3) Determine the annotation information of visual elements at multiple attention levels Based on the visual element annotation information at the 3D attention level and the visual element annotation information at the vehicle posture attention level, the visual element annotation information at multiple attention levels of the target visualization conversion strategy is determined. For example, the range and resolution of the 3D scene construction are combined with the tilt angle and steering direction of the vehicle posture to form a complete visual element annotation information set, which can be used for subsequent attention annotation feature generation.

[0122] 3. Generate Cross-Attention Annotation Features 1. Generating attention annotation features at each attention level Attention annotation features at multiple attention levels are generated based on the annotation information of each visual element and the visual positioning annotation information of the trajectory. For example, for the coordinates of the starting point and the end point in the visual positioning annotation information of the trajectory, and the scene range in the visual element annotation information at the 3D attention level, the relationship between the trajectory and the scene range is calculated by an algorithm based on distance measurement (such as the Euclidean distance algorithm), and this relationship is quantified into a vector. This vector is part of the attention annotation feature at the 3D attention level. For the vehicle posture attention level, a similar algorithm is used to generate attention annotation features based on the relative relationship between the vehicle posture and the trajectory (such as the influence of the posture angle on the trajectory direction).

[0123] (II) Fully connected processing to obtain cross-attention annotation features The attention annotation features of multiple attention levels are fully connected. For example, the attention annotation feature of the 3D attention level is a 16-dimensional vector, and the attention annotation feature of the vehicle posture attention level is a 12-dimensional vector. The full connection processing can be implemented through a fully connected layer, and the weight matrix of the fully connected layer can be initialized to a random value (such as randomly drawn from a normal distribution with a mean of 0 and a standard deviation of 0.01). After the calculation of the fully connected layer, the cross-attention annotation feature is obtained. This feature integrates the information of different attention levels and can more comprehensively describe the various element relationships under the target visualization conversion strategy.

[0124] 4. Obtaining Visual Element Mapping Features through Spatiotemporal Graph Neural Network (I) Obtaining cross-attention residual annotation features through the residual processing subnet 1) Residual processing subnet structure and principle The residual processing subnet of the spatiotemporal graph neural network is used to process cross-attention annotation features. The residual processing subnet can adopt a residual block structure similar to the ResNet architecture. For example, a residual block contains two 3*3 convolutional layers and a shortcut connection.

[0125] When the cross-attention annotation feature is input to the residual processing subnet, it first passes through the first convolution layer, and the convolution kernel slides on the feature map to perform convolution operations and extract features. For example, the step size of the convolution kernel is 1, and the padding method is the same padding, so that the size of the feature map can be kept unchanged. Then, after batch normalization, the data is normalized to a distribution with a mean of 0 and a variance of 1, and then the nonlinear characteristics are introduced through the ReLU activation function. Then, it passes through the second convolution layer for further feature extraction, and finally the input and the output of the convolution layer are added through a shortcut connection to obtain the cross-attention residual annotation feature.

[0126] 2) The significance of cross-attention residual annotation features This feature represents the remaining information or difference information after the cross-attention annotation features have been processed by residual processing. It can mine deeper feature relationships and provide more meaningful input for subsequent feature mapping.

[0127] 2. Feature Mapping via Bidirectional Long Short-Term Memory Subnetwork 1) Characteristics of Bi-LSTM The bidirectional long short-term memory subnet of the spatiotemporal graph neural network performs feature mapping on the cross-attention residual annotation features. Bi-LSTM consists of a forward LSTM and a backward LSTM. There are input gates, forget gates, and output gates inside the LSTM unit. The input gate determines which new information can enter the cell state, the forget gate determines which information in the cell state needs to be forgotten, and the output gate controls which information in the cell state can be output.

[0128] For sequence data such as cross-attention residual annotation features (for example, if the features are arranged in a certain order to form a time series), the forward LSTM processes the data from the beginning to the end of the sequence in chronological order, and the backward LSTM processes the data from the end to the beginning of the sequence. This bidirectional processing method enables Bi-LSTM to better capture long-term dependencies in sequence data.

[0129] 2) Obtain visual element mapping features After being processed by the Bi-LSTM subnet, the visual element mapping feature of the target visualization conversion strategy is obtained. This visual element mapping feature can accurately reflect the adaptation relationship between the target visualization conversion strategy and the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile, thereby providing a guarantee for the realization of high-quality running trajectory visualization display.

[0130] With this design, firstly, by co-debugging the spatiotemporal graph neural network and the running trajectory visualization processing network, it is possible to ensure the adaptation of the visual element mapping features and the fully connected quantitative knowledge of the running trajectory, thereby improving the accuracy of the visualization conversion. Secondly, the process of obtaining the three-dimensional mixed-mode visual annotation information set and generating the cross-attention annotation features comprehensively considers multiple factors such as trajectory positioning and visual elements at different attention levels, so that the visualization display can more comprehensively reflect the running status and environmental relationship of the snowmobile. Furthermore, the residual processing subnet and the bidirectional long short-term memory subnet in the spatiotemporal graph neural network are used to obtain the visual element mapping features, which can dig out the deep feature relationship, improve the quality and effectiveness of the visualization display, and provide a comprehensive, accurate and efficient technical solution for the visualization display of the running trajectory of the intelligent electric snowmobile.

[0131] On the basis of the above content, the method further includes: (1) The visualization conversion strategy management server transfers the visualization conversion strategy generated by the edge side and the visualization vehicle status data of the visualization conversion strategy to the visualization conversion strategy pool; (2) performing a strategy feasibility assessment by instructing a strategy feasibility assessment server, wherein the strategy feasibility assessment performed includes any one or both of the following: instructing the strategy feasibility assessment server to perform a visual conversion feasibility assessment on the visualization conversion strategy, instructing the strategy feasibility assessment server to perform a visual conversion feasibility assessment on the visualization vehicle status data of the visualization conversion strategy, and instructing the strategy optimization server to optimize the visualization conversion strategy.

[0132] Among them, the strategy feasibility assessment server obtains the visualization conversion strategy and / or the visualization vehicle status data of the visualization conversion strategy through the visualization conversion strategy management server; the strategy optimization server obtains the visualization conversion strategy and / or the visualization vehicle status data of the visualization conversion strategy through the visualization conversion strategy management server.

[0133] In the next step, the instruction strategy optimization server optimizes the visualization conversion strategy, including: generating visual element mapping features of each reported visualization conversion strategy through a spatiotemporal graph neural network; performing adaptability analysis based on the visual element mapping features of each reported visualization conversion strategy to obtain adaptation analysis information, wherein the adaptation analysis information represents the correlation between any two visualization conversion strategies in each reported visualization conversion strategy; merging and optimizing the reported visualization conversion strategies based on the correlation between any two visualization conversion strategies in each reported visualization conversion strategy.

[0134] Among them, when generating visual element mapping features of each reported visualization conversion strategy through the spatiotemporal graph neural network, the input information of the spatiotemporal graph neural network includes: trajectory visualization connection features of each reported visualization conversion strategy and 3D visual conversion task events obtained by task sampling of each reported visualization conversion strategy, and the trajectory visualization connection features include visualization vehicle status data identified from the visualization conversion strategy pool and vehicle posture conversion task events identified from the 3D visual conversion task events obtained from task sampling.

[0135] It can be understood that the above technical solution involves relevant design ideas for visual conversion strategy management, evaluation and optimization. The following is an exemplary introduction to these design ideas.

[0136] 1. Decentralization of Visual Conversion Strategy 1. The role of the visual conversion strategy management server The visualization conversion strategy management server plays an important role in the whole process. It is responsible for transferring the visualization conversion strategies generated by the edge side and the visualization vehicle status data of the visualization conversion strategies to the visualization conversion strategy pool. For example, the edge side may generate multiple visualization conversion strategies according to different snowmobile driving environments and requirements. For example, the edge side generates two visualization conversion strategies, namely strategy A and strategy B. Strategy A is suitable for relatively flat snow, focusing on showing the relationship between the snowmobile and the surrounding exemplary obstacles (such as a small amount of snow piles). Its visualization vehicle status data may include information such as the snowmobile speed range of 0-5 m / s and small steering angle changes; Strategy B is suitable for snowy environments with a certain slope, focusing on showing the relationship between the snowmobile's driving trajectory and the terrain slope, and its visualization vehicle status data includes information such as the snowmobile speed range of 3-8 m / s and often a certain tilt angle. The visualization conversion strategy management server accurately transfers these strategies and their related visualization vehicle status data to the visualization conversion strategy pool, providing a basis for subsequent evaluation and optimization.

[0137] 2. Strategic Feasibility Assessment 1. Operation of the Policy Feasibility Assessment Server 1) Visual conversion feasibility assessment Instruct the strategy feasibility evaluation server to perform a visual conversion feasibility evaluation on the visualization conversion strategy. This evaluation process may involve multiple aspects. For example, for strategy A that is delegated to the visualization conversion strategy pool, evaluate whether it can accurately convert the fully connected quantitative knowledge of the snowmobile's running trajectory into a visual display. A sample data-based evaluation method can be used to select a certain number (e.g., 100) of snowmobile running trajectory samples, which have known fully connected quantitative knowledge of the running trajectory and actual running trajectory conditions.

[0138] For each sample, use strategy A to perform visualization transformation, and then calculate the matching degree between the transformed visualization trajectory and the actual trajectory. The matching degree can be calculated using a variety of algorithms, such as calculating the mean absolute error (MAE) between trajectory coordinate points. For example, the coordinate points of the actual trajectory are [(0, 0, 0), (1, 0, 0), (2, 0, 0.1)], and the coordinate points of the trajectory after visualization transformation using strategy A are [(0, 0, 0), (1.1, 0, 0), (2.2, 0, 0.15)], then the mean absolute error is: (frac{|0-0|+|1-1.1|+|2-2.2|+|0-0|+|0-0|+|0.1-0.15|}{6}). If the mean absolute error exceeds a certain threshold (such as 0.1), strategy A is considered to have low feasibility in terms of visual transformation; otherwise, it is considered to have high feasibility.

[0139] 2) Feasibility assessment of visual conversion of visualized vehicle status data The strategy feasibility evaluation server is instructed to perform a visual conversion feasibility evaluation on the visualized vehicle state data of the visualization conversion strategy. Taking the visualized vehicle state data of strategy B as an example, it contains information such as the speed range and tilt angle of the snowmobile. Evaluate the feasibility of these data in the visualization conversion process. For example, check whether the visualization representation of the speed range is reasonable and whether it can accurately reflect the driving state of the snowmobile in the visualization display. The evaluation can be performed by comparing the corresponding relationship between the motion characteristics of the snowmobile model in the visualization display at different speeds (such as the visual perception of the moving speed) and the actual speed. For the visualization of the tilt angle, check whether the tilt angle of the snowmobile in the 3D visualization display matches the actual tilt angle data. The error between the tilt angle in the visualization display and the actual tilt angle can be calculated (such as using cosine similarity to measure the angle error). If the error exceeds a certain limit (such as 0.2), it is considered that there is a problem in the visual conversion of the visualized vehicle state data and the feasibility is low.

[0140] 3) Strategy optimization server operation When the policy feasibility evaluation finds a problem, the policy optimization server is instructed to optimize the visualization conversion policy. For example, when policy A performs poorly in the visualization conversion feasibility evaluation, the policy optimization server intervenes.

[0141] 3. Optimization of Visual Conversion Strategy 1. Generate visual element mapping features 1) Input information of spatiotemporal graph neural network When generating visual element mapping features of each reported visualization transformation strategy (such as strategy A and strategy B) through the spatiotemporal graph neural network, the input information of the spatiotemporal graph neural network includes the trajectory visualization connection features of each reported visualization transformation strategy and the 3D visual transformation task events obtained by task sampling for each reported visualization transformation strategy.

[0142] The trajectory visualization connection feature includes the visualized vehicle state data identified from the visualization conversion strategy pool and the vehicle posture conversion task event identified from the 3D visual conversion task event obtained from the task sampling. Taking strategy A as an example, for example, the speed in the visualized vehicle state data identified from the visualization conversion strategy pool is 3 meters per second, and this speed information is used as part of the trajectory visualization connection feature. Among the 3D visual conversion task events obtained from the task sampling, if the sampled task is to build a 3D scene around the snowmobile, the identified vehicle posture conversion task event may be that the steering angle of the snowmobile at the current speed is 5 degrees, and this steering angle information is also included in the trajectory visualization connection feature.

[0143] 2) The generation process of visual element mapping features The spatiotemporal graph neural network processes these input information to generate visual element mapping features. For example, the spatiotemporal graph neural network adopts a multi-layer structure, the first layer is a convolution layer, the convolution kernel size is 3*3, the step size is 1, and the number of channels is 16. After the input trajectory visualization connection features and 3D visual transformation task events are processed by the convolution layer, preliminary features are extracted. Then, downsampling is performed through the pooling layer (such as the maximum pooling layer, the pooling window size is 2*2) to reduce the amount of data while retaining key features. Then, further feature fusion and abstraction are performed through the fully connected layer (for example, the number of neurons in the fully connected layer is 128), and finally the visual element mapping features are obtained. For strategy A and strategy B, their respective visual element mapping features are obtained. For example, the visual element mapping feature of strategy A may be a 64-dimensional vector, and the visual element mapping feature of strategy B may be an 80-dimensional vector.

[0144] 2. Adaptability analysis and merger optimization 1) Adaptability analysis The adaptability analysis is performed based on the visual element mapping features of each reported visualization conversion strategy to obtain the adaptation analysis information. The adaptation analysis information represents the correlation between any two visualization conversion strategies in each reported visualization conversion strategy. For example, the correlation between the visual element mapping features of strategy A and strategy B is calculated. The cosine similarity algorithm can be used to calculate the similarity between two vectors (i.e., the visual element mapping feature vectors of strategy A and strategy B). For example, the visual element mapping feature vector of strategy A is (a=[0.1, 0.2, -0.3, cdots, 0.05]), and the visual element mapping feature vector of strategy B is (b=[0.2, 0.3, -0.2, cdots, 0.1]), then the cosine similarity between them is (frac{acdotb}{|a|*|b|}). The correlation between the two is judged based on this similarity value. If the similarity is high (such as greater than 0.8), it means the correlation is strong; if the similarity is low (such as less than 0.3), it means the correlation is weak.

[0145] 2) Merge optimization The reported visualization conversion strategies are merged and optimized according to the correlation between any two visualization conversion strategies in each reported visualization conversion strategy. If the correlation between strategy A and strategy B is strong, it means that they are similar in some aspects and can be merged and optimized. For example, the parts about 3D scene construction in strategy A and strategy B are merged to take the advantages of both. If strategy A is more accurate in building a 3D model of an exemplary obstacle, and strategy B is better in showing the relationship between the snowmobile and the terrain, then in the merged and optimized strategy, the advantages of these two parts are integrated to obtain a more complete visualization conversion strategy, thereby improving the effect and efficiency of visualization conversion.

[0146] In this way, firstly, by delegating strategies and data through the visualization conversion strategy management server, the effective connection between the edge side and the visualization conversion strategy pool is realized, ensuring the effective integration of data and strategies. Secondly, the strategy feasibility evaluation server evaluates the visualization conversion strategy and its visualization vehicle status data, which can comprehensively and accurately identify possible problems and improve the quality of visualization conversion. Furthermore, the strategy optimization server performs adaptability analysis and merge optimization based on the visual element mapping characteristics of the spatiotemporal graph neural network, which can make full use of the advantages of different strategies, reduce redundancy, and improve the effectiveness and adaptability of the visualization conversion strategy, thereby providing better quality and more efficient strategy support for the visualization display of the running trajectory of the smart electric snowmobile.

[0147] In some scalable examples, the running trajectory visualization processing network includes a vector migration subnet, and the vector migration subnet is used to perform vector space migration on the original mixed-mode sensor monitoring information, and the original mixed-mode sensor monitoring information includes: vehicle-mounted environment perception sensor monitoring information, first laser radar point cloud sensor monitoring information, and any one or more of the mixed-mode sensor monitoring information in the associated mixed-mode sensor monitoring information set. When the original mixed-mode sensor monitoring information belongs to mixed-mode sensor monitoring information of a mutually exclusive relationship, the original mixed-mode sensor monitoring information is vector-space migrated through the vector migration subnet; when the original mixed-mode sensor monitoring information belongs to mixed-mode sensor monitoring information of an affiliation relationship, the original mixed-mode sensor monitoring information is vector-space migrated through the vector migration subnet, and the migration features obtained after the vector space migration are subjected to feature pyramid conversion, and the size of the migration features obtained after the feature pyramid conversion is a preset size.

[0148] In this embodiment, the vector migration subnet in the running trajectory visualization processing network is used to perform vector space migration on the original mixed-mode sensor monitoring information. When the original mixed-mode sensor monitoring information belongs to mixed-mode sensor monitoring information with a mutually exclusive relationship, the vector space migration is performed directly through the vector migration subnet. For example, the camera image data and the laser radar point cloud data in the vehicle-mounted environment perception sensor monitoring information can be regarded as a mutually exclusive relationship (the camera provides visual images, and the laser radar provides point cloud information. The two have essential differences in the way and content of obtaining information). For camera image data (for example, the image resolution is 1920*1080 pixels, stored in RGB three channels, the data volume is large and the features are complex), the vector migration subnet uses a specific algorithm (such as an algorithm based on a multi-layer perceptron MLP) to perform vector space migration. For example, MLP has 3 hidden layers, and the number of neurons is 1024, 512, and 256 respectively. The feature vector (high-dimensional vector after expansion) of the image data is input into the MLP, and the image data is migrated from the original feature space to the new vector space through the linear transformation and activation function (such as ReLU activation function) of each layer to extract features that are more conducive to subsequent processing.

[0149] When the original mixed-mode sensor monitoring information belongs to the mixed-mode sensor monitoring information of the belonging relationship, the vector space migration is also performed through the vector migration subnet, and then the migration features obtained after the vector space migration are converted into feature pyramids. For example, the same type of sensor data of different associated vehicles in the associated mixed-mode sensor monitoring information set (such as the lidar point cloud data of multiple associated vehicles) can be regarded as the belonging relationship. Taking the lidar point cloud data of a certain associated vehicle as an example, the migration features are obtained through the vector migration subnet (using an algorithm similar to the above-mentioned MLP-based algorithm or other suitable algorithms). For example, the migration feature is a vector with a dimension of 512.

[0150] Then, the migration feature is converted into a feature pyramid. The size of the migration feature obtained after the feature pyramid conversion is the preset size. For example, the preset size is 128 dimensions. The feature pyramid conversion process may use a hierarchical feature extraction and compression algorithm, such as first dividing the 512-dimensional migration feature into multiple sub-feature sets according to a certain rule (such as based on feature importance or spatial distribution), and then performing dimensionality reduction processing on each sub-feature set (the principal component analysis PCA algorithm can be used to retain a certain proportion of principal components, such as 80% of the principal components), and finally recombining the processed sub-feature sets into 128-dimensional migration features to meet the preset size requirements. This processing method can reduce the data dimension while maintaining key information and improve the efficiency of subsequent processing.

[0151] It can be seen that for the mixed-mode sensor monitoring information with mutually exclusive relationships, the vector space migration of the vector migration subnet helps to extract their respective effective features. For example, the camera image and lidar point cloud data can be better used for subsequent trajectory visualization related processing after processing. For the mixed-mode sensor monitoring information with affiliation relationships, the vector space migration is combined with the feature pyramid transformation to reduce the data dimension while retaining key information, and output the migration features with a preset size. This not only improves the data processing efficiency, but also enables subsequent operations based on these features (such as other subnet processing in the trajectory visualization processing network) to be performed more efficiently and accurately, which helps to improve the overall performance of the visualization display of the running trajectory of the smart electric snowmobile.

[0152] In an expandable embodiment, the associated mixed-mode sensor monitoring information set includes X associated mixed-mode sensor monitoring information corresponding to X associated intelligent electric snowmobiles, and obtaining the associated mixed-mode sensor monitoring information set of the associated intelligent electric snowmobile of the target intelligent electric snowmobile includes: obtaining the second laser radar point cloud sensor monitoring information of any associated intelligent electric snowmobile among the X associated intelligent electric snowmobiles, and the vehicle-borne cross-sensing monitoring information between any associated intelligent electric snowmobile and the target intelligent electric snowmobile, the vehicle-borne cross-sensing monitoring information including path intersection area features and signal intersection area features; determining the second laser radar point cloud sensor monitoring information and the vehicle-borne cross-sensing monitoring information as the associated mixed-mode sensor monitoring information corresponding to any associated intelligent electric snowmobile.

[0153] In detail, in the construction of the associated mixed-mode sensor monitoring information set, for any associated intelligent electric snowmobile among the X associated intelligent electric snowmobiles, its second laser radar point cloud sensor monitoring information is first obtained. The laser radar point cloud data can provide three-dimensional structural information about the surrounding environment. For example, the laser radar scans at a certain scanning frequency (such as 10Hz), and each scan obtains point cloud data within a certain range around (such as 20 meters). These point cloud data represent the position of surrounding objects in the form of three-dimensional coordinates. By analyzing a large amount of point cloud data, detailed information on the terrain, obstacles, etc. around the associated intelligent electric snowmobile can be obtained.

[0154] At the same time, vehicle-mounted cross-sensing monitoring information between any associated smart electric snowmobile and the target smart electric snowmobile is obtained, which includes path intersection area characteristics and signal intersection area characteristics.

[0155] Among them, the path intersection area feature is used to determine whether there is an intersection area between the paths of the two vehicles. For example, through the driving trajectory planning data of the two vehicles, if the planned trajectory of the target smart electric snowmobile is from the coordinate point (0, 0, 0) to (10, 5, 0) (the coordinates here are based on a certain global coordinate system, the x-axis represents the forward direction, the y-axis represents the lateral direction, and the z-axis represents the vertical direction), the planned trajectory of the associated smart electric snowmobile is from the coordinate point (5, -3, 0) to (15, 2, 0), by calculating the intersection of the two trajectories (the calculation method of straight line intersection in analytic geometry can be used), if there is an intersection, the path intersection area feature is determined.

[0156] The signal intersection area feature is used to consider the regional features of the signal interaction between two vehicles. For example, when wireless communication (such as Wi-Fi communication, the communication frequency band is 2.4GHz) is used between the two vehicles, the effective signal coverage range is determined according to the signal propagation model (such as the Friis transmission formula, by calculating the transmit power, receive power, antenna gain and other parameters). If the signal coverage ranges of the two vehicles overlap, this overlapping area is the signal intersection area feature.

[0157] The acquired second laser radar point cloud sensing monitoring information and the vehicle-mounted cross sensing monitoring information are determined as the associated mixed-mode sensing monitoring information corresponding to any associated smart electric snowmobile. In this way, for each associated smart electric snowmobile, associated mixed-mode sensing monitoring information containing its own laser radar point cloud information and cross information between it and the target smart electric snowmobile can be constructed, and this information will be used for subsequent operations such as operation trajectory visualization processing.

[0158] In this way, the associated mixed-mode sensor monitoring information is comprehensively constructed. By obtaining the laser radar point cloud information of the associated smart electric snowmobile, the perception data of the surrounding environment can be enriched. The path intersection area features in the on-board cross-sensing monitoring information can accurately determine whether the two vehicles are likely to meet or affect each other through trajectory calculation, which is of great significance in actual trajectory planning and collision avoidance. The signal intersection area features determine the signal interaction area based on the signal propagation model, which helps to analyze the communication status and coordinate the interaction between the two vehicles. This information combined provides a more comprehensive and accurate data basis for the visualization of the operation trajectory of the smart electric snowmobile.

[0159] By applying the above technical solution of the embodiment of the present invention, firstly, by acquiring the mixed-mode sensor monitoring information set of the target intelligent electric snowmobile and its associated intelligent electric snowmobiles, including multi-source information such as vehicle-mounted environmental perception sensor monitoring information and laser radar point cloud sensor monitoring information, it is possible to comprehensively cover the surrounding environment conditions and self-state information of the snowmobile during operation. The integration of this multi-source information lays a solid data foundation for the subsequent accurate trajectory visualization display.

[0160] In the process of information processing, the operation trajectory visualization processing network plays an important role. Among them, the vehicle-mounted environment perception mining subnet and the lidar point cloud mining subnet mine the corresponding sensor monitoring information respectively to obtain the vehicle-mounted environment perception quantitative knowledge and lidar point cloud quantitative knowledge. This helps to transform complex raw sensor data into more representative and analyzable quantitative knowledge, thereby extracting key environmental and target features. For example, the vehicle-mounted environment perception quantitative knowledge can accurately reflect the object type and road conditions around the snowmobile, and the lidar point cloud quantitative knowledge can accurately describe the terrain undulations, obstacle locations and other three-dimensional spatial information around the snowmobile.

[0161] The knowledge feature interaction subnet further obtains sensor monitoring interaction knowledge features. This process comprehensively considers the target vehicle's on-board environmental perception information and the mixed-mode sensor monitoring information of the associated vehicle, and can capture the interaction between vehicles and between vehicles and the environment. For example, in scenarios where multiple vehicles are driving in coordination or snowmobiles have complex interactions with the surrounding environment (such as passing through narrow passages or approaching other vehicles), this interactive knowledge feature can more comprehensively reflect the overall situation and improve the accuracy and practicality of trajectory visualization.

[0162] The fully connected subnet performs fully connected processing on multiple quantitative knowledge to obtain fully connected quantitative knowledge of the running trajectory. This processing method can integrate knowledge from different sources and types and fully explore the internal connections between them. The final fully connected quantitative knowledge of the running trajectory can comprehensively reflect various factors related to the snowmobile running trajectory, including environmental factors, the vehicle's own state, and the interaction between vehicles.

[0163] Finally, based on the fully connected quantitative knowledge of the running trajectory, the target visualization conversion strategy is obtained from the visualization conversion strategy pool, and the most suitable visualization method can be selected to display the running trajectory of the snowmobile according to the actual situation. This ensures the pertinence and effectiveness of the visualization display. For example, in different driving scenarios (such as open snowfields, complex mountain snowfields, etc.) and different task requirements (such as safety monitoring, path planning display, etc.), the most suitable visualization display can be provided, thereby providing strong support for the operation management, monitoring and decision-making of intelligent electric snowmobiles.

[0164] Further, Figure 2 FIG. 2 is a schematic diagram of a structure of a visual display system 200 provided in an embodiment of the present invention. Figure 2 The visual display system 200 shown includes a processor 210, and the processor 210 can call and run a computer program from a memory to implement the method in the embodiment of the present invention.

[0165] Alternatively, if Figure 2As shown, the visualization display system 200 may further include a memory 230. The processor 210 may call and run a computer program from the memory 230 to implement the method in the embodiment of the present invention.

[0166] The memory 230 may be a separate device independent of the processor 210, or may be integrated into the processor 210. Figure 2 As shown, the visualization display system 200 may also include a transceiver 220, and the processor 210 may control the transceiver 220 to interact with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. Optionally, the visualization display system 200 may implement the corresponding process corresponding to the storage engine or the components (such as processing modules) in the storage engine or the device deployed with the storage engine in each method of the embodiment of the present invention, and for the sake of brevity, it will not be repeated here. It should be understood that the processor of the embodiment of the present invention may be an integrated circuit chip with signal processing capabilities. It can be understood that the memory in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. It should be noted that the memory of the system and method described herein is intended to include but is not limited to a memory of a suitable type. On the basis of the above, a readable storage medium is provided, on which a program or instruction is stored, and the steps of the above method are implemented when the program or instruction is executed by the processor.

[0167] The embodiments of the embodiments of the present invention are described above in conjunction with the accompanying drawings, but the embodiments of the present invention are not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the embodiments of the present invention, ordinary technicians in this field can also make many forms without departing from the purpose of the embodiments of the present invention and the scope of protection of the embodiments of the present invention, all of which are within the protection of the embodiments of the present invention.

Claims

1. A method for visualizing the running trajectory of an autonomous driving vehicle, characterized in that: include: Obtaining a target mixed-mode sensor monitoring information set of a target intelligent electric snowmobile to be visualized for running trajectory display and an associated mixed-mode sensor monitoring information set of an associated intelligent electric snowmobile of the target intelligent electric snowmobile, wherein the target mixed-mode sensor monitoring information set includes: vehicle-mounted environment perception sensor monitoring information and first laser radar point cloud sensor monitoring information; The vehicle environment perception quantified knowledge of the vehicle environment perception sensor monitoring information is obtained by running the vehicle environment perception mining subnet of the trajectory visualization processing network, and the lidar point cloud quantified knowledge of the first lidar point cloud sensor monitoring information is obtained by running the lidar point cloud mining subnet of the trajectory visualization processing network; Obtaining sensor monitoring interaction knowledge features corresponding to the vehicle environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set by running the knowledge feature interaction subnet of the trajectory visualization processing network; The fully connected subnet of the running trajectory visualization processing network performs full connection processing on the vehicle environment perception quantitative knowledge, the laser radar point cloud quantitative knowledge and the sensor monitoring interaction knowledge features to obtain the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile; Based on the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile, a target visualization conversion strategy is obtained from a visualization conversion strategy pool; the target visualization conversion strategy is used to realize the visualization display of the running trajectory of the target intelligent electric snowmobile.

2. The method according to claim 1, characterized in that The visualization conversion strategy pool includes multiple visualization conversion strategies, each visualization conversion strategy generates a corresponding visual element mapping feature through a spatiotemporal graph neural network, and the visual element mapping feature of the target visualization conversion strategy obtained from the visualization conversion strategy pool is adapted to the fully connected quantitative knowledge of the running trajectory of the target intelligent electric snowmobile; The spatiotemporal graph neural network and the running trajectory visualization processing network are obtained through collaborative debugging. The error variables determined by the debugging error indicators used in the collaborative debugging link include: the fully connected quantitative knowledge of the running trajectory of the intelligent electric snowmobile sample obtained by the running trajectory visualization processing network during debugging, the visual element mapping characteristics of the visualization conversion strategy sample obtained by the spatiotemporal graph neural network during debugging, and the debugging guidance information determined by the quality evaluation of the visualization conversion strategy sample based on the intelligent electric snowmobile sample.

3. The method according to claim 2, characterized in that The method further comprises: Acquire a three-dimensional mixed-mode visual annotation information set of a target visualization conversion strategy, wherein the three-dimensional mixed-mode visual annotation information set includes: trajectory visual positioning annotation information and visual element annotation information at multiple attention levels; Generate multiple attention annotation features at multiple attention levels based on the annotation information of each visual element and the trajectory visual positioning annotation information, and perform full connection processing on the attention annotation features at the multiple attention levels to obtain cross-attention annotation features; Obtaining a cross-attention residual annotation feature corresponding to the cross-attention annotation feature through a residual processing subnet of the spatiotemporal graph neural network; The cross-attention residual annotation features are feature mapped through the bidirectional long short-term memory subnet of the spatiotemporal graph neural network to obtain the visual element mapping features of the target visualization conversion strategy; the target visualization conversion strategy is any one of the multiple visualization conversion strategies.

4. The method according to claim 3, characterized in that Obtain visual element annotation information at multiple attention levels of the target visualization conversion strategy, including: Performing task sampling on the 3D visual conversion task information included in the target visualization conversion strategy, and determining visual element annotation information at the 3D attention level according to the 3D visual conversion task events obtained by the task sampling; Performing vehicle posture conversion task mining on the 3D visual conversion task events obtained by the task sampling to obtain vehicle posture conversion task events, and determining visual element annotation information at the vehicle posture attention level based on the vehicle posture conversion task events and the visualized vehicle status data of the target visualization conversion strategy; Based on the visual element annotation information at the 3D attention level and the visual element annotation information at the vehicle posture attention level, the visual element annotation information at multiple attention levels of the target visualization conversion strategy is determined.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The visualization conversion strategy management server is used to store the visualization conversion strategy generated by the edge side and the visualization vehicle status data of the visualization conversion strategy into the visualization conversion strategy pool; Performing a strategy feasibility assessment by instructing a strategy feasibility assessment server, wherein the performed strategy feasibility assessment includes any one or two of the following: instructing the strategy feasibility assessment server to perform a visual conversion feasibility assessment on the visualization conversion strategy, instructing the strategy feasibility assessment server to perform a visual conversion feasibility assessment on the visualization vehicle status data of the visualization conversion strategy, and instructing the strategy optimization server to optimize the visualization conversion strategy; Among them, the strategy feasibility assessment server obtains the visualization conversion strategy and / or the visualization vehicle status data of the visualization conversion strategy through the visualization conversion strategy management server; the strategy optimization server obtains the visualization conversion strategy and / or the visualization vehicle status data of the visualization conversion strategy through the visualization conversion strategy management server.

6. The method according to claim 5, characterized in that The instructing strategy optimization server to optimize the visualization conversion strategy includes: Generate visual element mapping features of each reported visualization transformation strategy through spatiotemporal graph neural network; Performing adaptability analysis according to visual element mapping features of each reported visualization conversion strategy to obtain adaptability analysis information, wherein the adaptability analysis information indicates the correlation between any two visualization conversion strategies in each reported visualization conversion strategy; The reported visualization conversion strategies are merged and optimized according to the correlation between any two visualization conversion strategies among the reported visualization conversion strategies; Among them, when generating visual element mapping features of each reported visualization conversion strategy through the spatiotemporal graph neural network, the input information of the spatiotemporal graph neural network includes: trajectory visualization connection features of each reported visualization conversion strategy and 3D visual conversion task events obtained by task sampling of each reported visualization conversion strategy, and the trajectory visualization connection features include visualization vehicle status data identified from the visualization conversion strategy pool and vehicle posture conversion task events identified from the 3D visual conversion task events obtained from task sampling.

7. The method according to claim 1, characterized in that The running trajectory visualization processing network includes a vector migration subnet, the vehicle environment perception mining subnet includes a bidirectional long short-term memory subnet, and the vehicle environment perception quantitative knowledge of the vehicle environment perception sensor monitoring information obtained by the vehicle environment perception mining subnet of the running trajectory visualization processing network includes: The vehicle environment migration features obtained by performing vector space migration on the vehicle environment perception sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network are used as inputs of the vehicle environment perception mining subnet; The vehicle environment migration characteristics are quantified through the bidirectional long short-term memory subnet of the vehicle environment perception mining subnet to obtain the vehicle environment perception quantified knowledge of the vehicle environment perception sensor monitoring information.

8. The method according to claim 1, characterized in that The running trajectory visualization processing network includes a vector migration subnet, and the laser radar point cloud quantification knowledge of the first laser radar point cloud sensing monitoring information obtained by the laser radar point cloud mining subnet of the running trajectory visualization processing network includes: Using multiple laser radar point cloud migration features of the first laser radar point cloud sensor monitoring information obtained by the vector migration subnet of the running trajectory visualization processing network as inputs of the laser radar point cloud mining subnet of the running trajectory visualization processing network, wherein the first laser radar point cloud sensor monitoring information includes multiple laser radar scanning information obtained by sequentially sorting according to scanning cycles; Processing the migration features of multiple lidar point clouds by running the lidar point cloud mining subnet of the trajectory visualization processing network to obtain lidar point cloud quantitative knowledge of the first lidar point cloud sensing monitoring information; The laser radar point cloud mining subnet of the running trajectory visualization processing network obtains the laser radar point cloud quantitative knowledge of the first laser radar point cloud sensing monitoring information according to a plurality of laser radar scanning information whose scanning cycles meet the step length requirements; Alternatively, the lidar point cloud mining subnet of the running trajectory visualization processing network obtains multiple three-dimensional scene drawing element features corresponding to the multiple lidar point cloud migration features through the residual processing subnet of the lidar point cloud mining subnet, and processes the multiple three-dimensional scene drawing element features through the feature downsampling branch of the lidar point cloud mining subnet to obtain the lidar point cloud quantitative knowledge of the first lidar point cloud sensor monitoring information.

9. The method according to claim 1, characterized in that The associated mixed-mode sensor monitoring information set includes X associated mixed-mode sensor monitoring information corresponding to X associated intelligent electric snowmobiles, where X is a positive integer; the running trajectory visualization processing network includes a vector migration subnet, and the knowledge feature interaction subnet includes X feature enhancement branches, a residual processing subnet, and a feature downsampling branch; The sensor monitoring interaction knowledge features corresponding to the vehicle environment perception sensor monitoring information and the associated mixed-mode sensor monitoring information set are obtained by running the knowledge feature interaction subnet of the trajectory visualization processing network, including: Using the vehicle environment migration features obtained by performing vector space migration on the vehicle environment perception sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network, and the X associated migration features obtained by performing vector space migration on the X associated mixed-mode sensor monitoring information through the vector migration subnet of the running trajectory visualization processing network, as inputs of the knowledge feature interaction subnet; Processing the vehicle environment migration feature and the X associated migration features through the X feature enhancement branches of the knowledge feature interaction subnet to obtain X associated vehicle environment enhancement features; Obtaining the associated residual annotation features corresponding to the X associated vehicle environment enhancement features through the residual processing subnet of the knowledge feature interaction subnet; Processing the associated residual annotation features through the feature downsampling branch of the knowledge feature interaction subnet to obtain the sensor monitoring interaction knowledge features; The processing of the vehicle environment migration feature and the X associated migration features through the X feature enhancement branches of the knowledge feature interaction subnet to obtain X associated vehicle environment enhancement features includes: Obtaining a visible layer overlay feature between the vehicle environment migration feature and the u-th associated migration feature among the X associated migration features, where u is a positive integer not greater than X; Integrate the vehicle environment migration feature, the u-th associated migration feature and the visible layer overlay feature, and input the result obtained after the integration into the weighted processing module of the u-th feature enhancement branch among the X feature enhancement branches to obtain the output of the weighted processing module of the u-th feature enhancement branch; The output of the weighted processing module of the u-th feature enhancement branch is used as the input of the normalization module of the u-th feature enhancement branch, and the normalization module of the u-th feature enhancement branch outputs the u-th associated vehicle environment enhancement feature.

10. A visual display system, characterized in that: The method comprises at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method according to any one of claims 1 to 9.

Images