Automatic driving decision-making method, system and device and medium

Abstract

The invention discloses an automatic driving decision-making method, system and device and a medium. The method comprises the following steps: acquiring multi-modal information, encoding the multi-modal information into a unified scene representation through a liquid embedding technology, and visualizing the multi-modal information into a whole through volume rendering; field parameters are estimated in real time through a liquid time constant neural network LTC so as to dynamically update field information, and a recurrent neural network RNN is introduced into the liquid time constant neural network LTC so that the network can dynamically adapt to the internal state and weight when processing different tasks and time sequence data; planning and optimizing a driving path of the vehicle based on the field information updated in real time; the environment is perceived and understood in real time in the driving process, and environment information is extracted; and matching the environment information sensed in real time with the planned and optimized driving path so as to determine the action to be taken in actual driving. According to the method, multi-modal information can be effectively processed, and real-time decision making and efficient path planning are realized.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving path planning, and in particular relates to an autonomous driving decision-making method, system, device and medium. Background Art

[0002] Current autonomous driving technology is based on pre-built maps and real-time data from different sensors to build a high-precision model of the vehicle's surroundings and make real-time decisions. However, semantic maps are subject to high computing resource and energy requirements, insufficient real-time inference capabilities, and the susceptibility of models from fixed information sources to structurally limited performance scalability.

[0003] Traditional autonomous driving path planning methods have problems such as poor dynamic scalability (difficulty in dynamically introducing new modes or dynamically scaling model performance), high GPU computing requirements, and susceptibility to accidents. Summary of the invention

[0004] In order to solve at least one of the problems existing in the prior art, the present invention provides an information fusion decision-making method, system, device and medium based on autonomous driving technology to improve the safety and efficiency of the autonomous driving system.

[0005] In order to achieve the purpose of the present invention, the present invention provides an automatic driving decision-making method, comprising the following steps:

[0006] Acquire multimodal information, encode it into a unified scene representation through liquid embedding technology, and visualize it as a whole through volume rendering;

[0007] Real-time estimation of field parameters by liquid time constant neural network LTC to dynamically update field information, wherein a recurrent neural network RNN ​​is introduced into the liquid time constant neural network LTC to enable the network to dynamically adapt internal states and weights when processing different tasks and time series data;

[0008] Plan and optimize vehicle routes based on real-time updated field information;

[0009] Real-time perception and understanding of the environment while driving, and extraction of environmental information;

[0010] Match the real-time perceived environmental information with the planned optimized driving path to determine the actions to be taken during actual driving.

[0011] The present invention also provides an automatic driving decision system, the system comprising the following modules:

[0012] The multimodal information embedding module is used to obtain multimodal information, encode the multimodal information into a unified scene representation through liquid embedding technology, and visualize the multimodal information as a whole through volume rendering;

[0013] A real-time field parameter estimation module, used for estimating field parameters in real time through a liquid time constant neural network LTC to dynamically update field information, wherein a recurrent neural network RNN ​​is introduced into the liquid time constant neural network LTC to enable the network to dynamically adapt internal states and weights when processing different tasks and time series data;

[0014] Path planning and optimization module, used to plan and optimize the vehicle's driving path based on real-time updated field information;

[0015] The environment modeling and information extraction module is used to perceive and understand the environment in real time while driving, and to extract environmental information;

[0016] The path information acquisition module is used to match the real-time perceived environmental information with the planned optimized driving path to determine the actions to be taken during actual driving.

[0017] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0018] The present invention also provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0019] Compared with the prior art, the present invention can at least achieve the following beneficial effects:

[0020] (1) The present invention provides an innovative autonomous driving information fusion decision-making method that can effectively process multimodal information, achieve real-time decision-making and efficient path planning. By integrating multiple technologies and methods, the present invention has broad application prospects in the field of autonomous driving and can provide important impetus for the development and implementation of autonomous driving technology.

[0021] (2) The present invention predicts the grid field and weight field and combines the cosine transform of spherical harmonics to model the environment, fuse multimodal information, and make real-time decisions through a scene rendering algorithm.

[0022] (3) The present invention can adjust the computing occupancy rate of the system by adjusting the LTC network parameters, thereby solving the problem of high GPU computing requirements in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a step diagram of an automatic driving decision-making method provided by an embodiment of the present invention.

[0024] Figure 2It is a schematic diagram of the process flow in one time step in an embodiment of the present invention.

[0025] Figure 3 Schematic diagram of specific steps of a volume rendering algorithm in an embodiment of the present invention.

[0026] Figure 4 It is a visualized schematic diagram of the weight field and the grid field in an embodiment of the present invention.

[0027] Figure 5 It is a schematic diagram of the composition of an automatic driving decision-making system in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The core idea of ​​the invention is to integrate the various information acquired by the sensors into a unified field and transformation vector. This field can be regarded as an abstract representation that can accommodate multimodal information and is processed through volume rendering operations. In this way, the system is able to extract information about the vehicle's surroundings from different sensor sources and incorporate it into a unified framework for analysis and decision-making.

[0030] During the implementation, multimodal information is embedded into the field, which enables data from different sensors to be processed in a unified scene. Then, through technologies such as liquid embedding and volume rendering, the system can update scene information and make decision inferences in real time. These steps enable the system to make accurate decisions based on different scenes and real-time data, and realize autonomous navigation and path planning.

[0031] See also Figure 1 , the present invention provides an automatic driving decision-making method, comprising the following steps:

[0032] Step 1: Acquire multimodal information, encode it into a unified scene representation through liquid embedding technology, and visualize the multimodal information as a whole through volume rendering.

[0033] This step uses liquid embedding and volume rendering technology to efficiently embed multi-source information from various sensors into a unified scene, achieve efficient representation and extraction of multimodal information, and realize the fusion of multimodal information, which can provide strong information support for the system's intelligent decision-making.

[0034] The multimodal information is information from different sensors. In some embodiments of the present invention, the sensors include but are not limited to cameras, road signs, and lidar.

[0035] In step 1.2, the system uses liquid embedding technology and volume rendering to efficiently embed multi-source information from various sensors into a unified scene to form a weight field matrix and a grid field matrix.

[0036] The weight field matrix and grid field matrix are generated and updated by the following steps:

[0037] (1) Generation and updating of weight field matrix: Various factors in the environment (such as wind direction, terrain, traffic flow, etc.) are mapped into the weight field to form a weight field matrix that reflects the impact of the environment. The weight field is updated by the system continuously sensing and analyzing changes in the environment, thereby dynamically adjusting the weight field matrix to reflect the latest environmental status.

[0038] (2) Generation and update of grid field matrix: During the environmental modeling process, the continuous environment is discretized into discrete grids, each grid representing a local area in the environment. The grid field matrix is ​​generated by combining the information of the weight field and the gridding process, which enables the path planning algorithm to operate more efficiently on the discrete grid. The update of the grid field depends on the perception and changes of the environment to ensure that the grid matrix can accurately reflect the current environmental characteristics.

[0039] This step includes the following sub-steps:

[0040] Step 1.1: receiving multimodal data from each sensor and preprocessing the multimodal data to ensure data quality;

[0041] Step 1.2: Using liquid embedding technology, multiple information is integrated into a unified scene representation through nonlinear transformation to achieve efficient integration of different data sources;

[0042] In step 1.2, modeling is performed through the environmental modeling method of weight field and grid field, including:

[0043] Step 1.2.1: Map various factors in the environment (such as wind direction, terrain, traffic flow, etc.) to the weight field to form a weight field matrix that reflects the impact of the environment.

[0044] The design of the weight field takes into account various factors of vehicle driving and provides an effective reference for path planning.

[0045] Step 1.2.2: Discretize the environment modeling and divide the continuous environment into discrete grids to construct a grid field. Each grid represents a local area in the environment.

[0046] In this step, the establishment of the grid field not only considers the discrete representation of the environment, but also combines the information of the weight field. This discretization allows the path planning algorithm to be easier to operate and achieves efficient execution of path planning by optimizing the mapping of the vehicle's motion on the discrete grid. Discretize the environmental modeling and divide the continuous environment into discrete grids, each grid representing a local area in the environment. The establishment of the grid field not only considers the discrete representation of the environment, but also first establishes the weight field in step 1.2.1. This weight field reflects various factors in the environment, such as wind direction, terrain, traffic flow, etc. It forms a weight field matrix that reflects the impact of the environment. Therefore, when the grid field is established, the information of the weight field will be combined when considering the discrete representation of the environment, so that the degree of influence of different areas can be adjusted according to the weight information in the weight field.

[0047] Step 1.2.3: Efficiently abstract the autonomous driving environment through weight transformation and grid transformation.

[0048] In the grid transformation, the polar coordinate representation of the input point on the ellipsoid is transformed by the cosine transform of the spherical harmonic function. In this process, the system uses the embedding matrix and field state matrix of the previous moment to calculate the transformation value, which reflects the degree of change of the characteristics of the corresponding point in the environment.

[0049] The embedding matrix and field state matrix are generated during the liquid embedding process. The field state matrix is ​​generated during the liquid embedding process to reflect the state of the environment at the previous moment and is used in the weight transformation and grid transformation processes to ensure that the system can capture the dynamic changes of the environment. Through liquid embedding technology, the embedding matrix and field state matrix of the previous moment are used to encode environmental information and are used in the spherical harmonic function cosine transform process. The embedding matrix and field state matrix are used to calculate the transformation value, reflecting the degree of change of the characteristics of the corresponding points in the environment.

[0050] In the weight transformation stage, the system also uses the embedding matrix and field state matrix of the previous moment, adopts the environment modeling method of weight field and grid field, and calculates the weight transformation value through the cosine transformation of spherical harmonic function, which reflects the weight change of points in the environment. These transformed values ​​provide an efficient abstract representation of the autonomous driving environment.

[0051] The use of embedding matrix and field state matrix ensures that the system takes into account the dynamic information of the previous moment in the process of abstract representation of the environment, providing more accurate environmental understanding and description capabilities for subsequent real-time decision-making and path planning. Through fine grid transformation and weight transformation, the system can capture the key features of multimodal information, improve the system's adaptability and decision-making ability, and enable it to more effectively cope with complex driving scenarios.

[0052] Wherein, the spherical harmonic function cosine transform comprises:

[0053] The following deformation is applied to the surface of the field plane, which is complete for the transformation from the original ellipsoid to any closed 3D structure:

[0054] Assume that the input point is represented by polar coordinates (θ, φ) on the ellipsoid, where θ is the polar angle and φ is the azimuth. Select a given center direction (θ c ,φ c ), θ c is the polar angle in the direction of the center, φ c Indicates the azimuth of the center direction.

[0055] Define the transformation function T(θ, φ, θ c ,φ c , s), where s is the transformation strength parameter, and the transformation function is calculated by the following formula:

[0056]

[0057] in:

[0058] cos(θ-θ c ) represents the input point (θ, φ) and the center direction (θ c ,φ c ) is the cosine of the angle difference between s Express Perform s-th power operation;

[0059] In this way, the transformation function T(θ, φ, θ) expressed by the formula c ,φ c , s) can be calculated based on the input point (θ, φ) and the given center direction (θ c ,φ c ) and the transformation strength parameter s to calculate the transformation value. The result of this function is in the interval [0, 1], and as the transformation strength parameter s increases, the transformation value gradually decays. This is an important source of model nonlinearity, indicating that it has the potential to become a successor to traditional neural networks.

[0060] The polar coordinate representation of the input point on the ellipsoid can be transformed by the spherical harmonic function cosine transform. This transformation function is defined by the following formula:

[0061]

[0062] Among them, cos(θ-θ c ) represents the input point (θ, φ) and the center direction (θ c ,φ c). The transformation strength parameter s is the power that affects the transformation function. This transformation function calculates the corresponding grid transformation value and the weight transformation value of the weight transformation based on the weight field matrix and grid field matrix of the previous moment, which is used to reflect the degree of feature change and weight adjustment of points in the environment. These transformed values ​​provide an efficient abstract representation of the autonomous driving environment, which helps the system to plan paths and make decisions.

[0063] The key to liquid embedding technology lies in its ability to perform adaptive nonlinear transformations on data. Through technologies such as neural networks, the liquid embedding module can effectively encode the multimodal information input by sensors into a unified scene representation, allowing different types of data to coexist in this scene.

[0064] Step 1.3: Apply volume rendering technology to the fused field information, and visualize the complex multimodal information as a whole through ray emission, path sampling and color mapping. This comprehensive information extraction and presentation method provides a rich data foundation for subsequent real-time field parameter estimation and path planning.

[0065] In this step, a volume rendering environment modeling information extraction method is used to extract multimodal information. The volume rendering environment modeling information extraction method includes the following steps:

[0066] The volume rendering retrieves information from the field, takes the vehicle's current position as the ray-issuing point, sends a ray from that point until it intersects the grid field, samples volume density weight points along the path, obtains the RGB "color" of the path through volume rendering, and uses a fully connected layer to map the values ​​of the three channels into usable information.

[0067] Step 2: Estimate the field parameters in real time through the liquid time constant neural network (LTC) to dynamically update the field information.

[0068] The adaptability of the liquid time constant neural network LTC enables it to flexibly adapt to different scenarios and provide the system with hot-swap capabilities for multi-modal slots.

[0069] The structure of the liquid temporal neural network (LTC) achieves adaptability based on the simulation of liquid dynamics. The network contains multiple neurons whose morphology can change in real time through closed-form solutions of differential equations. This structure enables the liquid temporal neural network LTC to dynamically adjust its own morphology to adapt to different tasks and real-time data, providing high flexibility for field parameter estimation.

[0070] The process of updating the field information includes adjusting the entire scene information according to the field parameters estimated in real time by the liquid temporal neural network LTC. The field parameters include but are not limited to object positions (position information of each object in the scene, including the position of the vehicle itself and the positions of other objects around it), motion states (including speed, acceleration, direction, etc.), environmental characteristics (road conditions, traffic signs, traffic light status, etc.), involving updates of object positions, motion states, etc. to ensure that the scene representation is consistent with the actual environment. This dynamic field parameter update process works in conjunction with other modules through feedback and iteration mechanisms to ensure that the system accurately perceives and adapts to different driving scenarios, providing a reliable foundation for subsequent path planning and optimization.

[0071] Liquid-time continuous neural networks are able to adapt over time to better handle emergencies and provide flexible scalability for multimodal slots.

[0072] A recurrent neural network (RNN) is introduced into the liquid time constant neural network (LTC) so that the network can dynamically adapt internal states and weights when processing different tasks and time series data.

[0073] A key innovation of the present invention is the use of a liquid time constant neural network (LTC) based on RNN (recurrent neural network) to estimate field parameters. RNN is introduced into the recurrent connection established between the hidden layers inside the liquid time constant neural network LTC. This network is adaptive and can dynamically change its form according to different tasks and data. By introducing RNN, the liquid time constant neural network LTC enables the network to dynamically adapt its internal state and weight when processing different tasks and time series data, thereby achieving adaptability. This enables the system to adapt to different multimodal slots, and only a small number of neurons are required to achieve complex functions. By introducing RNN into the liquid time constant neural network LTC, the present invention can improve the representation ability and generalization ability of the LTC network when processing time series data, making it more suitable for tasks that require modeling and prediction of time series information.

[0074] Step 3: Plan and optimize the vehicle's driving path based on the real-time updated field information.

[0075] This step plans and optimizes the path based on the field information updated in real time, so that the optimal path can be selected in a complex environment. The liquid time constant neural network LTC receives the embedding matrix and field state matrix of the previous moment, and updates the field state, thereby generating real-time field information, that is, dynamic change characteristics.

[0076] Step 4: By adjusting the parameters of the LTC network, the network performance and computing occupancy rate can be dynamically adjusted. The system maintains similar linear complexity advantages and adapts to different computing needs.

[0077] Step 5: Environmental modeling and information extraction: Real-time perception and understanding of the environment during driving, and extraction of environmental information.

[0078] Step 6: Volume rendering to obtain path information: Match the real-time perceived environmental information with the driving path planned in step 3 to determine the actions to be taken during actual driving.

[0079] In actual driving, the environment may change dynamically, so it is necessary to perceive the environment in real time, adjust the path in real time, and fine-tune the environment according to actual needs to ensure that the autonomous driving system can complete the driving task safely and efficiently. Through the "environmental modeling and information extraction" step, the system can perceive and understand the changes in the surrounding environment in real time, which may affect the actual feasibility of the optimal path, so it is necessary to continuously perceive and understand the environment during driving. Through the "volume rendering to obtain path information" step, the system can match the real-time perceived environmental information with the pre-planned path information to determine the specific actions that should be taken in actual driving, such as adjusting the vehicle speed, changing lanes, etc.

[0080] In some embodiments of the present invention, an autonomous driving decision system based on spherical harmonic function field transformation is also provided, including the following modules: a multimodal information embedding module, a real-time field parameter estimation module, a path planning and optimization module, and a dynamic performance adjustment module.

[0081] The multimodal information embedding module is responsible for receiving multimodal information from various sensors (such as cameras, road notifications, lidar, etc.), and efficiently representing and extracting multimodal information through liquid embedding and volume rendering technology.

[0082] The multimodal information embedding module first models the environment through a method called weight field and grid field, and then extracts multimodal information through an environmental modeling information extraction method called volume rendering. Through liquid embedding and volume rendering technology, multi-source information from various sensors is efficiently embedded into a unified scene, which can provide strong information support for the system's intelligent decision-making.

[0083] In addition, the multimodal information embedding module also uses core components such as weight field and grid field to abstract the environment into scene information and provide a basis for path planning. Through these fields, the system can better understand and construct the autonomous driving environment, thereby achieving more accurate path planning.

[0084] The real-time field parameter estimation module is used to estimate the field parameters in real time through the liquid time constant neural network (LTC) so as to update the field information according to the real-time multimodal data.

[0085] The path planning and optimization module is used to plan and optimize the vehicle's driving path based on real-time updated field information.

[0086] The path planning and optimization module uses volume rendering technology to perform path planning and optimization based on the real-time updated field information. First, by introducing the real-time updated field information, including the highly abstract representation and dynamic change characteristics of the environment, it provides key input for the vehicle's path planning. Subsequently, the field information is visualized in the form of a three-dimensional entity using volume rendering technology, and an intuitive environmental expression is presented through ray emission, path sampling and color mapping. Under this three-dimensional scene representation, path planning algorithms (such as A* algorithm, Fast Marching Method, etc.) are applied to determine the optimal driving path of the vehicle, taking into account the real-time updated field information to cope with environmental changes. Finally, the preliminary planned path is further optimized through optimization algorithms (such as local search algorithms, genetic algorithms, etc.) to ensure that the generated path is not only the shortest, but also the safest and most suitable. The entire process realizes dynamic adjustment of path planning through the dynamic performance adjustment module, ensuring that the vehicle's driving path has real-time and adaptability, and improving the accuracy and intelligence level of path planning.

[0087] The environment modeling and information extraction module is used to perceive and understand the environment in real time while driving and to extract environmental information.

[0088] Dynamic performance adjustment module: By adjusting the LTC network parameters, the path planning is dynamically adjusted, and the performance and computational occupancy of the liquid time constant neural network can be dynamically adjusted while maintaining similar linear complexity advantages.

[0089] The dynamic performance adjustment module can realize dynamic adjustment of path planning by adjusting the LTC network parameters. The parameter adjustment of the LTC network not only affects the dynamic adjustment capability of path planning, but also affects the computing occupancy rate of the system.

[0090] The dynamic performance adjustment module uses field model parameter adjustment strategies to dynamically adjust the key parameters of LTC by real-time monitoring of the vehicle's surrounding environment and task requirements. This covers real-time adjustment of key parameters such as weights and biases to adapt to changes in different driving scenarios and tasks. At the same time, in order to achieve flexible adjustment of network performance, the module dynamically adjusts the structure, connection weights, activation functions, etc. of the neural network while adjusting the field model parameters to meet changes in real-time task load and computing resource conditions. During the adjustment process, the system maintains a similar linear complexity advantage, ensuring that while performance is improved, the computational complexity remains within a controllable range, achieving flexible adjustment of the autonomous driving decision network and balancing performance and efficiency.

[0091] Key parameter adjustment: This includes real-time adjustment of key parameters in the LTC network, such as learning rate, time constant, etc. The adjustment of learning rate can affect the convergence speed and stability of the network, while the adjustment of time constant can affect the update speed of field status and the degree of memory of historical information. The adjustment of these parameters can be dynamically adjusted according to the real-time monitoring of the vehicle's surrounding environment and task requirements.

[0092] Network structure adjustment: Adjustment of the network structure includes increasing or decreasing the number of hidden layer nodes, changing the hierarchical structure or adding new layers. This adjustment can optimize the network's expressiveness and performance under different driving scenarios and task requirements.

[0093] Connection weight adjustment: The connection weight adjustment can be achieved by changing the weights between connected neurons. This can adjust the weights of different connections in the network according to the real-time monitored data and task requirements to optimize the input-output mapping and performance of the network.

[0094] Activation function adjustment: The selection of activation function and parameter adjustment have an important impact on the nonlinear modeling and performance of the network. By selecting a suitable activation function and adjusting its parameters, the representation ability and performance of the network can be improved. In some embodiments of the present invention, the activation function can be ReLU or Leaky ReLU.

[0095] The path information acquisition module is used to match the real-time perceived environmental information with the planned optimized driving path to determine the actions to be taken during actual driving.

[0096] Some embodiments of the present invention further include a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method provided in the aforementioned embodiments.

[0097] Some embodiments of the present invention further include a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the steps of the method provided in the aforementioned embodiment are implemented.

[0098] In some embodiments of the present invention, the method of the present invention is compared with similar decision-making methods through experimental data, wherein the indicators in Table 1 are capability type (such as detection, tracking, scene modeling, etc.), parameter quantity, and reasoning delay. The indicators in Table 2 are model type and distance from the center of the lane, and the scenarios of planning once every second, two seconds, and three seconds are evaluated respectively. (The last column of data in Tables 1 and 2 is the data of the method of the present invention)

[0099] The data in Table 1 and Table 2 compare the differences between the method of the present invention and other methods in terms of path planning accuracy and computational cost.

[0100] Table 1 compares the capability types, parameter quantities, and reasoning delays of different decision-making methods. As can be seen from Table 1, the method of the present invention achieves a significant reduction in the number of parameters, and the reasoning delay is also significantly shortened. This shows that the method of the present invention is superior to other methods in path planning accuracy and also has a significant advantage in computational cost.

[0101] Table 2 compares the distance indicators of the deviation from the center of the lane for different model types. The method of the present invention is significantly better than the traditional method in the planning scenarios of 1 second, 2 seconds and 3 seconds, and the deviation distance is smaller, which further confirms the improvement of the path planning accuracy of the method of the present invention.

[0102] Table 1

[0103]

[0104] Table 2

[0105]

[0106] The method of the present invention is a significant improvement over traditional methods. The method of the present invention has demonstrated an increase of up to 28% in path planning accuracy and an exponential reduction in floating-point operation costs in actual driving scenarios, highlighting its low hardware dependence and real-time decision-making capabilities. In addition, the scalability of the method is demonstrated by easily integrating additional sensors and dynamically adjusting performance during reasoning. Through the above design, the present invention achieves the potential for high scalability, low computational dependence, and robustness, and is particularly suitable for information processing and decision-making that requires real-time, efficient, and scalable in autonomous driving systems.

[0107] Compared with traditional methods, the invention has achieved significant improvements in path planning accuracy and a significant reduction in computational costs. In addition, the system also demonstrates excellent scalability, can easily integrate new sensors, and dynamically adjust performance during reasoning.

[0108] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automatic driving decision-making method, characterized in that: The following steps are involved: Acquire multimodal information, encode it into a unified scene representation through liquid embedding technology, and visualize it as a whole through volume rendering; Real-time estimation of field parameters by liquid time constant neural network LTC to dynamically update field information, wherein a recurrent neural network RNN ​​is introduced into the liquid time constant neural network LTC to enable the network to dynamically adapt internal states and weights when processing different tasks and time series data; Plan and optimize vehicle routes based on real-time updated field information; Real-time perception and understanding of the environment while driving, and extraction of environmental information; Match the real-time perceived environmental information with the planned optimized driving path to determine the actions to be taken during actual driving.

2. The automatic driving decision-making method according to claim 1, characterized in that: The method also includes the steps of: dynamically adjusting the network performance and computing occupancy by adjusting the parameters of the liquid time constant neural network LTC.

3. An automatic driving decision-making method according to any one of claims 1-2, characterized in that: The acquiring of multimodal information, encoding the multimodal information into a unified scene representation by liquid embedding technology, and visualizing the multimodal information as a whole by volume rendering, includes: Receiving multimodal data from various sensors; Using liquid embedding technology, multiple information is fused into a unified scene representation through nonlinear transformation; Volume rendering technology is applied to the fused field information to visualize the complex multimodal information as a whole through ray emission, path sampling and color mapping.

4. The automatic driving decision-making method according to claim 3, characterized in that: Modeling is done through the environmental modeling method of weight field and grid field, including: Mapping various factors in the environment to weight fields to form a weighted place matrix that reflects environmental impacts; Discretize the environment modeling, divide the continuous environment into discrete grids to construct a grid field, each grid represents a local area in the environment; The autonomous driving environment is efficiently abstracted through weight transformation and grid transformation.

5. The automatic driving decision-making method according to claim 4, characterized in that: In the grid transformation, the polar coordinate representation of the input point on the ellipsoid is transformed by the cosine transform of the spherical harmonic function. In this process, the system uses the embedding matrix and field state matrix of the previous moment to calculate the transformation value, which reflects the degree of change of the characteristics of the corresponding point in the environment. In the weight transformation stage, the system also uses the embedding matrix and field state matrix of the previous moment, adopts the environmental modeling method of weight field and grid field, and calculates the weight transformation value through the cosine transform of spherical harmonic function, which reflects the weight change of points in the environment.

6. An automatic driving decision system, characterized in that: The system is used to implement any of the methods described in claims 1-5, and the system includes the following modules: The multimodal information embedding module is used to obtain multimodal information, encode the multimodal information into a unified scene representation through liquid embedding technology, and visualize the multimodal information as a whole through volume rendering; A real-time field parameter estimation module, used for estimating field parameters in real time through a liquid time constant neural network LTC to dynamically update field information, wherein a recurrent neural network RNN ​​is introduced into the liquid time constant neural network LTC to enable the network to dynamically adapt internal states and weights when processing different tasks and time series data; Path planning and optimization module, used to plan and optimize the vehicle's driving path based on real-time updated field information; The environment modeling and information extraction module is used to perceive and understand the environment in real time while driving, and to extract environmental information; The path information acquisition module is used to match the real-time perceived environmental information with the planned optimized driving path to determine the actions to be taken during actual driving.

7. The automatic driving decision system according to claim 1, characterized in that: It also includes a dynamic performance adjustment module, which is used to dynamically adjust the path planning and the network performance and computing occupancy by adjusting the parameters of the liquid time constant neural network LTC.

8. The automatic driving decision system based on spherical harmonic function field transformation according to claim 7, characterized in that: Adjust the parameters of the liquid time constant neural network LTC, including the learning rate and the time constant.

9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of any one of claims 1-5.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images