Intelligent driving method, device, equipment, medium and program product

By automatically identifying and classifying technical debts in intelligent driving systems, combining multimodal data fusion and reinforcement learning, the performance degradation caused by technical debt is solved, the performance and stability of the system is improved, and the driving behavior analysis ability in complex driving scenarios is enhanced.

CN120382913APending Publication Date: 2025-07-29HEFEI IFLY DIGITAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510313949.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-03-17
Publication Date
2025-07-29

AI Technical Summary

Technical Problem

The accumulation of technical debt in intelligent driving systems leads to performance degradation, maintenance difficulties, and affects safety. The existing technology lacks an effective automated management mechanism.

Method used

By identifying the operating data of the intelligent driving system, automatically classifying technical debts, and generating optimization suggestions for different categories, including simplifying the model structure, cleaning up redundant data, etc., combining multimodal data fusion and reinforcement learning to improve system performance.

Benefits of technology

Improve the efficiency of technical debt identification, reduce missed identification, optimized suggestions help manage technical debt, improve system performance and stability, and enhance the semantic analysis ability of driving behavior in complex driving scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120382913A_ABST
    Figure CN120382913A_ABST
Patent Text Reader

Abstract

The invention provides an intelligent driving method and device, equipment, a medium and a program product, and the method comprises the steps: recognizing a technical debt in an intelligent driving system based on the operation data of the intelligent driving system, and determining the type of the technical debt; and based on the category of the technology debt, generating an optimization suggestion of the technology debt. The performance of the intelligent driving system can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an intelligent driving method, device, equipment, medium and program product. Background Art

[0002] With the development of smart cars, autonomous driving technology has gradually become an important part of modern transportation systems. To achieve safe and efficient autonomous driving, cars need to have complex environmental perception, driving decision-making, and behavior prediction capabilities.

[0003] As intelligent driving systems evolve over time, the accumulation of technical debt has become a significant issue plaguing their development. Technical debt refers to potential issues left in an intelligent driving system in pursuit of short-term gains. This can lead to performance degradation, maintenance difficulties, and even safety concerns. Summary of the invention

[0004] The present application provides an intelligent driving method, apparatus, device, medium, and program product for improving the performance of an intelligent driving system.

[0005] According to a first aspect of an embodiment of the present application, there is provided an intelligent driving method, including:

[0006] identifying technical debt in the intelligent driving system and determining a category of the technical debt based on operating data of the intelligent driving system;

[0007] Based on the category of the technical debt, an optimization suggestion for the technical debt is generated.

[0008] Optionally, identifying technical debt in the intelligent driving system and determining a category of the technical debt based on operating data of the intelligent driving system includes:

[0009] If the operating data of the intelligent driving system satisfies at least one preset debt identification condition, determining that a technical debt exists in the intelligent driving system and determining a target debt identification condition satisfied by the operating data of the intelligent driving system;

[0010] According to the mapping relationship between the preset debt identification condition and the preset debt category, the category of the technical debt corresponding to the target debt identification condition is determined.

[0011] Optionally, the intelligent driving system includes an intelligent driving model; the intelligent driving model is used to generate intelligent driving behavior;

[0012] The target debt identification conditions include that the number of network layers of the intelligent driving model is greater than the preset number of network layers, and the inference speed of the intelligent driving model decreases; the category of the technical debt corresponding to the target debt identification conditions includes model debt; and the optimization suggestion for the technical debt includes simplifying the model structure of the intelligent driving model.

[0013] Optionally, generating optimization suggestions for the technical debt based on the category of the technical debt includes:

[0014] Determining a priority for handling the technical debt based on the category of the technical debt and the impact of the technical debt;

[0015] Based on the category of the technical debt and the processing priority of the technical debt, an optimization suggestion for the technical debt is generated.

[0016] Optionally, generating an optimization suggestion for the technical debt based on the category of the technical debt and the processing priority of the technical debt includes:

[0017] Before the intelligent driving system is updated, generating optimization suggestions for the technical debt based on the category of the technical debt and the processing priority of the technical debt;

[0018] The method further comprises:

[0019] During the update of the intelligent driving system, based on the optimization suggestions for the technical debt, the technical debt is paid off;

[0020] determining, based on the operating data of the intelligent driving system before the update and the operating data of the intelligent driving system after the update, a status of repayment of the technical debt;

[0021] Based on the repayment status of the technical debt, it is determined whether to update the intelligent driving system again.

[0022] Optionally, the intelligent driving system includes an intelligent driving model;

[0023] The method further comprises:

[0024] Extracting data features of driving data of each modality;

[0025] Fusing the data features of the driving data of each modality to obtain fused features;

[0026] The fused features are input into the intelligent driving model to obtain intelligent driving behavior.

[0027] Optionally, fusing the data features of the driving data of each modality to obtain fused features includes:

[0028] Based on the data characteristics of driving data at different moments of the same modality, obtain the time characteristics of the same modality;

[0029] Based on the data characteristics of driving data at the same moment of each modality, obtain the spatial characteristics at the same moment;

[0030] Based on the fusion of the time characteristics and the spatial characteristics, obtain the fused characteristics.

[0031] Optionally, the intelligent driving system includes an intelligent driving model; the intelligent driving model is used to generate intelligent driving behaviors;

[0032] The training process of the intelligent driving model includes:

[0033] Based on the sample driving data in each driving scenario, obtain the predicted intelligent driving behaviors in each driving scenario through the intelligent driving model;

[0034] Obtain the feedback data for the predicted intelligent driving behaviors in each driving scenario;

[0035] Based on the predicted intelligent driving behaviors in each driving scenario and the feedback data for the predicted intelligent driving behaviors in each driving scenario, train the intelligent driving model.

[0036] Optionally, the sample driving data includes multi-modal sample driving data;

[0037] The obtaining the predicted intelligent driving behaviors in each driving scenario through the intelligent driving model based on the sample driving data in each driving scenario includes:

[0038] Input the multi-modal sample driving data in each driving scenario into a multi-modal data fusion model to obtain the fused sample characteristics;

[0039] Input the fused sample characteristics into the intelligent driving model to obtain the predicted intelligent driving behaviors in each driving scenario;

[0040] The method further includes:

[0041] Based on the predicted intelligent driving behaviors in each driving scenario and the feedback data for the predicted intelligent driving behaviors in each driving scenario, train the multi-modal data fusion model.

[0042] According to the second aspect of the embodiments of the present application, there is provided an intelligent driving device, including:

[0043] An identification unit, configured to identify the technical debt in the intelligent driving system and determine the category of the technical debt based on the operation data of the intelligent driving system;

[0044] A generating unit is configured to generate an optimization suggestion for the technical debt based on the category of the technical debt.

[0045] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including a memory and a processor;

[0046] The memory is connected to the processor and is used to store programs;

[0047] The processor is used to implement the intelligent driving method as described in the first aspect by running the program in the memory.

[0048] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the intelligent driving method as described in the first aspect is implemented.

[0049] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to execute the intelligent driving method as described in the first aspect.

[0050] In this application, based on the operating data of the intelligent driving system, it is possible to accurately identify technical debt in the intelligent driving system and determine the category of technical debt during the operation of the intelligent driving system. Compared with manual identification of technical debt in the intelligent driving system, this improves the efficiency of identifying technical debt in the intelligent driving system and can reduce the occurrence of missing identification of some technical debt in the intelligent driving system. Based on the category of technical debt, optimization suggestions for technical debt are generated. For different categories of technical debt, corresponding optimization suggestions are generated to help developers effectively manage technical debt in subsequent updates or iterations of the intelligent driving system, improve the performance of the intelligent driving system, and reduce performance bottlenecks caused by technical debt. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0052] Figure 1 A schematic diagram of a flow chart of an intelligent driving method provided in an embodiment of the present application;

[0053] Figure 2 This is a flow chart of step 102 provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a flow chart of an intelligent driving method provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of a flow chart of an intelligent driving method provided in an embodiment of the present application;

[0056] Figure 5 This is a flow chart of step 402 provided in an embodiment of the present application;

[0057] Figure 6 A schematic diagram of a process flow for training an intelligent driving model provided in an embodiment of the present application;

[0058] Figure 7 This is a flow chart of step 601 provided in an embodiment of the present application;

[0059] Figure 8 This is a schematic structural diagram of an intelligent driving device provided in an embodiment of the present application;

[0060] Fig. 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] As intelligent driving systems evolve over time, the accumulation of technical debt has become a significant issue plaguing their development. Technical debt refers to potential issues left in an intelligent driving system in pursuit of short-term gains. This can lead to performance degradation, maintenance difficulties, and even safety concerns.

[0062] Currently, most intelligent driving systems rely on manual identification and repair during their development and iteration processes to manage technical debt, lacking effective automated mechanisms. This can easily lead to the accumulation of potential issues with updates, impacting long-term performance and maintenance costs.

[0063] In order to improve the performance of an intelligent driving system, the present application provides an intelligent driving method, apparatus, device, medium and program product.

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] Exemplary Implementation Environment

[0066] The intelligent driving method according to the embodiments of the present application can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user device, a mobile device, a computing device, a wearable device, etc. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a cloud server capable of cloud computing. The method can be implemented by a processor calling computer-readable program instructions stored in a memory.

[0067] Exemplary Methods

[0068] See also Figure 1 In an exemplary embodiment, an intelligent driving method is provided. Figure 1 As shown, the process of the intelligent driving method mainly includes:

[0069] Step 101: Identify technical debt in the intelligent driving system and determine the category of the technical debt based on operating data of the intelligent driving system.

[0070] In the exemplary embodiments, technical debt refers to potential problems introduced by software systems during the development process in pursuit of short-term benefits, which may be caused by time constraints, imperfect design, or improper technology selection. The accumulation of technical debt in intelligent driving systems will affect the accuracy and real-time performance of the semantic parsing of driving behavior. Common technical debt in intelligent driving systems includes complex code structure, data redundancy, and lengthy computational time for training models. Technical debt in intelligent driving systems not only reduces the performance of intelligent driving systems, but also increases the difficulty of maintaining and expanding intelligent driving systems. Technical debt in intelligent driving systems may lead to reduced model reasoning efficiency, decreased parsing accuracy, and even unexpected errors. Therefore, identifying and managing technical debt is crucial to maintaining the long-term stability of intelligent driving systems.

[0071] In an exemplary embodiment, step 101 may be performed by a monitoring module within the intelligent driving system. Unlike traditional manual identification of technical debt, this application automatically identifies technical debt within the intelligent driving system through the intelligent driving system. Of course, step 101 may also be performed by other electronic devices, and this application does not limit this.

[0072] In an exemplary embodiment, the operating data of the intelligent driving system may include code complexity indicators, model execution time, decision accuracy, etc., and the operating data of the intelligent driving system may be continuously collected during the operation of the intelligent driving system.

[0073] In some embodiments, step 101 includes:

[0074] When the operating data of the intelligent driving system satisfies at least one preset debt identification condition, determining that technical debt exists in the intelligent driving system and determining a target debt identification condition satisfied by the operating data of the intelligent driving system;

[0075] According to the mapping relationship between the preset debt identification condition and the preset debt category, the category of the technical debt corresponding to the target debt identification condition is determined.

[0076] In an exemplary embodiment, the preset debt identification conditions may include but are not limited to lower model accuracy, increased decision delay, code complexity greater than a preset complexity, resource consumption greater than a preset consumption, and the like.

[0077] In exemplary embodiments, technical debt categories include, but are not limited to, structural debt, data debt, model debt, and resource debt. Structural debt, such as redundant code and inappropriate data flow design, directly impacts the readability and maintainability of the intelligent driving system. Data debt, such as redundancy in data processing and incomplete datasets, impacts model training effectiveness and decision accuracy. Model debt, including overly complex and inadequately optimized models, impacts inference speed and scalability. Resource debt, such as excessive resource consumption by the intelligent driving system when processing tasks, creates hardware performance bottlenecks.

[0078] In an exemplary embodiment, there is a predefined mapping relationship between the preset debt identification conditions and the preset debt categories. After determining the target debt identification conditions satisfied by the operating data of the intelligent driving system, the category of technical debt corresponding to the target debt identification conditions can be determined according to the mapping relationship between the preset debt identification conditions and the preset debt categories, thereby realizing real-time classification of technical debts.

[0079] For example, the preset debt identification conditions include code complexity greater than a preset complexity, reduced model accuracy, the number of network layers in the model greater than a preset number of network layers and a decrease in model inference speed, and resource consumption greater than a preset consumption. The preset debt identification condition of code complexity greater than a preset complexity corresponds to structural debt, the preset debt identification condition of reduced model accuracy corresponds to data debt, the preset debt identification condition of the number of network layers in the model greater than a preset number of network layers and a decrease in model inference speed corresponds to model debt, and the preset debt identification condition of resource consumption greater than a preset consumption corresponds to resource debt. These are merely examples and this application is not intended to limit this.

[0080] According to the mapping relationship between the preset debt identification conditions and the preset debt categories, the category of the technical debt corresponding to the target debt identification condition satisfied by the operation data of the intelligent driving system is determined, fully considering the impact of the category of technical debt on the operation data of the intelligent driving system. Different preset debt categories can make the operation data of the intelligent driving system satisfy different preset debt identification conditions. Therefore, the category of technical debt can be accurately determined according to the target debt identification condition satisfied by the operation data of the intelligent driving system, improving the identification efficiency and accuracy of technical debt in the intelligent driving system. Furthermore, for different categories of technical debt, corresponding optimization suggestions are generated to help developers effectively manage technical debt in the subsequent update or iteration of the intelligent driving system, improving the performance of the intelligent driving system and reducing performance bottlenecks caused by technical debt.

[0081] In an exemplary embodiment, in the intelligent driving system, as the model is updated and iterated, technical debt gradually accumulates. Step 101 may include: identifying the technical debt in the intelligent driving system and determining the category of the technical debt based on the historical update records of the intelligent driving system and the operation data of the intelligent driving system.

[0082] Step 102, generating an optimization suggestion for the technical debt based on the category of the technical debt.

[0083] In some embodiments, step 102 includes: directly generating an optimization suggestion for the technical debt based on the category of the technical debt.

[0084] For different categories of technical debt, corresponding optimization suggestions are generated to help developers effectively manage technical debt in the subsequent update or iteration of the intelligent driving system. It can automatically optimize the architecture and reduce complexity during the model training and inference stages. For example, when it is detected that the number of model layers is too large and the inference speed decreases, it is recommended to reduce redundant layers or simplify the model structure through pruning techniques. Moreover, by reducing data redundancy and improving the data preprocessing process, the intelligent driving system can improve the real-time performance and accuracy of decision-making during driving behavior semantic parsing, thereby reducing performance bottlenecks caused by technical debt.

[0085] In some embodiments, the intelligent driving system includes an intelligent driving model; the intelligent driving model is used to generate intelligent driving behaviors;

[0086] The target debt identification condition includes that the number of network layers of the intelligent driving model is greater than the preset number of network layers, and the inference speed of the intelligent driving model decreases; the category of the technical debt corresponding to the target debt identification condition includes model debt; the optimization suggestion for the technical debt includes simplifying the model structure of the intelligent driving model.

[0087] When the operating data of the intelligent driving system meets the target debt identification conditions, including that the number of network layers of the intelligent driving model is greater than the preset number of network layers and the inference speed of the intelligent driving model decreases, according to the mapping relationship between the preset debt identification conditions and the preset debt categories, it is determined that the category of the technical debt corresponding to the target debt identification conditions includes model debt, and optimization suggestions for the model debt are generated. The optimization suggestions for the model debt include simplifying the model structure of the intelligent driving model. For example, the model structure of the intelligent driving model can be simplified by reducing the redundant layers of the intelligent driving model or by using pruning techniques, which can automatically optimize the architecture of the intelligent driving model during the training and inference stages of the intelligent driving model, reduce the complexity of the intelligent driving model, and thereby improve the inference speed of the intelligent driving model.

[0088] In some other embodiments, as Figure 2 shown, step 102 includes:

[0089] Step 201, based on the category of the technical debt and the impact degree of the technical debt, determine the processing priority of the technical debt.

[0090] In an exemplary embodiment, the processing priority may include high priority and low priority, may also include high priority, medium priority and low priority, and may also include other situations. This application does not limit this. In the following embodiments, the example where the processing priority includes high priority and low priority is used for explanation. For example, a technical debt with high priority refers to a technical debt that seriously affects the real-time performance or accuracy of the intelligent driving system and needs to be solved in the short term. A technical debt with low priority refers to a technical debt that does not have an obvious impact on the current operation of the intelligent driving system but may affect the expansion and maintenance of the intelligent driving system in the future.

[0091] Step 202, based on the category of the technical debt and the processing priority of the technical debt, generate optimization suggestions for the technical debt.

[0092] In an exemplary embodiment, the optimization suggestions for the technical debt may refer to the repayment plan for the technical debt. The repayment plan for the technical debt includes the processing order of each technical debt and the repayment suggestions for each debt.

[0093] Based on the category of the technical debt and the impact degree of the technical debt, determine the processing priority of the technical debt, and based on the category of the technical debt and the processing priority of the technical debt, generate optimization suggestions for the technical debt, which can accurately determine the processing order of each technical debt and how to perform targeted processing for the category of the technical debt, is conducive to quickly identifying and solving the most urgent problems, and reduces the risk of the intelligent driving system crashing or performance degradation caused by the accumulation of technical debts.

[0094] In some embodiments, step 202 includes: before the intelligent driving system is updated, generating optimization suggestions for the technical debt based on the category of the technical debt and the processing priority of the technical debt.

[0095] To ensure that the performance of the intelligent driving system after the update is not affected by the technical debt, the intelligent driving system automatically generates a technical debt repayment plan, that is, optimization suggestions for the technical debt, before each update. The technical debt repayment plan can determine which technical debts must be processed first and how to repay them without affecting the performance of the current intelligent driving system. For example, for high-priority structural debts, the intelligent driving system may recommend optimizing or restructuring the model architecture; for data debts, the intelligent driving system will clean up redundant data and rebalance the dataset. The technical debt repayment plan can ensure that the updated intelligent driving system maintains performance improvement while not introducing new technical debts.

[0096] In some embodiments, as Figure 3 shown, the intelligent driving method further includes:

[0097] Step 301, during the update of the intelligent driving system, repaying the technical debt based on the optimization suggestions for the technical debt.

[0098] Step 302, determining the repayment situation of the technical debt based on the running data before the update of the intelligent driving system and the running data after the update of the intelligent driving system.

[0099] Step 303, judging whether to update the intelligent driving system again based on the repayment situation of the technical debt.

[0100] Based on the running data before the update of the intelligent driving system and the running data after the update of the intelligent driving system, the performance comparison result before and after the update of the intelligent driving system can be obtained. For example, the improvement effects of the updated intelligent driving system in aspects such as driving behavior semantic parsing, decision-making accuracy, and resource consumption, and further obtain the repayment situation of the technical debt. The intelligent driving system will continuously monitor the running data of the updated intelligent driving system, and then dynamically obtain the repayment situation of the technical debt in real time, judge whether to update the intelligent driving system again, and ensure that the technical debt will not accumulate again to the extent that it affects the performance of the intelligent driving system. It ensures that each update of the intelligent driving system can steadily improve the performance while controlling the expansion of the technical debt, avoiding long-term impacts on the scalability and stability of the intelligent driving system, and improving the maintainability and upgrade efficiency of the intelligent driving system. Ensuring the long-term stability and continuous optimization of the performance of the intelligent driving system can help developers effectively manage and solve the technical debt and avoid the negative impacts of long-term accumulation of the technical debt.

[0101] Traditional intelligent driving systems typically rely on a single type of sensor data (such as cameras or lidar) or limited sensor data sources for environmental perception and behavior decision-making. However, a single data source often has limitations. For example, cameras are limited by lighting conditions, lidar performs poorly in rain and snow, and GPS (Global Positioning System) signals may fail in complex environments such as tunnels. This way of relying on a single sensor leads to limited breadth and depth of driving behavior semantic extraction, making it difficult to handle complex and changing driving scenarios. Limited sensor data sources also have limitations. For example, Company A's intelligent driving system mainly relies on camera and radar data, which can work effectively in simple driving scenarios, but in complex or extreme environments (such as rain and snow, at night, etc.), due to incomplete perception information, the accuracy and robustness of driving behavior parsing often decrease. To improve the driving behavior semantic parsing ability of intelligent driving systems in complex driving scenarios, the following embodiments propose a multi-modal driving data fusion solution.

[0102] In some embodiments, the intelligent driving system includes an intelligent driving model.

[0103] As Figure 4 shown, the intelligent driving method further includes:

[0104] Step 401, extract the data features of the driving data of each modality.

[0105] In an exemplary embodiment, the driving data of each modality may include the driving data collected by various sensors such as cameras, radars, lidars, and GPS.

[0106] Multi-modal driving data fusion can effectively improve the environmental perception ability of intelligent driving systems. The intelligent driving system can obtain rich environmental information from different sensors. Especially in complex driving scenarios, different sensors can complement each other to provide more comprehensive environmental information. For example, lidar can provide high-precision three-dimensional distance data, cameras can capture rich visual details, GPS provides geographical location information, and radars can work stably in extreme weather. The fusion of these data provides a more solid foundation for the semantic parsing of driving behavior, improving the accuracy of driving behavior semantic parsing and the system robustness of intelligent driving systems in complex scenarios.

[0107] In an exemplary embodiment, the driving behavior semantic parsing mentioned in this application may refer to the intelligent driving system predicting intelligent driving behaviors based on the driving data of each modality.

[0108] Step 402, fuse the data features of the driving data of each modality to obtain the fused features.

[0109] In an exemplary embodiment, the intelligent driving system includes an intelligent driving model and a multimodal data fusion model. Steps 401 and 402 can be implemented using the multimodal data fusion model. Steps 401 and 402 may include inputting driving data from each modality into the multimodal data fusion model to obtain fused features.

[0110] In an exemplary embodiment, the multimodal data fusion model includes two parts: the first part is a feature extraction network for driving data of different modalities. Step 401 can be implemented through the feature extraction network. For example, convolutional neural networks (CNN) are used to extract visual features, and point cloud processing networks are used to extract features of lidar data. Different feature extraction networks can be used for driving data of different modalities. The second part is a multimodal fusion network. Step 402 can be implemented through the multimodal fusion network. The multimodal fusion network uses feature splicing, weighted averaging and other methods to fuse data features of driving data of different modalities. Through the fusion network, a unified understanding and decision-making of the semantics of driving behavior can be achieved.

[0111] In some embodiments, as Figure 5 As shown, step 402 includes:

[0112] Step 501 : obtaining a time feature of the same modality based on data features of driving data of the same modality at different times.

[0113] Step 502 : Based on the data features of the driving data of each modality at the same time, obtain the spatial features at the same time.

[0114] Step 503: Fusing the temporal features and the spatial features to obtain fused features.

[0115] By combining the data features of driving data from the same modality at different times, we can obtain the temporal features of the same modality. This allows us to extract dynamic changes in driving behavior through continuous information in the temporal dimension. By combining the data features of driving data from different modalities at the same time, we can obtain spatial features at the same time. By fusing information from different sensors in the spatial dimension, we can parse more accurate driving behavior semantics. By fusing temporal and spatial features, we can obtain fused features that can be combined for joint learning, further improving the accuracy of driving behavior semantic parsing in intelligent driving systems. For example, convolutional neural networks are used to process camera and lidar data, combined with long short-term memory networks (LSTMs) to capture driving behavior changes over time series, thereby improving the intelligent driving system's ability to understand complex driving scenarios.

[0116] In some other embodiments, step 402 includes: fusing the data features of the driving data of each modality by methods such as feature splicing and weighted averaging to obtain the fused features.

[0117] Step 403, inputting the fused features into the intelligent driving model to obtain the intelligent driving behavior.

[0118] Extract the data features of the driving data of each modality, fuse the data features of the driving data of each modality to obtain the fused features, input the fused features into the intelligent driving model to obtain the intelligent driving behavior. In complex driving scenarios, the performance of different sensors may be affected by different factors, and the joint learning of multi-modal driving data can use the data of other sensors for compensation when a certain sensor fails or the data quality deteriorates, thereby ensuring the stability of the intelligent driving system and the accuracy of the semantic parsing of driving behaviors. For example, in night driving or bad weather, lidar and radar can make up for the lack of camera information, while GPS signals can provide the necessary data support for positioning in a short time, improving the robustness of the intelligent driving system. It can improve the semantic parsing ability of driving behaviors and the system robustness of the intelligent driving system in complex driving scenarios, not only improving the perception accuracy, but also ensuring the stability of the intelligent driving system when a single sensor fails.

[0119] Traditional intelligent driving systems use rule-based or experience-based semantic parsing of driving behaviors, relying on manually designed rules or predefined driving behavior libraries, which perform well in dealing with specific driving scenarios, but lack flexibility in dealing with complex, unknown, and variable driving scenarios and are difficult to adapt to variable and complex driving scenarios. In order to improve the adaptability of the intelligent driving system to different driving scenarios, the following embodiments propose a reinforcement learning solution.

[0120] Reinforcement Learning (RL) is a machine learning method that obtains feedback by interacting with the environment and optimizes the behavior strategy. The core idea of reinforcement learning is that the agent continuously tries to interact with the environment, obtains feedback (reward or punishment), and maximizes the long-term cumulative reward by optimizing the strategy. In intelligent driving decision-making, the semantic parsing of driving behaviors involves a diverse decision-making process in complex driving scenarios, which highly coincides with the goal of reinforcement learning and can effectively handle the decision-making challenges brought by the dynamic changes of the environment during driving.

[0121] In some embodiments, the intelligent driving system includes an intelligent driving model; the intelligent driving model is used to generate intelligent driving behaviors.

[0122] As Figure 6 shown, the training process of the intelligent driving model includes:

[0123] Step 601: Based on the sample driving data in each driving scenario, obtain the predicted intelligent driving behaviors in each driving scenario through the intelligent driving model.

[0124] In an exemplary embodiment, each driving scenario may include, but is not limited to, driving scenarios such as urban, suburban, highway, rain and snow weather, etc.

[0125] Step 602: Obtain the feedback data for the predicted intelligent driving behaviors in each driving scenario.

[0126] In an exemplary embodiment, the feedback data may include whether the predicted intelligent driving behaviors in each driving scenario are correct, may also include the consequences after executing the predicted intelligent driving behaviors in each driving scenario, and may further include other feedback data. The present application does not limit this.

[0127] Step 603: Train the intelligent driving model based on the predicted intelligent driving behaviors in each driving scenario and the feedback data for the predicted intelligent driving behaviors in each driving scenario.

[0128] By training the intelligent driving model based on the predicted intelligent driving behaviors in each driving scenario and the feedback data for the predicted intelligent driving behaviors in each driving scenario, and introducing reinforcement learning, it is possible to dynamically learn the driving behaviors in different driving scenarios by simulating the driver's behavior decisions in different driving scenarios, enabling the intelligent driving system to continuously optimize the driving behavior semantic parsing ability of the intelligent driving system through feedback in different driving scenarios, and having stronger adaptability and learning ability. Through the reinforcement learning algorithm, the intelligent driving system can make adaptive adjustments in diverse driving scenarios. For example, the intelligent driving system can adjust the strategy in each driving scenario (such as urban, suburban, highway, rain and snow weather, etc.) and learn the optimal driving behavior semantics for each driving scenario. Through this interactive learning, the intelligent driving system can automatically parse and output appropriate driving behaviors, improving the rationality and accuracy of decision-making.

[0129] The introduction of reinforcement learning greatly improves the adaptability of the intelligent driving system in complex scenarios. For example, in the face of complex scenarios such as traffic jams, pedestrian crossings, or emergencies, the intelligent driving system can generate a more reasonable driving behavior semantic analysis more quickly and judge the best driving behavior based on the experience accumulated through reinforcement learning. In addition, reinforcement learning can adapt to the real-time changing environment during driving and dynamically adjust the driving behavior strategy at each moment to ensure the efficiency and accuracy of driving behavior decisions. This adaptive ability effectively makes up for the limitations of traditional rule-based systems in dealing with complex and dynamic environments.

[0130] In some embodiments, the training process of the intelligent driving model is executed in a virtual simulation environment. The intelligent driving method further includes: after the intelligent driving model is trained, deploying the intelligent driving model to the actual driving environment.

[0131] To improve the actual adaptability of the intelligent driving system, the training of reinforcement learning can be first completed through a virtual simulation environment. The virtual simulation environment can provide diverse driving scenarios and feedback data for the intelligent driving system, and the reinforcement learning algorithm continuously adjusts and optimizes the driving behavior strategy in these virtual simulation environments. When the intelligent driving model reaches the expected stability, that is, after the intelligent driving model is trained, the intelligent driving model is then deployed to the actual driving environment for testing and verification, and finally applied to actual driving decisions. Through this "virtual-actual" combination method, reinforcement learning can quickly iterate and optimize the driving behavior semantic parsing ability of the intelligent driving model without sacrificing safety.

[0132] In some embodiments, the sample driving data includes multi-modal sample driving data;

[0133] As Figure 7 shown, step 601 includes:

[0134] Step 701, inputting the multi-modal sample driving data in each driving scenario into the multi-modal data fusion model to obtain the fused sample features.

[0135] Step 702, inputting the fused sample features into the intelligent driving model to obtain the predicted intelligent driving behaviors in each driving scenario.

[0136] In the process of implementing reinforcement learning, the driving behavior semantic parsing depends on the processing and understanding of multi-modal driving data (such as data fusion based on sensors such as cameras, radars, and lidar). Combining reinforcement learning with multi-modal driving data fusion can greatly improve the adaptability of the intelligent driving system to complex driving scenarios and the driving behavior semantic decision-making ability.

[0137] In some embodiments, the intelligent driving method further includes: training the multi-modal data fusion model based on the predicted intelligent driving behaviors in each driving scenario and the feedback data of the predicted intelligent driving behaviors in each driving scenario.

[0138] Based on the predicted intelligent driving behaviors and feedback data from each driving scenario, multimodal data fusion models and intelligent driving models are trained to learn how to parse the correct driving behavior semantics in different driving scenarios. Through trial and error and feedback, the interpretation of multimodal driving data is continuously improved, and the optimal driving behavior semantic parsing strategy is generated for the intelligent driving system. For example, the intelligent driving system can learn how to supplement insufficient visual information with lidar data in rainy and snowy weather, thereby optimizing obstacle recognition and avoidance.

[0139] In summary, in this application, based on the operating data of the intelligent driving system, it is possible to accurately identify the technical debt in the intelligent driving system and determine the category of the technical debt during the operation of the intelligent driving system. Compared with manual identification of technical debt in the intelligent driving system, this improves the efficiency of identifying technical debt in the intelligent driving system and can reduce the occurrence of missed identification of some technical debt in the intelligent driving system. Based on the category of technical debt, optimization suggestions for technical debt are generated. For different categories of technical debt, corresponding optimization suggestions are generated to help developers effectively manage technical debt in subsequent updates or iterations of the intelligent driving system, improve the performance of the intelligent driving system, and reduce performance bottlenecks caused by technical debt.

[0140] Exemplary devices

[0141] Accordingly, the embodiment of the present application also provides an intelligent driving device, such as Figure 8 As shown, the intelligent driving device includes:

[0142] an identification unit 801 for identifying technical debt in the intelligent driving system and determining a category of the technical debt based on operating data of the intelligent driving system;

[0143] The generating unit 802 is configured to generate an optimization suggestion for the technical debt based on the category of the technical debt.

[0144] Optionally, the identification unit 801 is specifically configured to:

[0145] If the operating data of the intelligent driving system satisfies at least one preset debt identification condition, determining that a technical debt exists in the intelligent driving system and determining a target debt identification condition satisfied by the operating data of the intelligent driving system;

[0146] According to the mapping relationship between the preset debt identification condition and the preset debt category, the category of the technical debt corresponding to the target debt identification condition is determined.

[0147] Optionally, the intelligent driving system includes an intelligent driving model; the intelligent driving model is used to generate intelligent driving behaviors;

[0148] The target debt identification condition includes that the number of network layers of the intelligent driving model is greater than a preset number of network layers, and the inference speed of the intelligent driving model decreases; the category of the technical debt corresponding to the target debt identification condition includes model debt; the optimization suggestion for the technical debt includes simplifying the model structure of the intelligent driving model.

[0149] Optionally, the generating unit 802 includes:

[0150] The priority determination subunit is used to determine the processing priority of the technical debt based on the category of the technical debt and the influence degree of the technical debt;

[0151] The generating subunit is used to generate an optimization suggestion for the technical debt based on the category of the technical debt and the processing priority of the technical debt.

[0152] Optionally, the generating subunit is specifically used for:

[0153] Before the intelligent driving system is updated, generate an optimization suggestion for the technical debt based on the category of the technical debt and the processing priority of the technical debt;

[0154] The intelligent driving device further includes:

[0155] The liquidation unit is used to liquidate the technical debt based on the optimization suggestion of the technical debt during the update process of the intelligent driving system;

[0156] The evaluation unit is used to determine the liquidation situation of the technical debt based on the operation data before the update of the intelligent driving system and the operation data after the update of the intelligent driving system;

[0157] The first processing unit is used to determine whether to update the intelligent driving system again based on the liquidation situation of the technical debt.

[0158] Optionally, the intelligent driving system includes an intelligent driving model;

[0159] The intelligent driving device further includes:

[0160] The extraction unit is used to extract the data features of the driving data of each modality;

[0161] The fusion unit is used to fuse the data features of the driving data of each modality to obtain the fused features;

[0162] The second processing unit is used to input the fused features into the intelligent driving model to obtain intelligent driving behavior.

[0163] Optionally, the fusion unit is specifically configured to:

[0164] Based on the data features of driving data of the same modality at different times, the temporal features of the same modality are obtained;

[0165] Based on the data features of the driving data of each modality at the same time, the spatial features at the same time are obtained;

[0166] The temporal feature and the spatial feature are fused to obtain a fused feature.

[0167] Optionally, the intelligent driving system includes an intelligent driving model; the intelligent driving model is used to generate intelligent driving behavior;

[0168] The intelligent driving device also includes:

[0169] a third processing unit, configured to obtain, based on the sample driving data in each driving scenario, a predicted intelligent driving behavior in each driving scenario using the intelligent driving model;

[0170] An acquisition unit, configured to acquire feedback data on predicted intelligent driving behaviors for various driving scenarios;

[0171] The first training unit is used to train the intelligent driving model based on the predicted intelligent driving behavior in each driving scenario and the feedback data of the predicted intelligent driving behavior in each driving scenario.

[0172] Optionally, the sample driving data includes multimodal sample driving data;

[0173] The third processing unit is specifically configured to:

[0174] Input the multimodal sample driving data under various driving scenarios into the multimodal data fusion model to obtain the sample fusion features;

[0175] Inputting the sample fusion features into the intelligent driving model to obtain predicted intelligent driving behaviors in various driving scenarios;

[0176] The intelligent driving device also includes:

[0177] The second training unit is used to train the multimodal data fusion model based on the predicted intelligent driving behavior in each driving scenario and the feedback data of the predicted intelligent driving behavior in each driving scenario.

[0178] The intelligent driving device provided in this embodiment belongs to the same inventive concept as the intelligent driving method provided in the above embodiments of the present application. It can execute the intelligent driving methods provided in any of the above embodiments of the present application and has corresponding functional modules and beneficial effects for executing the intelligent driving methods. For technical details not described in detail in this embodiment, reference can be made to the specific processing content of the intelligent driving method provided in the above embodiments of the present application, which will not be elaborated here.

[0179] The functions implemented by the above recognition unit 801 and generation unit 802 can be implemented by the same or different processors respectively, which is not limited in the embodiments of the present application.

[0180] It should be understood that the units in the above device can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or the functions of each unit of the device. The processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory inside the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of a hardware circuit. By designing the hardware circuit, part or all of the functions of the units can be implemented. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and through the design of the logical relationship of the components in the circuit, part or all of the functions of the above units are implemented. Another example is that in another implementation, the hardware circuit can be implemented by a PLD. Taking an FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured through a configuration file, so as to implement part or all of the functions of the above units. All units of the above device can be all implemented in the form of a processor calling software, or all implemented in the form of a hardware circuit, or part implemented in the form of a processor calling software, and the remaining part implemented in the form of a hardware circuit.

[0181] In the embodiments of the present application, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and running capabilities, such as a CPU, a microprocessor, a GPU, or a DSP, etc. In another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or can be reconfigured. For example, the processor is a hardware circuit implemented by an ASIC or a PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement part or all of the functions of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a kind of ASIC, such as an NPU, a TPU, a DPU, etc.

[0182] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, for example: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0183] In addition, each unit in the above device can be integrated in whole or in part, or can be independently implemented. In one implementation, these units are integrated together and implemented in the form of an SOC. The SOC can include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The types of the at least one processor can be different, for example, including CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0184] Exemplary electronic devices

[0185] An embodiment of the present application proposes an electronic device. Refer to Fig. 9 As shown, the device includes:

[0186] A memory 200 and a processor 210;

[0187] Wherein, the memory 200 is connected to the processor 210 and is used for storing programs;

[0188] The processor 210 is used to implement the intelligent driving method disclosed in any of the above embodiments by running the programs stored in the memory 200.

[0189] Specifically, the above electronic device may further include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0190] The processor 210, the memory 200, the communication interface 220, the input device 230, and the output device 240 are interconnected through the bus. Among them:

[0191] The bus may include a path for transmitting information between various components of the computer system.

[0192] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0193] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0194] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include program code, which includes computer operating instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), other types of dynamic storage devices that can store information and instructions, disk storage, flash memory, etc.

[0195] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0196] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speakers, etc.

[0197] The communication interface 220 may include any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0198] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of any intelligent driving method provided in the above embodiments of the present application.

[0199] Exemplary computer program products and storage media

[0200] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the intelligent driving method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0201] The computer program product may be written in any combination of one or more programming languages to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0202] In addition, an embodiment of the present application may also be a storage medium having a computer program stored thereon. The computer program is used by a processor to execute the steps of the intelligent driving method according to various embodiments of the present application described in any of the above embodiments of this specification. Specifically, the following steps may be implemented:

[0203] Step 101: Identify technical debt in the intelligent driving system and determine the category of the technical debt based on operating data of the intelligent driving system.

[0204] Step 102: Generate technical debt optimization suggestions based on the technical debt categories.

[0205] For the sake of simplicity, the aforementioned method embodiments are described as a series of action combinations. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0206] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similarities between the various embodiments can be referred to in conjunction with each other. For device embodiments, since they are generally similar to method embodiments, their description is relatively simple, and for relevant details, reference can be made to the description of the method embodiments.

[0207] The steps in the methods of each embodiment of the present application can be adjusted in sequence, merged, and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0208] The modules and sub-modules in the devices and terminals in the various embodiments of the present application can be merged, divided, and deleted according to actual needs.

[0209] In the several embodiments provided in this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative. For example, the division of modules or submodules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or module, which can be electrical, mechanical or other forms.

[0210] The modules or submodules described as separate components may or may not be physically separate, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules may be selected to achieve the purpose of this embodiment according to actual needs.

[0211] In addition, each functional module or submodule in each embodiment of the present application may be integrated into a processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into a single module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or software functional modules or submodules.

[0212] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0213] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, software units executed by a processor, or a combination of the two. The software units may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0214] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0215] The above description of the disclosed embodiments will enable those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is to be construed in the widest manner consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent driving method, characterized in that, Including: Identifying technical debts in the intelligent driving system based on the operation data of the intelligent driving system and determining the categories of the technical debts; Generating optimization suggestions for the technical debts based on the categories of the technical debts.

2. The intelligent driving method according to claim 1, wherein, The identifying technical debts in the intelligent driving system based on the operation data of the intelligent driving system and determining the categories of the technical debts includes: When the operation data of the intelligent driving system meets at least one preset debt identification condition, determining that there are technical debts in the intelligent driving system and determining the target debt identification condition that the operation data of the intelligent driving system meets; Determining the category of the technical debt corresponding to the target debt identification condition according to the mapping relationship between the preset debt identification condition and the preset debt category.

3. The intelligent driving method according to claim 2, wherein The intelligent driving system includes an intelligent driving model; the intelligent driving model is used to generate intelligent driving behaviors; The target debt identification condition includes that the number of network layers of the intelligent driving model is greater than the preset number of network layers and the inference speed of the intelligent driving model decreases; the category of the technical debt corresponding to the target debt identification condition includes model debt; the optimization suggestion for the technical debt includes simplifying the model structure of the intelligent driving model.

4. The intelligent driving method according to claim 1, characterized in that The generating optimization suggestions for the technical debts based on the categories of the technical debts includes: Determining the processing priority of the technical debt based on the category of the technical debt and the impact degree of the technical debt; Generating optimization suggestions for the technical debt based on the category of the technical debt and the processing priority of the technical debt.

5. The intelligent driving method according to claim 4, wherein The generating optimization suggestions for the technical debt based on the category of the technical debt and the processing priority of the technical debt includes: Before the intelligent driving system is updated, generating optimization suggestions for the technical debt based on the category of the technical debt and the processing priority of the technical debt; The method further includes: During the update of the intelligent driving system, clearing the technical debt based on the optimization suggestion for the technical debt; Determining the clearing situation of the technical debt based on the operation data before the update of the intelligent driving system and the operation data after the update of the intelligent driving system; Judging whether to update the intelligent driving system again based on the clearing situation of the technical debt.

6. The intelligent driving method according to claim 1, characterized in that The intelligent driving system includes an intelligent driving model; The method further includes: Extracting the data features of the driving data of each modality; Fusing the data features of the driving data of each modality to obtain the fused features; Inputting the fused features into the intelligent driving model to obtain intelligent driving behaviors.

7. The intelligent driving method according to claim 6, characterized in that The fusing the data features of the driving data of each modality to obtain the fused features includes: Obtaining the time features of the same modality based on the data features of the driving data of the same modality at different times; Obtaining the spatial features at the same moment based on the data features of the driving data of each modality at the same moment; Fusing the time features and the spatial features to obtain the fused features.

8. The intelligent driving method according to claim 1, wherein The intelligent driving system includes an intelligent driving model; the intelligent driving model is used to generate intelligent driving behaviors; The training process of the intelligent driving model includes: Based on the sample driving data in each driving scenario, obtaining the predicted intelligent driving behaviors in each driving scenario through the intelligent driving model; Obtaining feedback data for the predicted intelligent driving behaviors in each driving scenario; Training the intelligent driving model based on the predicted intelligent driving behaviors in each driving scenario and the feedback data for the predicted intelligent driving behaviors in each driving scenario.

9. The intelligent driving method according to claim 8, wherein The sample driving data includes multi-modal sample driving data; The step of obtaining the predicted intelligent driving behaviors in each driving scenario through the intelligent driving model based on the sample driving data in each driving scenario includes: Inputting the multi-modal sample driving data in each driving scenario into a multi-modal data fusion model to obtain the features after sample fusion; Inputting the features after sample fusion into the intelligent driving model to obtain the predicted intelligent driving behaviors in each driving scenario; The method further includes: Training the multi-modal data fusion model based on the predicted intelligent driving behaviors in each driving scenario and the feedback data for the predicted intelligent driving behaviors in each driving scenario.

10. An intelligent driving device, characterized in that, It includes: An identification unit, configured to identify the technical debt in the intelligent driving system and determine the category of the technical debt based on the operation data of the intelligent driving system; A generation unit, configured to generate optimization suggestions for the technical debt based on the category of the technical debt.

11. An electronic device, characterized in that, It includes a memory and a processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the intelligent driving method according to any one of claims 1 to 9 by running the programs in the memory.

12. A storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is run by the processor, the intelligent driving method according to any one of claims 1 to 9 is implemented.

13. A computer program product, characterized in that, It includes computer program instructions, and when the computer program instructions are run by the processor, the processor is caused to execute the intelligent driving method according to any one of claims 1 to 9.