Method and system for determining a control strategy, vehicle, storage medium and program product

Abstract

The application relates to a control strategy determination method and system, a vehicle, a storage medium and a program product, which comprise the following steps: a first perception module in a first system on chip is used to process perception information of a surrounding environment of a vehicle by using a first perception algorithm to obtain a first perception result corresponding to the vehicle; a second perception module in a second system on chip is used to process the perception information by using a second perception algorithm to obtain a second perception result corresponding to the vehicle; a perception result checking module is used to check and process the first perception result and the second perception result to obtain a perception checking result; in the case that the perception checking result is the second perception checking result, a recommended decision processing module is used to determine a perception processing result based on the perception information; and a regulation and control protection module in a processor is used to determine a target control strategy based on the perception processing result. The scheme provided by the application can improve the safety of intelligent driving.

Description

Technical Field

[0001] This application relates to the field of intelligent driving safety technology, specifically to a method and system for determining control strategies, a vehicle, a storage medium, and a program product. Background Technology

[0002] With the development of intelligent driving technology, driver assistance systems are widely used. However, anticipated functional safety issues, such as insufficient functionality of these systems or the ineffectiveness of drivers, are also increasing. In related technologies, these driver assistance systems typically only consider functional safety issues caused by the failure of electronic and electrical components within the vehicle, but do not consider anticipated functional safety issues in other aspects (e.g., under non-fault conditions), resulting in lower safety levels for intelligent driving systems. Summary of the Invention

[0003] One objective of this application is to provide a method for determining a control strategy to address the problem of low safety in existing intelligent driving technologies; another objective is to provide a system for determining a control strategy; a third objective is to provide a vehicle; a fourth objective is to provide a computer-readable storage medium; and a fifth objective is to provide a computer program product.

[0004] To achieve the above objectives, the technical solution adopted in this application is as follows:

[0005] A method for determining a control strategy, applied in a vehicle, the vehicle including a first system-on-a-chip, a second system-on-a-chip, a third system-on-a-chip, and a processor, the method comprising:

[0006] The first sensing module in the first chip system processes the sensing information of the vehicle's surrounding environment using the first sensing algorithm to obtain the first sensing result corresponding to the vehicle.

[0007] The second sensing module in the second chip system processes the sensing information using the second sensing algorithm to obtain the second sensing result corresponding to the vehicle.

[0008] The first and second perception results are processed by the perception result processing module in the third chip system to obtain the perception processing result.

[0009] The control and protection module in the processor determines the target control strategy based on the perception processing results; the target control strategy is used to control the vehicle.

[0010] Based on the aforementioned technical means, on the one hand, by integrating different perception modules into two on-chip systems respectively, the perceived information is processed separately to obtain corresponding perception results. Since this application integrates perception modules into the newly added on-chip system on the basis of the existing intelligent driving main function, hardware redundancy of the on-chip system and redundancy design of the perception modules are realized, ensuring the accuracy and robustness of perception. On the other hand, the perception result processing module in the third on-chip system processes the perception results output by the two perception modules, realizing redundancy verification of the perception results to further improve the accuracy of the perception results. Furthermore, the control and protection module in the processor determines the vehicle's target control strategy based on the perception processing results, realizing the construction of a redundant safety link outside the intelligent driving main function link. This allows the target control strategy output by the vehicle's driving assistance system to not only solve safety problems caused by electronic and electrical failures, but also solve expected functional safety problems under non-fault conditions, thereby improving the safety of intelligent driving and the comprehensiveness of expected functional safety protection.

[0011] Furthermore, the perception result processing module includes a perception result verification module and a recommendation decision processing module. The perception result processing module in the third-chip system processes the first and second perception results to obtain a perception processing result, including: verifying the first and second perception results through the perception result verification module to obtain a perception verification result; if the perception verification result is the first perception verification result, the first perception processing result is used as the perception processing result; wherein, the first perception verification result indicates that the first and second perception results match, and the first perception processing result indicates successful perception verification; if the perception verification result is the second perception verification result, the recommendation decision processing module determines the perception processing result based on the perception information; wherein, the second perception verification result indicates that the first and second perception results do not match.

[0012] Based on the above technical means, by verifying the first perception result and the second perception verification result, when the two do not match, the perception information is used again for perception processing by the recommendation decision processing module. This realizes multi-level processing of perception information, which not only improves the accuracy of perception, but also solves the problem of uncertainty of perception information in complex scenarios such as long-tail scenarios, thereby greatly improving the reliability and accuracy of control strategies in complex scenarios.

[0013] Furthermore, the recommendation decision processing module includes a first recommendation decision module, a second recommendation decision module, and a recommendation decision verification module. The recommendation decision processing module determines the perception processing result based on perceived information, including: determining first recommendation decision information based on the front-view image information in the perceived information using a first visual language model in the first recommendation decision module; determining second recommendation decision information based on the surrounding-view image information in the perceived information using a second visual language model in the second recommendation decision module; and determining the perception processing result based on the first and second recommendation decision information using the recommendation decision verification module.

[0014] Based on the aforementioned technical means, on the one hand, by deploying two visual language models in the two recommendation decision modules respectively, recommendation decision information is determined through different image information, thus realizing the redundant design of the recommendation decision modules and ensuring the accuracy of the recommendation decision information; on the other hand, the recommendation decision verification module performs redundant verification on the first recommendation decision information output by the first visual language model and the second recommendation decision information output by the second visual language model, so as to make the perception accuracy higher and greatly improve the passability and safety level of intelligent driving in complex scenarios.

[0015] Furthermore, based on the first recommended decision information and the second recommended decision information, the perception processing result is determined, including: if the first recommended decision information and the second recommended decision information match, the first perception processing result is used as the perception processing result; if the first recommended decision information and the second recommended decision information do not match, the second perception processing result is used as the perception processing result; wherein, the second perception processing result indicates that the perception verification failed.

[0016] Based on the above technical means, the perception processing result is determined by comparing two recommendation decision information to ensure the accuracy of perception. At the same time, the verification process of recommendation decision information is simple and reduces computing resources.

[0017] Furthermore, the processor includes a first processor and a second processor, and the regulatory protection module includes at least one regulatory module deployed in the first processor and a minimum risk strategy module deployed in the second processor; the regulatory protection module in the processor determines the target control strategy based on the perception processing result, including: when the perception processing result is a first perception processing result, determining the target control strategy based on the first control strategy output by each regulatory module; when the perception processing result is a second perception processing result, using the second control strategy output by the minimum risk strategy module as the target control strategy; wherein the second control strategy includes at least one of the following: intervention reminder, safe parking control.

[0018] Based on the aforementioned technical means, on the one hand, two processors, one primary and one redundant, are introduced at the control and protection end to achieve chip hardware redundancy design. A redundant control and protection link is built outside the main intelligent driving function link to ensure the accuracy and robustness of control and protection. On the other hand, the target control strategy is determined according to different perception processing results, which improves the accuracy of the target control strategy. Furthermore, if the perception processing result obtained after multi-level perception processing is a perception verification failure, the output of the first processor is terminated, and the second control strategy output by the minimum risk strategy module deployed in the second processor is immediately adopted as the target control strategy. This reduces the safety risks during vehicle driving and can actively enter a safe state when the vehicle malfunctions, achieving the expected functional safety fallback.

[0019] Furthermore, the control and protection module also includes at least one verification module and an arbitration module deployed in the second processor, with each verification module corresponding to one control module; based on the first control strategy output by each control module, a target control strategy is determined, including: for each control module, performing security boundary verification on the first control strategy output by the control module through the verification module corresponding to the control module to obtain the verification result of the control module; and determining the target control strategy based on the verification result of each control module through the arbitration module.

[0020] Based on the above technical means, on the one hand, the security boundary of the control strategy output by each verification module to its corresponding control module is verified to ensure the security of each control strategy; on the other hand, the target control strategy is determined by integrating the verification results of each control module through the arbitration module, thereby improving the accuracy of the target control strategy and realizing the protection of the control module.

[0021] Furthermore, based on the verification results of each regulatory control module, a target control strategy is determined, including: if the verification result of any regulatory control module indicates a verification failure, the second control strategy output by the minimum risk strategy module is used as the target control strategy; if the verification result of each regulatory control module indicates a verification success, the first control strategy output by each regulatory control module is used as the target control strategy.

[0022] According to the above technical means, when the verification result of any control module indicates that the verification fails, it means that the control strategy output by the module exceeds the safety boundary. In order to reduce safety risks, the second control strategy output by the minimum risk strategy module is directly used as the target control strategy to ensure that the vehicle can enter the minimum risk state. When the verification result of each control module indicates that the verification is successful, it means that the control strategies output by each control module are within the safety boundary. At this time, the vehicle can be controlled according to the first control strategies output by the first processor to ensure the safety of the vehicle.

[0023] Furthermore, if the perception processing result is the second perception processing result or the verification result of any regulation control module indicates a verification failure, the determination method further includes: uploading the intelligent driving data corresponding to the perception information to the cloud; wherein, the intelligent driving data is used for at least one of the following: training the visual language model in the perception result processing module, or updating the verification module corresponding to each regulation control module.

[0024] Based on the aforementioned technical means, if the perception processing result is the second perception processing result or the verification result of any regulation control module indicates a verification failure, it indicates that the verification modules corresponding to the first perception module, the second perception module, and the regulation control module may have problems. Therefore, the intelligent driving data corresponding to the perception information is uploaded to the cloud for iterative optimization of the visual language model and the verification modules corresponding to each regulation control module, continuously reducing the residual safety risks in intelligent driving and realizing real-time reasoning and human-like driving in complex scenarios.

[0025] A control strategy determination system, applied in a vehicle, the determination system includes:

[0026] The first perception module, deployed on the first chip system of the vehicle, is used to process the perception information of the vehicle's surrounding environment using the first perception algorithm to obtain the first perception result corresponding to the vehicle.

[0027] The second perception module, deployed on the second chip system of the vehicle, is used to process the perception information using the second perception algorithm to obtain the second perception result corresponding to the vehicle.

[0028] The perception result processing module, deployed on the third chip system of the vehicle, is used to process the first perception result and the second perception result to obtain the perception processing result;

[0029] The control and protection module, deployed in the vehicle's processor, is used to determine the target control strategy based on the perception processing results; the target control strategy is used to control the vehicle.

[0030] Based on the aforementioned technical means, on the one hand, by integrating different perception modules into two on-chip systems respectively, the perceived information is processed separately to obtain corresponding perception results. Since this application integrates perception modules into the newly added on-chip system on the basis of the existing intelligent driving main function, it realizes hardware redundancy of the on-chip system and redundancy design of the perception modules, ensuring the accuracy and robustness of perception. On the other hand, the perception result processing module in the third on-chip system processes the perception results output by the two perception modules, realizing redundancy verification of the perception results, thereby further improving the accuracy of the perception results. Furthermore, the control and protection module in the processor determines the vehicle's target control strategy based on the perception processing results, realizing the construction of a redundant safety link outside the intelligent driving main function link. This allows the target control strategy output by the vehicle's driving assistance system to not only solve safety problems caused by electronic and electrical failures, but also solve expected functional safety problems under non-fault conditions, thereby improving the safety of intelligent driving and the comprehensiveness of expected functional safety protection.

[0031] Furthermore, the perception result processing module includes a perception result verification module, a first recommendation decision module, a second recommendation decision module, and a recommendation decision verification module. The perception result verification module is used to verify the first perception result and the second perception result to obtain a perception verification result. The first recommendation decision module is used to determine the first recommendation decision information based on the front view image information in the perception information using a first visual language model. The second recommendation decision module is used to determine the second recommendation decision information based on the surrounding view image information in the perception information using a second visual language model. The recommendation decision verification module is used to determine the perception processing result based on the first recommendation decision information and the second recommendation decision information.

[0032] Based on the aforementioned technical means, on the one hand, by deploying two visual language models in the two recommendation decision modules respectively, recommendation decision information is determined through different image information, thus realizing the redundant design of the recommendation decision modules and ensuring the accuracy of the recommendation decision information; on the other hand, the recommendation decision verification module performs redundant verification on the first recommendation decision information output by the first visual language model and the second recommendation decision information output by the second visual language model, so as to make the perception accuracy higher and greatly improve the passability and safety level of intelligent driving in complex scenarios.

[0033] Furthermore, the processor includes a first processor and a second processor. The control and protection module includes at least one control module, a verification module, an arbitration module, and a minimum risk strategy module corresponding to each control module. Each control module is deployed in the first processor and is used to output a corresponding first control strategy. Each verification module is deployed in the second processor and is used to verify the first control strategy output by the corresponding control module to obtain the verification result of each control module. The arbitration module is deployed in the second processor and is used to take the second control strategy output by the minimum risk strategy module as the target control strategy if the verification result of any control module indicates verification failure; and to take the first control strategy output by each control module as the target control strategy if the verification result of each control module indicates verification success.

[0034] Based on the above technical means, on the one hand, the security boundary of the control strategy output by each verification module to its corresponding control module is verified to ensure the security of each control strategy; on the other hand, the target control strategy is determined by integrating the verification results of each control module through the arbitration module, thereby improving the accuracy of the target control strategy and realizing the protection of the control module.

[0035] A vehicle includes a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the program to implement any of the methods described above.

[0036] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods described above.

[0037] A computer program product includes a computer program or instructions that, when executed by a processor, implement the method described in any of the preceding claims.

[0038] The beneficial effects of this application are:

[0039] (1) By integrating different perception modules into two on-chip systems respectively, the perception information is processed to obtain the corresponding perception results. Since this application integrates a perception module into the newly added on-chip system on the basis of the existing intelligent driving main function, the hardware redundancy of the on-chip system and the redundancy design of the perception module are realized, ensuring the accuracy and robustness of perception. On the other hand, the perception result processing module in the third on-chip system processes the perception results output by the two perception modules, realizing the redundancy verification of the perception results, so as to further improve the accuracy of the perception results. Moreover, it solves the problem of uncertainty of perception information in complex scenarios such as long-tail scenarios, thereby greatly improving the reliability and accuracy of the control strategy in complex scenarios.

[0040] (2) By deploying two visual language models in the two recommendation decision modules of the third chip system respectively, the recommendation decision information is determined by different image information, thus realizing the redundant design of the recommendation decision module and ensuring the accuracy of the recommendation decision information. The recommendation decision verification module performs redundant verification on the first recommendation decision information output by the first visual language model and the second recommendation decision information output by the second visual language model, so as to make the perception accuracy higher and greatly improve the passability and safety level of intelligent driving in complex scenarios.

[0041] (3) Two processors, one main and one redundant, are introduced at the control and protection end to realize the chip hardware redundancy design. A redundant control and protection link is built outside the main function link of intelligent driving to ensure the accuracy and robustness of control and protection. This enables the target control strategy output by the vehicle's driving assistance system to not only solve the safety problems caused by electronic and electrical failures, but also solve the expected functional safety problems under non-fault conditions, thereby improving the safety of intelligent driving and the comprehensiveness of expected functional safety protection.

[0042] (4) If the perception processing result obtained after multi-level perception processing is a perception verification failure, the output of the first processor is terminated, and the second control strategy output by the minimum risk strategy module deployed in the second processor is immediately adopted as the target control strategy, which reduces the safety risks during vehicle driving and can actively enter a safe state when the vehicle malfunctions, thus achieving the expected functional safety fallback.

[0043] (5) The control strategies output by the corresponding regulatory control modules in the first processor are verified according to the verification modules in the second processor to ensure the safety of each control strategy; the target control strategy is determined by integrating the verification results of each regulatory control module through the integrated arbitration module, which improves the accuracy of the target control strategy and realizes the security protection of the regulatory control module.

[0044] (6) If the perception verification fails or the verification result of any regulation control module fails, it indicates that the verification modules corresponding to the first perception module, the second perception module, and the regulation control module may have problems. In this case, the intelligent driving data corresponding to the perception information is uploaded to the cloud to iteratively optimize the visual language model and the verification modules corresponding to each regulation control module, continuously reduce the residual safety risks in intelligent driving, and realize real-time reasoning and human-like driving in complex scenarios. Attached Figure Description

[0045] Figure 1 A schematic diagram of the implementation process of a method for determining a control strategy provided in this application embodiment. Figure 1 ;

[0046] Figure 2A schematic diagram of the composition structure of a control strategy determination system provided in this application embodiment. Figure 1 ;

[0047] Figure 3 A schematic diagram of the composition structure of a control strategy determination system provided in this application embodiment. Figure 2 ;

[0048] Figure 4 A schematic diagram of the composition structure of a control strategy determination system provided in this application embodiment. Figure 3 ;

[0049] Figure 5 A schematic diagram of the composition structure of a control strategy determination system provided in this application embodiment. Figure 4 ;

[0050] Figure 6 A schematic diagram of the implementation process of a method for determining a control strategy provided in this application embodiment. Figure 2 ;

[0051] Figure 7 This is a schematic diagram of the hardware entity of a vehicle provided in an embodiment of this application. Detailed Implementation

[0052] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0053] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0054] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0055] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0056] In this embodiment, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0058] The method provided in this application embodiment can be executed by a vehicle, which may include, but is not limited to, sedans, sports cars, SUVs, commercial vehicles, engineering vehicles, etc. The vehicle includes at least a first system-on-a-chip (SOC), a second system-on-a-chip, a third system-on-a-chip, and a processor. The computing power levels of the first SOC, second SOC, and third SOC may be the same or different. For example, the first SOC and second SOC may have the same level of computing power, but the computing power level of the first SOC may be higher than that of the third SOC.

[0059] The technical solutions in the embodiments of this application will now be clearly and completely described with reference to the accompanying drawings.

[0060] Figure 1 A schematic diagram of the implementation process of a method for determining a control strategy provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the method includes steps S11 to S14, wherein:

[0061] Step S11: The first sensing module of the first on-chip system processes the sensing information of the vehicle's surrounding environment using the first sensing algorithm to obtain the first sensing result corresponding to the vehicle.

[0062] Here, the first SOC is equipped with at least a first perception module, which integrates a first perception algorithm. The first perception algorithm can be any suitable perception algorithm.

[0063] During vehicle operation, perception information about the surrounding environment can be acquired. This perception information may include, but is not limited to, images, radar, etc. The method of acquiring this perception information can be any suitable method. In some examples, the vehicle can acquire perception information in real time through perception components, which may include, but are not limited to, cameras, radar, etc. In some examples, the vehicle can acquire perception information archived by the perception components from a local archive.

[0064] The perception result (including the first and second perception results) refers to the identification result of the perceived information. This perception result may include, but is not limited to, the presence of a target and the target's attributes. The target can be any suitable object, such as the road surface, other vehicles, pedestrians, lane lines, road signs, etc. The target's attributes may include, but are not limited to, type, speed, position, acceleration, etc. It is understood that different types of targets have corresponding attributes.

[0065] In some implementations, the sensing component can be connected to a first SOC so that the first SOC can acquire the sensing information collected by the sensing component.

[0066] Step S12: The second sensing module in the second on-chip system processes the sensing information using the second sensing algorithm to obtain the second sensing result corresponding to the vehicle.

[0067] Here, the second SOC deploys at least a second sensing module, which integrates a second sensing algorithm. The second sensing algorithm can be any suitable sensing algorithm. In some embodiments, the sensing component can be connected to the second SOC, enabling the second SOC to acquire the sensing information collected by the sensing component.

[0068] In the embodiments of this application, the first perception algorithm and the second perception algorithm may be different algorithms, but they are both used to process the perceived information to obtain the identified target.

[0069] Step S13: The first and second perception results are processed by the perception result processing module in the third chip system to obtain the perception processing result.

[0070] Here, the third SOC deploys at least a perception result processing module, which can be any suitable module capable of performing this function. This perception result processing module is mainly used to process the first perception result output by the first SOC and the second perception result output by the second SOC.

[0071] The perception processing results may include, but are not limited to, a first perception processing result and a second perception processing result. The first perception processing result indicates that the perception verification was successful, and the second perception processing result indicates that the perception verification failed.

[0072] The method for determining the result of the perception processing can be any suitable method.

[0073] In some implementations, when the first perception result and the second perception result match, the first perception processing result is taken as the perception processing result. Matching the first perception result and the second perception result means that the similarity between the first perception result and the second perception result is greater than a preset similarity threshold. For example, if the first perception result and the second perception result are the same, then the first perception result and the second perception result are considered to match.

[0074] In some implementations, when the first and second perception results do not match, the second perception processing result can be used as the perception processing result. Alternatively, the recommendation decision processing module in the perception result processing module can be used to further determine the perception processing result based on the perception information.

[0075] Understandably, due to the different amounts of data being processed, the first, second, and third SOCs can be on-chip systems with varying computing power to avoid wasting computing resources. For example, the first and second SOCs can have the same level of high computing power, while the third SOC can have medium computing power.

[0076] Step S14: Based on the perception processing results, the control and protection module in the processor determines the target control strategy.

[0077] Here, the target control strategy is used to control the vehicle. The processor includes at least a control protection module, which can be any suitable module capable of outputting the control strategy. In implementation, different perception processing results will output different control strategies.

[0078] In one possible implementation, if the perception processing result indicates successful perception verification, it can be determined that the current perception function is normal. At this point, a conventional control strategy can be used as the target control strategy. Conventional control strategies may include, but are not limited to, acceleration, deceleration, braking, lane changing, and constant speed.

[0079] In another possible implementation, if the perception processing result indicates a perception verification failure, a preset minimum risk strategy can be used as the target control strategy to reduce safety risks. The minimum risk strategy may include, but is not limited to, intervention reminders and safe parking control. Safe parking control includes parking in the current lane and parking on the side of the road. In practice, if a safe lane change is met, parking on the side of the road is executed; if a safe lane change is not met, parking in the current lane is executed.

[0080] In this application embodiment, on the one hand, by integrating different perception modules into two on-chip systems respectively, the perception information is processed to obtain corresponding perception results. Since this application integrates perception modules into the newly added on-chip system on the basis of the existing intelligent driving main function, it realizes hardware redundancy of the on-chip system and redundancy design of the perception module, ensuring the accuracy and robustness of perception. On the other hand, the perception result processing module in the third on-chip system processes the perception results output by the two perception modules, realizing redundancy verification of the perception results, so as to further improve the accuracy of the perception results. Furthermore, the control and protection module in the processor determines the vehicle's target control strategy based on the perception processing results, realizing the construction of a redundant safety link outside the intelligent driving main function link. This allows the target control strategy output by the vehicle's driving assistance system to not only solve safety problems caused by electronic and electrical failures, but also solve expected functional safety problems under non-fault conditions, thereby improving the safety of intelligent driving and the comprehensiveness of expected functional safety protection.

[0081] In some embodiments, the perception result processing module includes a perception result verification module and a recommendation decision processing module, and step S13 may include steps S21 to 23, wherein:

[0082] Step S21: The first and second perception results are verified by the perception result verification module to obtain the perception verification result.

[0083] Here, the perception result verification module can be any suitable module capable of implementing this function. This module primarily verifies the first and second perception results to determine if they match. The perception verification results may include, but are not limited to, the first and second perception verification results. The first perception verification result indicates that the first and second perception results match, while the second perception verification result indicates that the first and second perception results do not match.

[0084] In some examples, it is possible to detect whether the difference between the first perception result and the second perception result meets a preset threshold. If the preset threshold is not met, it is determined that the first perception result and the second perception result do not match. If the preset threshold is met, it is determined that the first perception result and the second perception result match. Specifically, the detection process may be to detect whether the attributes of the target are too different.

[0085] In some examples, consistency matching between the first and second perception results can be detected, such as whether the target is the same or whether the target type is the same. If all targets are the same, the first and second perception results are considered to match; if there are differences among the targets, the first and second perception results are considered to not match.

[0086] Step S22: If the perception verification result is the first perception verification result, the first perception processing result is used as the perception processing result.

[0087] Here, if the first perception result and the second perception result match, it indicates that the perception verification is successful. In this case, the first perception processing result is taken as the perception processing result.

[0088] Step S23: If the perception verification result is the second perception verification result, the recommendation decision processing module determines the perception processing result based on the perception information.

[0089] Here, if the first perception result and the second perception result do not match, the perception information needs to be further processed to obtain the perception processing result. The recommendation decision processing module can be any suitable module capable of implementing this function. In the embodiments of this application, to avoid misjudgment by the perception result verification module and to further protect the perception function, the vehicle can also further process the perception information through the recommendation decision module to obtain the perception processing result.

[0090] In this embodiment of the disclosure, by verifying the first perception result and the second perception verification result, when the two do not match, the perception information is used again for perception processing by the recommendation decision processing module. This realizes multi-level processing of perception information, which not only improves the accuracy of perception, but also solves the problem of uncertainty of perception information in complex scenarios such as long-tail scenarios, thereby greatly improving the reliability and accuracy of control strategies in complex scenarios.

[0091] In some implementations, the recommendation decision processing module includes a first recommendation decision module, a second recommendation decision module, and a recommendation decision verification module. Step S23, "determining the perception processing result based on the perception information through the recommendation decision processing module," may include the following steps S231 to S233, wherein:

[0092] Step S231: Determine the first recommendation decision information based on the front view image information in the perceived information through the first visual language model in the first recommendation decision module.

[0093] Step S232: Determine the second recommendation decision information based on the surrounding image information in the perceived information through the second visual language model in the second recommendation decision module.

[0094] Step S233: The perception processing result is determined by the recommendation decision verification module based on the first recommendation decision information and the second recommendation decision information.

[0095] Here, the recommendation decision module (including the first recommendation decision module and the second recommendation decision module) can be any suitable module capable of implementing this function. This recommendation decision module primarily generates recommendation decision information based on perceived information.

[0096] The visual language model (including the first visual language model and the second visual language model) can be any suitable neural network model. The visual language model is used to reason about the input image to obtain recommendation decision information. Recommendation decision information may include, but is not limited to, weather conditions, road type, target type, target features, and simple decision information. Decision information may include, but is not limited to, constant speed driving, acceleration, deceleration, braking, changing lanes left, and changing lanes right.

[0097] The first recommendation decision module integrates at least a first visual language model. By inputting the front view image information into the first visual language model, the first recommendation decision information can be obtained.

[0098] The second recommendation decision module integrates at least a second visual language model. By inputting the panoramic image information into the second visual language model, the second recommendation decision information can be obtained.

[0099] The recommendation decision verification module can be any suitable module that can implement this function. The recommendation decision verification module is mainly used to verify the first recommendation decision information output by the first visual language model and the second recommendation decision information output by the second visual language model in order to obtain the final perception processing result.

[0100] In some examples, the first and second recommendation decision information can be verified based on the pre-trained recommendation decision verification model in the recommendation decision verification module.

[0101] In some examples, a simple matching and verification can be performed on the first and second recommended decision information.

[0102] During implementation, the perception processing result is determined based on the verification results of the two recommendation decision information.

[0103] In this embodiment, on the one hand, by deploying two visual language models in the two recommendation decision modules respectively, recommendation decision information is determined through different image information, thus realizing the redundant design of the recommendation decision modules and ensuring the accuracy of the recommendation decision information; on the other hand, the recommendation decision verification module performs redundant verification on the first recommendation decision information output by the first visual language model and the second recommendation decision information output by the second visual language model, so as to make the perception accuracy higher and greatly improve the passability and safety level of intelligent driving in complex scenarios.

[0104] In some implementations, the step S233, "determining the perception processing result based on the first recommendation decision information and the second recommendation decision information," may include steps S2331 and S2332, wherein:

[0105] Step S2331: If the first recommendation decision information and the second recommendation decision information match, the first perception processing result is used as the perception processing result;

[0106] Step S2332: If the first recommended decision information and the second recommended decision information do not match, the second perception processing result shall be used as the perception processing result.

[0107] Here, matching the first and second recommended decision information means that the similarity between the first and second recommended decision information is greater than a preset similarity threshold. For example, if the first and second recommended decision information are the same, then the first and second recommended decision information are considered to match; otherwise, they are considered not to match.

[0108] In this way, the perception processing result is determined by comparing two recommendation decision information to ensure the accuracy of perception. At the same time, the verification process of recommendation decision information is simple and reduces computing resources.

[0109] In some embodiments, the processor includes a first processor and a second processor, and the regulatory protection module includes at least one regulatory module deployed in the first processor and a minimum risk strategy module deployed in the second processor. Step S14 may include steps S31 and S32, wherein:

[0110] Step S31: If the perception processing result is the first perception processing result, determine the target control strategy based on the first control strategy output by each control module.

[0111] Here, when the perception processing result is the first perception processing result, it can be determined that the perception verification is successful and the current vehicle's perception function is normal. At this time, the vehicle can perform overall vehicle strategy planning based on the various control modules in the first processor.

[0112] The first processor can be the main control chip, primarily responsible for calculating normal planning and control instructions. It can be understood that the various planning and control modules within the first processor can be existing planning and control modules in intelligent driving systems. These modules can include, but are not limited to, planning modules, decision-making modules, lateral planning and control modules, and longitudinal planning and control modules. The planning module is mainly used to plan the driving trajectory. The decision-making module is mainly used to determine whether to change lanes. The lateral planning and control module is mainly used to plan and control the vehicle's lateral driving information. The longitudinal planning and control module is mainly used to plan and control the vehicle's longitudinal driving information.

[0113] The target control strategy can be any of the first control strategies, or it can be the second control strategy output by the minimum risk strategy module.

[0114] The first control strategy can be a conventional control strategy, such as acceleration request, deceleration request, steering wheel angle request, etc.

[0115] In some embodiments, the vehicle can fuse various first control strategies to obtain a target control strategy with richer objectives.

[0116] Step S32: If the perception processing result is the second perception processing result, the second control strategy output by the minimum risk strategy module is used as the target control strategy.

[0117] Here, when the perception processing result is the second perception processing result, the second control strategy output by the minimum risk strategy module can be used as the target control strategy. The second control strategy includes at least one of the following: intervention reminder and safe parking control. Intervention reminders may include, but are not limited to, displaying alarm text and images through the vehicle's terminal equipment, outputting alarm sounds through the vehicle's audio system, and flashing warning lights. Safe parking control may include parking in the same lane or parking on the side of the road.

[0118] In this embodiment, on the one hand, two processors, one primary and one redundant, are introduced at the control and protection end to realize the chip hardware redundancy design. A redundant control and protection link is built outside the main intelligent driving function link to ensure the accuracy and robustness of the control and protection. On the other hand, the target control strategy is determined according to different perception processing results, which improves the accuracy of the target control strategy. Furthermore, if the perception processing result obtained after multi-level perception processing is a perception verification failure, the output of the first processor is terminated, and the second control strategy output by the minimum risk strategy module deployed in the second processor is immediately adopted as the target control strategy. This reduces the safety risks during vehicle driving and can actively enter a safe state when the vehicle malfunctions, achieving the expected functional safety fallback.

[0119] In some embodiments, the control and protection module further includes at least one verification module and an arbitration module deployed in the second processor, each verification module corresponding to one control module. The step S31, "determining the target control strategy based on the first control strategy output by each control module," may include the following steps S311 and S312, wherein:

[0120] Step S311: For each planning and control module, the first control strategy output by the planning and control module is verified by the corresponding verification module to obtain the verification result of the planning and control module.

[0121] Step S312: The target control strategy is determined through the arbitration module based on the verification results of each control module.

[0122] Here, the second processor can be a redundant chip, and it deploys a verification module corresponding to each planning and control module. This verification module is used to verify the first control strategy output by the corresponding planning and control module. It is understood that the number of verification modules is the same as the number of planning and control modules. In some implementations, this verification module may include, but is not limited to, a planning verification module, a decision verification module, a horizontal verification module, and a vertical verification module.

[0123] The planning verification module is mainly used to verify the rationality of trajectory coefficients, the safety of trajectory coefficients, and the rationality of deviations between actual and expected lane-changing trajectories. Here, trajectory coefficients are preset parameters used to characterize the vehicle's running trajectory, such as trajectory curvature and trajectory path points. The decision verification module is mainly used to verify the rationality of system-initiated lane-changing decisions, driver-triggered lane-changing decisions, driving strategies within ramps, and decisions to stop and return to the original lane after a lane change. The lateral verification module is mainly used to verify the rationality of lateral control strategies when driving within a lane and when changing lanes. Lateral control includes steering wheel angle control and steering wheel angular velocity control. The longitudinal verification module is mainly used to verify the rationality of acceleration, speed, and following distance.

[0124] Here, the verification results of the control module may include, but are not limited to, a first verification result and a second verification result. The first verification result indicates that the verification was successful, and the second verification result indicates that the verification failed.

[0125] The arbitration module can be any suitable module capable of performing this function. Its primary purpose is to determine the target control strategy.

[0126] In this embodiment, a corresponding verification module is also provided for each planning and control module, so that each verification module sets different safety boundaries for different planning and control modules, and performs safety boundary verification on the first control strategy output by each planning and control module according to the safety boundaries, so as to ensure the accuracy of planning.

[0127] In some implementations, the step S31, "determining the target control strategy based on the verification results of each control module," includes steps S311 and S312, wherein:

[0128] Step S311: If the verification result of any control module indicates a verification failure, the second control strategy output by the minimum risk strategy module shall be used as the target control strategy.

[0129] The second control strategy can be a minimum risk strategy, which may include, but is not limited to, intervention reminders and safe parking control. Safe parking control includes parking in the same lane and parking on the side of the road.

[0130] Step S312: If the verification result of each control module indicates that the verification is successful, the first control strategy output by each control module shall be used as the target control strategy.

[0131] Thus, when the verification result of any control module indicates a verification failure, meaning that its output control strategy exceeds the safety boundary, then, in order to reduce safety risks, the second control strategy output by the minimum risk strategy module is directly used as the target control strategy to ensure that the vehicle can enter the minimum risk state. When the verification result of each control module indicates a verification success, meaning that the control strategies output by each control module are within the safety boundary, then the vehicle can be controlled according to the first control strategies output by the first processor to ensure the safety of the vehicle.

[0132] In some embodiments, if the sensing processing result is a second sensing processing result or the verification result of any regulatory control module indicates a verification failure, the determination method provided in this application further includes step S15:

[0133] Step S15: Upload the intelligent driving data corresponding to the perceived information to the cloud.

[0134] Among them, intelligent driving data is used for at least one of the following: training the visual language model in the perception result processing module, and updating the verification module corresponding to each regulation and control module.

[0135] In this embodiment, if the perception processing result is either the second perception processing result or the verification result of any control module indicating a verification failure, the vehicle can determine that there is a defect in the current perception and control functions. The vehicle can then upload the intelligent driving data perception result processing module corresponding to the perception information to the cloud. The intelligent driving data can be data within a preset time range before and after the perception information is received. The preset time range can be any suitable range, such as 10 seconds. The intelligent driving data can include, but is not limited to, video image data, perception processing results, and the first control strategy.

[0136] The vehicle uploads intelligent driving data to the cloud so that the backend (which can be the OEM's data closed-loop platform) can obtain the intelligent driving data from the cloud, update and optimize the visual language model in the perception result processing module or the verification module corresponding to each control module, and train the first perception algorithm and the second perception algorithm.

[0137] For example, video data from intelligent driving data can be input into the visual language model for training.

[0138] For example, the training of the first perception algorithm and the second perception algorithm can be carried out using perception information from different times in the intelligent driving data until the first perception result and the second perception result output by the two are consistent.

[0139] For example, in the update and optimization process of each verification module, the false trigger rate of security boundary verification failure is calculated, and the security boundary threshold is continuously optimized until the false trigger rate meets the preset threshold.

[0140] In the embodiments of this application, if the perception processing result is the second perception processing result or the verification result of any regulation control module indicates that the verification module corresponding to the first perception module, the second perception module, and the regulation control module may have problems, then the intelligent driving data corresponding to the perception information is uploaded to the cloud for iterative optimization of the visual language model and the verification module corresponding to each regulation control module, continuously reducing the residual safety risks in intelligent driving, and realizing real-time reasoning and human-like driving in complex scenarios.

[0141] Based on the above embodiments, this application also provides a system for determining control strategies. Figure 2 A schematic diagram of the composition structure of a control strategy determination system provided in this application embodiment. Figure 1 ,like Figure 2 As shown, the determining system 200 includes:

[0142] The first perception module 201 is deployed on the first on-chip system of the vehicle and is used to process the perception information of the vehicle's surrounding environment using the first perception algorithm to obtain the first perception result corresponding to the vehicle.

[0143] The second perception module 202 is deployed on the second chip system of the vehicle and is used to process the perception information using the second perception algorithm to obtain the second perception result corresponding to the vehicle.

[0144] The perception result processing module 203 is deployed on the third chip system of the vehicle and is used to process the first perception result and the second perception result to obtain the perception processing result.

[0145] The control and protection module 204 is deployed in the vehicle's processor and is used to determine the target control strategy based on the perception processing results; wherein, the target control strategy is used to control the vehicle.

[0146] Here, the first perception module 201 is deployed in the vehicle's first SOC. The first perception module 201 integrates a first perception algorithm, which can be any suitable perception algorithm. In implementation, the first SOC can be connected to any perception component in any connection method to obtain real-time perception information of the vehicle's surrounding environment output by the perception component. This perception component can include, but is not limited to, cameras, radar, etc.; the first SOC can also obtain perception information archived by the perception component from a local archive.

[0147] The sensing component can also be connected to the second SOC in any manner, enabling the second SOC to acquire sensing information about the vehicle's surrounding environment through the sensing component. The second SOC deploys a second sensing module 202, which integrates a second sensing algorithm. This second sensing algorithm can be any suitable sensing algorithm; it is understood that the first sensing algorithm and the second sensing algorithm can be different algorithms, but both are used to process the sensing information to obtain the identified target.

[0148] The first perception result is obtained by processing the perceived information using the first perception algorithm in the first perception module 201, and the second perception result is obtained by processing the perceived information using the second perception algorithm in the second perception module 202. Here, the perception result (including the first and second perception results) refers to the recognition result of the perceived information. This perception result may include, but is not limited to, the existence of a target and the attributes of the target. The target can be any suitable object, such as the road surface, other vehicles, pedestrians, lane lines, road signs, etc. The attributes of the target may include, but are not limited to, type, speed, position, acceleration, etc. It is understood that different types of targets have corresponding attributes.

[0149] Both the first SOC and the second SOC have arbitrary connection relationships with the third SOC. When the first sensing module 201 obtains the first sensing processing result, the first SOC will input the first sensing result into the third SOC; when the second sensing module 202 obtains the second sensing processing result, the second SOC will input the second sensing result into the third SOC.

[0150] Here, the third SOC is equipped with at least a perception result processing module 203, which can be any suitable module capable of performing this function. This perception result processing module 203 is mainly used to process the first and second perception results to obtain the perception processing result.

[0151] The perception processing results may include, but are not limited to, a first perception processing result and a second perception processing result. The first perception processing result indicates that the perception verification was successful, and the second perception processing result indicates that the perception verification failed.

[0152] The process by which the perception result processing module 203 in the third SOC determines the perception processing result can be referred to the specific implementation of step S13 above, and will not be repeated here.

[0153] Understandably, due to the different amounts of data being processed, the first, second, and third SOCs can be on-chip systems with varying computing power to avoid wasting computing resources. For example, the first and second SOCs can have the same level of high computing power, while the third SOC can have medium computing power.

[0154] The third SOC has an arbitrary connection to the processor. After the perception result processing module 203 determines the perception processing result, the third SOC will output the perception processing result to the processor for further processing. Here, the processor is equipped with at least a control and protection module 204, which can be any suitable module capable of outputting control strategies. In implementation, the control and protection module 204 outputs different control strategies based on different perception processing results.

[0155] The process by which the control and protection module 204 in the processor determines the target control strategy based on the perception processing results can be referred to the specific implementation of step S14 above, and will not be repeated here.

[0156] In some embodiments, such as Figure 3 As shown, the control strategy determination system provided in this application includes a first SOC, a second SOC, a third SOC, and a processor. The first SOC is equipped with at least a first sensing module 201, the second SOC is equipped with at least a second sensing module 202, the third SOC is equipped with at least a sensing result processing module 203, and the processor is equipped with at least a rule and control protection module 204.

[0157] In some embodiments, such as Figure 4 As shown, the perception result processing module 203 includes a perception result verification module 2031, a first recommendation decision module 2032, a second recommendation decision module 2033, and a recommendation decision verification module 2034, wherein:

[0158] The perception result verification module 2031 is used to verify the first perception result and the second perception result to obtain the perception verification result.

[0159] The first recommendation decision module 2032 is used to determine the first recommendation decision information based on the front view image information in the perceived information using the first visual language model.

[0160] The second recommendation decision module 2033 is used to determine the second recommendation decision information based on the surrounding image information in the perceived information using the second visual language model.

[0161] The recommendation decision verification module 2034 is used to determine the perception processing result based on the first recommendation decision information and the second recommendation decision information.

[0162] Here, the perception result verification module 2031 can be any suitable module capable of implementing this function. After the third SOC obtains the first perception result output by the first SOC and the second perception result output by the second SOC, it will perform verification processing on the first and second perception results through the perception result verification module to verify whether the two perception results match. The perception verification result may include, but is not limited to, the first perception verification result and the second perception verification result. Among them, the first perception verification result indicates that the first and second perception results match, and the second perception verification result indicates that the first and second perception results do not match.

[0163] The process by which the perception result verification module 2031 verifies the first perception result and the second perception result can be referred to the specific implementation of the aforementioned step S21, and will not be repeated here.

[0164] During implementation, if the perception verification result is the first perception result, the third SOC will output the first perception processing result as the perception processing result; if the perception verification result is the second perception result, the first perception result and the second perception result do not match. In order to avoid misjudgment by the perception result verification module 2031 and to further protect the perception function, the third SOC will continue to process the data through the first recommendation decision module 2032 and the second recommendation decision module 2033 to obtain the final perception processing result.

[0165] Here, the recommendation decision module (including the first recommendation decision module 2032 and the second recommendation decision module 2033) can be any suitable module capable of implementing this function. Through the visual language model (including the first visual language model and the second visual language model) in the recommendation decision module, image information in the perceived information can be processed. Here, the visual language model (including the first visual language model and the second visual language model) can be any suitable neural network model. The visual language model is used to reason about the input image to obtain recommendation decision information. The recommendation decision information may include, but is not limited to, weather environment, road type, target type, target features, and simple decision information. Simple decision information may include, but is not limited to, constant speed driving, acceleration, deceleration, braking, changing lanes left, and changing lanes right.

[0166] After obtaining the first recommendation decision information output by the first recommendation decision module 2032 and the second recommendation decision information output by the second recommendation decision module 2033, the third SOC will verify the first and second recommendation decision information through the recommendation decision verification module 2034. Here, the recommendation decision verification module 2034 can be any suitable module capable of implementing this function. The recommendation decision verification module 2034 is mainly used to verify the first recommendation decision information output by the first visual language model and the second recommendation decision information output by the second visual language model to obtain the final perception processing result.

[0167] The process by which the recommendation decision verification module 2034 verifies the first recommendation decision information and the second recommendation decision information can be referred to the specific implementation of the aforementioned step S233, and will not be repeated here.

[0168] In this embodiment, on the one hand, by deploying two visual language models in the two recommendation decision modules respectively, recommendation decision information is determined through different image information, thus realizing the redundant design of the recommendation decision module and ensuring the accuracy of the recommendation decision information; on the other hand, the recommendation decision verification module performs redundant verification on the first recommendation decision information output by the first visual language model and the second recommendation decision information output by the second visual language model, so as to make the perception accuracy higher and greatly improve the passability and safety level of intelligent driving in complex scenarios.

[0169] In some embodiments, such as Figure 5 As shown, the processor includes a first processor and a second processor, and the regulation and protection module 204 includes at least one regulation and protection module 2041, a verification module 2042 corresponding to each regulation and protection module, an arbitration module 2043, and a minimum risk strategy module 2044, wherein:

[0170] Each control module 2041 is deployed in the first processor and is used to output the corresponding first control strategy.

[0171] Each verification module 2042 is deployed in the second processor and is used to verify the first control strategy output by the corresponding control module 2041 to obtain the verification result of each control module 2041.

[0172] Arbitration module 2043, deployed in the second processor, is used to take the second control strategy output by minimum risk strategy module 2044 as the target control strategy when the verification result of any control module 2041 indicates verification failure; and to take the first control strategy output by each control module 2041 as the target control strategy when the verification result of each control module 2041 indicates verification success.

[0173] Here, the first processor can be the main control chip, which primarily performs normal planning and control instruction calculations. It can be understood that the various planning and control modules 2041 within the first processor can be existing planning and control modules 2041 in intelligent driving systems. These planning and control modules 2041 may include, but are not limited to, planning modules, decision-making modules, lateral planning and control modules, and longitudinal planning and control modules. The planning module is mainly used to plan the driving trajectory. The decision-making module is mainly used to determine whether to change lanes. The lateral planning and control module is mainly used to plan and control the vehicle's lateral driving information. The longitudinal planning and control module is mainly used to plan and control the vehicle's longitudinal driving information.

[0174] When the perception processing result is the first perception processing result during implementation, it can be determined that the perception verification is successful and the current vehicle's perception function is normal. At this time, the vehicle can perform overall vehicle strategy planning according to each control module 2041 in the first processor to obtain the first control strategy output by each control module 2041. Here, the first control strategy can be a conventional control strategy, such as acceleration request, deceleration request, steering wheel angle request, etc.

[0175] After obtaining the first control strategy output by each control module 2041, in order to ensure the correctness of the target control strategy, the first control strategy will be verified by the verification module 2042 deployed in the second processor, which corresponds to each control module 2041.

[0176] Here, the second processor can be a redundant chip, and it deploys a verification module 2042 corresponding to each planning and control module 2041. This verification module 2042 is used to perform safety boundary verification on the first control strategy output by the corresponding planning and control module 2041. It is understood that the number of verification modules 2042 is the same as the number of planning and control modules 2041. In some embodiments, the verification module 2042 may include, but is not limited to, a planning verification module, a decision verification module, a horizontal verification module, and a vertical verification module.

[0177] The process by which the verification module 2042 performs security boundary verification on the first control strategy can be referred to the specific implementation of step S312 mentioned above, and will not be repeated here.

[0178] After each verification module 2042 performs safety boundary verification on the first control strategy output by the corresponding control module 2041, it can obtain the verification result corresponding to each control module 2041. The verification result may include, but is not limited to, the first verification result and the second verification result. The first verification result indicates that the verification was successful, and the second verification result indicates that the verification failed.

[0179] After each verification module 2042 receives the verification results from each control module 2041, the second processor can determine the target control strategy based on the verification results through the arbitration module 2043. Here, the arbitration module 2043 can be any suitable module capable of implementing this function. The arbitration module 2043 is mainly used to determine the target control strategy. Here, the target control strategy can be any of the first control strategies, or it can be the second control strategy output by the minimum risk strategy module. The second control strategy includes at least one of the following: intervention reminder and safe parking control. Intervention reminder can include, but is not limited to, displaying alarm text and images through the vehicle's terminal equipment, outputting alarm sounds through the vehicle's audio system, and flashing warning lights. Safe parking control can include parking in the same lane and parking on the side of the road.

[0180] The process by which the arbitration module 2043 determines the target control strategy based on the verification results of each control module 2041 may include: if the verification result of any control module 2041 indicates a verification failure, the second control strategy output by the minimum risk strategy module 2044 shall be used as the target control strategy; if the verification result of each control module 2041 indicates a verification success, the first control strategy output by each control module 2041 shall be used as the target control strategy.

[0181] It is understandable that, during implementation, when the perception processing result is the second perception processing result, the second processor can directly use the second control strategy output by the minimum risk strategy module 2044 as the target control strategy.

[0182] In this embodiment, on the one hand, two processors, one primary and one redundant, are introduced at the control and protection end to achieve chip hardware redundancy design. A redundant control and protection link is built outside the main intelligent driving function link to ensure the accuracy and robustness of control and protection. On the other hand, the control strategy output by the corresponding control module 2041 is verified by each verification module 2042 to ensure the security of the control strategy. On the other hand, the target control strategy is determined by integrating the verification results of each control module 2041 through the arbitration module 2043, which improves the accuracy of the target control strategy and realizes the protection of the control module 2041.

[0183] Figure 6 A schematic diagram of the implementation process of a method for determining a control strategy provided in this application embodiment. Figure 2 , can be adopted Figure 6 The control strategy shown is executed by the system, such as... Figure 6 As shown, the method includes the following steps, wherein:

[0184] Step S601: The first sensing module of the first on-chip system processes the sensing information of the vehicle's surrounding environment using the first sensing algorithm to obtain the first sensing result corresponding to the vehicle.

[0185] In step S602, the second sensing module in the second on-chip system processes the sensing information using the second sensing algorithm to obtain the second sensing result corresponding to the vehicle.

[0186] Step S603: The first and second perception results are verified by the perception result verification module to obtain the perception verification result.

[0187] Step S604: If the perception verification result is the first perception verification result, the first perception processing result is used as the perception processing result.

[0188] Among them, the first perception verification result indicates that the first perception result and the second perception result match, and the first perception processing result indicates that the perception verification is successful.

[0189] Step S605: If the perception verification result is the second perception verification result, the first recommendation decision information is determined based on the front view image information in the perception information through the first visual language model in the first recommendation decision module.

[0190] Step S606: Determine the second recommendation decision information based on the surrounding image information in the perceived information through the second visual language model in the second recommendation decision module.

[0191] Step S607: If the first recommendation decision information and the second recommendation decision information match, the first perception processing result is taken as the perception processing result by the recommendation decision verification module.

[0192] Step S608: If the first recommended decision information and the second recommended decision information do not match, the second perception processing result shall be used as the perception processing result.

[0193] The second perception processing result indicates that the perception verification failed.

[0194] The second perception verification result indicates that the first perception result and the second perception result do not match.

[0195] Step S609: If the perception processing result is the first perception processing result, for each regulation and control module, the first control strategy output by the regulation and control module is verified by the verification module corresponding to the regulation and control module to obtain the verification result of the regulation and control module.

[0196] In step S610, through the arbitration module, if the verification result of any control module indicates a verification failure, the second control strategy output by the minimum risk strategy module is taken as the target control strategy.

[0197] Step S611: If the verification result of each control module indicates that the verification is successful, the first control strategy output by each control module shall be used as the target control strategy.

[0198] Step S612: If the perception processing result is the second perception processing result, the second control strategy output by the minimum risk strategy module is used as the target control strategy.

[0199] The second control strategy includes at least one of the following: intervention reminders and safe parking control.

[0200] Among them, the target control strategy is used to control the vehicle.

[0201] Step S613: If the perception processing result is the second perception processing result or the verification result of any regulatory control module indicates a verification failure, the intelligent driving data corresponding to the perception information is uploaded to the cloud.

[0202] Among them, intelligent driving data is used for at least one of the following: training the visual language model in the perception result processing module, and updating the verification module corresponding to each regulation and control module.

[0203] In this application embodiment, on the one hand, by integrating different perception modules into two on-chip systems respectively, the perception information is processed to obtain corresponding perception results. Since this application integrates perception modules into the newly added on-chip system on the basis of the existing intelligent driving main function, it realizes hardware redundancy of the on-chip system and redundancy design of the perception module, ensuring the accuracy and robustness of perception. On the other hand, the perception result processing module in the third on-chip system processes the perception results output by the two perception modules, realizing redundancy verification of the perception results, so as to further improve the accuracy of the perception results. Furthermore, the control and protection module in the processor determines the vehicle's target control strategy based on the perception processing results, realizing the construction of a redundant safety link outside the intelligent driving main function link. This allows the target control strategy output by the vehicle's driving assistance system to not only solve safety problems caused by electronic and electrical failures, but also solve expected functional safety problems under non-fault conditions, thereby improving the safety of intelligent driving and the comprehensiveness of expected functional safety protection.

[0204] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages does not have to be sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0205] This application also provides a vehicle including a memory and a processor, the memory storing a computer program that can run on the processor, and the processor executing the computer program to implement any of the methods described above.

[0206] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. The computer-readable storage medium can be transient or non-transient.

[0207] This application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement some or all of the steps in any of the above-described methods. The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0208] It should be noted that, Figure 7 This is a schematic diagram of the hardware entity of a vehicle provided in the embodiments of this application, such as... Figure 7 As shown, the hardware entity of the vehicle 700 includes: a processor 701, a communication interface 702, and a memory 703, wherein:

[0209] The processor 701 typically controls the overall operation of the vehicle 700.

[0210] Communication interface 702 enables the vehicle to communicate with other terminals or servers via a network.

[0211] The memory 703 is configured to store instructions and applications executable by the processor 701, and can also cache data to be processed or already processed by the processor 701 and various modules in the vehicle 700 (e.g., image data, audio data, voice communication data, and video communication data). It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 701, the communication interface 702, and the memory 703 can be performed via bus 704.

[0212] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0213] The above embodiments are merely preferred embodiments provided to fully illustrate this application, and the scope of protection of this application is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on this application are all within the scope of protection of this application.

Claims

1. A method for determining a control strategy, characterized in that, Applied to a vehicle, the vehicle includes a first system-on-a-chip, a second system-on-a-chip, a third system-on-a-chip, and a processor, the determination method includes: The first sensing module in the first on-chip system processes the sensing information of the vehicle's surrounding environment using the first sensing algorithm to obtain the first sensing result corresponding to the vehicle. The second sensing module in the second on-chip system processes the sensing information using a second sensing algorithm to obtain the second sensing result corresponding to the vehicle. The first perception result and the second perception result are processed by the perception result processing module in the third on-chip system to obtain the perception processing result; wherein, the perception result processing module includes a perception result verification module and a recommendation decision processing module; The control and protection module in the processor determines a target control strategy based on the perception processing results; wherein the target control strategy is used to control the vehicle. The sensing result processing module in the third on-chip system processes the first sensing result and the second sensing result to obtain a sensing processing result, including: The perception result verification module verifies the first perception result and the second perception result to obtain the perception verification result. If the perception verification result is the second perception verification result, the recommendation decision processing module determines the perception processing result based on the perception information; wherein, the second perception verification result indicates that the first perception result and the second perception result do not match, and the perception processing result indicates whether the perception verification was successful.

2. The determination method according to claim 1, characterized in that, The method further includes: If the perception verification result is the first perception verification result, the first perception processing result shall be used as the perception processing result; wherein, the first perception verification result indicates that the first perception result and the second perception result match, and the first perception processing result indicates that the perception verification is successful.

3. The determination method according to claim 1, characterized in that, The recommendation decision processing module includes a first recommendation decision module, a second recommendation decision module, and a recommendation decision verification module; The step of determining the perception processing result based on the perception information through the recommendation decision processing module includes: Based on the front view image information in the perceived information, the first recommendation decision information is determined using the first visual language model in the first recommendation decision module. Based on the circumferential image information in the perceived information, the second recommendation decision information is determined using the second visual language model in the second recommendation decision module. The recommendation decision verification module determines the perception processing result based on the first recommendation decision information and the second recommendation decision information.

4. The determination method according to claim 3, characterized in that, The step of determining the perception processing result based on the first recommendation decision information and the second recommendation decision information includes: If the first recommendation decision information and the second recommendation decision information match, the first perception processing result shall be used as the perception processing result. If the first recommendation decision information and the second recommendation decision information do not match, the second perception processing result shall be used as the perception processing result; wherein, the second perception processing result indicates that the perception verification failed.

5. The determining method according to any one of claims 1 to 4, characterized in that, The processor includes a first processor and a second processor, and the control and protection module includes at least one control module deployed in the first processor and a minimum risk strategy module deployed in the second processor; The step of determining the target control strategy based on the perception processing result through the control and protection module in the processor includes: If the perception processing result is the first perception processing result, the target control strategy is determined based on the first control strategy output by each of the control modules. If the perception processing result is the second perception processing result, the second control strategy output by the minimum risk strategy module shall be used as the target control strategy; wherein, the second control strategy includes at least one of the following: intervention reminder, safe parking control.

6. The determination method according to claim 5, characterized in that, The regulation and protection module further includes at least one verification module and an arbitration module deployed in the second processor, with each verification module corresponding to one regulation and protection module. The determination of the target control strategy based on the first control strategy output by each of the control modules includes: For each control module, the first control strategy output by the control module is verified by the corresponding verification module to obtain the verification result of the control module. The arbitration module determines the target control strategy based on the verification results of each of the control modules.

7. The determination method according to claim 6, characterized in that, The determination of the target control strategy based on the verification results of each of the control modules includes: If the verification result of any of the aforementioned control modules indicates a verification failure, the second control strategy output by the minimum risk strategy module shall be used as the target control strategy. If the verification result of each of the aforementioned control modules indicates successful verification, the first control strategy output by each of the aforementioned control modules shall be used as the target control strategy.

8. The determination method according to claim 5, characterized in that, If the sensing processing result is the second sensing processing result or the verification result of any of the control modules indicates a verification failure, the determination method further includes: The intelligent driving data corresponding to the perceived information is uploaded to the cloud; wherein the intelligent driving data is used for at least one of the following: training the visual language model in the perception result processing module, or updating the verification module corresponding to each of the control modules.

9. A system for determining a control strategy, characterized in that, The determining system, when applied in a vehicle, includes: The first perception module, deployed in the first on-chip system of the vehicle, is used to process the perception information of the surrounding environment of the vehicle using the first perception algorithm to obtain the first perception result corresponding to the vehicle. The second perception module, deployed in the second on-chip system of the vehicle, is used to process the perception information using a second perception algorithm to obtain the second perception result corresponding to the vehicle. A perception result processing module, deployed on the third chip system of the vehicle, is used to process the first perception result and the second perception result to obtain a perception processing result; A control and protection module, deployed in the vehicle's processor, is used to determine a target control strategy based on the perception processing results; wherein the target control strategy is used to control the vehicle. The perception result processing module includes a perception result verification module and a recommendation decision processing module. The perception result verification module is used to verify the first perception result and the second perception result to obtain a perception verification result. The recommendation decision processing module is used to determine the perception processing result based on the perception information when the perception verification result is the second perception verification result; wherein, the second perception verification result indicates that the first perception result and the second perception result do not match, and the perception processing result indicates whether the perception verification was successful.

10. The determining system according to claim 9, characterized in that, The recommendation decision processing module includes a first recommendation decision module, a second recommendation decision module, and a recommendation decision verification module, wherein: The first recommendation decision module is used to determine the first recommendation decision information based on the front view image information in the perceived information using the first visual language model; The second recommendation decision module is used to determine the second recommendation decision information based on the panoramic image information in the perceived information using the second visual language model; The recommendation decision verification module is used to determine the perception processing result based on the first recommendation decision information and the second recommendation decision information.

11. The determining system according to claim 9 or 10, characterized in that, The processor includes a first processor and a second processor, and the regulation and protection module includes at least one regulation and control module, a verification module, an arbitration module, and a minimum risk strategy module corresponding to each regulation and control module, wherein: Each of the aforementioned control modules is deployed in the first processor and is used to output the corresponding first control strategy; Each of the aforementioned verification modules is deployed in the second processor and is used to verify the first control strategy output by the corresponding control module to obtain the verification result of each of the aforementioned control modules. The arbitration module, deployed in the second processor, is used to take the second control strategy output by the minimum risk strategy module as the target control strategy when the verification result of any of the control modules indicates verification failure; and to take the first control strategy output by each control module as the target control strategy when the verification result of each control module indicates verification success.

12. A vehicle comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 8.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

14. A computer program product comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

Images