Method and device for intelligent driving vehicle safety control and electronic equipment

Abstract

The invention provides a method and device for intelligent driving vehicle safety control and electronic equipment. The method comprises the steps that perception information is acquired; a master control model is obtained, and the master control model unit outputs an initial instruction based on the sensing information; acquiring a parameter regulator, and outputting safety parameters in real time by the parameter regulator based on the sensing information; acquiring a safety core, and calculating a safety correction value by the safety core based on the initial instruction, the safety parameters and the real-time state information of the vehicle; and fusing the safety correction value with the initial instruction to generate a final control instruction. The obtained parameter regulator outputs safety parameters in real time, so that the intelligent driving automobile can continuously learn the safety parameters in the whole product life cycle and adapt to complex and changeable driving scenes, such as an operation design domain (ODD) automatically adapting to road surface friction force changes under various weather conditions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to a method and device for safety control of a smart driving vehicle and an electronic device. BACKGROUND

[0002] The safety module of the prior art cannot adapt to the optimization of parameter variables in different scenarios. Most safety modules of existing automatic driving systems are designed with rules or static parameters. Once deployed on a smart driving vehicle, the parameters remain fixed throughout the product's life cycle, and cannot adapt to complex and variable driving scenarios.

[0003] Therefore, there is an urgent need for a method, device and electronic equipment for safety control of a smart driving vehicle to improve the above problems. SUMMARY

[0004] The present application aims to provide a method and device for safety control of a smart driving vehicle.

[0005] In a first aspect, the present application provides a method for safety control of a smart driving vehicle, comprising:

[0006] obtaining perception information;

[0007] obtaining a master model, based on the perception information, the master model unit outputs an initial instruction;

[0008] obtaining a parameter regulator, based on the perception information, the parameter regulator outputs safety parameters in real time;

[0009] obtaining a safety core, based on the initial instruction, the safety parameters and the real-time state information of the vehicle, the safety core calculates a safety correction amount; the safety correction amount and the initial instruction are fused with each other to generate a final control instruction.

[0010] The beneficial effects of the present application are that the parameter regulator outputs safety parameters in real time, so that the smart driving vehicle can continuously learn safety parameters throughout the product life cycle, and adapt to complex and variable driving scenarios, such as automatically adapting to the changes in road friction under various weather conditions.

[0011] Preferably, the safety core includes a system of physical equations using the safety parameters.

[0012] Preferably, the parameter regulator includes a parameter adjustment neural network.

[0013] Preferably, the safety parameters include at least a gain coefficient for scaling the influence degree of different safety items and a correction coefficient for basic intervention bias calculation.

[0014] Preferably, the gain coefficient is one or both of a relative speed gain and a distance error gain.

[0015] The correction coefficient is one or both of a longitudinal bias and a lateral bias.

[0016] Preferably, a context perception vector is generated based on the perception information, the parameter adjustment neural network acquires the context perception vector, and based on an embedded learning model, outputs a set of safety parameters in real time for use by the safety core;

[0017] The context perception vector includes information in at least one of the following dimensions:

[0018] ① Vehicle dynamic state

[0019] ② Road geometry state

[0020] ③ Traffic flow state

[0021] ④ Environmental weather state

[0022] Preferably, the parameter adjustment neural network aims to minimize the gap between the output of the safety correction unit and the safety expert driving behavior, while also aiming to maintain a small correction amplitude.

[0023] In a second aspect, the present application provides a device for intelligent driving vehicle safety control, which comprises units for executing any one of the possible designs of the method of the first aspect. These units can be realized by hardware, or realized by hardware executing corresponding software, and the device comprises:

[0024] A perception unit for acquiring perception information;

[0025] A master control unit for acquiring a master control model, which outputs an initial instruction based on the perception information;

[0026] A parameter adjustment unit for acquiring a parameter adjuster, which outputs safety parameters in real time based on the perception information;

[0027] A safety correction unit for acquiring a safety core, which calculates a safety correction amount based on the initial instruction, the safety parameters, and real-time state information of the vehicle; and fuses the safety correction amount and the initial instruction with each other to generate a final control instruction.

[0028] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a program capable of running on the processor, and when the program is executed by the processor, the electronic device implements the method of any one of the possible designs of any one of the aspects.

[0029] In a fourth aspect, the present application provides a readable storage medium, wherein a program is stored in the readable storage medium, and the program, when executed, implements the method of any possible design of any one of the above aspects.

[0030] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described above.

[0031] The beneficial effects of the above-mentioned second to fifth aspects can be referred to the description of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 A flowchart of a method for intelligent driving vehicle safety control is provided for an embodiment of the present application.

[0033] Figure 2 A structural diagram of an apparatus for intelligent driving vehicle safety control is provided for an embodiment of the present application.

[0034] Figure 3 A system architecture diagram of a method for intelligent driving vehicle safety control is provided for another embodiment of the present application.

[0035] Figure 4 A block diagram of an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numbers in different drawings represent the same or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0037] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0038] The present application provides a method for intelligent driving vehicle safety control, as shown in Figure 1 The method comprises:

[0039] S1, acquiring perception information;

[0040] S2, acquiring a master control model, and outputting initial instructions by the master control model unit based on the perception information;

[0041] S3, acquiring a parameter regulator, and outputting safety parameters by the parameter regulator in real time based on the perception information;

[0042] S4, a safety core is acquired, based on the initial instruction, the safety parameter, and vehicle real-time state information, the safety core calculates a safety correction amount; the safety correction amount and the initial instruction are fused with each other to generate a final control instruction.

[0043] It should be noted that the order of S2 and S3 in the above method can be changed, or they can be performed simultaneously.

[0044] For example, the parameter regulator outputs the safety parameter as a static parameter in real time. During the working and reasoning of the automatic driving master control model, the parameter regulator outputs this set of static parameters in real time. It should be noted that this set of static parameters can be updated by learning through the OTA mode pushed by the background cloud. Specifically, when the intelligent driving vehicle detects a representative key dangerous scene, the intelligent driving vehicle pushes the related scene data to the background, the cloud background reconstructs the digital scene through digital twinning, and re-trains the safety core static parameters with the new scene. The model training aims to minimize the difference between the output of the safety correction unit and the safety expert driving behavior, while keeping a small correction amplitude. After training a new set of parameters, the intelligent driving vehicle is pushed back to update the static parameters through the OTA mode. There is no frequency limit for parameter updating, and the development team can update it daily, weekly, or monthly as needed.

[0045] Among them, the key dangerous scene is a scene where the vehicle almost collides but safely flashes. This condition can be fed back by the user or detected by the sensors of the autonomous vehicle, such as sudden triggering of AEB emergency braking. The system will record and review the uploaded historical environment perception data.

[0046] It should be noted that in some preferred embodiments, the parameter regulator can also output the safety parameter in real time as a dynamic parameter. The real-time output safety parameter is not a pre-set static fixed value, but a dynamic parameter generated in real time based on perception information.

[0047] In some embodiments, the safety core includes a physical equation set using the safety parameter. For example, the physical equation set outputs longitudinal correction or lateral correction, or both. Among them, the correction amount output by the physical equation set can be a correction amount for acceleration, a correction amount for speed, or a correction amount for relative displacement, etc.

[0048] In some embodiments, the parameter regulator includes a parameter adjustment neural network.

[0049] For example, the parameter adjustment neural network adopts a 3-5 layer MLP network. The parameter adjustment neural network is trained together with the master control model, and after training, it is deployed on the intelligent driving vehicle. During the working and reasoning of the master control model, the parameter adjustment neural network outputs a new set of dynamic parameters to the safety core in real time according to different scenes and ODDs.

[0050] It should be noted that the parameter tuning neural network can also work in conjunction with the training model deployed in the cloud for continuous optimization and updates. When the autonomous driving system detects a critical and representative dangerous scenario that may lead to an accident, the system will push the relevant scenario data to the cloud. Developers can then use the new data to retrain the dynamic parameter modulation network in the background cloud. After training, the new parameter tuning neural network will be pushed to the autonomous vehicle via OTA. There is no frequency limit for network updates, and the development team can update it daily, weekly, or monthly as needed.

[0051] Among them, the key dangerous scenario is a situation where the vehicle almost collided but was able to avoid it safely.

[0052] In some embodiments, the safety parameters include at least a gain coefficient for scaling the degree of influence of different safety items and a correction coefficient for calculating the basic intervention bias.

[0053] In some embodiments, the gain coefficient is one or both of relative velocity gain and distance error gain;

[0054] The correction factor is one or both of the longitudinal and lateral biases.

[0055] In some embodiments, a context-aware vector is generated based on the perception information, a parameter-adjusted neural network is used to obtain the context-aware vector, and a set of security parameters is output in real time for the security kernel to use based on the built-in learning model.

[0056] Context-aware vectors include information in at least one of the following dimensions:

[0057] ① Vehicle dynamic status

[0058] ② Road geometry

[0059] ③ Traffic flow status

[0060] ④ Environmental weather conditions

[0061] The road geometry includes road curvature and road slope; the traffic flow status includes traffic density and scene chaos index.

[0062] The built-in learning model can be selected according to the specific application scenario, such as MLP model, convolutional + MLP model, LSTM / GRU, Transformer, CNN, GCN, or a hybrid model of two or more models, or a specially tuned model.

[0063] Preferably, the context-aware vector includes at least the vehicle dynamic state, and one or more of the road geometry state, traffic flow state, and ambient weather state.

[0064] In some embodiments, the parameter-adjusted neural network aims to minimize the gap between the output of the safety correction unit and the driving behavior of a safety expert, while maintaining a small correction magnitude.

[0065] To facilitate understanding, this embodiment further elaborates on the specific implementation process of the above method in conjunction with a specific application scenario. Please refer to... Figure 3 To show understanding, specifically:

[0066] Application scenario: Following another vehicle on a highway in rainy or foggy weather.

[0067] Scenario description: A vehicle is driving on a highway and suddenly enters a rain and fog area. Visibility decreases, the road surface friction coefficient μ drops sharply, and a slow-moving vehicle appears ahead.

[0068] Generate a context-aware vector C, which specifically includes the following dimensions:

[0069] ① Vehicle status: The vehicle's speed is 110 km / h, and the current acceleration is 0 m / s².

[0070] ②Environmental conditions: Estimated friction coefficient μ = 0.4, visibility = 200 meters, precipitation = moderate rain;

[0071] ③ Traffic conditions: Distance to the vehicle in front is 60 meters, relative speed is -20 km / h, traffic density is high;

[0072] ④ Road conditions: Road curvature = low, slope = 0%;

[0073] ⑤Scene Index: The scene chaos index calculated by the system is 0.8 (high risk).

[0074] Training method for parameter modulation unit:

[0075] This embodiment uses a three-layer MLP fully connected small parameter-tuning neural network.

[0076] Training objective (loss function): The parameter modulation unit is trained by reducing the following loss function: L=α*|Δ|+β*|Y_corrected-Y_expert|, where Δ is the correction amount, and the smaller the value, the smoother the intervention;

[0077] Y_corrected is the corrected output, and Y_expert is expert safe driving data, which is used to ensure the safety of the correction.

[0078] The source and nature of the training data: The training data comes from real driving logs and simulation environments, and includes a large number of edge scenarios, such as emergency braking, obstacle avoidance, and low-friction road surfaces.

[0079] The basic training process is as follows: During training, the context-aware vector C is used as input to make the generated parameters make the output of the safety correction core as close as possible to the driving behavior of a safety expert, while maintaining a small correction range.

[0080] The process of a parameter-tuning neural network generating dynamic parameters in real time:

[0081] Input vector C into the trained parameters to adjust the neural network.

[0082] In this embodiment, the network outputs dynamic parameters, including:

[0083] k1 (relative velocity gain): 0.95 (high gain, more sensitive to velocity changes);

[0084] k2 (distance error gain): 0.88 (high gain, more sensitive to insufficient distance);

[0085] ε_long (longitudinal offset): -0.3m / s 2 (Negative bias, tending to decelerate earlier);

[0086] The longitudinal physical equations for core calculations and fine-tuning of instructions for safety correction are as follows:

[0087] a_long=clip(-k,(C)·Δv / d-k2(C)·d-d_safe+ε_long(C))

[0088] The safety correction core substitutes the dynamic parameters k1 = 0.95, k2 = 0.88, and ε_long = -0.3 into its longitudinal physical equation to calculate a longitudinal correction amount ΔX = -3.5 m / s. 2 Subsequently, this correction value was compared with the original acceleration command X_ai = -1.5 m / s² output by the main model. 2 Adding them together, we get the final instruction X_final = -5.0 m / s 2 .

[0089] Most existing autonomous driving systems employ rule-based or statically parameterized designs for their safety modules. Once deployed in an intelligent vehicle, these parameters remain fixed throughout the product's lifecycle. Current technologies cannot predict the decrease in road friction and can only execute braking according to fixed braking algorithms (such as AEB). In actual use, executing model output commands often leads to wheel slippage. However, using the fused output of this embodiment allows for earlier braking, reducing or avoiding slippage and significantly improving safety.

[0090] In some other embodiments, the lateral correction can also be calculated using a similar method to that used to calculate the longitudinal correction. Specifically, the network output dynamic parameters also output k3, k4, and ε_lat. The lateral physical equations calculated using the safety correction core and fine-tuned by instructions are as follows:

[0091] a_lat=clip(-k3(C)·e_lat-k4(C)·1 / d_lat+ε_lat(C))

[0092] Where e_lat represents the estimated lateral distance error relative to the center of the lane.

[0093] Y_final is calculated accordingly.

[0094] The beneficial effects of this invention are as follows: The core advantage of this adaptive unrestricted ODD scenario autonomous driving safety kernel learning method is that it can simultaneously meet the requirements of controllability and interpretability of safety technology, as well as the generalizable learning and reasoning requirements for adapting to more unknown scenarios for high-level autonomous driving, and can effectively achieve the goal of L5 level autonomous driving.

[0095] Secondly, the present invention provides a device for safety control of intelligent driving vehicles, such as... Figure 2 As shown, the apparatus includes units for performing any of the possible design methods described in the first aspect above. These units can be implemented in hardware or by hardware executing corresponding software. The apparatus includes:

[0096] Sensing units are used to acquire sensory information;

[0097] The main control unit is used to acquire the main control model. Based on the perception information, the main control model outputs initial instructions.

[0098] The parameter adjustment unit is used to acquire the parameter regulator. Based on the sensing information, the parameter regulator outputs safety parameters in real time.

[0099] The safety correction unit is used to acquire the safety core. Based on the initial command, safety parameters and real-time vehicle status information, the safety core calculates the safety correction amount. The safety correction amount is then integrated with the initial command to generate the final control command.

[0100] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a program that can run on the processor, and when the program is executed by the processor, the electronic device implements a method for performing any of the possible designs described above.

[0101] Figure 4A block diagram of an electronic device 400 suitable for implementing embodiments of the present invention is shown. Device 400 includes a processor 410. Processor 410 controls the operation and functions of device 400. For example, in some embodiments, processor 410 may perform various operations by means of a program 430 stored in a memory 420 coupled thereto. Memory 420 may be any suitable type suitable for the local technical environment and may be implemented using any suitable data storage technology, including but not limited to semiconductor-based memory devices, magnetic storage devices and systems, optical storage devices and systems. Although Figure 4 Only one memory cell is shown in the diagram, but there may be multiple physically different memory cells in device 400.

[0102] Processor 410 can be any suitable type for the local technical environment and can include, but is not limited to, one or more of a general-purpose computer, a special-purpose computer, a microcontroller, a digital signal controller (DSP), a system-on-a-chip (SoC), a GPU, and a controller-based multi-core controller architecture. Device 400 may also include multiple processors 410.

[0103] When device 400 is deployed on an intelligent driving vehicle, processor 410, when executing instruction 430, causes device 400 to perform actions to achieve... Figures 1-3 The method described.

[0104] Fourthly, the present invention provides a readable storage medium storing a program, which, when executed, implements a method of any possible design of any of the above aspects.

[0105] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0106] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0107] The computer program described herein refers to a program that can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and forwards those instructions to the computer-readable storage medium stored in the respective computing / processing device.

[0108] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0109] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0110] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0112] It should also be noted that each box in a block diagram and / or flowchart, as well as combinations of boxes in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0113] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0114] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0115] For the beneficial effects of the second to fifth aspects mentioned above, please refer to the description of the first aspect mentioned above.

Claims

1. A method for intelligent vehicle safety control, characterized in that, The method comprises: acquiring perception information; acquiring a master model, based on the perception information, the master model unit outputs initial instructions; acquiring a parameter regulator, based on the perception information, the parameter regulator outputs safety parameters in real time; acquiring a safety core, based on the initial instructions, the safety parameters and the real-time state information of the vehicle, the safety core calculates safety correction amounts; the safety correction amounts and the initial instructions are fused with each other to generate final control instructions.

2. The method of claim 1, wherein, The safety core includes a physical equation set using the safety parameters.

3. The method of claim 1, wherein, The parameter regulator includes a parameter adjustment neural network.

4. The method of claim 2, wherein, The safety parameters at least include gain coefficients for scaling the influence degree of different safety items and correction coefficients for basic intervention bias calculation.

5. The method of claim 4, wherein, The gain coefficients are one or both of relative speed gain and distance error gain; The correction coefficients are one or both of longitudinal bias and lateral bias.

6. The method of claim 3, wherein, Based on the perception information, a context perception vector is generated, the parameter adjustment neural network acquires the context perception vector, and based on the built-in learning model, a set of safety parameters is output in real time for use by the safety core; The context perception vector includes information in at least one of the following dimensions: ① Vehicle dynamic state ② Road geometry state ③ Traffic flow state ④ Environmental weather state.

7. The method of claim 6, wherein, The parameter adjustment neural network aims to minimize the gap between the output of the safety correction unit and the safety expert driving behavior, while keeping a small correction amplitude.

8. A device for safety control of intelligent driving vehicles, characterized in that, The device comprises: a perception unit for acquiring perception information; a master unit for acquiring a master model, based on the perception information, the master model outputs initial instructions; a parameter adjustment unit for acquiring a parameter regulator, based on the perception information, the parameter regulator outputs safety parameters in real time; a safety correction unit for acquiring a safety core, based on the initial instructions, the safety parameters and the real-time state information of the vehicle, the safety core calculates safety correction amounts; the safety correction amounts and the initial instructions are fused with each other to generate final control instructions.

9. An electronic device, comprising: The program is executed by the processor to implement the method of any one of claims 1-7.

10. A readable storage medium, the readable storage medium having stored therein a program, characterized in that, The program is executed to implement the method of any one of claims 1-7.

11. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-7.

Images