Vehicle braking control method, system and device based on multi-mode perception and medium

Abstract

The invention discloses a vehicle braking control method, system and device based on multi-mode perception and a medium. The method comprises the steps that the power battery charge state, the vehicle speed, the current braking strength and real-time road condition information of a target vehicle are obtained; performing data fusion on the charge state of the power battery, the vehicle speed, the current braking strength and the real-time road condition information to obtain multi-mode sensing time sequence data; inputting the multi-modal sensing time sequence data into a pre-trained braking force distribution prediction model to obtain a braking force distribution result; and a target braking mode is determined according to the braking force distribution result, and braking control is conducted on the target vehicle according to the target braking mode. According to the embodiment of the invention, the corresponding braking mode can be adaptively switched according to the real-time driving scene and the energy demand, the braking performance of the vehicle and the energy recovery efficiency are improved, more flexible braking control is realized, the driving experience of a user is improved, and the method can be widely applied to the technical field of vehicle control.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to a vehicle braking control method, system, device and medium based on multimodal perception. Background Art

[0002] The braking energy recovery system includes a generator and a battery that are compatible with the vehicle model, as well as an intelligent battery management system that can monitor the battery power. The braking energy recovery system recovers the excess energy released by the vehicle during braking or coasting, and converts it into electrical energy through the generator, which is then stored in the battery for subsequent acceleration. In new energy vehicles, the recovery and utilization of braking energy greatly improves energy utilization, reduces carbon emissions, and achieves green travel.

[0003] In the prior art, most braking energy recovery systems adopt a single-pedal mode and a hybrid braking mode. In the single-pedal mode, the driver only needs to operate the accelerator pedal. When the pedal is released, the system automatically starts energy recovery braking, and the vehicle slows down until it stops. In the hybrid braking mode, the system uses energy recovery braking and mechanical braking at the same time. During light braking, energy recovery braking is used first, and during heavy braking, mechanical braking intervenes to ensure safety.

[0004] Although the existing braking energy recovery scheme has achieved the recovery and utilization of braking energy to a certain extent, the braking mode is relatively simple and cannot be adaptively switched according to the real-time driving scenario and energy demand, which affects the vehicle's braking performance and energy recovery efficiency, thereby affecting the user's driving experience. Summary of the invention

[0005] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0006] To this end, an object of an embodiment of the present invention is to provide a vehicle braking control method based on multimodal perception, which can adaptively switch the corresponding braking mode according to the real-time driving scenario and energy demand, thereby improving the vehicle's braking performance and energy recovery efficiency, thereby improving the user's driving experience.

[0007] Another object of an embodiment of the present invention is to provide a vehicle braking control system based on multimodal perception.

[0008] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0009] In a first aspect, an embodiment of the present invention provides a vehicle braking control method based on multimodal perception, comprising the following steps:

[0010] Obtain the target vehicle's power battery charge status, vehicle speed, current braking intensity, and real-time road condition information;

[0011] Performing data fusion on the power battery state of charge, the vehicle speed, the current braking intensity, and the real-time road condition information to obtain multimodal perception time series data;

[0012] Inputting the multimodal perception time series data into a pre-trained braking force distribution prediction model to obtain a braking force distribution result;

[0013] A target braking mode is determined according to the braking force distribution result, and braking control is performed on the target vehicle according to the target braking mode.

[0014] Further, in one embodiment of the present invention, the acquiring of the power battery charge state, vehicle speed, current braking intensity and real-time road condition information of the target vehicle specifically includes:

[0015] Obtaining the state of charge of the power battery through a battery management system;

[0016] Determine the vehicle speed by using a GPS positioning system and a wheel speed sensor;

[0017] Acquiring brake line pressure through a brake pressure sensor, calculating current braking force according to the brake line pressure, and determining current braking intensity according to the current braking force;

[0018] Acquire the real-time traffic information through vehicle-mounted cameras, inertial measurement units, millimeter-wave radars, and V2X systems;

[0019] The real-time traffic information includes road type, road slope, road adhesion and traffic flow information.

[0020] Furthermore, in one embodiment of the present invention, the real-time road condition information is obtained through a vehicle-mounted camera, an inertial measurement unit, a millimeter-wave radar, and a V2X system, which specifically includes:

[0021] Acquiring road image information through the vehicle-mounted camera, and identifying the road type according to the road image information;

[0022] Acquiring the body posture information of the target vehicle through the inertial measurement unit, and determining the road surface slope according to the body posture information;

[0023] Detecting the road surface water film thickness by the millimeter wave radar, and determining the road surface adhesion according to the road surface water film thickness;

[0024] The vehicle density information within a preset distance range is received through the V2X system, and the traffic flow information is determined according to the vehicle density information.

[0025] Further, in one embodiment of the present invention, the data fusion of the power battery state of charge, the vehicle speed, the current braking intensity and the real-time road condition information is performed to obtain multimodal perception time series data, which specifically includes:

[0026] Performing spatiotemporal alignment and data encoding on the road type, the road surface slope, the road surface adhesion, and the traffic flow information to obtain road condition encoding information;

[0027] Generate multimodal perception information according to the power battery state of charge, the vehicle speed, the current braking intensity, and the road condition coding information;

[0028] The multimodal perception data at multiple consecutive moments are subjected to time series processing to obtain the multimodal perception time series data.

[0029] Further, in one embodiment of the present invention, the braking force distribution prediction model is trained by the following steps:

[0030] Acquire multiple multimodal perception sample time series data of the test vehicle, and determine the braking force distribution label corresponding to each of the multimodal perception sample time series data through manual labeling;

[0031] Inputting the multimodal sensing sample time series data into a pre-built CNN-LSTM hybrid neural network to obtain a braking force distribution prediction result;

[0032] determining a loss value according to the braking force distribution prediction result and the braking force distribution label;

[0033] Updating the parameters of the CNN-LSTM hybrid neural network through a back propagation algorithm according to the loss value to obtain the trained braking force distribution prediction model;

[0034] The braking force distribution label includes a distribution ratio of the regenerative braking force and the mechanical braking force.

[0035] Furthermore, in one embodiment of the present invention, the CNN-LSTM hybrid neural network includes an input layer, a CNN convolutional layer, a feature fusion layer, an LSTM layer, an attention layer and an output layer, the input layer is used to input the multimodal perception sample time series data, the CNN convolutional layer is used to perform feature extraction on the multimodal perception sample time series data to obtain local time series features, the feature fusion layer is used to perform feature fusion on the local time series features to obtain fused time series features, the LSTM layer is used to generate a hidden state sequence according to the fused time series features, the attention layer is used to dynamically assign weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism, and the output layer is used to map the hidden state sequence after dynamic weight assignment to the braking force distribution prediction result.

[0036] Further, in one embodiment of the present invention, determining a target braking mode according to the braking force distribution result, and performing braking control on the target vehicle according to the target braking mode specifically includes:

[0037] Determining a regenerative braking force distribution ratio and a mechanical braking force distribution ratio according to the braking force distribution result;

[0038] When the regenerative braking force distribution ratio is 100%, determining that the target braking mode is the regenerative braking mode, and controlling the target vehicle to switch to the regenerative braking mode for braking;

[0039] When the mechanical braking force distribution ratio is 100%, determining that the target braking mode is the mechanical braking mode, and controlling the target vehicle to switch to the mechanical braking mode for braking;

[0040] When the regenerative braking force distribution ratio and the mechanical braking force distribution ratio are both not 0, the target braking mode is determined to be a hybrid braking mode, and the target regenerative braking force and the target mechanical braking force in the hybrid braking mode are determined, and then the target vehicle is controlled to switch to the hybrid braking mode for braking.

[0041] In a second aspect, an embodiment of the present invention provides a vehicle braking control system based on multimodal perception, comprising:

[0042] A data acquisition module is used to obtain the target vehicle's power battery charge state, vehicle speed, current braking intensity, and real-time road condition information;

[0043] A data fusion module, used for fusing the power battery state of charge, the vehicle speed, the current braking intensity and the real-time road condition information to obtain multi-modal perception time series data;

[0044] A braking force distribution prediction module, used for inputting the multimodal perception time series data into a pre-trained braking force distribution prediction model to obtain a braking force distribution result;

[0045] The braking mode determination module is used to determine a target braking mode according to the braking force distribution result, and perform braking control on the target vehicle according to the target braking mode.

[0046] In a third aspect, an embodiment of the present invention provides a vehicle braking control device based on multimodal perception, comprising:

[0047] at least one processor;

[0048] at least one memory for storing at least one program;

[0049] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned vehicle braking control method based on multimodal perception.

[0050] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used to execute the above-mentioned vehicle braking control method based on multimodal perception when executed by the processor.

[0051] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0052] The embodiment of the present invention obtains the power battery charge state, vehicle speed, current braking intensity and real-time road condition information of the target vehicle, performs data fusion on the power battery charge state, vehicle speed, current braking intensity and real-time road condition information to obtain multimodal perception time series data, inputs the multimodal perception time series data into a pre-trained braking force distribution prediction model to obtain a braking force distribution result, determines the target braking mode according to the braking force distribution result, and performs braking control on the target vehicle according to the target braking mode. The embodiment of the present invention monitors the battery charge state, vehicle speed, current braking intensity and real-time road condition information in real time and performs data fusion to obtain multimodal perception time series data, and can obtain the corresponding braking force distribution result in combination with the pre-trained braking force distribution prediction model, and then can adaptively switch the corresponding braking mode according to the real-time driving scene and energy demand, thereby improving the braking performance of the vehicle and the efficiency of energy recovery, and realizing more flexible braking control, thereby improving the driving experience of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 A flowchart of a vehicle braking control method based on multimodal perception provided by an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of the structure of a CNN-LSTM hybrid neural network provided in an embodiment of the present invention;

[0056] Figure 3 A structural block diagram of a vehicle braking control system based on multimodal perception provided by an embodiment of the present invention;

[0057] Figure 4 A structural block diagram of a vehicle braking control device based on multimodal perception provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0059] In the description of the present invention, the meaning of "a plurality" is two or more than two. If there is a description of "a first" or "a second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by those skilled in the art.

[0060] Reference Figure 1 The embodiment of the present invention provides a vehicle braking control method based on multimodal perception, which specifically includes the following steps:

[0061] S101, obtaining the power battery charge state, vehicle speed, current braking intensity and real-time road condition information of the target vehicle;

[0062] S102, fusing the power battery state of charge, vehicle speed, current braking intensity, and real-time road condition information to obtain multi-modal perception time series data;

[0063] S103, inputting the multimodal perception time series data into a pre-trained braking force distribution prediction model to obtain a braking force distribution result;

[0064] S104: Determine a target braking mode according to the braking force distribution result, and perform braking control on the target vehicle according to the target braking mode.

[0065] The embodiment of the present invention monitors the battery state of charge, vehicle speed, current braking intensity and real-time road condition information in real time and performs data fusion to obtain multimodal perception time series data. Combined with a pre-trained braking force distribution prediction model, the corresponding braking force distribution result can be obtained, and then the corresponding braking mode can be adaptively switched according to the real-time driving scenario and energy demand, thereby improving the vehicle's braking performance and energy recovery efficiency, achieving more flexible braking control, and thus improving the user's driving experience.

[0066] As an optional implementation, obtaining the power battery charge state, vehicle speed, current braking intensity and real-time road condition information of the target vehicle specifically includes:

[0067] S1011. Obtaining the state of charge of the power battery through a battery management system;

[0068] S1012, determining the vehicle speed through a GPS positioning system and a wheel speed sensor;

[0069] S1013, obtaining a brake line pressure through a brake pressure sensor, calculating a current braking force according to the brake line pressure, and determining a current braking intensity according to the current braking force;

[0070] S1014, obtain real-time traffic information through vehicle-mounted cameras, inertial measurement units, millimeter-wave radars and V2X systems;

[0071] Among them, real-time traffic information includes road type, road slope, road adhesion and traffic flow information.

[0072] Specifically, the state of charge (SOC) of the power battery is obtained through the battery management system, and a temperature compensation model (-30℃~60℃ operating conditions) can be added at the same time, that is, the SOC value obtained by the battery management system is corrected according to the real-time battery temperature, thereby improving the accuracy of the power battery state of charge; the vehicle position change is determined by the GPS positioning system, and the tire speed measured by the wheel speed sensor is combined to determine the vehicle speed, thereby improving the accuracy of the vehicle speed; the brake line pressure is obtained by the brake pressure sensor, and the current braking force F is estimated according to the brake line pressure, and the current braking intensity Z=(F / m)g is determined according to the current braking force F and the vehicle mass m; the road type, road slope, road adhesion and traffic flow information are determined by the on-board camera, inertial measurement unit, millimeter wave radar and V2X system respectively, thereby obtaining real-time road condition information.

[0073] As an optional implementation, real-time road condition information is obtained through a vehicle-mounted camera, an inertial measurement unit, a millimeter-wave radar, and a V2X system, which specifically includes:

[0074] S10141. Obtain road image information through a vehicle-mounted camera, and identify the road type according to the road image information;

[0075] S10142, obtaining body posture information of the target vehicle through an inertial measurement unit, and determining a road slope according to the body posture information;

[0076] S10143, detecting the thickness of the water film on the road surface by using a millimeter wave radar, and determining the road adhesion according to the thickness of the water film on the road surface;

[0077] S10144. Receive vehicle density information within a preset distance range through the V2X system, and determine traffic flow information based on the vehicle density information.

[0078] Specifically, road image information is acquired through the on-board camera, and the corresponding road type can be obtained by detecting and identifying the road image information based on a preset road type recognition model; the body posture information of the target vehicle is acquired through the inertial measurement unit, and the body posture information includes the three-axis speed and three-axis angular velocity of the target vehicle, so that the road slope of the target vehicle can be calculated; the road surface is covered with water film due to rainfall, and the tires of the vehicle squeeze the water film on the road surface when the vehicle is driving at high speed on the road. The dynamic water pressure generated reduces the contact area between the tire and the road surface, thereby reducing the adhesion of the road surface. The thickness of the water film on the road surface is detected by millimeter wave radar, and the road adhesion is determined according to the thickness of the water film on the road surface; the vehicle density information within a range of 500 meters is received through the vehicle's V2X system, and then integrated to obtain traffic flow information.

[0079] As an optional implementation, data fusion is performed on the power battery state of charge, vehicle speed, current braking intensity, and real-time road condition information to obtain multimodal perception time series data, which specifically includes:

[0080] S1021, performing spatiotemporal alignment and data encoding on road type, road surface slope, road surface adhesion, and traffic flow information to obtain road condition encoding information;

[0081] S1022, generating multimodal perception information according to the power battery state of charge, vehicle speed, current braking intensity, and road condition coding information;

[0082] S1023. Perform time series processing on the multimodal perception data at multiple consecutive moments to obtain multimodal perception time series data.

[0083] Specifically, taking into account the time delay of data collected by different sensors, the embodiments of the present invention need to perform spatiotemporal alignment after acquiring road condition information such as road type, road slope, road adhesion, and traffic flow information, so as to obtain a spatiotemporal alignment matrix about the real-time road condition, and then generate road condition coding information through data coding; a multi-dimensional feature tensor, namely, multimodal perception information, can be generated according to the power battery state of charge, vehicle speed, current braking intensity, and road condition coding information; and multimodal perception data at multiple consecutive moments are subjected to time series processing to obtain multimodal perception time series data.

[0084] As an optional implementation, the braking force distribution prediction model is trained by the following steps:

[0085] S201, obtaining multiple multimodal perception sample time series data of the test vehicle, and determining the braking force distribution label corresponding to each multimodal perception sample time series data through manual labeling;

[0086] S202, inputting the multimodal perception sample time series data into a pre-built CNN-LSTM hybrid neural network to obtain a braking force distribution prediction result;

[0087] S203, determining a loss value according to the braking force distribution prediction result and the braking force distribution label;

[0088] S204, updating the parameters of the CNN-LSTM hybrid neural network through a back propagation algorithm according to the loss value to obtain a trained braking force distribution prediction model;

[0089] The braking force distribution label includes the distribution ratio of the regenerative braking force and the mechanical braking force.

[0090] Specifically, multiple multimodal perception sample time series data of the test vehicle in the test scenario are obtained, which need to include data under different SOCs, different vehicle speeds, different braking intensities and different road conditions. The same processing is performed based on the aforementioned steps of generating multimodal perception time series data to obtain multimodal perception sample time series data, and the braking force distribution label of each multimodal perception sample time series data is determined by manual labeling (the distribution ratio of regenerative braking force and mechanical braking force is expressed as a percentage). For example, in the scenario of SOC=98%, congested road conditions, and braking intensity of 0.4g, the corresponding braking force distribution label can be (mechanical braking force 100%, regenerative braking force 0%), that is, when the power battery has a high state of charge and braking safety requirements For example, when the SOC is high and the braking intensity is large, the mechanical braking mode is used to ensure braking stability. For example, in the scenario of SOC=38%, vehicle speed 60km / h, and dry asphalt road, the corresponding braking force distribution label can be (mechanical braking force 0%, regenerative braking force 100%), that is, when the power battery state of charge is low, the vehicle speed is moderate, and the road conditions are good, the regenerative braking mode is used to maximize energy recovery. For example, in the scenario of SOC=75%, road slope -8%, and water film 2mm, the corresponding braking force distribution label can be (mechanical braking force 30%, regenerative braking force 70%), that is, when the power battery state of charge is moderate, anti-slip is required, and the road adhesion is small, the hybrid braking mode is used to take into account both energy recovery and braking stability.

[0091] The multimodal perception sample time series data is input into the pre-built CNN-LSTM hybrid neural network to obtain the braking force distribution prediction result. According to the braking force distribution prediction result and the braking force distribution label, the loss value is determined using the preset loss function. Then, based on the loss value, the parameters of the CNN-LSTM hybrid neural network are updated through the back propagation algorithm. After a preset number of iterations or the loss value reaches a preset threshold or the accuracy on the validation set reaches a preset threshold, a trained braking force distribution prediction model can be obtained.

[0092] The CNN-LSTM hybrid neural network combines the advantages of convolutional neural network (CNN) in spatial feature extraction and long short-term memory network (LSTM) in temporal dependency modeling, and is widely used in scenarios such as time series prediction and video analysis. The following introduces its structure and training process.

[0093] Further as an optional implementation, the CNN-LSTM hybrid neural network includes an input layer, a CNN convolutional layer, a feature fusion layer, an LSTM layer, an attention layer and an output layer, the input layer is used to input multimodal perception sample time series data, the CNN convolutional layer is used to extract features from the multimodal perception sample time series data to obtain local time series features, the feature fusion layer is used to perform feature fusion on the local time series features to obtain fused time series features, the LSTM layer is used to generate a hidden state sequence based on the fused time series features, the attention layer is used to dynamically assign weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism, and the output layer is used to map the hidden state sequence after dynamic weight assignment to a braking force distribution prediction result.

[0094] like Figure 2 The figure shows a structural schematic diagram of a CNN-LSTM hybrid neural network provided by an embodiment of the present invention. The input data is standardized through the input layer and then input into the CNN convolution layer. The local time series features are extracted using a one-dimensional convolution kernel. The extracted local time series features are subjected to the feature fusion operation of the feature fusion layer to generate fused time series features. The fused time series features are then used as the feature input of the LSTM layer to capture long-term dependencies and generate hidden state sequences. Then, based on the multi-head self-attention mechanism, the attention weight is calculated through the SoftMax function (for example, the slope feature weight is increased by 40% when going downhill), and the dimensions of the hidden state sequence output by the LSTM layer are dynamically weighted. The hidden state sequence after the dynamic weight allocation of the output layer is mapped to the braking force distribution prediction result, and then the loss value is determined in combination with the braking force distribution label. The loss value is back-propagated using the Adam algorithm, and the model parameters are gradually updated layer by layer. The loss function can use binary cross entropy or weighted loss function (to deal with data imbalance).

[0095] The above describes the training process of the braking force distribution prediction model. By inputting the multimodal perception time series data of the target vehicle into the trained braking force distribution prediction model, the real-time braking force distribution result of the target vehicle inferred by the model can be obtained.

[0096] As an optional implementation, a target braking mode is determined according to the braking force distribution result, and braking control is performed on the target vehicle according to the target braking mode, which specifically includes:

[0097] S1041, determining a regenerative braking force distribution ratio and a mechanical braking force distribution ratio according to the braking force distribution result;

[0098] S1042, when the regenerative braking force distribution ratio is 100%, determining that the target braking mode is the regenerative braking mode, and controlling the target vehicle to switch to the regenerative braking mode for braking;

[0099] S1043, when the mechanical braking force distribution ratio is 100%, determining that the target braking mode is the mechanical braking mode, and controlling the target vehicle to switch to the mechanical braking mode for braking;

[0100] S1044. When the regenerative braking force distribution ratio and the mechanical braking force distribution ratio are both not 0, determine that the target braking mode is the hybrid braking mode, and determine the target regenerative braking force and the target mechanical braking force in the hybrid braking mode, and then control the target vehicle to switch to the hybrid braking mode for braking.

[0101] Specifically, the regenerative braking force distribution ratio and the mechanical braking force distribution ratio can be obtained according to the braking force distribution result. When the regenerative braking force distribution ratio is 100% (the mechanical braking force distribution ratio is 0%), the target braking mode is determined to be the regenerative braking mode, and the target vehicle is controlled to switch to the regenerative braking mode for braking to maximize energy recovery; when the mechanical braking force distribution ratio is 100% (the regenerative braking force distribution ratio is 0%), the target braking mode is determined to be the mechanical braking mode, and the target vehicle is controlled to switch to the mechanical braking mode for braking to ensure braking stability; when the regenerative braking force distribution ratio and the mechanical braking force distribution ratio are both not 0, the target braking mode is determined to be the hybrid braking mode, and the target regenerative braking force and the target mechanical braking force in the hybrid braking mode are determined according to the regenerative braking force distribution ratio, the mechanical braking force distribution ratio and the real-time braking force demand, and then the target vehicle is controlled to switch to the hybrid braking mode and brake under the action of the target regenerative braking force and the target mechanical braking force to take into account both energy recovery and braking stability.

[0102] The above is an explanation of the method steps of the embodiment of the present invention. It can be understood that the embodiment of the present invention monitors the battery state of charge, vehicle speed, current braking intensity and real-time road condition information in real time and performs data fusion to obtain multi-modal perception time series data, and combines the pre-trained braking force distribution prediction model to obtain the corresponding braking force distribution result, and then can adaptively switch the corresponding braking mode according to the real-time driving scene and energy demand, improve the vehicle's braking performance and energy recovery efficiency, achieve more flexible braking control, and thus improve the user's driving experience.

[0103] Reference Figure 3 , an embodiment of the present invention provides a vehicle braking control system based on multimodal perception, comprising:

[0104] A data acquisition module is used to obtain the target vehicle's power battery charge state, vehicle speed, current braking intensity, and real-time road condition information;

[0105] The data fusion module is used to fuse the power battery charge state, vehicle speed, current braking intensity and real-time road condition information to obtain multi-modal perception time series data;

[0106] A braking force distribution prediction module is used to input the multimodal perception time series data into a pre-trained braking force distribution prediction model to obtain a braking force distribution result;

[0107] The braking mode determination module is used to determine a target braking mode according to the braking force distribution result, and perform braking control on the target vehicle according to the target braking mode.

[0108] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0109] Reference Figure 4 , an embodiment of the present invention provides a vehicle braking control device based on multimodal perception, comprising:

[0110] at least one processor;

[0111] at least one memory for storing at least one program;

[0112] When the at least one program is executed by the at least one processor, the at least one processor implements the vehicle braking control method based on multimodal perception.

[0113] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0114] An embodiment of the present invention further provides a computer-readable storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to execute the above-mentioned vehicle braking control method based on multimodal perception.

[0115] A computer-readable storage medium according to an embodiment of the present invention can execute a vehicle braking control method based on multimodal perception provided by an embodiment of the method of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0116] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0117] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0118] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0119] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0120] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0121] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0122] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0123] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0124] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0125] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A vehicle braking control method based on multimodal perception, characterized in that: The following steps are involved: Obtain the target vehicle's power battery charge status, vehicle speed, current braking intensity, and real-time road condition information; Performing data fusion on the power battery state of charge, the vehicle speed, the current braking intensity, and the real-time road condition information to obtain multimodal perception time series data; Inputting the multimodal perception time series data into a pre-trained braking force distribution prediction model to obtain a braking force distribution result; A target braking mode is determined according to the braking force distribution result, and braking control is performed on the target vehicle according to the target braking mode.

2. The vehicle braking control method based on multimodal perception according to claim 1, characterized in that: The step of obtaining the target vehicle's power battery charge state, vehicle speed, current braking intensity, and real-time road condition information specifically includes: Obtaining the state of charge of the power battery through a battery management system; Determine the vehicle speed by using a GPS positioning system and a wheel speed sensor; Acquiring brake line pressure through a brake pressure sensor, calculating current braking force according to the brake line pressure, and determining current braking intensity according to the current braking force; Acquire the real-time traffic information through vehicle-mounted cameras, inertial measurement units, millimeter-wave radars, and V2X systems; The real-time traffic information includes road type, road slope, road adhesion and traffic flow information.

3. The vehicle braking control method based on multimodal perception according to claim 2 is characterized in that: The real-time traffic information is obtained by using a vehicle-mounted camera, an inertial measurement unit, a millimeter-wave radar, and a V2X system, which specifically includes: Acquiring road image information through the vehicle-mounted camera, and identifying the road type according to the road image information; Acquiring the body posture information of the target vehicle through the inertial measurement unit, and determining the road surface slope according to the body posture information; Detecting the road surface water film thickness by the millimeter wave radar, and determining the road surface adhesion according to the road surface water film thickness; The vehicle density information within a preset distance range is received through the V2X system, and the traffic flow information is determined according to the vehicle density information.

4. The vehicle braking control method based on multimodal perception according to claim 2, characterized in that: The data fusion of the power battery state of charge, the vehicle speed, the current braking intensity and the real-time road condition information to obtain multi-modal perception time series data specifically includes: Performing spatiotemporal alignment and data encoding on the road type, the road surface slope, the road surface adhesion, and the traffic flow information to obtain road condition encoding information; Generate multimodal perception information according to the power battery state of charge, the vehicle speed, the current braking intensity, and the road condition coding information; The multimodal perception data at multiple consecutive moments are subjected to time series processing to obtain the multimodal perception time series data.

5. The vehicle braking control method based on multimodal sensing according to claim 1, characterized in that: The braking force distribution prediction model is trained by the following steps: Acquire multiple multimodal perception sample time series data of the test vehicle, and determine the braking force distribution label corresponding to each of the multimodal perception sample time series data through manual labeling; Inputting the multimodal sensing sample time series data into a pre-built CNN-LSTM hybrid neural network to obtain a braking force distribution prediction result; determining a loss value according to the braking force distribution prediction result and the braking force distribution label; Updating the parameters of the CNN-LSTM hybrid neural network through a back propagation algorithm according to the loss value to obtain the trained braking force distribution prediction model; The braking force distribution label includes a distribution ratio of the regenerative braking force and the mechanical braking force.

6. The vehicle braking control method based on multimodal sensing according to claim 5, characterized in that: The CNN-LSTM hybrid neural network includes an input layer, a CNN convolutional layer, a feature fusion layer, an LSTM layer, an attention layer and an output layer. The input layer is used to input the multimodal perception sample time series data, the CNN convolutional layer is used to extract features from the multimodal perception sample time series data to obtain local time series features, the feature fusion layer is used to perform feature fusion on the local time series features to obtain fused time series features, the LSTM layer is used to generate a hidden state sequence according to the fused time series features, the attention layer is used to dynamically assign weights to each dimension of the hidden state sequence based on a multi-head self-attention mechanism, and the output layer is used to map the hidden state sequence after dynamic weight assignment to the braking force distribution prediction result.

7. A vehicle braking control method based on multimodal sensing according to any one of claims 1 to 6, characterized in that: Determining a target braking mode according to the braking force distribution result, and performing braking control on the target vehicle according to the target braking mode, specifically includes: Determining a regenerative braking force distribution ratio and a mechanical braking force distribution ratio according to the braking force distribution result; When the regenerative braking force distribution ratio is 100%, determining that the target braking mode is the regenerative braking mode, and controlling the target vehicle to switch to the regenerative braking mode for braking; When the mechanical braking force distribution ratio is 100%, determining that the target braking mode is the mechanical braking mode, and controlling the target vehicle to switch to the mechanical braking mode for braking; When the regenerative braking force distribution ratio and the mechanical braking force distribution ratio are both not 0, the target braking mode is determined to be a hybrid braking mode, and the target regenerative braking force and the target mechanical braking force in the hybrid braking mode are determined, and then the target vehicle is controlled to switch to the hybrid braking mode for braking.

8. A vehicle braking control system based on multimodal perception, characterized in that: include: A data acquisition module is used to obtain the target vehicle's power battery charge state, vehicle speed, current braking intensity, and real-time road condition information; A data fusion module, used for fusing the power battery state of charge, the vehicle speed, the current braking intensity and the real-time road condition information to obtain multi-modal perception time series data; A braking force distribution prediction module, used for inputting the multimodal perception time series data into a pre-trained braking force distribution prediction model to obtain a braking force distribution result; The braking mode determination module is used to determine a target braking mode according to the braking force distribution result, and perform braking control on the target vehicle according to the target braking mode.

9. A vehicle braking control device based on multimodal perception, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a vehicle braking control method based on multimodal perception as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to execute a vehicle braking control method based on multimodal perception as described in any one of claims 1 to 7 when executed by the processor.

Images