Vehicle control method and device, electronic equipment, storage medium and vehicle

Abstract

The invention discloses a vehicle control method and device, electronic equipment, a storage medium and a vehicle, and relates to the technical field of vehicles. The method comprises the following steps: acquiring multi-modal data of a vehicle based on a long-distance radar array, infrared thermal imaging detection and inertia detection modes; the multi-modal data is fused, three-dimensional space information around the vehicle is generated, the three-dimensional space information around the vehicle comprises at least one obstacle existing in a scanning range, and the scanning range is determined based on a long-distance radar array; performing risk level evaluation on at least one obstacle existing in the three-dimensional space information; and adjusting a vehicle control strategy based on the risk level. Compared with a radar detection mode in the prior art, the method has the advantages that the multi-modal data of the vehicle is collected and processed, the three-dimensional space information is generated, the risk level of the surrounding obstacles is judged on the basis of the three-dimensional space information, more accurate information is provided for a driver, and therefore driving safety is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle control method, device, electronic device, storage medium and vehicle. Background Art

[0002] As the global car population continues to grow, the proportion of traffic accidents caused by vehicle blind spots is increasing year by year. Blind spots primarily occur in conditions with poor visibility, such as curves, parallel start situations, and in inclement weather or at night. Traditional methods rely solely on radar sensing for detection. However, due to radar frequency limitations and vehicle obstruction, it is difficult to quickly and accurately identify obstacles in these blind spots, leading to traffic accidents.

[0003] Therefore, how to improve the vehicle's obstacle recognition accuracy in scenarios such as blind spots has become a problem that needs to be solved urgently. Summary of the Invention

[0004] In view of this, the present application provides a vehicle control method, device, electronic device, storage medium and vehicle to solve the problem of how to improve the obstacle recognition accuracy in scenarios such as blind spots.

[0005] In a first aspect, the present application provides a vehicle control method, comprising:

[0006] Collect multimodal data of the vehicle based on long-range radar array, infrared thermal imaging detection and inertial detection methods;

[0007] fusing the multimodal data to generate three-dimensional spatial information about the vehicle's surroundings, the three-dimensional spatial information about the vehicle's surroundings including at least one obstacle within a scanning range determined based on the long-range radar array;

[0008] Performing a risk level assessment on at least one obstacle in the three-dimensional space information;

[0009] Based on the risk level, a vehicle control strategy is adjusted.

[0010] Optionally, the multimodal data includes radar point cloud data, infrared thermal map data and inertial detection data within the vehicle's scanning range, and the inertial detection data is used to eliminate obstacle offset caused by vehicle movement; the multimodal data is subjected to modal fusion processing to generate three-dimensional spatial information around the vehicle, including: fusing the radar point cloud data and the infrared thermal map data using an extended Kalman filter to obtain a fusion result; and performing coordinate transformation on the inertial detection data and the fusion result to generate three-dimensional spatial information around the vehicle.

[0011] Optionally, the risk level assessment of at least one obstacle existing in the three-dimensional spatial information includes: obtaining the distance, relative speed, and movement direction to the target obstacle in the three-dimensional spatial information, and calculating the predicted collision time; judging whether the target obstacle is a living body based on thermal imaging data to obtain a judgment result; and dividing the risk level based on the predicted collision time and the judgment result.

[0012] Optionally, adjusting the vehicle control strategy based on the risk level includes: triggering emergency braking or steering assist functions when the risk level is high and the predicted collision time is less than a preset duration; limiting vehicle acceleration and issuing a voice reminder when the risk level is medium; and voice broadcasting obstacle information when the risk level is low.

[0013] Optionally, the method further includes: adjusting the scanning strategy of the long-range radar array when a curve scene is detected; adjusting the scanning strategy of the long-range radar array when a curve scene is detected includes: extending the detection range of the long-range radar array and obtaining the curvature radius of the current curve; and increasing the beam scanning density on the inside of the curve when the curvature radius is less than a preset radius threshold.

[0014] Optionally, the method further includes: executing a penetration detection strategy when vehicles are detected to be side by side; the penetration detection strategy includes: establishing a multipath propagation model with the bottom space of adjacent vehicles based on the long-range radar array, and calculating the actual position of the obscured life form through the radar echo phase difference; judging the behavioral intention of the obscured life form based on infrared thermal image data and human posture recognition algorithm; and issuing a warning broadcast to the driver based on the actual position and behavioral intention of the obscured life form.

[0015] In a second aspect, the present application provides a vehicle control device, comprising:

[0016] an acquisition unit configured to collect multimodal data of the vehicle based on a long-range radar array, infrared thermal imaging detection, and inertial detection methods;

[0017] a fusion unit configured to fuse the multimodal data to generate three-dimensional spatial information around the vehicle, wherein the three-dimensional spatial information around the vehicle includes at least one obstacle within a scanning range determined based on the long-range radar array;

[0018] an evaluation unit configured to perform a risk level evaluation on at least one obstacle existing in the three-dimensional space information;

[0019] The processing unit is configured to adjust the vehicle control strategy based on the risk level.

[0020] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements the vehicle control method described in the first aspect when executing the computer program.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle control method described in the first aspect.

[0022] In a fifth aspect, the present application provides a vehicle, comprising the vehicle control device as mentioned in the second aspect or the electronic device as mentioned in the fourth aspect.

[0023] Based on the above technical solution, the present application provides a vehicle control method, device, electronic device, storage medium, and vehicle. First, multimodal data from the vehicle is collected using a long-range radar array, infrared thermal imaging detection, and inertial detection. The multimodal data is then fused to generate three-dimensional spatial information surrounding the vehicle. The three-dimensional spatial information surrounding the vehicle includes at least one obstacle within the scanning range, which is determined based on the long-range radar array. A risk level assessment is then performed for the at least one obstacle present in the three-dimensional spatial information, and the vehicle control strategy is adjusted based on the risk level. Compared to related art methods that rely solely on radar detection, multimodal vehicle data is collected using a long-range radar array, infrared thermal imaging detection, and inertial detection. This distinguishes it from a single radar detection method and improves recognition accuracy. The multimodal data is then fused to generate three-dimensional spatial information, establishing an area within the scanning range surrounding the vehicle and providing information about obstacles within the scanning area. Based on this three-dimensional spatial information, the risk level of surrounding obstacles is determined and the vehicle control strategy is adjusted, providing the driver with more accurate information and improving driving safety.

[0024] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] Figure 1 A schematic flow chart of a vehicle control method provided in an embodiment of the present application is shown;

[0028] Figure 2 A schematic flow chart of another vehicle control method provided in an embodiment of the present application is shown;

[0029] Figure 3 A schematic structural diagram of a vehicle control device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] In order to be able to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In addition, in order to be able to understand the characteristics and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through a plurality of details. However, in the absence of these details, one or more embodiments can still be implemented. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0031] Based on the aforementioned background, current practical applications of traditional in-vehicle perception technology present numerous challenges that require urgent resolution. In terms of detection capabilities, the angular resolution of traditional 24GHz millimeter-wave radar is only greater than 2°, resulting in a positioning error exceeding 3 meters at a distance of 100 meters. This makes it difficult to accurately detect obstacles on long, curved roads, significantly limiting its reliability in complex road conditions. Regarding environmental adaptability, according to relevant data, visible light cameras experience a sharp drop in recognition accuracy to below 50% at nighttime illumination levels below 10 lux, and their penetration range is less than 30 meters in rainy and foggy conditions, severely impacting perception in low-light and adverse weather conditions. Multi-target processing capabilities also present significant shortcomings. When obstructed by a vehicle approaching the vehicle, existing systems are unable to penetrate the obstruction to detect pedestrians and effectively identify living beings, reducing their effectiveness in ensuring life safety and posing significant risks to driving safety. Regarding human-computer interaction, two-dimensional warning maps fail to intuitively present target height information, making it difficult to accurately distinguish between objects of varying heights, such as children and animals. This increases the driver's spatial cognitive load and hinders their ability to make informed decisions. These technical problems seriously restrict the role of in-vehicle perception systems in improving driving safety and reliability.

[0032] The vehicle control method provided in this embodiment is applied to a vehicle control device, which can be installed in an electronic control unit (ECU) in a vehicle or an extended domain control unit (XCU) of the vehicle, and is used to control the entire vehicle to execute any vehicle control method mentioned in this embodiment.

[0033] In order to improve the obstacle recognition accuracy in scenes such as blind spots, this embodiment proposes a vehicle control method. Figure 1 As shown, the method includes:

[0034] S101 collects multimodal data from vehicles using long-range radar arrays, infrared thermal imaging, and inertial sensing.

[0035] Among them, the long-range radar array is different from the existing 24GHz millimeter wave ordinary radar. The 77GHz millimeter wave radar array used in this embodiment is combined with multiple-input multiple-output (MIMO) technology to achieve a mirror velocity measurement error of less than 0.5m / s for targets 300 meters away. The radar array is composed of multiple radar units arranged in a certain pattern. It perceives the surrounding environment by transmitting and receiving radio waves. It is a radar system combination capable of detecting targets at long distances. Infrared thermal imaging detection uses infrared thermal imaging technology to detect infrared radiation emitted by objects. In the automotive field, infrared thermal imaging detection can clearly identify objects with different temperatures, such as pedestrians, animals, and other living things, at night, in low light, or in bad weather conditions, providing vehicles with target recognition and perception capabilities in complex environments. Inertial detection is a detection method that uses an inertial measurement unit (IMU) to detect the motion state of an object. Among them, the IMU includes an accelerometer and a gyroscope. The accelerometer is used to measure the vehicle's acceleration, and the gyroscope is used to measure the vehicle's angular velocity. This allows real-time acquisition of information such as the vehicle's motion trajectory and posture changes. It can also be used to eliminate the position offset of obstacles caused by vehicle movement, providing key motion status data for vehicle navigation, stability control, and intelligent driving.

[0036] A vehicle's multimodal data reflects the surrounding environment and its own motion from different perspectives. This includes, but is not limited to, long-range radar arrays providing geometric information such as the distance and speed of target obstacles; infrared thermal imaging providing information on the temperature distribution of obstacles; and inertial sensing providing information on the vehicle's motion. These multimodal data complement and synergize to more comprehensively and accurately describe the complex environment in which the vehicle finds itself, providing rich information support for intelligent decision-making and control.

[0037] S102, fusing the multimodal data to generate three-dimensional spatial information around the vehicle;

[0038] Fusion processing mainly includes the unification of data formats and mapping data information into the same system (such as three-dimensional space). First, these raw data need to be preprocessed, such as filtering and denoising the data of the long-range radar array to remove interference signals; temperature calibration of infrared thermal imaging data to ensure the accuracy of temperature data; error compensation of inertial detection data to eliminate the deviation of the sensor itself, etc.

[0039] Furthermore, the three-dimensional spatial information around the vehicle includes at least one obstacle within the scanning range, and the scanning range is determined based on the long-range radar array. The three-dimensional spatial information around the vehicle refers to a three-dimensional description of the vehicle's surrounding environment constructed by fusing multimodal data. Specifically, the three-dimensional spatial information around the vehicle includes but is not limited to obstacle information (for example, identifying a car on the road ahead, clarifying its specific position, length, width, height and driving direction in three-dimensional space), environmental feature information (terrain information such as the ups and downs, slope, and curvature of the road, as well as the position and shape of surrounding fixed objects such as buildings and trees), life form information (pedestrians are detected and their walking direction and speed are known) and dynamic change information (vehicle acceleration, deceleration, and turning). This is to facilitate the subsequent acquisition of possible dangers in the surrounding area based on the three-dimensional spatial information, and then control the vehicle.

[0040] S103, performing a risk level assessment on at least one obstacle in the three-dimensional space information;

[0041] Obstacles include static objects such as roadside buildings, parked vehicles, and traffic signs, as well as dynamic ones such as other moving vehicles, pedestrians, and animals. Based on parameters such as distance, relative speed, and direction of movement from the target obstacle in three-dimensional space, the system assesses the likelihood and severity of each obstacle posing a hazard to the vehicle, categorizing each obstacle into different risk levels, such as high, medium, and low. Based on this assessment, the vehicle control system can take appropriate measures, such as warnings or emergency braking, deceleration, and avoidance, to ensure driving safety.

[0042] S104: Adjust the vehicle control strategy based on the risk level.

[0043] Based on the different risk levels assessed from potential risks, the system will adjust the vehicle's control methods and strategies accordingly.

[0044] In this embodiment, multimodal vehicle data is first collected using a long-range radar array, infrared thermal imaging, and inertial detection. This multimodal data is then fused to generate three-dimensional spatial information surrounding the vehicle. This three-dimensional spatial information surrounding the vehicle includes at least one obstacle within the scanning range, which is determined by the long-range radar array. A risk level assessment is then performed for the at least one obstacle present in the three-dimensional spatial information, and the vehicle control strategy is adjusted based on the risk level. Compared to related art methods that rely solely on radar detection, multimodal vehicle data is collected using a long-range radar array, infrared thermal imaging, and inertial detection. This distinguishes it from a single radar detection method and improves recognition accuracy. This multimodal data is then fused to generate three-dimensional spatial information, establishing the area within the vehicle's scanning range and providing information about obstacles within that area. This three-dimensional spatial information is then used to determine the risk level of surrounding obstacles and adjust the vehicle control strategy, providing the driver with more accurate information and improving driving safety.

[0045] Optionally, the multimodal data includes radar point cloud data, infrared thermal map data and inertial detection data within the vehicle's scanning range, and the inertial detection data is used to eliminate obstacle offset caused by vehicle movement; the multimodal data is modally fused to generate three-dimensional spatial information around the vehicle, including: fusing the radar point cloud data and the infrared thermal map data using an extended Kalman filter to obtain a fusion result; and performing coordinate transformation on the inertial detection data and the fusion result to generate three-dimensional spatial information around the vehicle.

[0046] In this embodiment, because vehicle motion may affect the measurement of obstacle positions by other sensors (such as radar and infrared sensors), inertial detection data is used to eliminate obstacle position offsets caused by vehicle motion (such as acceleration, deceleration, and turning). Specifically, a motion compensation algorithm is used to obtain vehicle speed and steering angle via the vehicle CAN bus, and Doppler shift correction is performed on the radar echo to eliminate target offsets caused by vehicle motion.

[0047] Modal fusion processing aims to fuse data from different modalities to generate more comprehensive and accurate environmental information. Radar point cloud data provides precise 3D geometric information about the vehicle's surroundings, while infrared thermal image data provides information on the temperature distribution of objects. The extended Kalman filter (EKF) is used to fuse these two types of data. The EKF is a commonly used state estimation method that can handle nonlinear systems and improves state estimation accuracy by fusing data from different sensors. The fusion of radar point cloud data and infrared thermal image data combines the advantages of both radar and infrared sensors, providing richer environmental information. Coordinate transformation is performed between inertial detection data (which provides information on the vehicle's motion state) and the fusion results (the fusion of radar and infrared data) to generate 3D spatial information about the vehicle's surroundings, providing a key basis for vehicle decision-making and control.

[0048] Optionally, a risk level assessment is performed on at least one obstacle existing in the three-dimensional spatial information, including: obtaining the distance, relative speed, and movement direction to the target obstacle in the three-dimensional spatial information, and calculating the predicted collision time; judging whether the target obstacle is a living body based on thermal imaging data, and obtaining a judgment result; and dividing the risk level based on the predicted collision time and the judgment result.

[0049] In this embodiment, in the three-dimensional spatial information generated by the vehicle, the distance, relative speed and direction of movement of a specific target obstacle to the vehicle are obtained. The predicted collision time is calculated using the acquired distance, relative speed and direction of movement data. The predicted collision time refers to the time when a collision is expected to occur if the vehicle and the obstacle continue to move at the current speed and direction. Infrared thermal image data (thermal imaging data) is used to determine whether the target obstacle is a living thing. Living things (such as pedestrians, animals, etc.) usually have a temperature distribution that is different from the surrounding environment, and these characteristics are identified by thermal imaging technology. Combined with the predicted collision time and the judgment result of whether the obstacle is a living thing, the target obstacle is divided into risk levels. The risk level division can be set according to the specific application scenario and needs, for example, it can be divided into low risk, medium risk and high risk. The shorter the predicted collision time and the higher the risk level when the obstacle is a living thing.

[0050] Optionally, based on the risk level, the vehicle control strategy is adjusted, including: triggering emergency braking or steering assist functions when the risk level is high and the predicted collision time is less than a preset duration; limiting vehicle acceleration and issuing voice reminders when the risk level is medium; and voice broadcasting of obstacle information when the risk level is low.

[0051] In this embodiment, when the obstacle risk level is determined to be high and the predicted collision time is less than the preset safe time, indicating that the current driving situation poses a traffic hazard, immediate action is taken to avoid a potential collision. Specifically, this includes triggering emergency braking or activating steering assistance to help the driver adjust the vehicle's direction to avoid the obstacle. These measures can minimize the likelihood of a collision or mitigate its severity. For obstacles with a medium risk level, the vehicle system adopts a relatively mild control strategy. Firstly, the system limits the vehicle's acceleration to prevent the driver from increasing the risk of collision through excessive acceleration. Secondly, the system informs the driver of the potential risk through voice prompts, encouraging the driver to adopt more cautious driving behavior. When the obstacle risk level is low, the vehicle system focuses on providing information feedback. The system uses voice announcements to inform the driver of surrounding obstacle information, such as its location and type. This approach allows the driver to maintain awareness of their surroundings without excessively interfering with normal driving.

[0052] Optionally, the method also includes: adjusting the scanning strategy of the long-range radar array when a curve scene is detected; adjusting the scanning strategy of the long-range radar array when a curve scene is detected, including: extending the detection range of the long-range radar array and obtaining the curvature radius of the current curve; when the curvature radius is less than a preset radius threshold, increasing the beam scanning density on the inside of the curve.

[0053] This embodiment further optimizes the system for certain special scenarios. For example, when a curve is detected, the steering wheel angle can be used to determine whether the vehicle is about to enter or is currently in the curve. For example, if the angle between the steering wheel angle (i.e., and the initial position) is greater than 15°, this allows the system to accurately determine whether the vehicle is about to enter or is currently in the curve. Furthermore, the scanning strategy of the long-range radar array is adjusted to accommodate the special requirements of driving on a curve. On a curve, the driver's line of sight is obstructed by the terrain, making it impossible to detect pedestrians or non-motor vehicles on the opposite side of the curve in advance. Therefore, extending the detection range of the long-range radar array can help the vehicle detect potential obstacles in the curve earlier, allowing it to react in advance. The radius of curvature is a parameter that describes the curvature of a curve. By obtaining the current radius of curvature, the vehicle system can more accurately understand the shape and difficulty of the curve, providing a basis for subsequent adjustments to the scanning strategy. If the radius of curvature is less than a preset radius threshold (100 meters), it means the curve is sharp and more prone to obstacles or emergencies, requiring the vehicle to pay more attention to the inside of the curve. Therefore, the vehicle system will increase the beam scanning density on the inside of the curve, that is, increase the transmission and reception frequency of the radar beam on the inside of the curve to improve the perception of the environment inside the curve.

[0054] Optionally, the method also includes: when vehicles are detected side by side, executing a penetration detection strategy; the penetration detection strategy includes: based on a long-range radar array, establishing a multipath propagation model with the bottom space of adjacent vehicles, and calculating the actual position of the obscured life form through the radar echo phase difference; judging the behavioral intention of the obscured life form based on infrared thermal map data and human posture recognition algorithm; and issuing a warning broadcast to the driver based on the actual position and behavioral intention of the obscured life form.

[0055] This embodiment optimizes another special scenario: vehicles traveling side by side. This refers to situations such as waiting at a traffic light or starting a vehicle. In this case, the primary obstacle is a person or animal. Therefore, the obscured obstacle here is the obscured living being. The vehicle system first detects the presence of other vehicles traveling side by side in the current driving environment. This can be achieved using sensors such as radar and cameras. Sensor data is analyzed to determine whether other vehicles are in the same driving plane and relatively close to the vehicle. When vehicles traveling side by side are detected, a penetration detection strategy is implemented. A long-range radar array is used to establish a multipath propagation model for the space beneath adjacent vehicles. Multipath propagation refers to the fact that when radar waves encounter obstacles (such as adjacent vehicles) during propagation, they reach the target area through multiple paths, such as reflection and refraction. By establishing a multipath propagation model, the system can better understand the propagation characteristics of radar waves in complex environments. Based on this multipath propagation model, the system analyzes the phase difference of radar echoes. This phase difference is caused by radar waves propagating along different paths. By measuring and analyzing this phase difference, the actual location of living beings (such as pedestrians or animals) obscured by adjacent vehicles can be inferred. For example, a multipath propagation model can be established for the space beneath adjacent vehicles (0.3-1.2 meters above the ground), and the target position can be calculated using the phase difference of radar echoes. In addition to determining the location of a living being, the system also combines infrared thermal image data and a human gesture recognition algorithm to determine the behavioral intentions of the obscured living being. Infrared thermal image data can provide information on the temperature distribution of living beings, while the human gesture recognition algorithm can predict the living being's likely next move by analyzing its posture and movements. For example, the heat source (temperature > 32°C) on the legs of a pedestrian is detected through the tire gap (0.2-0.6 meters in height), and the human gesture recognition algorithm is used to determine behavioral intentions. Finally, based on the actual location and behavioral intentions of the obscured living being, the vehicle system will issue a warning to the driver. For example, when the radar detects a multipath reflection signal and the thermal imaging identifies a heat source, a "penetrating warning" is triggered with a response time of <500ms.

[0056] Specifically, based on the actual application of different scenarios, Figure 2 FIG. 1 is a flow chart of another vehicle control method proposed in this embodiment, and the specific details are as follows:

[0057] 1. Radar array (with integrated thermal imaging processing):

[0058] The process starts with the top "radar array (integrated thermal imaging processing)", which is the data input source for the entire system and is responsible for collecting environmental information.

[0059] For example, the radar array hardware is a 77GHz millimeter-wave radar array, combined with MIMO (multiple-input, multiple-output) technology, and the intelligent driving system is integrated through a beamforming algorithm. The above means can achieve a radial velocity measurement error of less than 0.5m / s for targets 300 meters away; infrared thermal imaging is based on an uncooled microbolometer, which identifies the thermal characteristics of living organisms through dual-band (8-12μm / 12-14μm) temperature difference analysis.

[0060] 2. Scene recognition:

[0061] The collected data enters the "scene recognition" module, and the system will determine the current scene type.

[0062] 3. Scenario branch:

[0063] Traffic light start:

[0064] If the scene is recognized as "starting at a traffic light", the system will perform "infrared scanning" and "life detection".

[0065] If a living being is detected, the system will "generate dynamic three-dimensional coordinates."

[0066] Corner:

[0067] If the scene is identified as a "curve", the system will further determine whether "curve adaptive scanning" is required.

[0068] If necessary, perform "infrared scan" and "life detection".

[0069] If a living being is detected, the system will "generate dynamic three-dimensional coordinates."

[0070] Rainy / foggy days:

[0071] If the scene is identified as "rainy / foggy", the system will perform "infrared scanning" and "life detection".

[0072] If a living being is detected, the system will "generate dynamic three-dimensional coordinates."

[0073] 4. Data transmission:

[0074] Regardless of the scenario, as long as dynamic three-dimensional coordinates are generated, the data will enter the "Data Transmission" module.

[0075] 5. Cockpit domain controller:

[0076] The transmitted data is sent to the "cockpit domain controller" for further processing.

[0077] 6.3D scene construction:

[0078] The "cockpit domain controller" uses the received data to "build a 3D scene."

[0079] 7. Reminder method:

[0080] Finally, the system can provide information to the driver through various methods, including "HUD image reminders," "center console display scenes," and "AI voice broadcast reminders." Voice warning strategies: See Table 1 below.

[0081]

[0082] Table 1 Voice warning strategy

[0083] For example, the cockpit 3D display:

[0084] (1) Targets are marked with different colors according to their threat level (red - high, yellow - medium, blue - low);

[0085] (2) Dynamically display the target motion vector (the direction / length of the arrow represents the direction / magnitude of the speed).

[0086] HUD Projection:

[0087] (1) Real-time marking of target azimuth (e.g., "30° left front") and distance (unit: meters);

[0088] (2) Display a red warning box in dangerous scenarios (response time < 100ms).

[0089] In summary, the intelligent driving domain controller achieves precise perception and intelligent response in multiple scenarios, with significant technical advantages. In terms of complex scene recognition, it can quickly judge different situations such as starting at traffic lights, curves, and rainy / foggy days, and take targeted infrared scanning and life detection measures to effectively respond to various road conditions. By generating dynamic three-dimensional coordinates, it provides the vehicle with accurate environmental spatial information, greatly improving the vehicle's perception of the surrounding environment. In terms of data transmission and processing, the process is efficient and smooth, and the cockpit domain controller can quickly build a 3D scene, allowing the driver to intuitively understand the situation outside the car. A variety of reminder methods, such as HUD images, central control screen display and AI voice broadcast, fully guarantee the driver's timely access to information, enhance driving safety and convenience, and bring a more reliable and comfortable experience to intelligent driving.

[0090] Further, as Figure 1 and Figure 2 The specific implementation of the method shown in this embodiment provides a vehicle control device, such as Figure 3 As shown, the device includes: a collection unit 301, a fusion unit 302, an evaluation unit 303 and a processing unit 304.

[0091] The acquisition unit 301 is configured to collect multimodal data of the vehicle based on a long-range radar array, infrared thermal imaging detection, and inertial detection methods;

[0092] a fusion unit 302 configured to fuse the multimodal data to generate three-dimensional spatial information about the vehicle's surroundings, wherein the three-dimensional spatial information about the vehicle's surroundings includes at least one obstacle within a scanning range determined based on the long-range radar array;

[0093] An evaluation unit 303 is configured to perform a risk level evaluation on at least one obstacle in the three-dimensional space information;

[0094] The processing unit 304 is configured to adjust the vehicle control strategy based on the risk level.

[0095] In a specific application scenario, the fusion unit 302 is specifically configured to use an extended Kalman filter to fuse the radar point cloud data and the infrared thermal map data to obtain a fusion result; and to perform coordinate transformation on the inertial detection data and the fusion result to generate three-dimensional spatial information around the vehicle.

[0096] In a specific application scenario, the evaluation unit 303 is further configured to obtain the distance, relative speed, and movement direction from the target obstacle in the three-dimensional spatial information, and calculate the predicted collision time; determine whether the target obstacle is a living body based on the thermal imaging data, and obtain a judgment result; and divide the risk level based on the predicted collision time and the judgment result.

[0097] In a specific application scenario, the processing unit 304 is further configured to trigger an emergency braking or steering assist function when the risk level is high and the predicted collision time is less than a preset duration; when the risk level is medium, limit vehicle acceleration and issue a voice reminder; when the risk level is low, voice broadcast the obstacle information.

[0098] In a specific application scenario, the processing unit 304 is further configured to extend the detection range of the long-range radar array and obtain the curvature radius of the current curve; when the curvature radius is less than a preset radius threshold, increase the beam scanning density on the inner side of the curve.

[0099] In a specific application scenario, the processing unit 304 is further configured to establish a multipath propagation model with the bottom space of adjacent vehicles based on the long-range radar array, calculate the actual position of the obscured life form through the radar echo phase difference; determine the behavioral intention of the obscured life form based on the infrared thermal map data and the human posture recognition algorithm; and issue a warning broadcast to the driver based on the actual position and behavioral intention of the obscured life form.

[0100] It should be noted that for other corresponding descriptions of the functional units involved in the vehicle control device provided in this embodiment, please refer to Figure 1 and Figure 2 The corresponding description in will not be repeated here.

[0101] Based on the above Figure 1 and Figure 2 The method shown in FIG. 1 is a method for performing the above-mentioned steps. Accordingly, this embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program can realize the above-mentioned steps. Figure 1 and Figure 2 The method shown.

[0102] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0103] Based on the above Figure 1 and Figure 2 The method shown, and Figure 3 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides an electronic device that can be configured on a computer terminal side or a vehicle terminal side, etc. The device memory, processor and computer program stored on the memory, the processor executes the computer program to achieve the above-mentioned Figure 1 and Figure 2 The method shown.

[0104] Based on the above electronic device, the embodiment of the present application further provides a vehicle, which may specifically include: Figure 3 The device shown or the electronic device as described above. The vehicle can be a new energy vehicle or a traditional vehicle.

[0105] Optionally, the physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and may optionally include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface), etc.

[0106] Those skilled in the art will understand that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0107] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device, supporting the execution of information processing programs and other software and / or programs. The network communication module is used to enable communication between components within the storage medium, as well as with other hardware and software within the physical information processing device.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by means of hardware. Compared with the related art, the solution of this embodiment collects multimodal data of the vehicle through long-range radar array, infrared thermal imaging detection and inertial detection, which is different from the single radar detection method and improves the recognition accuracy. The multimodal data is then fused to generate three-dimensional spatial information, an area within the scanning range around the vehicle is established, and the obstacle information within the scanning area can be understood. Based on the three-dimensional spatial information, the risk level of the surrounding obstacles is judged and the vehicle control strategy is adjusted to provide the driver with more accurate information, thereby improving driving safety.

[0109] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0110] The above description is only a specific embodiment of the present application, which enables those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features of the present application.

[0111] The above description and accompanying drawings sufficiently illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The embodiments represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or replaced with portions and features of other embodiments. As used in this application, the term "and / or" means including any and all possible combinations of one or more associated listed items. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitation, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or device that includes the element. In this document, each embodiment may focus on the differences from other embodiments, and similar parts between the embodiments can be referenced. For methods, devices, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be referenced in the description of the method part.

[0112] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0113] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of the present disclosure may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0114] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A vehicle control method, characterized in that: include: Collect multimodal data of the vehicle based on long-range radar array, infrared thermal imaging detection and inertial detection methods; fusing the multimodal data to generate three-dimensional spatial information about the vehicle's surroundings, the three-dimensional spatial information about the vehicle's surroundings including at least one obstacle within a scanning range determined based on the long-range radar array; Performing a risk level assessment on at least one obstacle in the three-dimensional space information; Based on the risk level, a vehicle control strategy is adjusted.

2. The method according to claim 1, characterized in that The multimodal data includes radar point cloud data, infrared thermal image data, and inertial detection data within the vehicle scanning range, wherein the inertial detection data is used to eliminate obstacle offset caused by vehicle movement; Performing modal fusion processing on the multimodal data to generate three-dimensional spatial information around the vehicle, including: The radar point cloud data and the infrared thermal map data are fused using an extended Kalman filter to obtain a fusion result; Coordinate transformation is performed on the inertial detection data and the fusion result to generate three-dimensional spatial information around the vehicle.

3. The method according to claim 1, characterized in that The performing risk level assessment on at least one obstacle in the three-dimensional space information includes: Obtaining the distance, relative speed, and movement direction from the target obstacle in the three-dimensional spatial information, and calculating the predicted collision time; Determine whether the target obstacle is a living being based on the thermal imaging data, and obtain a determination result; Based on the predicted collision time and the judgment result, risk levels are divided.

4. The method according to claim 3, characterized in that The adjusting the vehicle control strategy based on the risk level includes: When the risk level is high and the predicted collision time is less than a preset time, triggering an emergency braking or steering assist function; When the risk level is medium, the vehicle will be restricted from accelerating and a voice reminder will be issued; When the risk level is low, the obstacle information will be announced by voice.

5. The method according to claim 4, characterized in that The method further includes: adjusting a scanning strategy of the long-range radar array when a curve scene is detected; When a curve scene is detected, adjusting the scanning strategy of the long-range radar array includes: Extending the detection range of the long-range radar array and obtaining the curvature radius of the current curve; When the curvature radius is smaller than a preset radius threshold, the beam scanning density on the inner side of the curve is increased.

6. The method according to claim 4, characterized in that The method further includes: executing a penetration detection strategy when vehicles are detected to be side by side; The penetration detection strategy includes: establishing a multipath propagation model with the space beneath adjacent vehicles based on the long-range radar array, and calculating the actual position of the obscured life form through the radar echo phase difference; Determining the behavioral intention of the obscured life form based on infrared thermal image data and a human posture recognition algorithm; Based on the actual position and behavioral intention of the obscured life form, a warning is broadcast to the driver.

7. A vehicle control device, characterized in that: include: an acquisition unit configured to collect multimodal data of the vehicle based on a long-range radar array, infrared thermal imaging detection, and inertial detection methods; a fusion unit configured to fuse the multimodal data to generate three-dimensional spatial information around the vehicle, wherein the three-dimensional spatial information around the vehicle includes at least one obstacle within a scanning range determined based on the long-range radar array; an evaluation unit configured to perform a risk level evaluation on at least one obstacle existing in the three-dimensional space information; The processing unit is configured to adjust the vehicle control strategy based on the risk level.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A vehicle, characterized in that: include: The apparatus according to claim 7, or the electronic device according to claim 9.

Images