A temperature regulation system and method based on multi-modal perception fusion

Abstract

The application discloses a temperature regulation system and method based on multi-modal perception fusion, which comprises the following steps: S1: obtaining vehicle driving data and performing data preprocessing to obtain various perception data; S2: fusing the various perception data to obtain fused data; S3: performing environment analysis on the fused data to obtain an environment category set; S4: formulating a regulation strategy for each environment category in the environment category set; and S5: dynamically regulating the air conditioning system of the vehicle according to the regulation strategy. The application adopts a multi-modal fusion technology, can effectively fuse different perception information, comprehensively considers the influence of various information on the air conditioning system, helps to improve the adaptability of the system to complex driving scenes, and enables the air conditioning system to more accurately make regulation, thereby improving driving comfort and safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle air conditioning regulation, and particularly relates to a temperature regulation system and method based on multi-modal perception fusion. BACKGROUND

[0002] Traditional vehicle air conditioning systems usually only perceive the temperature inside the vehicle based on temperature sensors, without considering the surrounding environment of the vehicle. This may lead to the air conditioning system being unable to adjust in a timely manner according to changes in the external environment in certain driving situations, such as driving on the highway or in the city, resulting in a decrease in driving comfort. Lack of understanding of individual needs of drivers: Traditional vehicle air conditioning systems usually lack understanding of the individual needs of drivers and are unable to adjust the temperature and air volume according to the preferences or physical feelings of the driver. This may lead to different drivers in the same vehicle having different comfort experiences with the air conditioning system. Poor adaptability to changes in external environment: Traditional vehicle air conditioning systems usually cannot effectively adapt to changes in the external environment, such as weather changes, traffic conditions, etc. This may lead to the air conditioning system being unable to make timely adjustments in different driving scenarios, affecting the driving experience of the driver. SUMMARY

[0003] The purpose of the present application is to provide a temperature regulation system and method based on multi-modal perception fusion to solve the technical problem of how to improve the regulation accuracy of vehicle air conditioning.

[0004] The present application is implemented by adopting the following technical solution: A temperature regulation method based on multi-modal perception fusion, comprising the following steps:

[0005] S1: Obtain vehicle driving data and perform data preprocessing to obtain multiple perception data;

[0006] S2: Fuse the multiple perception data to obtain fused data;

[0007] S3: Perform environment analysis on the fused data to obtain an environment category set;

[0008] S4: Develop a regulation strategy for each environment category in the environment category set;

[0009] S5: Dynamically adjust the air conditioning system of the vehicle according to the regulation strategy.

[0010] Further, step S1 comprises the following sub-steps:

[0011] S1: Obtain vehicle driving data through sensors, the vehicle driving data including the temperature inside the vehicle, the temperature outside the vehicle, road information, vehicle location, and vehicle speed;

[0012] S2: Preprocess the vehicle driving data by computer vision technology and / or radar signal processing technology to obtain multiple perception data.

[0013] Further, the sensor includes a temperature sensor, a camera, and a radar. For the data obtained by the temperature sensor, the temperature sensor data is directly read. For the data obtained by the camera, a convolutional neural network is used to extract features from the image. For the data obtained by the radar, a fast Fourier transform is used to convert the radar data into a spatial domain or a frequency domain representation.

[0014] Further, step S2 specifically comprises: using a multi-modal data fusion algorithm to fuse the multiple perception data to obtain fusion data, wherein the fusion data includes an indoor-outdoor temperature difference, a vehicle surrounding traffic situation, and a road state.

[0015] Further, the calculation method of the multi-modal data fusion algorithm is as follows:

[0016]

[0017] wherein F fusion represents the fused feature representation; σ represents an activation function; N represents the number of perception information; α i is an attention weight, representing the contribution degree of the i-th perception information to the fused feature; F i represents the feature representation of the i-th perception information.

[0018] Further, step S3 specifically comprises: performing environment analysis based on the fusion data to obtain an environment category set, wherein the environment category set includes multiple different environment categories, and the environment categories include weather conditions and traffic conditions.

[0019] Further, the machine learning method used for environment analysis includes decision trees and support vector machines.

[0020] Further, step S4 specifically comprises: using a rule engine to customize adjustment strategies for different environment categories, wherein the rule engine includes an IF-THEN rule engine.

[0021] Further, step S5 specifically comprises: collecting vehicle driving data in real time and dynamically adjusting the parameters of the vehicle air conditioning system through a PID controller, wherein the adjustment strategy of the PID controller is the adjustment strategy customized in step S4.

[0022] A temperature regulation system based on multi-modal perception fusion is used to implement the temperature regulation method based on multi-modal perception fusion described above, which includes a data perception module, a data fusion module, an environment analysis module, an adjustment strategy module, and a real-time adjustment module, wherein,

[0023] A perception module is configured to obtain vehicle driving data and perform data preprocessing to obtain various perception data.

[0024] A data fusion module is configured to fuse the various perception data to obtain fused data.

[0025] An environment analysis module is configured to perform environment analysis on the fused data to obtain an environment category set.

[0026] An adjustment strategy module is configured to formulate an adjustment strategy for each environment category in the environment category set.

[0027] A real-time adjustment module is configured to dynamically adjust the air conditioning system of the vehicle according to the adjustment strategy.

[0028] The present application has the beneficial effect that the present application combines vehicle sensor data, camera images, radar data and other various perception information, which can realize comprehensive perception of the environment around the vehicle. This enables the air conditioning system to more comprehensively understand the environment in which the vehicle is located, including weather conditions, road conditions, traffic conditions, etc., so as to more accurately adjust the temperature and air volume.

[0029] The present application adopts a multi-modal fusion technology, which can effectively fuse different perception information, so as to comprehensively consider the influence of various information on the air conditioning system. This helps to improve the adaptability of the system to complex driving scenarios, so that the air conditioning system can make more accurate adjustments, improving driving comfort and safety.

[0030] The present application has intelligent adjustment function, which can adjust the air conditioning parameters in real time according to the perception information, to adapt to different driving scenarios and the needs of the driver, which makes the air conditioning system better meet the individual needs of the driver, and improves the driving experience. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from the structures shown in these drawings without creative labor.

[0032] Figure 1 The flowchart of the present application. DETAILED DESCRIPTION

[0033] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0034] It should be noted that similar reference numerals and letters refer to like items in the accompanying drawings, and thus, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings.

[0035] The following will be combined with the accompanying drawings to make a detailed description of some embodiments of the present application. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0036] Referring to Figure 1 A temperature regulation method based on multi-modal perception fusion, comprising the following steps:

[0037] S1: acquiring vehicle driving data and performing data preprocessing to obtain multiple perception data;

[0038] S2: fusing the multiple perception data to obtain fused data;

[0039] S3: performing environment analysis on the fused data to obtain an environment category set;

[0040] S4: formulating a regulation strategy for each environment category in the environment category set;

[0041] S5: dynamically regulating the air conditioning system of the vehicle according to the regulation strategy.

[0042] In the present embodiment, step S1 is specifically data perception. When driving at high speed, the automobile air conditioning system perceives that the external temperature is high, which can be obtained through an external temperature sensor. At the same time, the air conditioning system also perceives that the temperature in the vehicle is low, which can be obtained through an in-vehicle temperature sensor. The camera image can show the road conditions, such as whether there is traffic congestion or whether the road conditions are good. The radar data can show the positions and speeds of surrounding vehicles and the positions of possible obstacles. Data acquisition of the external temperature sensor and the in-vehicle temperature sensor: directly reading the sensor data and representing them as T ext and T int Image processing of the camera: using computer vision techniques such as convolutional neural networks (CNN) to extract features from the image. For example, CNN can be used to extract image feature representation I. Radar data processing: using radar signal processing techniques such as fast Fourier transform (FFT) to convert radar data into spatial or frequency domain representation.

[0043] In this embodiment, step S2 is specifically: data fusion, fusing the acquired various perception data, for example, combining the data of the external temperature sensor and the in-vehicle temperature sensor to analyze the temperature difference between inside and outside the vehicle. At the same time, combining the camera image and radar data to analyze the traffic situation and road state around the vehicle. Multi-modal data fusion algorithm: using a deep learning model, such as a multi-modal fusion attention network. The fused feature is represented as F fusion , and the calculation formula is as follows:

[0044]

[0045] , where F fusion represents the fused feature representation; σ represents the activation function; N represents the number of perception information; α i is the attention weight, indicating the contribution degree of the i-th perception information to the fused feature; F i represents the feature representation of the i-th perception information.

[0046] In this embodiment, step S3 is specifically: environmental analysis, analyzing the fused data, and concluding that the external temperature is high, the in-vehicle temperature is low, and the traffic situation around the vehicle is complex, which may exist traffic congestion or other driving challenges. Environmental analysis based on fused data: using statistical analysis or machine learning methods, such as decision tree, support vector machine (SVM), etc. For example, SVM can be used to classify the environment around the vehicle to obtain the environment category, which represents the classification result of the environment around the vehicle. For example, it can be represented as different environment categories such as high-temperature weather, traffic congestion, etc.

[0047] In this embodiment, step S4 is specifically: adjustment strategy, based on the result of environmental analysis, formulating the corresponding adjustment strategy: improving the refrigeration effect to cope with the high external temperature; increasing the air volume to ensure the air circulation inside the vehicle; considering the traffic situation, it may take more active refrigeration measures to ensure the comfort and safety of the driver. Formulate adjustment strategy based on the result of environmental analysis: design adjustment strategy according to the environment around the vehicle and perception information. Rule engine: use IF-THEN rules to formulate adjustment strategy. For example, if T ext is high and T int is low, increase the refrigeration effect and air volume. Reinforcement learning-based method: learn the optimal strategy to maximize driving comfort and safety, which can use Q-learning, deep reinforcement learning, etc.

[0048] In this embodiment, step S5 is specifically: real-time adjustment, dynamically adjusting the parameters of the air conditioning system according to the real-time monitored vehicle state and environmental changes. For example, during continuous driving, the air conditioning system may gradually adjust the refrigeration effect and air volume according to changes in vehicle speed and external temperature to maintain optimal driving comfort and safety. Dynamic adjustment of air conditioning system parameters: adjusting the parameters of the air conditioning system according to the real-time monitored vehicle state and environmental changes. PID controller: using a PID controller to adjust the refrigeration effect and air volume. The calculation formula of the control variable u(t) is:

[0049]

[0050] where K p , K i , K d : proportional, integral and derivative coefficients of the PID controller, used to adjust the control variable of the refrigeration effect and air volume. e(t): error signal, representing the difference between the target value and the current value, used to calculate the control variable of the PID controller. u(t): control variable, representing the output control variable of the air conditioning system, used to adjust the refrigeration effect and air volume.

[0051] The application also provides a temperature regulation system based on multi-modal perception fusion, which implements the temperature regulation method based on multi-modal perception fusion described above, and includes a data perception module, a data fusion module, an environment analysis module, a regulation strategy module, and a real-time adjustment module.

[0052] The perception module is used to obtain vehicle driving data and perform data preprocessing to obtain various perception data.

[0053] The data fusion module is used to fuse the various perception data to obtain fused data.

[0054] The environment analysis module is used to analyze the fused data to obtain an environment category set.

[0055] The regulation strategy module is used to develop a regulation strategy for each environment category in the environment category set.

[0056] The real-time adjustment module is used to dynamically adjust the air conditioning system of the vehicle according to the regulation strategy.

[0057] Based on the above embodiment, the application has at least the following technical effects:

[0058] The application combines vehicle sensor data, camera images, radar data, and other multi-modal perception information to comprehensively perceive the environment around the vehicle. This enables the air conditioning system to more comprehensively understand the environment in which the vehicle is located, including weather conditions, road conditions, traffic conditions, etc., thereby more accurately adjusting the temperature and air volume.

[0059] The application adopts multi-modal fusion technology, which can effectively fuse different perception information, and comprehensively consider the influence of various information on the air conditioning system. This helps to improve the adaptability of the system to complex driving scenarios, so that the air conditioning system can make more accurate adjustments, and improve driving comfort and safety.

[0060] The application has intelligent adjustment function, which can adjust the air conditioning parameters in real time according to the perception information, to adapt to different driving scenarios and the needs of the driver, which makes the air conditioning system better meet the individual needs of the driver and improve the driving experience.

[0061] For the foregoing embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the application is not limited by the order of the described actions, because according to the application, some steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily required by the application.

[0062] In the above embodiments, the basic principles and main features of the application and the advantages of the application are described. Those skilled in the art should understand that the application is not limited by the above embodiments, and the above embodiments and the description in the specification are only to illustrate the principles of the application. Any modification and change made by those skilled in the art without departing from the spirit and scope of the application should be within the protection scope of the claims of the application.

Claims

1. A temperature regulation method based on multi-modal perception fusion, characterized in that, The method comprises the following steps: S1: obtaining vehicle driving data through a temperature sensor, a camera and a radar, and performing data preprocessing to obtain multiple perception data; S2: fusing the multiple perception data to obtain fused data; specifically, a multi-modal data fusion algorithm is used to fuse the multiple perception data to obtain the fused data, wherein the fused data comprises an indoor-outdoor temperature difference, a vehicle surrounding traffic condition and a road state; and a calculation method of the multi-modal data fusion algorithm is as follows: wherein F fusion represents the fused feature representation; σ represents an activation function; N represents the number of perception information; α i is an attention weight, representing the contribution degree of the i-th perception information to the fused feature; F i represents the feature representation of the i-th perception information; S3: performing environment analysis on the fused data by using a machine learning method to obtain an environment category set; the environment category set comprises multiple different environment categories, and the environment categories comprise weather conditions and traffic conditions; S4: formulating an adjustment strategy for each environment category in the environment category set; S5: collecting vehicle driving data in real time, and dynamically adjusting an air conditioning system of the vehicle according to the adjustment strategy.

2. The temperature regulation method based on multi-modal perception fusion according to claim 1, characterized in that, Step S1 comprises the following sub-steps: S11: the vehicle driving data comprises an indoor temperature, an outdoor temperature, road information, a vehicle position and a vehicle speed; S12: computer vision technology and / or radar signal processing technology are used to preprocess the vehicle driving data to obtain the multiple perception data.

3. The temperature regulation method based on multi-modal perception fusion according to claim 2, characterized in that, For the data obtained by the temperature sensor, the temperature sensor data is directly read; for the data obtained by the camera, a convolutional neural network is used to extract features of the image; and for the data obtained by the radar, fast Fourier transform is used to convert the radar data into a spatial domain or a frequency domain representation.

4. The temperature regulation method based on multi-modal perception fusion as claimed in claim 1, wherein, The machine learning method comprises a decision tree and a support vector machine.

5. The temperature regulation method based on multi-modal perception fusion according to claim 4, characterized in that, Step S4 specifically comprises: using a rule engine to customize the adjustment strategy for different environment categories, wherein the rule engine comprises an IF-THEN rule engine.

6. The temperature regulation method based on multi-modal perception fusion according to claim 5, wherein, Step S5 specifically comprises: dynamically adjusting parameters of the vehicle air conditioning system by using a PID controller, and the adjustment strategy of the PID controller is the adjustment strategy customized in step S4.

7. A temperature regulation system based on multi-modal perception fusion, used to implement the temperature regulation method based on multi-modal perception fusion according to any one of claims 1-6, characterized in that, The system comprises a data perception module, a data fusion module, an environment analysis module, an adjustment strategy module and a real-time adjustment module, wherein, The perception module obtains vehicle driving data through a temperature sensor, a camera and a radar, and performs data preprocessing to obtain multiple perception data; The data fusion module is configured to fuse the multiple perception data to obtain fused data; specifically, a multi-modal data fusion algorithm is used to fuse the multiple perception data to obtain the fused data, wherein the fused data comprises an indoor-outdoor temperature difference, a vehicle surrounding traffic condition and a road state; and a calculation method of the multi-modal data fusion algorithm is as follows: wherein F fusion represents the fused feature representation; σ represents an activation function; N represents the number of perception information; a i is an attention weight, representing the contribution degree of the i-th perception information to the fused feature; F i represents the feature representation of the i-th perception information; The environment analysis module is configured to perform environment analysis on the fused data by using a machine learning method to obtain an environment category set; the environment category set comprises multiple different environment categories, and the environment categories comprise weather conditions and traffic conditions; The adjustment strategy module is configured to formulate an adjustment strategy for each environment category in the environment category set; The real-time adjustment module is configured to collect vehicle driving data in real time, and dynamically adjust an air conditioning system of the vehicle according to the adjustment strategy.

Images