Vehicle-mounted air conditioner control method, device and system, electronic equipment and storage medium

Abstract

The invention provides a vehicle-mounted air conditioner control method, device and system, electronic equipment and a storage medium. The method comprises the steps that a capacitance change signal reflecting the fluctuation change of a thoracic and abdominal area of a driver on a vehicle and a thermal imaging image of a face area of the driver are obtained; breathing phase prediction is carried out based on the capacitance change signal and the thermal imaging image, and the breathing phase of the driver is obtained; on the basis of the breathing phase, an air conditioner control instruction is generated and used for adjusting at least one of the air outlet direction, the air outlet speed and the air outlet temperature of the vehicle-mounted air conditioner, accurate capture of the breathing phase of the driver is achieved, direct blowing can be automatically avoided or the air speed can be automatically reduced at the key moment when the driver inhales the air, and the driving safety is improved. According to the system and the method, non-inductive synchronization of air-conditioning airflow and human respiratory rhythm is realized, discomfort caused by direct blowing of a traditional vehicle-mounted air conditioner is effectively solved, and the intelligent level and the driving comfort of a cockpit are greatly improved under the condition that a driver does not need to wear any wearable equipment.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a vehicle air conditioning control method, device, system, electronic device, and storage medium. Background Technology

[0002] In car driving, an air conditioning system is indispensable for creating a comfortable in-car environment. Currently, the air vents of car air conditioners are mostly located near the steering wheel and facing the driver, which means the driver's face is easily exposed to direct airflow. Especially in cold air mode, continuous direct airflow can easily cause headaches, dry and uncomfortable nasal mucosa, and other problems for the driver, seriously affecting their health and driving experience.

[0003] Currently, most car air conditioners, while equipped with an automatic mode, primarily rely on traditional temperature and humidity sensors to detect the temperature and humidity inside the vehicle, or combine this with manually set preferences by passengers to adjust the airflow and temperature. However, this type of automatic mode, based on fixed program control, such as the pedestrian avoidance mode, simply controls the fan blades to swing left and right. When strong winds blow directly on the face, it cannot automatically adjust, still requiring the driver to manually intervene. This is not only cumbersome and extremely inconvenient, but also fails to fundamentally solve the physical discomfort caused by cold air. Summary of the Invention

[0004] This invention provides a vehicle air conditioning control method, device, system, electronic device, and storage medium to solve the problem that the automatic mode of vehicle air conditioning in the prior art is not intelligent enough, and manual adjustment is still required when strong wind blows directly, which cannot solve the problem of physical discomfort caused by cold air.

[0005] This invention provides a vehicle air conditioning control method, comprising: Acquire capacitance change signals reflecting the undulation changes in the driver's chest and abdomen region on the vehicle, as well as thermal imaging images of the driver's facial region; The driver's breathing phase is obtained by predicting the breathing phase based on the capacitance change signal and the thermal imaging image. Based on the breathing phase, an air conditioning control command is generated, which is used to adjust at least one of the following: air outlet direction, air outlet speed, and air outlet temperature of the vehicle air conditioner.

[0006] According to a vehicle air conditioning control method provided by the present invention, the breathing phase of the driver is predicted based on the capacitance change signal and the thermal imaging image; and an air conditioning control command is generated based on the breathing phase, including: Based on the thermal imaging image, the nose is located to obtain the driver's two-dimensional nose coordinates; The two-dimensional nose coordinates are transformed into spatial coordinates to obtain the three-dimensional nose tip coordinates relative to the air outlet of the vehicle air conditioner. The driver's breathing timing signal is obtained by fusing the breathing signal based on the capacitance change signal and the thermal imaging image. Based on the respiratory timing signal and the driver's average respiratory cycle, respiratory phase prediction is performed to obtain the respiratory phase; The air conditioning control command is generated based on the breathing phase and the three-dimensional nasal tip coordinates.

[0007] According to a vehicle air conditioning control method provided by the present invention, the step of fusing respiratory signals based on the capacitance change signal and the thermal imaging image to obtain the driver's respiratory timing signal includes: Based on the two-dimensional nasal coordinates, the temperature fluctuation characteristics of the driver's nasal region over time are extracted from the thermal imaging image to obtain the thermal imaging respiratory component. Frequency band filtering is performed on the capacitance change signal to obtain the capacitance breathing component; Based on the cross-correlation function between the capacitive breathing component and the thermal imaging breathing component, the relevant capacitive breathing component and the relevant thermal imaging breathing component are determined from the capacitive breathing component and the thermal imaging breathing component, respectively. The relevant capacitive breathing component and the relevant thermal imaging breathing component are fused to obtain the driver's breathing timing signal.

[0008] According to a vehicle air conditioning control method provided by the present invention, the thermal imaging image is acquired based on a thermal imaging acquisition device installed in the vehicle; the step of performing spatial coordinate transformation on the two-dimensional nose coordinates to obtain the three-dimensional nose tip coordinates relative to the air outlet of the vehicle air conditioner includes: Based on the relative positional relationship between the thermal imaging acquisition device and the air outlet of the vehicle air conditioner, the spatial coordinate transformation matrix is ​​determined; Based on the spatial coordinate transformation matrix, the two-dimensional nose tip coordinates in the two-dimensional nose coordinates are transformed to obtain the three-dimensional nose tip coordinates relative to the air outlet of the vehicle air conditioner.

[0009] According to a vehicle air conditioning control method provided by the present invention, the method includes: predicting the respiratory phase based on the respiratory timing signal and the driver's average respiratory cycle to obtain the respiratory phase; and generating the air conditioning control command based on the respiratory phase and the three-dimensional nose tip coordinates, comprising: The respiratory timing signal, the average respiratory cycle, and the three-dimensional nasal tip coordinates are input into the respiratory prediction model to obtain the air conditioning control parameters output by the respiratory prediction model. Based on the air conditioning control parameters, the air conditioning control command is generated; The breathing prediction model is used to determine the angle between the driver's nose tip position and the air outlet based on the three-dimensional nose tip coordinates and the three-dimensional air outlet coordinates, to predict the breathing phase based on the breathing timing signal and the average breathing cycle, to obtain the breathing phase, and to generate air conditioning control parameters based on the breathing phase and the angle. The breathing prediction model is trained based on the sample capacitance change signal and sample thermal imaging image of the sample driver, as well as the corresponding sample air conditioning control parameters.

[0010] According to the vehicle air conditioning control method provided by the present invention, the breathing prediction model is trained based on the following steps: The sample capacitance change signal and the sample thermal imaging image are input into the initial prediction model to obtain the predicted air conditioning control parameters output by the initial prediction model; Based on the sample air conditioning control parameters and the predicted air conditioning control parameters, the predicted loss term is determined; Based on the comfort ratings provided by the sample drivers in response to the predicted air conditioning control parameters, a penalty loss term is constructed, which is negatively correlated with the comfort rating. The initial prediction model is trained based on the prediction loss term and the penalty loss term to obtain the breathing prediction model.

[0011] According to a vehicle air conditioning control method provided by the present invention, the air conditioning control command includes a wind avoidance control command and a wind supply control command; the step of generating the air conditioning control command based on the breathing phase includes: When the breathing phase corresponds to the inhalation phase, a wind avoidance control command is generated; the wind avoidance control command is used to control the angle between the driver's nose tip position and the air outlet of the vehicle air conditioner to deflect downward by a preset avoidance angle, so as to adjust the air outlet direction and control the air outlet speed of the vehicle air conditioner to be reduced by a first preset ratio. When the breathing phase corresponds to the exhalation phase, an air supply control command is generated; the air supply control command is used to control the angle between the driver's nose tip position and the air outlet of the vehicle air conditioner so that the air outlet direction is aligned with the driver's facial area, and to control the air outlet speed of the vehicle air conditioner to increase by a second preset ratio, and to control the air outlet temperature to decrease by a preset temperature.

[0012] According to a vehicle air conditioning control method provided by the present invention, the step of generating an air conditioning control command based on the breathing phase further includes: The air conditioning control command is synchronously sent to the vehicle's infotainment system via the vehicle communication protocol, so that the infotainment system controls the vehicle's interactive interface to display the air conditioning airflow direction mapping diagram in real time based on the air conditioning control command. The vehicle system is also used to control the interactive interface to display the driver's breathing timing signal in real time, the breathing timing signal being determined based on the capacitance change signal and the thermal imaging image.

[0013] The present invention also provides a vehicle air conditioning control device, comprising: The data acquisition unit is used to acquire capacitance change signals reflecting the undulation changes of the driver's chest and abdomen region on the vehicle, as well as thermal imaging images of the driver's facial region. A phase prediction unit is used to predict the breathing phase based on the capacitance change signal and the thermal imaging image to obtain the driver's breathing phase. The instruction generation unit is used to generate air conditioning control instructions based on the breathing phase. The air conditioning control instructions are used to adjust at least one of the following: the air outlet direction, the air outlet speed, and the air outlet temperature of the vehicle air conditioner.

[0014] The present invention also provides an in-vehicle air conditioning control system, including a capacitive sensor array, a thermal imaging acquisition device, and a processor; The capacitive sensor array is mounted on the vehicle's steering wheel, and the thermal imaging acquisition device is mounted on the top of the vehicle's dashboard or in the center console area. The capacitance sensor array is used to sense the capacitance change signal caused by the undulation of the driver's chest and abdomen area in the vehicle; the thermal imaging acquisition device is used to acquire thermal imaging images including the driver's facial area. The processor is used to acquire the capacitance change signal and the thermal imaging image, and to perform respiratory phase prediction based on the capacitance change signal and the thermal imaging image to obtain the driver's respiratory phase; based on the respiratory phase, it generates an air conditioning control command, which is used to adjust at least one of the following: air outlet direction, air outlet speed, and air outlet temperature of the vehicle air conditioning.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the vehicle air conditioning control method as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle air conditioning control method as described above.

[0017] The vehicle air conditioning control method, device, system, electronic device, and storage medium provided by this invention integrate capacitance change signals reflecting mechanical motion and thermal imaging images reflecting thermodynamic changes to predict the breathing phase, achieving accurate capture of the driver's breathing phase. Furthermore, based on the generation of control commands and vehicle air conditioning control according to the breathing phase, it can automatically avoid direct airflow or reduce the airflow speed at the critical moment of the driver's inhalation, achieving imperceptible synchronization between the air conditioning airflow and the human breathing rhythm. This effectively solves the discomfort caused by direct airflow in traditional vehicle air conditioning systems, greatly improving the intelligence level of the driver's cabin and the driving comfort without requiring the driver to wear any wearable devices. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the vehicle air conditioning control method provided by the present invention; Figure 2 This is a general flowchart of the vehicle air conditioning control method provided by the present invention; Figure 3 This is a schematic diagram of the structure of the vehicle air conditioning control device provided by the present invention; Figure 4 This is a schematic diagram of the structure of the vehicle air conditioning control system provided by the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] During driving, an air conditioning system is essential to ensure a comfortable in-car environment. However, because the air conditioning vents are generally located near the steering wheel and are usually directed towards the driver to ensure airflow circulation, the driver's face is easily exposed to direct airflow from the vents. This continuous direct airflow, especially in cold weather, can easily cause symptoms such as headaches and dry nasal mucosa, seriously affecting the driver's health and driving experience.

[0022] Currently, while most car air conditioners are equipped with an automatic mode, they primarily rely on traditional temperature and humidity sensors to detect the temperature and humidity inside the vehicle, or combine this with manually set preferences by passengers, such as seat heating / ventilation linkage, to adjust the overall airflow and temperature. However, this approach has significant limitations. Existing automatic modes are mostly based on fixed program control. For example, the pedestrian avoidance mode often only controls the fan blades to perform simple left and right sweeping, failing to recognize the driver's specific facial position or sense their real-time physiological breathing status. When strong airflow blows directly on the face, it cannot take targeted avoidance actions, still requiring the driver to manually intervene. This is not only cumbersome but also fails to fundamentally solve the problem of discomfort caused by inhaling cold air.

[0023] In response, this invention provides a vehicle air conditioning control method. It aims to construct a breathing-synchronized intelligent air conditioning control mechanism based on non-contact multimodal biosignal recognition by using capacitance change signals reflecting the fluctuations in the driver's chest and abdomen area and thermal imaging images reflecting changes in nasal breathing temperature. This mechanism accurately obtains the driver's exhalation / inhalation inflection points and durations, and dynamically adjusts the airflow direction, speed, and temperature of the vehicle air conditioning according to the driver's real-time breathing rhythm. This achieves energy saving while avoiding direct airflow onto the face, thereby significantly improving driving comfort.

[0024] Figure 1 This is a flowchart illustrating the vehicle air conditioning control method provided by the present invention, as shown below. Figure 1 As shown, the method includes: Step 110: Acquire capacitance change signals reflecting the undulation changes in the driver's chest and abdomen region on the vehicle, as well as thermal imaging images of the driver's facial region. Step 120: Based on the capacitance change signal and thermal imaging image, predict the breathing phase to obtain the driver's breathing phase; Step 130: Based on the breathing phase, generate air conditioning control commands. The air conditioning control commands are used to adjust at least one of the following: air outlet direction, air outlet speed, and air outlet temperature of the vehicle air conditioning.

[0025] Specifically, in order to achieve automatic control of the vehicle's air conditioning during driving and thus optimize the driver's driving experience, this embodiment of the invention utilizes non-contact multimodal sensing technology to perceive the driver's physiological characteristics from multiple dimensions, thereby achieving vehicle air conditioning adjustment synchronized with the driver's breathing rhythm. The vehicle here can be a car, truck, bus, or other means of transportation equipped with vehicle air conditioning and a driver's cab.

[0026] To capture breathing movements without interfering with driving operations, this invention proposes using a capacitive sensor array located within the driver's cabin, such as on the steering wheel rim or spokes, to collect capacitance change signals reflecting the fluctuations in the driver's chest and abdominal region. Specifically, based on the principle of near-field capacitive sensing, the human body is considered a conductor. When a driver breathes under natural driving conditions, their chest and abdomen periodically expand and contract with inhalation and exhalation. Although this fluctuation in the chest and abdominal region does not directly contact the steering wheel, it alters the distance between the chest / abdomen and the capacitive sensor on the steering wheel. Since capacitance is inversely proportional to distance, the parasitic capacitance value sensed by the sensor will fluctuate slightly, such as on the order of 0.1 pF. By continuously collecting this series of time-varying signals, a capacitance change signal accurately reflecting the fluctuations in the driver's chest and abdominal region can be obtained.

[0027] Meanwhile, thermal imaging acquisition devices, such as infrared miniature thermal imagers, installed in specific locations inside the vehicle, such as the top of the dashboard or the center console area, can acquire real-time thermal images of the driver's entire face. These images can then be segmented into high-temperature regions to extract thermal images of the facial area. For example, a thresholding method can be used to extract a region of 34-37°C (the temperature of the facial area) from the overall thermal image and use this region as the thermal image of the driver's face.

[0028] It is important to note here that the thermal imaging image of the driver's face not only provides the positional data of the driver's face, but more importantly, it also contains information on the temperature distribution of the face. Especially in the area of ​​the mouth and nose, the hot air exhaled during exhalation causes the local temperature to rise, while the cold ambient air inhaled during inhalation causes the local temperature to drop. These dynamic changes in the temperature field are all fully recorded in the thermal imaging image.

[0029] After acquiring data from the two different modalities mentioned above, this embodiment of the invention can perform fusion analysis on the two to accurately determine the driver's current breathing state. Specifically, since a single sensor may have limitations in the complex in-vehicle environment, such as capacitance change signals being easily interfered with by mechanical vibrations caused by road bumps, and thermal imaging images being easily affected by changes in lighting, this embodiment of the invention combines capacitance change signals and thermal imaging images for processing. That is, the temporal features characterizing the mechanical movement of the chest and abdomen in the capacitance change signals are extracted separately, such as the expansion and contraction of the chest cavity corresponding to peaks and troughs, and the temperature changes over time in specific facial areas of the thermal imaging images, such as the tip of the nose and the philtrum.

[0030] By comprehensively calculating or reasoning about these two sets of features, such as through logical verification, cross-correlation analysis, and neural network model processing, it is possible to accurately predict the driver's respiratory phase within a future time period, such as 300ms. This respiratory phase characterizes the driver's specific state in the respiratory cycle, mainly including the inhalation phase, the exhalation phase, and the transition phase between the two, such as breath-holding or the instant of transition.

[0031] After that, the corresponding air conditioning control command can be generated based on the predicted breathing phase. The air conditioning control command can be a digital message or an electrical signal and is sent to the actuator of the vehicle air conditioning control system, such as the damper stepper motor, blower driver, positive temperature coefficient heater, etc., to achieve vehicle air conditioning adjustment synchronized with the breathing rhythm.

[0032] Here, the air conditioning control commands are diverse, and can be used to adjust any one or more of the following: airflow direction, airflow speed, and airflow temperature. For example, when the predicted breathing phase is inhalation, to avoid the airflow directly hitting the driver's mouth and nose, causing choking or dryness of the nasal mucosa, the air conditioning control command can drive the air guide vanes to rotate, adjusting the airflow direction so that it is deviated from the driver's face. When the breathing phase changes to exhalation, the air conditioning control command can drive the air vents to reset or face the driver to assist in heat dissipation.

[0033] For example, when the predicted breathing phase is inhalation, the generated air conditioning control command can adjust the blower speed to reduce the airflow speed; while when the predicted breathing phase is exhalation, the air conditioning control command can adjust the blower speed to increase the airflow speed, thereby creating a pulsed, gentle airflow experience that matches the driver's breathing rhythm.

[0034] For example, based on the predicted breathing phase, the driver's breathing frequency corresponding to the capacitance change signal and thermal imaging image, and the body's metabolic state reflected by that breathing frequency, an air conditioning control command can be generated to fine-tune the temperature / coolness of the air outlet.

[0035] The vehicle air conditioning control method provided by this invention integrates capacitance change signals reflecting mechanical motion and thermal imaging images reflecting thermodynamic changes to predict the breathing phase, achieving accurate capture of the driver's breathing phase. Furthermore, based on the generation of control commands and vehicle air conditioning control according to the breathing phase, it can automatically avoid direct airflow or reduce the airflow speed at the critical moment of the driver's inhalation, achieving imperceptible synchronization between the air conditioning airflow and the human breathing rhythm. This effectively solves the discomfort caused by direct airflow in traditional vehicle air conditioning systems, greatly improving the intelligence level of the driver's cabin and the driving comfort without requiring the driver to wear any wearable devices.

[0036] Based on the above embodiments, steps 120 and 130 include: The driver's two-dimensional nose coordinates were obtained by locating the nose based on thermal imaging images. By performing a spatial coordinate transformation on the two-dimensional nose coordinates, the three-dimensional nose tip coordinates relative to the air outlet of the vehicle's air conditioner are obtained. The driver's respiratory timing signal is obtained by fusing the capacitance change signal and the thermal imaging image. Based on the respiratory timing signal and the driver's average respiratory cycle, respiratory phase prediction is performed to obtain the respiratory phase; Air conditioning control commands are generated based on respiratory phase and three-dimensional nasal tip coordinates.

[0037] Specifically, the process of predicting the respiratory phase based on the capacitance change signal and thermal imaging image, and generating air conditioning control commands based on the predicted respiratory phase, includes: After acquiring a thermal image of the driver's facial region, image processing algorithms, such as edge detection and machine learning object detection, can be used to analyze the image to locate the area where the driver's nose is located, i.e., the nasal region, and determine the two-dimensional coordinates of this region, i.e., two-dimensional nasal coordinates. Specifically, because the human nasal region exhibits significant and regular temperature characteristics during respiration—for example, the temperature is higher than the surrounding skin during exhalation and slightly lower during inhalation—this characteristic can be used to quickly identify the nasal region from the thermal image and determine the coordinates of multiple pixels within this region, such as the tip of the nose and the center of the nostrils, in the pixel coordinate system of the thermal image, i.e., the two-dimensional nasal coordinates. These coordinates visually reflect the planar position of the driver's nose in the thermal image.

[0038] Knowing only the two-dimensional pixels in the image is insufficient, because the air vents of a car's air conditioning system are located in a specific three-dimensional space. Therefore, it is also necessary to know the actual distance and angle of the driver's nose area relative to the air vents. That is, by combining the intrinsic parameters of the thermal imaging acquisition device, such as focal length and principal point, and the extrinsic parameters, such as the device's position relative to the air vents, a spatial coordinate transformation can be performed on the aforementioned two-dimensional nose coordinates. This geometric transformation maps the planar pixel coordinates back to the true three-dimensional spatial coordinates, thereby calculating the three-dimensional coordinates of the driver's nose tip relative to the center point of the air vent, i.e., the three-dimensional nose tip coordinates. For example, represented as (x, y, z) in a Cartesian coordinate system, these three-dimensional coordinates accurately describe the depth, height, and horizontal position of the driver's nose tip within the vehicle's interior space, providing a precise geometric basis for calculating the air vent angle when adjusting the air vent direction.

[0039] Meanwhile, to address the issue of when to blow air, this embodiment of the invention also requires obtaining extremely high-precision respiratory rhythm. Specifically, although both capacitance change signals and thermal imaging images can reflect respiratory rhythm, a single signal is easily affected by interference. Therefore, in this embodiment, the capacitance change signal reflecting chest and abdominal fluctuations is fused with the thermal imaging image reflecting facial temperature changes to obtain the driver's respiratory timing signal. Here, the respiratory signal fusion process aims to leverage the strengths of both signals, for example, by utilizing the high sensitivity of the capacitance change signal and the spatial determinism of the thermal imaging image to eliminate noise in the single signal, such as vehicle vibration noise and light thermal noise. The fused respiratory timing signal can more realistically and smoothly depict the driver's current breathing strength and rhythm.

[0040] After obtaining a high-quality respiratory timing signal, the future respiratory phase can be predicted based on this signal in this embodiment of the invention. Specifically, when predicting the respiratory phase, it is necessary not only to analyze the current waveform trend, but also to call historical data to determine the driver's average respiratory cycle, that is, the average duration required for the driver to take a complete breath over a period of time in the past; and based on these two, the driver's respiratory phase in the future can be predicted.

[0041] Here, the average respiratory cycle is introduced as a reference benchmark, which can effectively prevent misjudgments caused by the driver's occasional deep or rapid breathing. Furthermore, when predicting the respiratory phase, the current respiratory timing signal trend and the pattern of the average respiratory cycle are combined to use a prediction algorithm to infer which node in the respiratory cycle the driver will be in within a future time period, such as 300ms, thus obtaining the respiratory phase, such as being in the inhalation phase with a high probability.

[0042] After obtaining the breathing phase and three-dimensional nose tip coordinates through the above process, in this embodiment of the invention, instructions can be generated based on these two data to obtain air conditioning control instructions. That is, the prediction results in the time dimension are combined with the positioning results in the spatial dimension to generate air conditioning control instructions. For example, based on the calculated three-dimensional nose tip coordinates, a geometric mathematical model can be used to calculate the horizontal azimuth angle and vertical pitch angle that the air outlet needs to be adjusted to ensure that the airflow can accurately align with or avoid the nose tip position; at the same time, combined with the predicted breathing phase, it is determined whether to execute an alignment strategy (during exhalation) or an avoidance strategy (during inhalation), and the corresponding air outlet speed, air outlet temperature, etc. are determined; finally, these decisions are encoded into specific air conditioning control instructions and sent to the actuator to realize the air outlet control of the vehicle air conditioner.

[0043] In this embodiment of the invention, millimeter-level precise positioning with the tip of the nose as the target point is achieved by transforming coordinates from a two-dimensional image to a three-dimensional space. Regardless of the driver's height, weight, or posture, key parts can be locked. At the same time, by fusing dual-modal signals and combining them with the average respiratory cycle for prediction, the misjudgment rate caused by environmental interference is greatly reduced, ensuring the robustness and real-time performance of respiratory phase prediction. In addition, the air conditioning control command is generated by combining the prediction results in the time dimension and the positioning results in the spatial dimension, realizing intelligent control that moves with the breath.

[0044] Based on the above embodiments, respiratory signal fusion is performed based on capacitance change signal and thermal imaging image to obtain the driver's respiratory timing signal, including: Based on two-dimensional nasal coordinates, the characteristics of temperature fluctuation in the driver's nasal region over time are extracted from thermal imaging images to obtain thermal imaging respiratory components. Frequency band filtering is performed on the capacitance change signal to obtain the capacitance breathing component; Based on the cross-correlation function between the capacitive breathing component and the thermal imaging breathing component, the relevant capacitive breathing component and the relevant thermal imaging breathing component are determined from the capacitive breathing component and the thermal imaging breathing component, respectively. The driver's breathing timing signal is obtained by fusing the relevant capacitive breathing component and the relevant thermal imaging breathing component.

[0045] Specifically, considering that in complex driving environments, a single sensor is easily interfered with, such as vehicle vibration affecting capacitive sensors and changes in light and shadow affecting thermal imaging, this embodiment of the invention extracts the purest breathing timing signal through a cross-correlation verification mechanism when acquiring the driver's breathing timing signal.

[0046] In detail, this could involve using the two-dimensional nasal coordinates obtained from nasal localization as an index to locate the driver's nostrils and surrounding area in continuously acquired thermal imaging images, and designating this area as the region of interest. Subsequently, since respiratory airflow causes regular temperature fluctuations in this area (e.g., warming during exhalation and cooling during inhalation), the average or highest temperature within this area can be continuously tracked. The waveform data showing this temperature variation over time can then be extracted, thus forming the thermal imaging respiratory component. This component is a signal that characterizes the driver's respiratory state from a thermodynamic perspective.

[0047] Meanwhile, in this embodiment of the invention, the acquired capacitance change signal can also be processed. Specifically, considering that the original capacitance change signal contains high-frequency vibration noise generated by vehicle movement, such as engine vibration and road bumps, as well as extremely low-frequency body posture adjustment signals, a digital filter can be used to filter out the frequency bands of the original capacitance change signal, retaining the effective signal within the normal adult breathing frequency range, such as 0.1Hz to 0.5Hz, while filtering out interference noise. The waveform data retained after this processing is the capacitance breathing component. This component is a signal that characterizes the driver's chest and abdomen rise and fall state from a mechanical motion perspective.

[0048] Furthermore, to verify whether the two components truly reflect the same breathing action, rather than some coincidental interference, this embodiment of the invention introduces a cross-correlation function for analysis. This function measures the similarity and lag relationship between two time-series signals at different times. Specifically, this can involve first calculating the cross-correlation between the capacitive breathing component and the thermal imaging breathing component. If both indeed originate from the same driver's breathing, they should exhibit a high correlation on the time axis, i.e., peak to peak, or a fixed physiological phase difference. Based on the calculation results, uncorrelated time periods or signal segments can be removed from both. For example, a moment with only capacitive fluctuations but no change in nasal temperature might be identified as accidental hand contact with the steering wheel, retaining the effective signal segments with significant correlation. These effective signal portions retained after cross-correlation verification are the correlated capacitive breathing component and the correlated thermal imaging breathing component.

[0049] Following this, the relevant capacitive breathing component and the relevant thermal imaging breathing component can be fused to obtain the driver's breathing time-series signal. That is, to obtain the most accurate and smoothest breathing curve, in this embodiment of the invention, the breathing components after cross-correlation verification can be fused. This fusion process can employ methods such as weighted averaging, Kalman filtering fusion, and Bayesian estimation. For example, considering that the relevant capacitive breathing component has a fast time response and the relevant thermal imaging breathing component has high spatial determinism, the two can be combined according to a certain strategy to generate the final breathing time-series signal. This signal is a high-confidence breathing waveform that eliminates single-mode noise while retaining the true rhythmic characteristics of breathing, providing a data foundation for subsequent phase prediction.

[0050] In this embodiment of the invention, a cross-correlation verification mechanism is introduced. By comparing the synchronicity of mechanical motion and thermodynamic changes, false breathing signals, such as simple body swaying or changes in lighting, are effectively filtered out. Furthermore, the signal fusion combines the sensitivity of the capacitive sensor and the vibration resistance of thermal imaging, so that the generated breathing timing signal can remain stable and accurate under various complex conditions such as vehicle bumps and changes in lighting, greatly improving robustness.

[0051] Based on the above embodiments, the thermal imaging image is acquired using a thermal imaging acquisition device installed inside the vehicle; a spatial coordinate transformation is performed on the two-dimensional nose coordinates to obtain the three-dimensional nose tip coordinates relative to the air vents of the vehicle's air conditioning system, including: Based on the relative positional relationship between the thermal imaging acquisition device and the air outlet of the vehicle air conditioner, the spatial coordinate transformation matrix is ​​determined; Based on the spatial coordinate transformation matrix, the two-dimensional nose tip coordinates in the two-dimensional nose coordinate system are transformed to obtain the three-dimensional nose tip coordinates relative to the air vent of the vehicle's air conditioning system.

[0052] Specifically, in practical applications, the thermal imaging acquisition device used to collect thermal images can be installed in a location within the driver's cab with a wide field of vision and directly facing the driver's seat, such as the top of the sun visor above the dashboard or the top edge of the central control display screen. This fixed installation method ensures that the device has a stable field of view and can continuously cover the driver's facial area.

[0053] Accordingly, the process of performing spatial coordinate transformation on the two-dimensional nose coordinates to obtain the three-dimensional nose tip coordinates relative to the air vents of the vehicle's air conditioning system can specifically include: To convert the image position under the field of view of the thermal imaging acquisition device into the spatial position under the field of view of the vehicle air conditioner vent, this embodiment of the invention requires first determining the relative relationship between the two in three-dimensional space. That is, during the vehicle factory calibration or air conditioning control system initialization phase, the relative positional relationship between the optical center of the thermal imaging acquisition device and the air conditioner vent is already set. Here, the spatial coordinate transformation matrix can be directly calculated using this positional relationship. This process involves the transformation of two coordinate systems: the camera coordinate system of the thermal imaging acquisition device (with the camera optical center as the origin), and the air conditioner vent coordinate system (with the vent center point as the origin). The relative positional relationship describes the installation distance of the camera relative to the vent, such as the camera being 10cm above the vent and 5cm to the left, as well as the deviation of the installation angle. Through the rigid body transformation principle, these geometric parameters can be solidified into a spatial coordinate transformation matrix, serving as the reference for subsequent spatial coordinate transformations.

[0054] Next, using the spatial coordinate transformation matrix calculated in the previous step, the two-dimensional nose tip coordinates in the two-dimensional nose coordinate system can be transformed to obtain the required three-dimensional nose tip coordinates. Specifically, the two-dimensional nose tip coordinates on the image plane can be back-projected onto the camera coordinate system using the camera's intrinsic parameters, such as focal length and principal point, to recover its preliminary three-dimensional coordinates from the camera's perspective. Then, using the spatial coordinate transformation matrix, matrix multiplication is performed on the preliminary three-dimensional coordinates to translate and rotate the nose tip position from the camera coordinate system to the air vent coordinate system, thus obtaining the three-dimensional nose tip coordinates in the air vent coordinate system, i.e., the three-dimensional nose tip coordinates relative to the vehicle's air vent. This coordinate system provides highly intuitive spatial positional information, such as the driver's nose being located x millimeters directly in front of the air vent, y millimeters above, and z millimeters to the right, providing a precise geometric basis for calculating the air vent angle.

[0055] In this embodiment of the invention, a spatial coordinate transformation matrix is ​​introduced, which not only decouples the coordinate system, allowing for flexible selection of the installation position of the thermal imaging acquisition device without coinciding with the air outlet, but also compensates for the deviation of the physical position simply by matrix calibration, increasing the freedom of hardware layout, but also realizes the coordinate transformation from the image plane to three-dimensional space, enabling accurate calculation of the three-dimensional nose tip coordinates, thereby achieving precise wind direction control.

[0056] Based on the above embodiments, respiratory phase prediction is performed based on the respiratory timing signal and the driver's average respiratory cycle to obtain the respiratory phase; based on the respiratory phase and three-dimensional nose tip coordinates, air conditioning control commands are generated, including: The respiratory timing signal, average respiratory cycle, and three-dimensional nasal tip coordinates are input into the respiratory prediction model to obtain the air conditioning control parameters output by the respiratory prediction model. Generate air conditioning control commands based on air conditioning control parameters; The breathing prediction model is used to determine the angle between the driver's nose tip position and the air outlet based on the three-dimensional nose tip coordinates and the three-dimensional air outlet coordinates. It predicts the breathing phase based on the breathing time sequence signal and the average breathing cycle, obtains the breathing phase, and generates air conditioning control parameters based on the breathing phase and the angle. The breathing prediction model is trained based on the sample capacitance change signal and sample thermal imaging image of the sample driver, as well as the corresponding sample air conditioning control parameters.

[0057] Specifically, the process of predicting the respiratory phase based on the respiratory timing signal and the driver's average respiratory cycle, obtaining the respiratory phase, and generating air conditioning control commands based on this respiratory phase and the three-dimensional nose tip coordinates can be achieved with the help of a respiratory prediction model.

[0058] Specifically, this could involve providing the respiratory timing signal reflecting the driver's current breathing waveform, the average respiratory cycle providing the driver's baseline breathing rhythm, and the three-dimensional nasal tip coordinates representing the target's spatial location as input to the respiratory prediction model. Upon receiving these inputs, the model performs complex nonlinear calculations internally, ultimately outputting a set of optimized air conditioning control parameters. These parameters are the values ​​that generate air conditioning control commands and drive the actuators; for example, they could be the angle of the air guide vanes used to adjust the airflow direction, or the speed of the blower used to adjust the airflow speed.

[0059] Specifically, after inputting the respiratory timing signal, average respiratory cycle, and three-dimensional nasal tip coordinates into the respiratory prediction model, the model will perform the following two-dimensional calculation tasks: Spatial computation task: The model uses the input 3D nose tip coordinates, combined with the known 3D air outlet coordinates (i.e., the origin in the air outlet coordinate system), to determine the angle between the driver's nose tip position and the air outlet through geometric calculations, including the horizontal azimuth and vertical pitch angles. This angle clarifies the geometric direction in which the airflow needs to blow.

[0060] Time prediction task: The model uses the input breathing timing signal (capturing the current instantaneous state) and the average breathing cycle (capturing long-term patterns) to make a forward-looking inference about the driver's future breathing state, that is, to perform breathing phase prediction, in order to predict whether the driver's breathing state in the future period of time is in the inhalation stage, the exhalation stage, or the transition stage between the two, thereby obtaining the breathing phase.

[0061] Furthermore, the model comprehensively considers the predicted breathing phase and the calculated angle, and generates the final air conditioning control parameters through an internal policy mapping layer, such as a fully connected layer or a decision tree. For example, if the predicted breathing phase corresponds to the inhalation phase, the air conditioning control parameters output by the model will indicate a downward deflection of the angle, such as deflecting the air guide vanes downward by a specific avoidance angle; while if the predicted breathing phase corresponds to the exhalation phase, the air conditioning control parameters output by the model will indicate an adjustment of the angle, such as adjusting the angle of the air guide vanes to align with the driver's nose.

[0062] However, it is worth noting that, in this embodiment of the invention, to ensure model performance before applying the respiratory prediction model for respiratory phase prediction, the model can be pre-trained using a large amount of real data. Here, the training process of the respiratory prediction model includes: First, a large amount of data was collected from sample drivers of different body types and driving postures. This data includes sample capacitance change signals (including capacitance change signals caused by chest and abdominal movements under various breathing patterns), sample thermal imaging images (including thermal imaging images of facial thermal features and positional changes under various ambient temperatures), and corresponding sample air conditioning control parameters (labels during training, i.e., optimal air conditioning control parameters, calibrated by experts or measured through comfort experiments, such as the optimal air outlet angle and air outlet speed at a certain moment of inhalation).

[0063] Next, the collected data can be used to train the initial prediction model. The initial prediction model is built on Long Short-Term Memory (LSTM) networks, Transformer-based neural networks, and other technologies.

[0064] Specifically, this could involve inputting the collected data into an initial prediction model and iteratively training it based on the model's output and labels. For example, the backpropagation algorithm could be used to continuously adjust the model's internal parameters, enabling it to learn the complex mapping relationship between the breathing waveform and the optimal air conditioning control parameters. The resulting trained model would then possess the ability to infer the optimal air conditioning control parameters based on real-time input data.

[0065] After obtaining the specific air conditioning control parameters output by the model, in this embodiment of the invention, they can be converted into electrical signals or digital messages that the underlying hardware can recognize, i.e., air conditioning control commands, and sent to the actuator through the vehicle bus to complete the intelligent adjustment of the vehicle air conditioning.

[0066] In this embodiment of the invention, breathing phase prediction is performed by a breathing prediction model, which can predict the breathing phase in advance, thereby offsetting the delay of motor action and airflow transmission, and achieving true zero-time-difference breathing synchronization. Furthermore, by training the model with a large amount of sample driver data of different body types and driving postures, the model can adapt to the breathing habits of different drivers, thus being able to cope with various scenarios in practical applications and greatly improving robustness.

[0067] Based on the above embodiments, the respiratory prediction model is trained using the following steps: The sample capacitance change signal and sample thermal imaging image are input into the initial prediction model to obtain the predicted air conditioning control parameters output by the initial prediction model. Based on the sample air conditioning control parameters and the predicted air conditioning control parameters, the predicted loss term is determined; Based on the comfort ratings of sample drivers in response to the predicted air conditioning control parameters, a penalty loss term is constructed, which is negatively correlated with the comfort rating. The initial prediction model is trained based on the prediction loss term and the penalty loss term to obtain the breathing prediction model.

[0068] Traditional model training often relies solely on fitting standard data, resulting in models with limited and unremarkable capabilities. Therefore, this invention introduces a reinforcement training strategy based on comfort ratings, incorporating the driver's subjective experience into the model's optimization objective.

[0069] Specifically, the training process of the above-mentioned respiratory prediction model includes: First, the initial model for the training process needs to be determined, i.e., the initial prediction model to be optimized. This model can be built on the basis of long short-term memory networks, neural networks based on the Transformer architecture, etc. Next, a large amount of collected data can be fed into the initial prediction model as input. This data includes sample capacitance change signals recording the chest and abdominal undulation characteristics of the sample drivers and sample thermal imaging images recording the facial thermal characteristics of the sample drivers.

[0070] Correspondingly, after receiving this data, the initial prediction model will perform forward propagation to give a prediction result, namely, the predicted air conditioning control parameters.

[0071] Next, based on the predicted air conditioning control parameters output by the initial prediction model and the labels of the input data (i.e., the sample air conditioning control parameters), the model's prediction error can be calculated. This involves calculating the distance between the predicted air conditioning control parameters and the sample air conditioning control parameters. Using mean squared error or cross-entropy loss functions, the model's prediction error, or prediction loss term, can be obtained. This term reflects the model's accuracy on the air conditioning control parameters, specifically how much the model's predicted air outlet angle, air outlet velocity, and air outlet temperature deviate from the optimal values.

[0072] It is worth noting that, considering that in practical applications there may be situations where the numerical values ​​are accurate but the driver's experience is not good, such as when the air conditioning control parameters predicted by the model are consistent with the theory, but the driver's subjective feeling is not good, a subjective evaluation mechanism is introduced in this embodiment of the invention to solve this problem.

[0073] Specifically, during the data collection process from sample drivers, their comfort ratings for the current in-vehicle air conditioning control parameters are recorded. This can be achieved through voice feedback or by clicking the "Comfortable / Uncomfortable" button on the screen, with scores ranging from 0 to 10. Based on these comfort ratings, a dedicated penalty loss term can be constructed. This loss term is negatively correlated with the comfort rating. In other words, the lower the sample driver's score, i.e., the more uncomfortable they feel, the larger the penalty loss term, and the heavier the penalty imposed on the model; conversely, the higher the score, i.e., the more comfortable they feel, the smaller the penalty loss term. This reward / reward mechanism allows the model to avoid predictions that lead to low scores.

[0074] Finally, the weighted sum of the prediction loss term (representing objective error) and the penalty loss term (representing subjective experience) yields the total loss function. Gradient descent or its variants can then be used to backpropagate and update the parameters of the initial prediction model based on this total loss function. Through multiple rounds of iterative training, the model continuously adjusts its parameters, striving not only to numerically fit the label but also to improve the driver's experience and avoid generating high-penalty outputs. Ultimately, the fully trained and converged model becomes a breathing prediction model that balances accuracy and comfort.

[0075] In this embodiment of the invention, a composite training objective comprising a penalty loss term and a prediction loss term is constructed for model training. This breaks through the limitation of traditional control focusing only on physical parameters, directly mapping comfort scores to the model's optimization objective, making the trained model more in line with the subjective comfort needs of the human body. Furthermore, through a negative correlation penalty mechanism during training, the model can quickly correct erroneous strategies, such as mistakenly increasing the wind speed during inhalation and causing choking, and automatically avoid similar behaviors in subsequent predictions, achieving human-like intelligent evolution at the algorithmic level.

[0076] Based on the above embodiments, the air conditioning control commands include windshield control commands and air supply control commands; step 130 includes: When the breathing phase corresponds to the inhalation phase, a wind avoidance control command is generated; the wind avoidance control command is used to control the angle between the driver's nose tip position and the air outlet of the vehicle air conditioner to deflect downward by a preset avoidance angle, so as to adjust the air outlet direction and control the air outlet speed of the vehicle air conditioner to be reduced by a first preset ratio. When the breathing phase corresponds to the exhalation phase, an air supply control command is generated; the air supply control command is used to control the angle between the driver's nose tip position and the air outlet of the vehicle air conditioner so that the air outlet direction is aimed at the driver's face area, and to control the air outlet speed of the vehicle air conditioner to increase by a second preset ratio, and to control the air outlet temperature to decrease by a preset temperature.

[0077] Specifically, based on the predicted breathing phase, the air conditioning control commands can include two different types: draft avoidance control commands and air supply control commands. The process of generating these two commands is as follows: Scenario 1: When the breathing phase corresponds to the inhalation phase, generate a wind avoidance control command.

[0078] That is, when it is predicted that the driver will be in the inhalation phase in the future, in order to prevent the cold air from the air conditioner from being directly inhaled into the nasal cavity, causing the nasal mucosa to dry out, become irritated, or cause coughing, in this embodiment of the invention, a wind avoidance control command can be generated to adjust the air outlet direction and air outlet speed accordingly.

[0079] Here, regarding airflow direction control, the command instructs the vehicle's air conditioning vents to deflect. Specifically, using the calculated angle between the driver's nose and the vent as a reference vector, it controls the vent's guide vanes, particularly the vertical vanes, to deflect downwards at a preset avoidance angle, such as 15°. This downward deflection is chosen to direct airflow towards the driver's chest or abdomen, avoiding the sensitive areas of the mouth and nose without completely cutting off the cooling supply. Regarding airflow speed control, to further reduce the impact of the airflow, the command simultaneously reduces the vehicle's air conditioning speed by a first preset percentage, such as reducing the current blower voltage or duty cycle by 30%.

[0080] This downward pressure and wind speed softening control strategy can create a relatively stable and gentle breathing environment during the driver's critical breathing periods in the future.

[0081] Scenario 2: When the breathing phase corresponds to the exhalation phase, generate air supply control commands.

[0082] Conversely, when it is predicted that the driver will be in the exhalation phase in the near future, the body exhales hot air, heat accumulates on the face, and the airflow in the nasal cavity is outward, making it less susceptible to cold wind stimulation. This can generate air supply control commands to adjust the airflow direction, airflow speed, and airflow temperature accordingly.

[0083] Here, regarding airflow direction control, this command instructs the air vents to return to center and lock onto the target. That is, it drives the air guide vanes to rotate, adjusting the angle between the driver's nose and the vehicle's air conditioning vents, ensuring the airflow is directed towards the driver's face, especially the nose, thus guaranteeing precise coverage of the hot facial area with cool air. Regarding airflow speed control, this command increases the vehicle's air conditioning speed by a second preset percentage, such as increasing it by 20% from the current or base speed. This instantaneous increase in airflow helps to quickly dissipate facial heat generated during exhalation. For airflow temperature control, to enhance the cooling sensation, this command can also lower the preset temperature, such as temporarily lowering it by 1°C, or adjusting the mixing damper position to increase the proportion of cool air. Through this precise direct blowing, increased airflow speed, and instantaneous cooling control strategy, maximum comfort and heat dissipation efficiency can be achieved during the exhalation phase.

[0084] In this embodiment of the invention, by using two distinct control modes, namely wind avoidance and air supply, the airflow of the vehicle's air conditioning system is precisely arranged. During inhalation, the airflow direction is automatically reduced and the speed is decreased, eliminating the discomfort of cold air entering the nose and protecting the driver's respiratory health. During exhalation, the airflow is precisely directed at the face and the cooling speed is accelerated. By utilizing the period of insensitivity of human breathing, concentrated heat dissipation is achieved, which not only ensures a cool experience but also avoids the risk of facial paralysis or headache caused by continuous direct airflow. This truly achieves intelligent adjustment synchronized with breathing.

[0085] Based on the above embodiments, in step 130, an air conditioning control command is generated based on the breathing phase, and then the following steps are also included: The air conditioning control commands are synchronously sent to the vehicle's infotainment system via the vehicle communication protocol, so that the infotainment system can control the vehicle's interactive interface to display the air conditioning airflow direction mapping diagram in real time based on the air conditioning control commands. The vehicle infotainment system is also used to control the interactive interface to display the driver's breathing timing signals in real time. The breathing timing signals are determined based on capacitance change signals and thermal imaging images.

[0086] Specifically, to allow drivers to intuitively understand the current operating status of the vehicle's air conditioning, this embodiment of the invention constructs a real-time visualization display system. That is, after generating specific air conditioning control commands, this embodiment of the invention also utilizes vehicle communication protocols, such as CAN FD (Controller Area Network Flexible Data-Rate) and FlexRay, high-speed vehicle bus protocols, to synchronously send the commands to the vehicle's infotainment system. The infotainment system, as the core carrier for displaying vehicle cabin information, possesses powerful graphics rendering capabilities and a large display screen.

[0087] After receiving a command, the vehicle's infotainment system invokes its internal human-machine interface rendering engine. Based on the airflow angle, speed, and temperature specified in the command, it creates and updates a dynamic airflow mapping on the vehicle's interface, such as the central control screen or the fully digital instrument cluster. This mapping is an animation that visually reflects the airflow trajectory. For example, when the air conditioning control command is a draft avoidance command, the airflow lines on the interface will curve downwards, avoiding the icon representing the driver's head; while when the command is a direct airflow control command, the airflow lines will appear to be directed straight towards the driver's face. This real-time mapping allows the driver to clearly see the air conditioning responding to their breathing, greatly enhancing their perception of the intelligent system.

[0088] It's worth noting that, in addition to displaying the air conditioning vent direction map, the vehicle's infotainment system can also control the interface to display the driver's breathing waveform in real time. Specifically, the system receives a fused breathing timing signal (the determination of this signal has been explained in detail above and will not be repeated here), and can render this signal as a dynamic waveform or breathing ball animation in a specific area of ​​the interface, such as a corner of the screen or a dedicated health monitoring card page.

[0089] For example, the waveform on the screen fluctuates in sync with the driver's actual breathing, or a light sphere expands (inhalation) and contracts (exhalation) in sync. This not only allows the driver to confirm the accuracy of the detection but also provides a certain degree of biofeedback, helping the driver adjust their breathing rhythm and relieve driving stress.

[0090] In this embodiment of the invention, the abstract control logic is transformed into an intuitive air conditioning airflow direction mapping diagram, eliminating the driver's sense of unknown and distrust of the system. The real-time display of breathing timing signals not only increases driving pleasure but also provides the driver with visual guidance on healthy breathing, thereby greatly optimizing the user's driving experience.

[0091] Based on the above embodiments, Figure 2 This is a general flowchart of the vehicle air conditioning control method provided by the present invention, as follows: Figure 2As shown, after the air conditioning control command is sent to the actuator, such as the damper stepper motor or blower, and converted into actual physical airflow adjustment, this embodiment of the invention continuously monitors the driver's actual experience. If the driver feels discomfort, for example, if the predicted breathing phase corresponds to the exhalation stage and the air supply control command is executed, but the driver is actually in the inhalation stage in the predicted future time period, causing choking, the driver can provide feedback through the interactive interface, such as clicking the "Discomfort" button or entering a lower comfort rating.

[0092] Upon receiving this feedback, the vehicle control system will treat it as an error correction signal and, in conjunction with the current capacitance change signal and thermal imaging image, trigger a calibration mechanism. This mechanism may involve using the driver's feedback to construct or update the penalty loss term and fine-tune the parameters of the breathing prediction model.

[0093] In this embodiment of the invention, through a closed-loop calibration mechanism, the system can gradually become familiar with the driver's unique breathing patterns and sensory preferences. As the number of uses increases, the control strategy will become more and more precise and personalized.

[0094] The vehicle air conditioning control device provided by the present invention is described below. The vehicle air conditioning control device described below can be referred to in correspondence with the vehicle air conditioning control method described above.

[0095] Figure 3 This is a structural schematic diagram of the vehicle air conditioning control device provided by the present invention, as shown below. Figure 3 As shown, the device includes: The data acquisition unit 310 is used to acquire capacitance change signals reflecting the undulation changes of the driver's chest and abdomen region on the vehicle, as well as thermal imaging images of the driver's facial region. Phase prediction unit 320 is used to predict the breathing phase based on the capacitance change signal and the thermal imaging image to obtain the driver's breathing phase; The instruction generation unit 330 is used to generate an air conditioning control instruction based on the breathing phase. The air conditioning control instruction is used to adjust at least one of the air outlet direction, air outlet speed, and air outlet temperature of the vehicle air conditioner.

[0096] The vehicle air conditioning control device provided by this invention integrates capacitance change signals reflecting mechanical motion and thermal imaging images reflecting thermodynamic changes to predict the breathing phase, achieving accurate capture of the driver's breathing phase. Furthermore, based on the generation of control commands and vehicle air conditioning control according to the breathing phase, it can automatically avoid direct airflow or reduce the airflow speed at the critical moment when the driver inhales, achieving imperceptible synchronization between the air conditioning airflow and the human breathing rhythm. This effectively solves the discomfort caused by direct airflow in traditional vehicle air conditioning systems, greatly improving the intelligence level of the driver's cabin and the comfort of driving without requiring the driver to wear any wearable devices.

[0097] Based on the above embodiments, the phase prediction unit 320 is used for: Based on the thermal imaging image, the nose is located to obtain the driver's two-dimensional nose coordinates; The two-dimensional nose coordinates are transformed into spatial coordinates to obtain the three-dimensional nose tip coordinates relative to the air outlet of the vehicle air conditioner. The driver's breathing timing signal is obtained by fusing the breathing signal based on the capacitance change signal and the thermal imaging image. Based on the respiratory timing signal and the driver's average respiratory cycle, respiratory phase prediction is performed to obtain the respiratory phase; Instruction generation unit 330 is used for: The air conditioning control command is generated based on the breathing phase and the three-dimensional nasal tip coordinates.

[0098] Based on the above embodiments, the phase prediction unit 320 is used for: Based on the two-dimensional nasal coordinates, the temperature fluctuation characteristics of the driver's nasal region over time are extracted from the thermal imaging image to obtain the thermal imaging respiratory component. Frequency band filtering is performed on the capacitance change signal to obtain the capacitance breathing component; Based on the cross-correlation function between the capacitive breathing component and the thermal imaging breathing component, the relevant capacitive breathing component and the relevant thermal imaging breathing component are determined from the capacitive breathing component and the thermal imaging breathing component, respectively. The relevant capacitive breathing component and the relevant thermal imaging breathing component are fused to obtain the driver's breathing timing signal.

[0099] Based on the above embodiments, the thermal imaging images are acquired using a thermal imaging acquisition device installed inside the vehicle; Phase prediction unit 320 is used for: Based on the relative positional relationship between the thermal imaging acquisition device and the air outlet of the vehicle air conditioner, the spatial coordinate transformation matrix is ​​determined; Based on the spatial coordinate transformation matrix, the two-dimensional nose tip coordinates in the two-dimensional nose coordinates are transformed to obtain the three-dimensional nose tip coordinates relative to the air outlet of the vehicle air conditioner.

[0100] Based on the above embodiments, the phase prediction unit 320 is used for: The respiratory timing signal, the average respiratory cycle, and the three-dimensional nasal tip coordinates are input into the respiratory prediction model to obtain the air conditioning control parameters output by the respiratory prediction model. The breathing prediction model is used to determine the angle between the driver's nose tip position and the air outlet based on the three-dimensional nose tip coordinates and the three-dimensional air outlet coordinates, to predict the breathing phase based on the breathing timing signal and the average breathing cycle, to obtain the breathing phase, and to generate air conditioning control parameters based on the breathing phase and the angle. The breathing prediction model is trained based on the sample capacitance change signal and sample thermal imaging image of the sample driver, as well as the corresponding sample air conditioning control parameters. Instruction generation unit 330 is used for: Based on the air conditioning control parameters, the air conditioning control command is generated.

[0101] Based on the above embodiments, the device further includes a model training unit, used for: The sample capacitance change signal and the sample thermal imaging image are input into the initial prediction model to obtain the predicted air conditioning control parameters output by the initial prediction model; Based on the sample air conditioning control parameters and the predicted air conditioning control parameters, the predicted loss term is determined; Based on the comfort ratings provided by the sample drivers in response to the predicted air conditioning control parameters, a penalty loss term is constructed, which is negatively correlated with the comfort rating. The initial prediction model is trained based on the prediction loss term and the penalty loss term to obtain the breathing prediction model.

[0102] Based on the above embodiments, the air conditioning control commands include wind avoidance control commands and air supply control commands; Instruction generation unit 330 is used for: When the breathing phase corresponds to the inhalation phase, a wind avoidance control command is generated; the wind avoidance control command is used to control the angle between the driver's nose tip position and the air outlet of the vehicle air conditioner to deflect downward by a preset avoidance angle, so as to adjust the air outlet direction and control the air outlet speed of the vehicle air conditioner to be reduced by a first preset ratio. When the breathing phase corresponds to the exhalation phase, an air supply control command is generated; the air supply control command is used to control the angle between the driver's nose tip position and the air outlet of the vehicle air conditioner so that the air outlet direction is aligned with the driver's facial area, and to control the air outlet speed of the vehicle air conditioner to increase by a second preset ratio, and to control the air outlet temperature to decrease by a preset temperature.

[0103] Based on the above embodiments, the device further includes an instruction synchronization unit, used for: The air conditioning control command is synchronously sent to the vehicle's infotainment system via the vehicle communication protocol, so that the infotainment system controls the vehicle's interactive interface to display the air conditioning airflow direction mapping diagram in real time based on the air conditioning control command. The vehicle system is also used to control the interactive interface to display the driver's breathing timing signal in real time, the breathing timing signal being determined based on the capacitance change signal and the thermal imaging image.

[0104] The present invention also provides a vehicle air conditioning control system. Figure 4 This is a structural schematic diagram of the vehicle air conditioning control system provided by the present invention, as shown below. Figure 4 As shown, the system includes a capacitive sensor array 410, a thermal imaging acquisition device 420, and a processor 430. The capacitive sensor array 410 is disposed on the steering wheel of the vehicle, and the thermal imaging acquisition device 420 is disposed on the top of the dashboard or the center console area of ​​the vehicle. The capacitance sensor array 410 is used to sense the capacitance change signal caused by the undulation of the driver's chest and abdomen area in the vehicle; the thermal imaging acquisition device 420 is used to acquire thermal imaging images containing the driver's facial area. The processor 430 is used to acquire the capacitance change signal and the thermal imaging image, and to perform respiratory phase prediction based on the capacitance change signal and the thermal imaging image to obtain the driver's respiratory phase; based on the respiratory phase, it generates an air conditioning control command, which is used to adjust at least one of the following: air outlet direction, air outlet speed, and air outlet temperature of the vehicle air conditioner.

[0105] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute an in-vehicle air conditioning control method. This method includes: acquiring a capacitance change signal reflecting the fluctuations in the driver's chest and abdomen region, and a thermal imaging image of the driver's facial region; performing respiratory phase prediction based on the capacitance change signal and the thermal imaging image to obtain the driver's respiratory phase; and generating an air conditioning control command based on the respiratory phase, the air conditioning control command being used to adjust at least one of the following: the air outlet direction, air outlet speed, and air outlet temperature of the in-vehicle air conditioning system.

[0106] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the vehicle air conditioning control method provided by the above methods, the method comprising: acquiring a capacitance change signal reflecting the fluctuation changes of the driver's chest and abdomen region in the vehicle, and a thermal imaging image of the driver's facial region; performing respiratory phase prediction based on the capacitance change signal and the thermal imaging image to obtain the driver's respiratory phase; and generating an air conditioning control instruction based on the respiratory phase, the air conditioning control instruction being used to adjust at least one of the following: the air outlet direction, the air outlet speed, and the air outlet temperature of the vehicle air conditioning.

[0108] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the vehicle air conditioning control method provided by the above methods. The method includes: acquiring a capacitance change signal reflecting the fluctuation of the driver's chest and abdomen region in the vehicle, and a thermal imaging image of the driver's facial region; performing respiratory phase prediction based on the capacitance change signal and the thermal imaging image to obtain the driver's respiratory phase; and generating an air conditioning control command based on the respiratory phase, the air conditioning control command being used to adjust at least one of the vehicle air conditioning's air outlet direction, air outlet speed, and air outlet temperature.

[0109] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle air-conditioning control method characterized by comprising: The method comprises: acquiring a capacitance change signal reflecting chest and abdomen area fluctuation of a driver on a vehicle, and a thermal imaging image of a face area of the driver; performing breathing phase prediction based on the capacitance change signal and the thermal imaging image to obtain a breathing phase of the driver; generating an air conditioner control instruction based on the breathing phase, the air conditioner control instruction being used to adjust at least one of an air outlet direction, an air outlet speed, and an air outlet temperature of a vehicle-mounted air conditioner.

2. The vehicle-mounted air conditioning control method according to claim 1, characterized by The method further comprises: performing breathing signal fusion based on the capacitance change signal and the thermal imaging image to obtain a breathing time sequence signal of the driver. The method further comprises: performing nose positioning based on the thermal imaging image to obtain two-dimensional nose coordinates of the driver; performing spatial coordinate transformation on the two-dimensional nose coordinates to obtain three-dimensional nose tip coordinates relative to an air outlet of the vehicle-mounted air conditioner. The method further comprises: performing breathing phase prediction based on the breathing time sequence signal and an average breathing period of the driver to obtain the breathing phase.

3. The vehicle-mounted air conditioning control method according to claim 2, characterized by, The method further comprises: generating the air conditioner control instruction based on the breathing phase and the three-dimensional nose tip coordinates. The method further comprises: extracting, based on the two-dimensional nose coordinates, a feature of temperature fluctuation over time of a nose area of the driver from the thermal imaging image to obtain a thermal imaging breathing component; performing frequency band filtering on the capacitance change signal to obtain a capacitance breathing component; 4. The vehicle-mounted air conditioning control method according to claim 2, characterized by determining, based on a cross-correlation function between the capacitance breathing component and the thermal imaging breathing component, a relevant capacitance breathing component and a relevant thermal imaging breathing component from the capacitance breathing component and the thermal imaging breathing component, respectively; performing fusion on the relevant capacitance breathing component and the relevant thermal imaging breathing component to obtain the breathing time sequence signal of the driver. The thermal imaging image is acquired based on a thermal imaging acquisition device arranged in the vehicle.

5. The vehicle-mounted air conditioning control method according to any one of claims 2 to 4, characterized by, The method further comprises: determining a spatial coordinate transformation matrix based on a relative position relationship between the thermal imaging acquisition device and the air outlet of the vehicle-mounted air conditioner; performing spatial coordinate transformation on two-dimensional nose tip coordinates of a nose tip position in the two-dimensional nose coordinates based on the spatial coordinate transformation matrix to obtain the three-dimensional nose tip coordinates relative to the air outlet of the vehicle-mounted air conditioner. The method further comprises: inputting the breathing time sequence signal, the average breathing period, and the three-dimensional nose tip coordinates into a breathing prediction model to obtain an air conditioner control parameter output by the breathing prediction model; generating the air conditioner control instruction based on the air conditioner control parameter. The breathing prediction model is used to determine an included angle between the nose tip position of the driver and the air outlet based on the three-dimensional nose tip coordinates and three-dimensional air outlet coordinates of the air outlet, to perform breathing phase prediction based on the breathing time sequence signal and the average breathing period to obtain the breathing phase, and to generate air conditioner control parameters based on the breathing phase and the included angle. The breathing prediction model is trained based on sample capacitance change signals and sample thermal imaging images of sample drivers, and corresponding sample air conditioner control parameters.

6. The vehicle-mounted air conditioning control method according to claim 5, characterized by The breathing prediction model is trained based on the following steps: The sample capacitance change signals and the sample thermal imaging images are input into an initial prediction model to obtain predicted air conditioner control parameters output by the initial prediction model; A prediction loss term is determined based on the sample air conditioner control parameters and the predicted air conditioner control parameters; A penalty loss term is constructed based on a comfort score fed back by the sample driver with respect to the predicted air conditioner control parameters, and the penalty loss term is negatively correlated with the comfort score; The initial prediction model is trained based on the prediction loss term and the penalty loss term to obtain the breathing prediction model.

7. The in-vehicle air conditioning control method according to any one of claims 1 to 4, characterized by, The air conditioner control instructions include wind-avoiding control instructions and air-sending control instructions; and the air conditioner control instructions are generated based on the breathing phase, including: In a case where the breathing phase corresponds to an inhaling stage, wind-avoiding control instructions are generated; the wind-avoiding control instructions are used to control the included angle between the nose tip position of the driver and the air outlet of the vehicle-mounted air conditioner to deflect downward by a preset avoidance angle, to adjust the air outlet direction, and to control the air outlet speed of the vehicle-mounted air conditioner to decrease by a first preset proportion; In a case where the breathing phase corresponds to an exhaling stage, air-sending control instructions are generated; the air-sending control instructions are used to control the included angle between the nose tip position of the driver and the air outlet of the vehicle-mounted air conditioner to make the air outlet direction align with the face region of the driver, and to control the air outlet speed of the vehicle-mounted air conditioner to increase by a second preset proportion, and to control the air outlet temperature to decrease by a preset temperature.

8. The vehicle-mounted air conditioning control method according to any one of claims 1 to 4, characterized by, The air conditioner control instructions are generated based on the breathing phase, and then further include: The air conditioner control instructions are synchronously sent to a vehicle infotainment system of the vehicle through a vehicle communication protocol, so that the vehicle infotainment system controls an interactive interface of the vehicle to display an air conditioner air direction mapping diagram of the vehicle-mounted air conditioner in real time based on the air conditioner control instructions; The vehicle infotainment system is further used to control the interactive interface to display a breathing time sequence signal of the driver in real time, the breathing time sequence signal being determined based on the capacitance change signal and the thermal imaging image.

9. A vehicle air-conditioning control device characterized by comprising: The method includes: A data acquisition unit is configured to acquire a capacitance change signal reflecting fluctuation changes of a chest and abdomen region of a driver in a vehicle, and a thermal imaging image of a face region of the driver; A phase prediction unit is configured to perform breathing phase prediction based on the capacitance change signal and the thermal imaging image to obtain a breathing phase of the driver; An instruction generation unit is configured to generate air conditioner control instructions based on the breathing phase, the air conditioner control instructions being used to adjust at least one of an air outlet direction, an air outlet speed, and an air outlet temperature of a vehicle-mounted air conditioner.

10. A vehicle air conditioning control system characterized by comprising: The method comprises: a capacitive sensor array, a thermal imaging acquisition device and a processor; The capacitive sensor array is arranged on a steering wheel of a vehicle, and the thermal imaging acquisition device is arranged on a top of an instrument panel or a center console area of the vehicle, The capacitive sensor array is used to sense a capacitive change signal caused by fluctuation of a chest and abdomen area of a driver on the vehicle; the thermal imaging acquisition device is used to acquire a thermal imaging image containing a face area of the driver; The processor is used to acquire the capacitive change signal and the thermal imaging image, and perform breathing phase prediction based on the capacitive change signal and the thermal imaging image to obtain a breathing phase of the driver; Based on the breathing phase, an air conditioner control instruction is generated, and the air conditioner control instruction is used to adjust at least one of an air outlet direction, an air outlet speed and an air outlet temperature of a vehicle-mounted air conditioner.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the vehicle-mounted air conditioner control method according to any one of claims 1 to 8.

12. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the vehicle-mounted air conditioner control method according to any one of claims 1 to 8.

Images