A cockpit thermal comfort monitoring and regulation method and system

Abstract

The application relates to the field of automobile cabin thermal management and artificial intelligence technology, in particular to a cabin thermal comfort monitoring and adjusting method and system. The method comprises the following steps: acquiring facial thermal imaging images of a vehicle occupant at multiple continuous time points; performing local feature extraction on each frame of facial thermal imaging image to obtain a frame-level feature vector of a preset dimension; adding frame-level time position coding to the frame-level feature vector; performing time sequence dependence modeling on the frame-level feature vector added with the frame-level time position coding to obtain a sequence fusion feature representation; determining a thermal comfort state and / or a thermal comfort change trend of the vehicle occupant according to the sequence fusion feature representation; and constructing an optimization problem with the thermal comfort degree, air conditioner energy consumption and response time as targets according to the thermal comfort state and / or the thermal comfort change trend, combining the current running state of the vehicle and the state of the air conditioning system, and solving to obtain an air conditioner control sequence to adjust the cabin air conditioner.

Description

Technical Field

[0001] This application relates to the fields of automotive cabin thermal management and artificial intelligence technology, and more specifically, to a cabin thermal comfort monitoring and adjustment method and system. Background Technology

[0002] With the increasing intelligence of automobiles, cabin thermal management centered on the thermal comfort experience of drivers and passengers has become a research hotspot. Traditional automotive heating, ventilation, and air conditioning (HVAC) systems usually require manual adjustment by the driver, which can distract the driver and increase driving risks. Although some automatic air conditioning systems can automatically adjust according to the set temperature, their control logic often relies mainly on environmental parameters such as the in-vehicle air temperature, resulting in insufficient attention to the actual thermal comfort status of individual occupants.

[0003] Research on thermal comfort assessment can be traced back to the Predictive Average Voting (PMV) model and its improvements. However, the PMV model, based on population statistics, aims to predict the average thermal sensation of most people, but it is difficult to accurately reflect individual differences caused by factors such as gender, metabolic rate, and clothing thermal resistance. In addition, parameters such as air velocity and mean radiant temperature required by the PMV model are difficult to measure accurately in real time in the dynamically changing automotive cabin environment, further limiting its application effectiveness.

[0004] To obtain more accurate individual thermal comfort data, researchers have explored various methods for monitoring physiological signals. Invasive measurements (such as attaching temperature sensors or electrodes to the skin) provide accurate data, but the devices are cumbersome to wear and cause significant discomfort to occupants, failing to meet the needs of real-world driving scenarios. Wearable devices (such as smartwatches) reduce invasiveness, but still rely on user wearing habits, making large-scale deployment difficult for all occupants.

[0005] In recent years, non-invasive infrared thermal imaging technology has attracted attention due to its ability to directly reflect the temperature distribution of the human body surface. Existing thermal imaging-based methods typically focus on extracting the absolute temperature values ​​of facial skin from thermal images and establishing a correlation between temperature and thermal comfort. However, this method requires extremely high temperature measurement accuracy, usually necessitating expensive high-precision infrared thermal imagers and complex blackbody calibration to eliminate environmental interference. On the other hand, existing deep learning-based thermal comfort recognition schemes mostly use single-frame facial thermal imaging images as input, primarily classifying based on static heat distribution at a single moment, which is insufficient to fully characterize the dynamic process of occupant thermal comfort changing over time. When there are changes in head posture, wearing glasses, partial occlusion, or transient thermal noise interference, the single-frame recognition results are prone to fluctuations, thus affecting the stability of subsequent air conditioning adjustments. The high hardware cost and complex calibration process hinder the widespread application of this technology in mass-produced vehicles.

[0006] In summary, there is an urgent need for a non-invasive monitoring method that does not rely on high-precision temperature measurement hardware, can combine continuous thermal imaging time-series information to accurately predict individual thermal comfort, and can further transform the thermal comfort state into a closed-loop mechanism for air conditioning to execute control commands, in order to solve the above-mentioned technical problems. Summary of the Invention

[0007] In view of this, this application provides a cabin thermal comfort monitoring and adjustment method and system to solve the technical problem that existing thermal comfort recognition methods based on single-frame thermal imaging cannot fully utilize the information on the evolution of heat distribution over time, and are prone to prediction fluctuations under changes in head posture, local occlusion and instantaneous thermal noise interference.

[0008] The technical solution provided in this application is as follows: In a first aspect, this application provides a method for monitoring and adjusting cabin thermal comfort, including: Acquire facial thermal images of vehicle occupants at multiple consecutive moments; Local features are extracted from each frame of facial thermal imaging image to obtain frame-level feature vectors of a preset dimension; Add frame-level temporal location encoding to the frame-level feature vector; the frame-level temporal location encoding is determined by the temporal order of acquiring the corresponding facial thermal imaging images. Temporal dependency modeling is performed on the frame-level feature vector with the added frame-level temporal position encoding to obtain the sequence fusion feature representation; The thermal comfort status and / or thermal comfort change trend of the occupants in the vehicle are determined based on the sequence fusion feature representation. Based on the thermal comfort state and / or the thermal comfort change trend, combined with the current vehicle operating state and the air conditioning system state, an optimization problem with thermal comfort, air conditioning energy consumption and response time as objectives is constructed, and the air conditioning control sequence is obtained by solving it to adjust the cabin air conditioning.

[0009] One possible implementation of acquiring facial thermal images of vehicle occupants at multiple consecutive moments further includes: Multiple frames of facial thermal imaging images are continuously acquired within a preset time window corresponding to multiple preset labeled times, and then combined according to the acquisition time order. Obtain the subjective thermal sensation vote TSV corresponding to each of the preset labeled times, and map the subjective thermal sensation vote TSV to a plurality of preset thermal sensation tags; the thermal sensation tags are used to characterize the thermal comfort state of the occupants in the vehicle corresponding to the preset labeled times; Multiple frames of facial thermal imaging images combined within the same preset time window are configured with the same thermal sensing label.

[0010] In one possible implementation, after acquiring facial thermal imaging images of the vehicle occupants at multiple consecutive moments, the method further includes: Facial regions are cropped from each frame of facial thermal imaging images; Scale the cropped facial thermal image to a preset size; The scaled facial thermal imaging image is subjected to pixel value normalization and format conversion; wherein, the format conversion is used to convert the facial thermal imaging image into an image format that meets preset requirements.

[0011] In one possible implementation, local features are extracted from each frame of the facial thermal imaging image to obtain a frame-level feature vector of a preset dimension, including: A residual network is used to extract local features from each frame of facial thermal imaging images to obtain high-dimensional feature vectors. The high-dimensional feature vector is mapped to a frame-level feature vector of the preset dimension through linear mapping.

[0012] In one possible implementation, temporal dependency modeling is performed on the frame-level feature vector with the added frame-level temporal position encoding to obtain a sequence fusion feature representation, including: The frame-level feature vector with the added frame-level temporal position encoding is input into the Transformer encoder, and the correlation between different time frames is modeled through a multi-head self-attention mechanism; The sequence features output by the Transformer encoder are averaged along the time dimension to obtain the sequence fusion feature vector.

[0013] In one possible implementation, determining the occupant's current thermal comfort state and / or thermal comfort change trend based on the sequence fusion feature representation includes: The probability distribution of the occupant in the preset thermal comfort state category is calculated based on the sequence fusion feature representation, and the thermal comfort state corresponding to the highest probability in the probability distribution is taken as the thermal comfort state of the occupant in the vehicle. And / or, based on the sequence fusion feature representation, a prediction is made to obtain the trend of changes in the thermal comfort of the occupants within a preset time period in the future.

[0014] In one possible implementation, before solving the air conditioning control sequence based on the thermal comfort state and / or future thermal comfort change trends, the method further includes: The system determines whether there is a valid human face target in the facial thermal imaging image, whether the sampling interval between adjacent frames meets the preset requirements, and whether the confidence level of the thermal comfort state prediction result is higher than the preset threshold. If the judgment result does not meet any preset condition, maintain the air conditioning parameters of the previous control cycle, trigger re-acquisition, or delay the update of the current control quantity.

[0015] In one possible implementation, the air conditioning control sequence is obtained by solving for: Based on the thermal comfort state and / or thermal comfort change trend in the next T time steps, and combined with the vehicle operating status and air conditioning system status, a multi-objective optimization problem is constructed with thermal comfort index, air conditioning energy consumption and response time as objectives. Solving the multi-objective optimization problem yields multiple non-dominated air conditioning control sequences on the Pareto front; The air conditioning control sequence is selected from the plurality of non-dominated air conditioning control sequences based on user preset preferences or preset control strategies; wherein, the air conditioning control sequence includes one or more of compressor frequency, air volume and air direction; According to the air conditioning control sequence, the control quantity at the current moment is executed in a rolling optimization manner, and the solution is re-solved based on the updated thermal comfort state in the next control cycle.

[0016] One possible implementation involves combining the vehicle's current operating status and the air conditioning system status, including: The number of occupants and their seating positions are determined using infrared thermal imaging. When no valid occupant is detected in the target seating area, the corresponding area will be marked as an unoccupied area; For unoccupied areas, zoned air conditioning control driven by thermal comfort state prediction is not executed, and / or the air supply volume of the corresponding area is reduced, the cooling and heating output of the area is turned off, or the energy-saving control parameters of the corresponding area in the previous control cycle are maintained.

[0017] Secondly, this application provides a cabin thermal comfort monitoring and adjustment system, including a sequence image acquisition module, a thermal comfort prediction model, a vehicle state acquisition module, and a multi-objective optimization solution module; The sequential image acquisition module is used to acquire facial thermal imaging images of occupants at multiple consecutive moments. The thermal comfort prediction module includes a pre-trained thermal comfort monitoring model, used to extract local features from each frame of facial thermal imaging images to obtain a frame-level feature vector of a preset dimension; add frame-level temporal location encoding to the frame-level feature vector; the frame-level temporal location encoding is determined by the temporal order of acquiring the corresponding facial thermal imaging images; perform temporal dependency modeling on the frame-level feature vector with the added frame-level temporal location encoding to obtain a sequence fusion feature representation; and determine the thermal comfort state and / or thermal comfort change trend of the occupants in the vehicle based on the sequence fusion feature representation. The vehicle status acquisition module is used to acquire the current operating status of the vehicle and the status of the air conditioning system; The multi-objective optimization solution module is used to generate an air conditioning control sequence based on the thermal comfort state and / or future thermal comfort change trend, combined with the current vehicle operating state and air conditioning system state, using a model predictive control strategy.

[0018] Compared with the prior art, the technical solution provided in this application has the following beneficial effects: This application predicts thermal comfort status based on a continuous sequence of facial thermal imaging frames, rather than relying solely on the static thermal distribution features of a single frame. Furthermore, it eliminates the need to extract the absolute temperature value of the facial skin, thus reducing reliance on the accuracy and calibration process of the infrared thermal imager and facilitating low-cost deployment in vehicles. By setting frame-level temporal location coding and utilizing a Transformer to model the temporal dependencies between features from multiple frames of thermal imaging, the model can distinguish thermal imaging features at different acquisition times, more fully representing the dynamic changes in occupant thermal comfort status, thereby improving the stability and accuracy of the prediction results.

[0019] After obtaining the current thermal comfort status of the occupants and / or the future thermal comfort change trend, this application further combines the vehicle operating status, air conditioning system status, number of occupants and seat positions, and uses multi-objective optimization and rolling optimization methods to generate air conditioning optimization control commands for the corresponding area, thereby realizing automatic closed-loop control from thermal comfort status perception, trend prediction to temperature control execution, which helps to balance thermal comfort, response efficiency and air conditioning energy consumption, and reduce ineffective cooling or heating in unoccupied areas.

[0020] Furthermore, by determining the validity, temporal continuity, and prediction confidence of the thermal imaging frame sequence, this application can reduce frequent misadjustments caused by invalid inputs or instantaneous fluctuations, thereby improving control reliability. Attached Figure Description

[0021] Figure 1 This is a flowchart of a cabin thermal comfort monitoring and adjustment method provided in Embodiment 1 of this application.

[0022] Figure 2 This is a flowchart of a method for acquiring facial thermal imaging images of vehicle occupants at multiple consecutive moments, as provided in Embodiment 1 of this application.

[0023] Figure 3 This is a structural schematic diagram of a cabin thermal comfort monitoring and adjustment system provided in Embodiment 2 of this application.

[0024] Figure 4 This is a schematic diagram of the process and prediction model of the cabin thermal comfort monitoring method provided in Embodiment 2 of this application.

[0025] Figure 5This is a flowchart of a cabin air conditioning adjustment method based on thermal comfort prediction results provided in Embodiment 3 of this application. Detailed Implementation

[0026] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0027] Example 1 See Figure 1 This is a flowchart of a cabin thermal comfort monitoring and adjustment method provided in Embodiment 1 of this application. Figure 1 As shown, the specific implementation steps of the above method include: Step 101: Acquire facial thermal images of the occupants at multiple consecutive moments.

[0028] In this embodiment, a facial thermal imaging frame sequence is formed from facial thermal imaging images at multiple consecutive moments. Each frame in the facial thermal imaging frame sequence is equipped with a thermal sensory tag characterizing the thermal comfort state of the vehicle occupants at that moment.

[0029] Specifically, such as Figure 2 As shown, the specific implementation steps of the above method include: Step 1011: Within the preset time windows corresponding to multiple preset labeled times, continuously acquire multiple frames of facial thermal imaging images and combine them according to the acquisition time order.

[0030] Step 1012: Obtain the subjective thermal sensation vote (TSV) corresponding to each of the preset annotation times.

[0031] Step 1013: Map the above subjective thermal sensation voting TSV to multiple preset thermal sensation labels to characterize the thermal comfort state of the occupants in the vehicle at the preset labeled time.

[0032] Step 1014: Configure multiple frames of facial thermal imaging images combined within the same preset time window with the same thermal sensing label.

[0033] In some embodiments, after obtaining the aforementioned facial thermal imaging frame sequence, preprocessing is performed on each frame of the facial thermal imaging image in the sequence. The preprocessing method includes further cropping of the facial region of the image, scaling it to a preset size (e.g., 224×224), and pixel value normalization. Additionally, the preprocessed single-channel grayscale thermal imaging image can be converted into an image format that meets preset requirements.

[0034] Step 102: Extract local features from each frame of facial thermal imaging image to obtain a frame-level feature vector of a preset dimension.

[0035] In some embodiments, a residual network is used to extract local features from each frame of facial thermal imaging images to obtain a high-dimensional feature vector. Further, the high-dimensional feature vector is mapped to a frame-level feature vector of a preset dimension using a linear mapping.

[0036] Step 103: Add frame-level temporal location encoding to the above frame-level feature vector. The above frame-level temporal location encoding is determined by the temporal order of acquiring the corresponding facial thermal imaging images.

[0037] In this embodiment, frame-level temporal position encoding is added to the frame-level feature vector corresponding to each frame image according to the temporal order of each frame image in the facial thermal imaging frame sequence, so as to distinguish thermal imaging features at different time positions.

[0038] Step 104: Perform temporal dependency modeling on the frame-level feature vector with the above-mentioned frame-level temporal position coding to obtain the sequence fusion feature representation.

[0039] Specifically, a frame-level feature sequence with added frame-level temporal location encoding is input into a Transformer encoder. A multi-head self-attention mechanism is used to model the correlation between different time frames, thereby extracting the temporal evolution of facial thermal distribution as a sequence feature. The sequence features are then averaged along the temporal dimension to obtain the fused sequence feature representation.

[0040] Step 105: Determine the thermal comfort status and / or thermal comfort change trend of the occupants in the vehicle based on the above sequence fusion feature representation.

[0041] Specifically, the probability distribution of occupants in preset thermal comfort state categories is calculated based on the aforementioned sequence fusion feature representation. The thermal comfort state corresponding to the highest probability in the aforementioned probability distribution is taken as the thermal comfort state of the occupants in the vehicle. And / or, based on the aforementioned sequence fusion feature representation, a prediction is made to obtain the trend of thermal comfort changes of occupants in a preset future time period.

[0042] Step 106: Based on the above thermal comfort state and / or the above thermal comfort change trend, combined with the current vehicle operating state and air conditioning system state, construct an optimization problem with thermal comfort, air conditioning energy consumption and response time as objectives, solve it to obtain the air conditioning control sequence, and adjust the cabin air conditioning.

[0043] The aforementioned thermal comfort state and / or thermal comfort change trend are the predicted thermal comfort state and / or thermal comfort change trend for the next T time steps. The aforementioned current vehicle operating status includes, but is not limited to, vehicle speed and window status. The air conditioning system status includes, but is not limited to, air conditioning mode.

[0044] In this embodiment, based on the predicted thermal comfort state and / or thermal comfort change trend over the next T time steps, and combined with the current vehicle operating state and air conditioning system state, a multi-objective optimization problem is constructed with thermal comfort index, air conditioning energy consumption, and response time as objectives. This problem yields multiple non-dominated air conditioning control sequences on the Pareto front. A target control sequence is then selected from these multiple non-dominated control sequences based on user preset preferences or control strategies. Subsequently, the control quantity at the current moment is executed using a rolling optimization method, and the control sequence is re-solved based on the updated thermal comfort prediction results in the next control cycle. The aforementioned air conditioning control sequence includes at least one or more of the following: compressor frequency, air volume, and airflow direction.

[0045] Furthermore, in some implementations, the air conditioning zone control strategy can be adaptively adjusted based on the number of occupants and their seating positions detected by infrared thermal imaging, in order to avoid ineffective cooling or heating of unoccupied areas.

[0046] Specifically, based on infrared thermal imaging, the number of occupants and their seat positions are detected, and the occupant presence status information of the target seating area is output to provide input for the zoned air conditioning control. If no valid occupants are detected in the target seating area, the unoccupied area status information is output. Based on the above thermal comfort prediction results, vehicle status information, number and position of occupants, and unoccupied area status information, a corresponding target control strategy is generated. In the above target control strategy, when the target seating area is in an unoccupied area state, thermal comfort prediction-driven zoned air conditioning control is not executed in that area, and / or the air supply volume in that area is reduced, the cooling and heating output in that area is turned off, or the energy-saving control parameters of the previous control cycle in that area are maintained.

[0047] In other implementations, before generating air conditioning control commands based on thermal comfort prediction results, the method further includes determining the validity, temporal continuity, and prediction confidence of the acquired continuous facial thermal imaging frame sequence. Specifically, it determines whether there is a valid human face target in the facial thermal imaging frame sequence, whether the sampling interval between adjacent frames meets preset requirements, and whether the confidence of the thermal comfort state prediction result is higher than a preset threshold. When the determination results do not meet the above preset conditions, the air conditioning parameters of the previous control cycle are maintained, re-acquisition is triggered, or the update of the current control quantity is delayed, thereby reducing erroneous adjustments caused by invalid inputs, short-term occlusion, or transient thermal noise, and improving the stability and reliability of subsequent control decisions.

[0048] Compared with the prior art, the technical solution provided in Embodiment 1 of this application has the following beneficial effects: This application predicts thermal comfort status based on a continuous sequence of facial thermal imaging frames, rather than relying solely on the static thermal distribution features of a single frame. Since it eliminates the need to extract the absolute temperature value of facial skin, it reduces reliance on the accuracy and calibration process of the infrared thermal imager, making it easier to deploy in vehicles at low cost. By setting frame-level temporal location coding and using a Transformer to model the temporal dependencies between features from multiple frames of thermal imaging, the model can distinguish thermal imaging features at different acquisition times, more fully representing the dynamic changes in occupant thermal comfort status, thereby improving the stability and accuracy of the prediction results.

[0049] Furthermore, after obtaining the thermal comfort status of occupants and / or future thermal comfort change trends, this application can further combine the vehicle operating status, air conditioning system status, number of occupants and seat positions, and use model predictive control strategies to generate corresponding air conditioning optimization control commands for the area. These commands are then applied to the air conditioning execution end through rolling optimization, thereby achieving automatic closed-loop control from thermal comfort status perception and trend prediction to temperature control execution. This helps to balance thermal comfort, response efficiency and air conditioning energy consumption, and reduce ineffective cooling or heating in unoccupied areas.

[0050] Example 2 See Figure 3 This is a structural schematic diagram of a cabin thermal comfort monitoring and adjustment system provided in Embodiment 2 of this application. Figure 3 As shown, the system includes a sequence image acquisition module, a thermal comfort prediction model, a vehicle state acquisition module, and a multi-objective optimization solution module, used to implement the cabin thermal comfort monitoring and adjustment method provided in Embodiment 1. The specific functions of each module are described in detail below.

[0051] The aforementioned sequence image acquisition module is specifically used to continuously acquire facial thermal imaging images of occupants at multiple times, forming a sequence of facial thermal imaging frames for the corresponding seat area.

[0052] In some embodiments, the system further includes a preprocessing module, specifically used to perform facial region cropping, image scaling, pixel value normalization, and format conversion on each frame image in the frame sequence to form sequence data suitable for input to the thermal comfort prediction module.

[0053] The aforementioned thermal comfort prediction module includes a pre-trained thermal comfort monitoring model, specifically used to extract local features from each frame of facial thermal imaging images to obtain frame-level feature vectors of a preset dimension. Frame-level temporal location encoding is then added to these frame-level feature vectors. Temporal dependency modeling is performed on the frame-level feature vectors with the added temporal location encoding to obtain a sequence fusion feature representation. Based on this sequence fusion feature representation, the thermal comfort state and / or thermal comfort change trend of the vehicle occupants are determined.

[0054] Specifically, such as Figure 4 As shown, the pre-trained thermal comfort monitoring model includes a frame-level feature extraction submodule, a feature projection submodule, a frame-level temporal location encoding submodule, a Transformer temporal modeling submodule, and an output module. The output module further includes a classification output submodule and a trend prediction output submodule.

[0055] In one specific implementation, the aforementioned thermal comfort monitoring model takes a sequence of N consecutive facial thermal imaging images as input. First, an improved ResNet50 network is used to extract local features from each frame of the thermal imaging image. To adapt to grayscale thermal imaging input, the number of input channels in the first convolutional layer of the ResNet50 network is adjusted to 1. Each frame of the image outputs a high-dimensional feature vector after passing through the ResNet50 network. It is then projected into a 128-dimensional frame-level feature vector through a linear mapping layer. Its expression is:

[0056] In the formula, This represents the high-dimensional feature vector output by the single-frame feature extraction submodule of the t-th frame facial thermal imaging image. This represents the weight matrix in the aforementioned feature projection submodule. This represents the bias term in the feature projection submodule. Let represent the frame-level feature vector obtained after linear mapping of the t-th frame.

[0057] Subsequently, frame-level temporal position encoding is added to the frame-level feature vector to represent the temporal order of each frame in the sequence. This frame-level temporal position encoding is set after feature projection and before the Transformer encoder, and its expression is:

[0058] In the formula, This represents the frame-level feature vector of frame t. This represents the frame-level time position encoding vector corresponding to the time position of frame t. This represents the feature vector of the t-th frame sequence after adding frame-level temporal position coding.

[0059] Next, the sequence features are input into the Transformer temporal feature fusion submodule. In this embodiment, the Transformer temporal feature fusion submodule uses a two-layer Transformer encoder, with 8 attention heads in each layer and a hidden dimension of 128. It models the correlation between different time frames through a multi-head self-attention mechanism. Its core calculation expression is:

[0060] After the sequence features are output by the Transformer encoder, average pooling is performed along the time dimension to obtain the sequence fused feature vector. Its expression is:

[0061] Finally, the aforementioned sequence fusion feature vector is input into a three-layer fully connected classification output submodule, which outputs the probability distributions of three thermal comfort states: cold, comfortable, and hot. The expression for this distribution is:

[0062] like Figure 4 As shown, in some implementations, the thermal comfort change trend or prediction result for the next T time steps can also be output based on the aforementioned sequence fusion feature vector, as input for subsequent model predictive control. During training, the Adam optimizer can be used, with an initial learning rate set to 0.00005, a weight decay coefficient set to 0.01, and a batch size set to 16. The learning rate can be adjusted or training can be stopped based on changes in validation set performance.

[0063] After training, the sequence of continuous facial thermal imaging frames, which are collected and preprocessed in real time, is input into the trained model, and the model outputs the probability distribution of the current thermal comfort state of the occupant and / or the future thermal comfort change trend.

[0064] The aforementioned vehicle status acquisition module is specifically used to obtain the current operating status of the vehicle and the status of the air conditioning system.

[0065] Specifically, the vehicle CAN bus information is acquired through the aforementioned vehicle status acquisition module, including at least one of vehicle speed, air conditioning mode, and window status, to determine the current operating status of the vehicle and the status of the air conditioning system, and this information is used as the basis for control decisions to improve the adaptability of the control strategy to actual driving conditions.

[0066] The aforementioned multi-objective optimization solution module is specifically used to generate an air conditioning control sequence based on the aforementioned thermal comfort state and / or future thermal comfort change trends, combined with the current vehicle operating state and air conditioning system state, using a model predictive control strategy to adjust the cabin air conditioning.

[0067] In one specific implementation, the system further includes a control execution interface module, which is specifically used to generate corresponding air conditioning optimization control commands based on the target control strategy and send them to the vehicle controller or the corresponding regional air conditioning execution terminal.

[0068] In some embodiments, the system further includes an occupant detection and positioning module, specifically used to detect the number of occupants and their seat positions based on infrared thermal images, and output occupant presence status information for the target seating area to provide input for zoned air conditioning control. When the occupant detection and positioning module does not detect any valid occupants in the target seating area, it outputs unoccupied area status information.

[0069] Based on this, the aforementioned multi-objective optimization solution module is further used to construct a multi-objective optimization problem with thermal comfort index, air conditioning energy consumption, and response time as objectives, based on the aforementioned thermal comfort prediction results, vehicle status information, number and location of occupants, and the aforementioned unoccupied area status information, and to solve multiple non-dominated control solutions on the Pareto front. It also selects the target control strategy according to user preset preferences. When the target seating area is in an unoccupied area, thermal comfort prediction-driven zoned air conditioning control is not implemented in that area, and / or the air supply volume in that area is reduced, the cooling and heating output in that area is turned off, or the energy-saving control parameters of the previous control cycle in that area are maintained.

[0070] In another embodiment, the system further includes a sequence validity and prediction stability determination module, used to determine the validity, temporal continuity, and prediction confidence of the thermal imaging frame sequence. When the determination result does not meet the preset conditions, the air conditioning parameters of the previous control cycle can be maintained or a re-acquisition can be triggered to avoid frequent adjustments due to invalid input or instantaneous fluctuations. For a detailed implementation, please refer to the description in Embodiment 1; this embodiment will not elaborate further.

[0071] Example 3 Based on the thermal comfort monitoring method and cabin thermal comfort monitoring system described in Embodiments 1 and 2, Embodiment 3 of this application further provides a cabin air conditioning adjustment method based on thermal comfort prediction results. Figure 5 As shown, the above method includes the following steps: Step 201: Continuously acquire thermal imaging images of occupants' faces to form a sequence of thermal imaging frames, and obtain the number of occupants, their seat positions, and vehicle CAN bus information.

[0072] Step 202: Preprocess the above facial thermal imaging frame sequence, including facial region cropping, image scaling, pixel value normalization, and format conversion, to form sequence data suitable for input into the thermal comfort prediction model.

[0073] Specifically, for the data collection and preprocessing in the above steps, Example 3 of this application selected 22 healthy volunteers (10 women and 12 men), aged 19-26 years, all without a history of cardiovascular or skin diseases. They avoided alcohol, staying up late, taking medication, and strenuous activity within 12 hours before the experiment. The clothing thermal resistance was 0.40-0.50 clo, and the metabolic rate corresponding to sitting still in the vehicle was 1.0-1.1 met. The experiment was conducted inside the vehicle cabin, with the vehicle stationary. Before each experiment, the vehicle was parked in a shady place, and the cabin temperature was controlled at 25℃-26℃. Volunteers were required to sit still for 15 minutes to eliminate the effects of walking and sun exposure. Two volunteers were assigned to the driver's seat and the passenger seat for each experiment, with a recorder in the back seat. The experiment lasted 1.5 hours, with the cabin temperature dynamically changing within a preset range of 20℃-30℃. Volunteers were allowed to engage in low-metabolic activities such as talking, drinking water, and reading. To avoid the airflow from the vents directly affecting the facial area, the airflow was directed towards the volunteers' chests. Facial thermal imaging images were acquired, and subjective thermal sensation votes were simultaneously collected as labels. Multiple frames of facial thermal imaging images were continuously acquired within a preset time window before and after each labeling time to form a facial thermal imaging frame sequence corresponding to the same thermal sensation label. An example acquisition interval for facial thermal imaging images was 10 minutes, with labeling and acquisition occurring at the 5th, 10th, 20th, ..., 80th, and 90th minutes. A total of 1320 facial thermal imaging images were acquired, which could be combined into a sequence sample according to chronological order. After removing a small number of low-quality samples, a valid dataset was formed. In one example, 419 images were classified as "cold," 602 as "comfortable," and 240 as "hot."

[0074] After each acquisition of a continuous thermal imaging frame sequence corresponding to a labeled time point, volunteers are required to complete a thermal perception vote (TSV). To facilitate model training, the TSV is further mapped to a three-category label: hot (+1), comfortable (0), and cold (-1). In one implementation, multiple consecutive thermal imaging image sequences corresponding to the same labeled time point are assigned the same label.

[0075] For data preprocessing, this embodiment first crops each frame of facial thermal imaging images to 100×100 pixels, then scales them to 224×224 pixels. To adapt to the input of the feature extraction network, the single-channel grayscale thermal imaging images can be converted to an image format that meets the network's input requirements. The preprocessed dataset is randomly divided into a training set, a validation set, and a test set, with the training set accounting for 70%, the validation set accounting for 20%, and the remainder for the test set. To improve the model's adaptability to real-world scenarios, samples of different genders, whether or not glasses are worn, and different facial orientations are included in the training set, validation set, and test set. Data augmentation is performed only on the training set, with examples of augmentation methods including horizontal flipping, clockwise rotation, and counterclockwise rotation to enrich sample diversity.

[0076] Step 203: Input the preprocessed frame sequence into the thermal comfort prediction model and output the current thermal comfort state of the occupants, the probability distribution of thermal comfort state, and the thermal comfort change trend or prediction result in the next T time steps.

[0077] Step 204: Based on the thermal comfort prediction results of the next T time steps, and combined with the current operating status of the vehicle and the status of the air conditioning system, construct a multi-objective optimization problem with thermal comfort index, air conditioning energy consumption and response time as objectives, solve multiple non-dominated control solutions on the Pareto front, and select the target control strategy from them according to the user's preset preferences.

[0078] Step 205: Solve the air conditioning control sequence for the next T time steps based on the above target control strategy. The air conditioning control sequence includes at least one or more of the following: compressor frequency, air volume, and air direction. A rolling optimization method is used to execute only the control values ​​at the current moment.

[0079] Step 206: In the next control cycle, the air conditioning control sequence is re-solved based on the updated thermal comfort prediction results, the number and location of occupants, and the vehicle CAN bus information to achieve zoned adaptive thermal management.

[0080] In some implementations, when no valid occupants are detected in the target seating area, the area can be marked as an unoccupied area, and thermal comfort prediction-driven air conditioning adjustments will not be performed in the area, and / or the air supply volume in the area will be reduced, the cooling and heating output in the area will be turned off, or the energy-saving control parameters of the area in the previous control cycle will be maintained. When the sequence validity determination or prediction confidence determination does not meet the preset conditions, the air conditioning parameters of the previous control cycle can be kept unchanged to avoid frequent adjustments due to invalid inputs or instantaneous fluctuations.

[0081] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for monitoring and adjusting cabin thermal comfort, characterized in that, include: Acquire facial thermal images of vehicle occupants at multiple consecutive moments; Local features are extracted from each frame of facial thermal imaging image to obtain frame-level feature vectors of a preset dimension; Add frame-level temporal location encoding to the frame-level feature vector; the frame-level temporal location encoding is determined by the temporal order of acquiring the corresponding facial thermal imaging images. Temporal dependency modeling is performed on the frame-level feature vector with the added frame-level temporal position encoding to obtain the sequence fusion feature representation; The thermal comfort status and / or thermal comfort change trend of the occupants in the vehicle are determined based on the sequence fusion feature representation. Based on the thermal comfort state and / or the thermal comfort change trend, combined with the current vehicle operating state and the air conditioning system state, an optimization problem with thermal comfort, air conditioning energy consumption and response time as objectives is constructed, and the air conditioning control sequence is obtained by solving it to adjust the cabin air conditioning.

2. The cabin thermal comfort monitoring and adjustment method according to claim 1, characterized in that, Acquiring facial thermal images of vehicle occupants at multiple consecutive moments also includes: Multiple frames of facial thermal imaging images are continuously acquired within a preset time window corresponding to multiple preset labeled times, and then combined according to the acquisition time order. Obtain the subjective thermal sensation vote TSV corresponding to each of the preset labeled times, and map the subjective thermal sensation vote TSV to a plurality of preset thermal sensation tags; the thermal sensation tags are used to characterize the thermal comfort state of the occupants in the vehicle corresponding to the preset labeled times; Configure multiple frames of facial thermal imaging images combined within the same preset time window with the same thermal sensing label.

3. The cabin thermal comfort monitoring and adjustment method according to claim 1, characterized in that, After acquiring facial thermal imaging images of vehicle occupants at multiple consecutive time points, the method further includes: Facial regions are cropped from each frame of facial thermal imaging images; Scale the cropped facial thermal image to a preset size; The scaled facial thermal imaging image is subjected to pixel value normalization and format conversion; wherein, the format conversion is used to convert the facial thermal imaging image into an image format that meets preset requirements.

4. The cabin thermal comfort monitoring and adjustment method according to claim 1, characterized in that, Local features are extracted from each frame of facial thermal imaging to obtain frame-level feature vectors of a preset dimension, including: A residual network is used to extract local features from each frame of facial thermal imaging images to obtain high-dimensional feature vectors. The high-dimensional feature vector is mapped to a frame-level feature vector of the preset dimension through linear mapping.

5. The cabin thermal comfort monitoring and adjustment method according to claim 1, characterized in that, Temporal dependency modeling is performed on the frame-level feature vector with the added frame-level temporal position encoding to obtain a sequence fusion feature representation, including: The frame-level feature vector with the added frame-level temporal position encoding is input into the Transformer encoder, and the correlation between different time frames is modeled through a multi-head self-attention mechanism; The sequence features output by the Transformer encoder are averaged along the time dimension to obtain the sequence fusion feature vector.

6. The cabin thermal comfort monitoring and adjustment method according to claim 1, characterized in that, Determining the occupant's current thermal comfort state and / or thermal comfort change trend based on the sequence fusion feature representation includes: The probability distribution of the occupant in the preset thermal comfort state category is calculated based on the sequence fusion feature representation, and the thermal comfort state corresponding to the highest probability in the probability distribution is taken as the thermal comfort state of the occupant in the vehicle. And / or, based on the sequence fusion feature representation, a prediction is made to obtain the trend of changes in the thermal comfort of the occupants within a preset time period in the future.

7. The cabin thermal comfort monitoring and adjustment method according to claim 1, characterized in that, Before solving the air conditioning control sequence based on the thermal comfort state and / or future thermal comfort change trends, the method further includes: The system determines whether there is a valid human face target in the facial thermal imaging image, whether the sampling interval between adjacent frames meets the preset requirements, and whether the confidence level of the thermal comfort state prediction result is higher than the preset threshold. If the judgment result does not meet any preset condition, maintain the air conditioning parameters of the previous control cycle, trigger re-acquisition, or delay the update of the current control quantity.

8. The cabin thermal comfort monitoring and adjustment method according to claim 1, characterized in that, The air conditioning control sequence is obtained by solving for: Based on the thermal comfort state and / or thermal comfort change trend in the next T time steps, and combined with the vehicle operating status and air conditioning system status, a multi-objective optimization problem is constructed with thermal comfort index, air conditioning energy consumption and response time as objectives. Solving the multi-objective optimization problem yields multiple non-dominated air conditioning control sequences on the Pareto front; The air conditioning control sequence is selected from the plurality of non-dominated air conditioning control sequences based on user preset preferences or preset control strategies; wherein, the air conditioning control sequence includes one or more of compressor frequency, air volume and air direction; According to the air conditioning control sequence, the control quantity at the current moment is executed in a rolling optimization manner, and the solution is re-solved based on the updated thermal comfort state in the next control cycle.

9. The cabin thermal comfort monitoring and adjustment method according to claim 1, characterized in that, Before obtaining the air conditioning control sequence, the method further includes: The number of occupants and their seating positions are determined using infrared thermal imaging. When no valid occupant is detected in the target seating area, the corresponding area will be marked as an unoccupied area; For unoccupied areas, zoned air conditioning control driven by thermal comfort state prediction is not executed, and / or the air supply volume of the corresponding area is reduced, the cooling and heating output of the area is turned off, or the energy-saving control parameters of the corresponding area in the previous control cycle are maintained.

10. A cabin thermal comfort monitoring and adjustment system, characterized in that, It includes a sequence image acquisition module, a thermal comfort prediction model, a vehicle state acquisition module, and a multi-objective optimization solution module; The sequential image acquisition module is used to acquire facial thermal imaging images of occupants at multiple consecutive moments. The thermal comfort prediction module includes a pre-trained thermal comfort monitoring model, which is used to extract local features from each frame of facial thermal imaging images to obtain a frame-level feature vector of a preset dimension; add a frame-level temporal position code to the frame-level feature vector; the frame-level temporal position code is determined by the temporal order of acquiring the corresponding facial thermal imaging images; and perform temporal dependency modeling on the frame-level feature vector with the added frame-level temporal position code to obtain a sequence fusion feature representation. The thermal comfort status and / or thermal comfort change trend of the occupants in the vehicle are determined based on the sequence fusion feature representation. The vehicle status acquisition module is used to acquire the current operating status of the vehicle and the status of the air conditioning system; The multi-objective optimization solution module is used to generate an air conditioning control sequence based on the thermal comfort state and / or future thermal comfort change trend, combined with the current vehicle operating state and air conditioning system state, using a model predictive control strategy.

Images