A non-invasive human thermal comfort multivariate prediction system and method

Abstract

The present invention discloses a non-invasive multivariate prediction system and method for human thermal comfort, comprising a thermal comfort prediction module based on thermal adaptive action, a thermal comfort prediction module based on multi-region temperature of human face, and a PMV evaluation module dominated by ambient temperature. Compared with the prior art, the present invention does not require sensor skin contact and behavioral restrictions, and only uses a camera to remotely collect three-dimensional information and images of the human body, with little subjective influence of thermal sensation. The prediction results of the three prediction modules are comprehensively considered, and weights are introduced through prediction accuracy and voting probability to achieve characteristic complementarity between modules, with high prediction accuracy.

Description

Technical Field

[0001] The invention relates to a non-invasive multivariate prediction method for human thermal comfort, and belongs to the technical field of human thermal comfort prediction and computer vision. Background Art

[0002] Real-time prediction of human thermal comfort has important practical significance and application value for improving building environmental comfort and building energy efficiency. Using non-invasive methods to determine people's thermal comfort in real time and adjust the setting parameters of the HVAC system can not only better meet the thermal comfort of users, but also play an important role in saving building operation energy consumption.

[0003] Existing thermal comfort prediction methods are mainly divided into prediction voting model method and sensor measurement method. Prediction voting model method (such as PMV model) mainly evaluates the feeling of most people in the same environment through evaluation indicators that characterize human thermal response, without considering individual differences or individual real-time feelings. This type of model has reference significance for the evaluation of thermal comfort of the crowd, but there is a deviation from the actual situation of the individual; sensor measurement methods are divided into invasive and non-invasive methods. The invasive method mainly measures the physiological data such as real-time heart rate, electroencephalogram and skin temperature of the human body to establish a prediction model with the actual thermal comfort feeling of the human body. This method requires the skin of the person to contact the sensor, which is inconvenient for data collection and has a subjective impact on the thermal comfort evaluation; the non-invasive method introduces non-invasive measurement technologies such as computer vision, uses remote acquisition equipment to record the behavioral characteristics that characterize human thermal adaptation, and establishes a related prediction model with thermal comfort feeling. Chinese patent number CN112303861A discloses a thermal comfort prediction method based on human thermal adaptability behavior. This method predicts an individual's thermal state by monitoring human behavior and posture. Although it solves the problems of inconvenient data collection and large subjective influence caused by skin contact through non-invasive methods, the thermal adaptability of the human body cannot fully reflect the actual thermal comfort state of the individual, resulting in the difficulty in ensuring the accuracy of thermal comfort prediction; Chinese patent number CN114627539A discloses a thermal comfort prediction method based on infrared thermal images of the face. This method remotely recognizes the thermal image of the face, uses a data-driven model, and predicts the thermal comfort of the human body through the thermal image of the face. However, the temperature of the human face is affected by multiple factors such as psychology and emotions. The thermal comfort state of the human body cannot be fully reflected based on the temperature data of the face alone, and the prediction accuracy cannot be guaranteed. Summary of the invention

[0004] In view of the deficiencies of the above-mentioned prior art in actual prediction, the present invention proposes a non-invasive multivariate prediction method and system for human thermal comfort. This method predicts human thermal comfort by weighting information based on multivariate information reflecting the thermal comfort state of the human body. Compared with the current method, the characteristics of this method are: 1. It integrates three prediction methods reflecting human thermal comfort, namely the prediction voting method (PMV model), the thermal adaptation action prediction method and the facial temperature prediction method. The mapping relationship between thermal adaptation action and facial temperature and human thermal comfort is established by voting, and the comprehensive thermal comfort evaluation of the human body is obtained by the weighted coefficient method, with high prediction accuracy; 2. The action acquisition and facial temperature acquisition in this method do not require human contact, and belong to a non-invasive prediction method. It has the characteristics of convenient data collection and small subjective influence on thermal comfort, wide application scenarios, and great practical application value.

[0005] The system of the present invention adopts the following technical solutions: a non-invasive multivariate prediction system for human thermal comfort, including a thermal comfort prediction module based on thermal adaptability actions, a thermal comfort prediction module based on multi-region temperatures of the human face, an ambient temperature-dominated PMV evaluation module, and a multivariate human comfort probability model;

[0006] The thermal comfort prediction module based on thermal adaptability actions realizes thermal comfort evaluation by collecting thermal adaptability actions of human body;

[0007] The thermal comfort prediction module based on multi-region temperature of human face realizes thermal comfort evaluation by collecting thermal images of human face;

[0008] The PMV evaluation module dominated by ambient temperature realizes thermal comfort evaluation by collecting the ambient temperature of the human body;

[0009] The multivariate human comfort probability model integrates the above three modules to achieve thermal comfort evaluation.

[0010] The technical solution of a non-invasive multivariate prediction method for human thermal comfort of the present invention comprises the following steps: collecting human thermal adaptability movements through a thermal comfort prediction module based on thermal adaptability movements to achieve thermal comfort evaluation; collecting thermal images of the human face through a thermal comfort prediction module based on multi-region temperatures of the human face to achieve thermal comfort evaluation; collecting the ambient temperature of the human body through a PMV evaluation module dominated by ambient temperature to achieve thermal comfort evaluation; and finally, multivariate human comfort probability model and the above three modules are multivariately integrated to achieve thermal comfort evaluation.

[0011] Furthermore, the establishment process of the thermal comfort prediction module based on thermal adaptability action includes the following steps:

[0012] 3.1 Define the categories of human thermal adaptation actions;

[0013] 3.2 Use the depth camera to collect the skeletal trajectory data of thermal adaptive movements and establish a thermal adaptive movement database;

[0014] 3.3 Establish a thermal adaptive action recognition model;

[0015] 3.4 Use a questionnaire to map thermal adaptation actions to subjective thermal sensation levels. The specific content of the questionnaire is to ask the subjects about their subjective thermal sensations of thermal adaptation actions. Specifically, the thermal sensation levels are divided into seven levels: "comfortable", "a little warm", "a little cool", "warm", "cool", "hot" and "cold" according to the heat / cold adaptation actions. The questionnaire can be collected online and no less than 200 copies are collected. The corresponding thermal sensation level of each type of action and the voting probability P of each type of action corresponding to the thermal sensation are obtained through the collected questionnaire results. thermal1 ;

[0016] 3.5 Output the thermal perception prediction result and probability based on thermal adaptive action; input the action sample collected in real time into the thermal adaptive action recognition model described in step 3.3 to obtain the prediction result and probability P of the action category action Then, according to the thermal voting level corresponding to the action in the questionnaire in step 3.4, the thermal level prediction value L1 and thermal prediction probability P corresponding to this action are output. action ×P thermal1 .

[0017] Furthermore, the categories of human thermal adaptability actions specifically include: action 1 wiping sweat, action 2 fanning with objects, action 3 shaking the collar, action 4 taking off the coat, action 5 putting on the coat, action 6 wrapping the arms around, action 7 rubbing the hands, and action 8 blowing on the hands.

[0018] Furthermore, the action database is no less than 30 persons x 8 actions, and the skeletal data sets of the above actions are collected from three angles (0°, -45°, 45°) and three distances (1.5m, 2.5m, 3.5m) facing the person from the camera. The duration of a single action is 3 to 9 seconds, the frame number ranges from 90 to 270 frames, and each frame contains the three-dimensional coordinate values ​​of 25 skeletal points.

[0019] Furthermore, the main steps of establishing a thermal adaptive action recognition model include:

[0020] 1) Skeletal data preprocessing: convert the three-dimensional coordinate data of the joint points into the data centered on the base of the spine; standardize the length of the joints to unify the length of each joint in the sample; traverse the distance between the center of the clavicle and the base of the spine in each frame, and set the threshold T1 to remove abnormal frames in each sample;

[0021] 2) Key frame extraction: Calculate the cumulative value sum of the distance changes of the six joints, namely, left shoulder 5, left elbow 6, left wrist 7, right shoulder 9, right elbow 10, and right wrist 11, in space as the number of frames increases, set a threshold T2, reset the sum value whenever sum>T2, and retain the number of frames accumulated to this moment, thereby extracting all key frames;

[0022] 3) Spatiotemporal feature extraction: In each frame, the distance values ​​S = [s5, s6, s7, s9, s 10 ,s 11 ], the distance s0 from the head to the bottom of the spine in the sample is introduced to eliminate the data difference caused by the height factor. The calculation formula is:

[0023]

[0024] Get a set of standard distance values ​​by calculation: Calculate the time series vector of the above 6 joint points changing in the previous and next two frames:

[0025]

[0026] Thus, a time series vector set [β5, β6β7, β9, β 10 , β 11 ]; Fusion of the above two sets can obtain the feature set of each frame Finally, the spatiotemporal feature matrix F = {f1,f2…f n-1 ,f n}, n represents the number of key frames;

[0027] 4) Constructing a thermal adaptive motion recognition model: The recognition model uses a bidirectional long short-term memory neural network (Bi-LSTM). The data set obtained in step 3.2 is divided proportionally, 80% as a training set and 20% as a test set for model training and parameter adjustment, thereby constructing a thermal adaptive motion recognition model.

[0028] Furthermore, the establishment process of the thermal comfort prediction module based on the multi-region temperature of the human face includes the following steps:

[0029] 7.1 Set the camera position and facial collection area, and set up the thermal imaging camera position: it should be facing the person's face (-45°~45°), and the distance is about 1.5m~3m. Select multiple temperature collection areas on the human face: including forehead, left eye, right eye, left cheek, right cheek, mouth, nose and chin, a total of 8 areas;

[0030] 7.2 Sampling of multi-region facial temperature data set: Set the camera position and acquisition area according to step 7.1; adjust the room temperature in intervals of 2°C, so that the personnel experience the heat discomfort state, comfort state, and cold discomfort state in sequence; after each temperature adjustment, the personnel first sit quietly for 15 to 20 minutes to achieve a steady state of the body, and then use a thermal imaging camera to take 5 static thermal images of the face and save them. The data set requires facial temperature data of at least 20 people;

[0031] 7.3 Preprocessing of facial temperature data: extract the temperature values ​​of 8 facial regions from 5 thermal images respectively, and calculate their arithmetic mean as the temperature values ​​of 8 facial regions. Normalize the 8 values ​​and map them to (-1, 1) to form a feature vector T = [t1, t2, t3, t4, t5, t6, t7, t8];

[0032] 7.4 Use a questionnaire to associate facial temperature with the subjective thermal sensation level. The questionnaire is conducted simultaneously with the facial temperature data collection described in step 7.2. The content of the questionnaire includes: the name, gender, height, weight and age of the person, the actual thermal sensation of the person at the current room temperature and the corresponding level; Based on the results of the questionnaire, the thermal sensation level corresponding to different facial temperatures and its voting probability P can be obtained. thermal2 ;

[0033] 7.5 Establish a thermal comfort prediction model for multi-region temperature of the face;

[0034] 7.6 Output the thermal sensation prediction result and probability based on facial temperature. Input the real-time collected facial temperature data into the SVM model described in step 7.5 to obtain the thermal sensation prediction result based on facial temperature as L2 and the thermal sensation prediction probability P face ×P thermal2 , the prediction accuracy of the model is P face .

[0035] Furthermore, the thermal comfort prediction model of the multi-zone temperature of the face adopts a support vector machine SVM model, the model input is the vector T described in step 3.3, and the output is the voting result of the thermal sensation level described in step 3.4. The facial temperature data is preprocessed and divided into a training set: 80% as a training set and 20% as a test set. The SVM model is trained and key parameters are adjusted, including: penalty factor, kernel function, and gamma value. The test set data is input into the above SVM prediction model, and the prediction accuracy of the model is obtained as P face .

[0036] Furthermore, the steps for establishing the ambient temperature-dominated PMV evaluation module include:

[0037] 9.1 Use the temperature sensor to obtain the indoor ambient temperature value in real time;

[0038] 9.2 Use the PMV formula to estimate indoor thermal comfort. The formula is as follows:

[0039] PMV=[0.303exp(-0.036M)+0.028]TL (3)

[0040] PMV is the predicted value L3 of the thermal sensation level obtained based on the PMV evaluation module. TL is defined as the difference between the heat generated by the human body and the heat emitted by the human body to the outside world at three locations. The calculation formula of TL is:

[0041] TL=MW-3.96×10 -8 f cl [(T cl +273.15) 4 -(T r +273.15) 4 ]-f cl h c (T cl -T)-3.05×10 -3 ×[5733-6.99(MW)-P a ]-0.42(MW-58.15)-1.7×10 -5 [5876-P a ]-0.0014M(34-T) (4) Where M is the metabolic rate, W / m 2 ; W is the external work output by humans, W / m 2 ;f cl is the ratio of clothed body surface to naked body surface, W / m 2 ; T is the ambient temperature, ℃; T cl is the temperature of the outer surface of the clothes, ℃; T r is the average radiation temperature, °C; h c is the surface heat transfer coefficient between clothing and air, W / m 2 ;P a is the partial pressure of water vapor, P a ; Among them, f cl 、T cl and h c It can be calculated according to the following formula:

[0042] f cl =1.05+0.645I cl (I cl ≥0.078) (5)

[0043] f cl =1+1.29I cl (I cl <0.078) (6)

[0044] Tcl =35.7-0.028(MW)-I cl {3.96×10 -8 f cl [(T cl +273.15) 4 -(T r +273.15) 4 ]+f cl h c (T cl -T)} (7)

[0045] h c =2.38×(T cl -T) 0.25 (2.38(T cl -T) 0.25 ≥12.1u 0.5 ) (8)

[0046] In the formula, I cl is the thermal resistance of the clothing, m 2 ℃ / W; u is air velocity, m / s;

[0047] Other parameters of the PMV formula are manually estimated based on the measured indoor ambient temperature and the human basal metabolic level. The parameters that need to be estimated include: human metabolism, human external work output, clothing surface temperature, average radiation temperature, surface heat transfer coefficient between clothing and air, water vapor partial pressure, clothing thermal resistance and air flow rate.

[0048] Furthermore, the multivariate human comfort probability model establishment step includes:

[0049] The weight of the evaluation result of the thermal comfort prediction module based on thermal adaptability action is α1, and its calculation formula is:

[0050]

[0051] The weight of the evaluation result of the thermal comfort prediction module based on the multi-region temperature of the human face is α2, and its calculation formula is:

[0052]

[0053] The weight of the PMV evaluation module evaluation result based on ambient temperature is α3, and its calculation formula is:

[0054] α3=1-α1-α2 (11)

[0055] Add the three different prediction results by weight to get L total , the calculation formula is:

[0056] Ltotal =L1×α1+L2×α2+L3×α3 (12)

[0057] Where L3 is the PMV calculation result dominated by ambient temperature.

[0058] Compared with the prior art, the advantages of the present invention are:

[0059] 1. Personnel do not need to have skin contact with the sensor and have no behavioral restrictions. The camera is used to remotely collect three-dimensional information and images of the human body, and the subjective impact of thermal sensation is small.

[0060] 2. Accurate prediction results: The present invention comprehensively considers the prediction results of the three prediction modules, introduces weights through prediction accuracy and voting probability, realizes the complementary characteristics between modules, and improves the prediction accuracy.

[0061] 3. High prediction robustness. It can be seen from formula (12) that when the two thermal perception prediction accuracies are poor, the method of the present invention will degenerate into a basic thermal perception evaluation method based on PMV, thereby improving prediction robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 : System framework diagram of the present invention

[0063] Figure 2 : Schematic diagram of the distribution of skeletal joint points in the behavior prediction module of the present invention

[0064] Figure 3 : Schematic diagram of eight action key frames of the behavior prediction module of the present invention

[0065] Figure 4 : Sample questionnaire on heat adaptation behavior

[0066] Figure 5 : Facial temperature thermography

[0067] Figure 6 : Sample questionnaire on facial temperature and thermal sensation DETAILED DESCRIPTION

[0068] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0069] It should be noted that this embodiment is intended to specifically illustrate the present invention, and is not intended to limit the actual implementation of the present invention, including the product selection of equipment, feature extraction, and application of algorithms are not limited to this embodiment.

[0070] according to Figure 1This embodiment is divided into three parts and a result calculation module, including: the first part is a thermal comfort prediction module based on thermal adaptive actions: defining thermal adaptive action categories, conducting questionnaires, establishing a thermal adaptive action database, building a thermal adaptive behavior recognition model, inputting real-time action data, and predicting thermal sensation results; the second part is a thermal comfort prediction module based on multi-region temperature of the human face: collecting facial temperature data, conducting questionnaires, building a facial temperature data-driven recognition model, inputting real-time temperature data, and predicting thermal sensation results; the third part is an ambient temperature-dominated PMV evaluation module: setting up environmental data measurement equipment, obtaining ambient temperature values, and substituting them into formulas to calculate thermal sensation results.

[0071] The hardware device used by the thermal comfort prediction module based on thermal adaptive movements is a binocular depth camera capable of collecting human skeleton data, which is connected to a computer to achieve thermal comfort evaluation by collecting human thermal adaptive movements; the hardware device used by the thermal comfort prediction module based on multi-region temperature of the human face is a thermal imaging camera, which is connected to a computer to achieve thermal comfort evaluation by collecting thermal images of the human face; the hardware device used by the ambient temperature-dominated PMV evaluation module is an ambient temperature sensor, which is connected to a computer to achieve thermal comfort evaluation by collecting the ambient temperature of the human body; the multivariate human comfort probability model is a model set by a computer program, and the above three modules are multivariately integrated to achieve thermal comfort evaluation.

[0072] The following is a more detailed description of the sub-sections:

[0073] The first part is the thermal comfort prediction module based on thermal adaptive actions.

[0074] The hardware in this part is based on Microsoft's Kinect v2 camera. Through its depth detection and skeleton tracking functions, it can obtain the depth information of the person to construct the user's skeleton map, and construct a three-dimensional coordinate system with the camera as the coordinate center to obtain the three-dimensional coordinates of each joint point of the human body. The skeleton map contains a total of 25 joint points of the human body (see Figure 2 ).

[0075] 1. Define heat-related action categories: When people feel cold or hot, they usually have various behaviors, so it is feasible to predict thermal comfort based on behaviors. Based on thermal comfort theory, this embodiment defines 8 types of actions related to human body cold and hot sensations, including: Action 1 Wiping sweat, Action 2 Fanning with an object, Action 3 Shaking the collar, Action 4 Taking off the coat, Action 5 Putting on the coat, Action 6 Protecting the arms, Action 7 Rubbing hands, Action 8 Blowing on the hands. The above action diagrams are shown in the figure below. Figure 3 shown.

[0076] 2. Establish a thermal adaptive motion database: In this part, 30 volunteers were recruited through designed experiments. A data set containing the above eight types of motions was collected from three camera angles (0, -45, 45) and three distances (1.5m, 2.5m, 3.5m). The duration of each motion was 3 to 9 seconds, the number of frames was 90 to 270, and each frame contained the three-dimensional coordinate values ​​of 25 bone points.

[0077] 3. Build a thermally adaptive motion recognition model: The skeletal data collected for an action is very dense, and the action changes between the previous and next frames are very small, which easily generates redundant data and is not conducive to extracting key features. In addition, during the collection of skeletal data, the color depth of clothing and background will affect the recognition of skeletal points, causing the positions of some data joints to jump or deviate between the previous and next frames. Based on the above factors, the skeletal data needs to be preprocessed and feature extracted, including:

[0078] 1) In order to facilitate data processing and calculation, the three-dimensional bone data is converted into coordinate values ​​centered on the bottom of the spine (joint point 1).

[0079] 2) In order to reduce the interference caused by differences in height and body shape, the length of the joints needs to be standardized: traverse each frame, find the average length of each joint in all frames as the standard length, and unify the length of all joints in each frame.

[0080] 3) Select the clavicle center (joint 21) and the spine base (joint 1) joints, traverse the distance of these two points before and after each frame, set the threshold T1 = 0.3, and when the sum of the two distance values ​​is greater than T1, remove the next frame and keep the current frame; the next step is to select 6 joints that can reflect the core change trend of the action (including: left shoulder 5, left elbow 6, left wrist 7, right shoulder 9, right elbow 10, right wrist 11), and traverse the distance of each joint before and after. D t It represents the sum of the distance values ​​of the 6 joint points between the tth frame and the t+1th frame; the first frame is used as the first key frame index, and the D value calculated each time is accumulated to the total value sum, and the threshold T2 is set to 0.8. When sum is greater than the threshold T2, the sum value is reset, and the t+1 frame is used as the index of the next key frame, and the above process is repeated. Finally, all key frames are extracted after screening.

[0081] 4) Calculate the distances from the above six joints to the base of the spine (joint 1) to form a set S = [s5, s6, s7, s9, s 10 ,s 11 ]; introduce length s0 (the distance from joint point 4 to joint point 1) to remove the distance feature differences caused by different heights; calculate the standard distance value:

[0082]

[0083] Among them, i represents the number of the joint point, Represents the standard distance value from joint point i to the base of the spine.

[0084] The time vector of joint point i from frame t to frame t+1 is expressed as:

[0085]

[0086] in, Represents the three-dimensional coordinate vector of joint point i in the tth frame.

[0087] For each indexed keyframe, a feature vector can be constructed:

[0088]

[0089] Finally, the spatiotemporal feature matrix F = {f1,f2…f n-1 ,f n}, n represents the total number of key frames.

[0090] 5) Training the model and setting parameters: The constructed data set was divided into 80% as a training set and 20% as a test set, and input into a bidirectional long short-term memory neural network (Bi-LSTM) to train and establish a thermal adaptive behavior recognition model. The model parameters were set as follows: the learning rate was set to 0.5, the number of hidden layer units was set to 200, and the minimum training batch was set to 100.

[0091] 4. Conduct a questionnaire survey: Map the heat adaptation actions to the subjective thermal sensation level. The specific content of the questionnaire is to ask the subjects about their subjective thermal sensations for the eight heat adaptation actions described in Section 2.1. Specifically, the heat / cold adaptation actions are divided into seven thermal sensation levels: "comfortable", "a little warm", "a little cool", "warm", "cool", "hot", and "cold" (see the questionnaire format for details). Figure 4 ). This part of the survey was conducted in the form of an online questionnaire, with a total of 200 questionnaires collected. The questionnaire obtained the corresponding thermal sensation of each type of action through the collected questionnaire results. The results of the thermal sensation level corresponding to each action category are shown in Table 1.

[0092] Table 1

[0093]

[0094]

[0095] Instructions for filling out the questionnaire are as follows:

[0096] 1) This questionnaire targets some movements related to the human body's sense of cold and heat, with the aim of investigating the specific degree to which each movement reflects the sense of cold and heat.

[0097] 2) The action scene is "when you are studying or having fun indoors."

[0098] 3) The actions are not purposeful and are considered subconscious.

[0099] 4) All actions have been judged in the hot and cold categories, so the option categories are targeted. For example, for "wiping sweat", there are only options such as "slightly warm", "warm", and "hot", but there are no options related to "cold" and "cool".

[0100] 5. After the camera collects the skeleton data of the person in real time, the above feature matrix is ​​generated and input into the recognition model to obtain the prediction result of the action category (the action recognition accuracy is P action ), and then according to the probability value P of the thermal voting level corresponding to this action in the questionnaire 2. thermal1 , output the predicted value L1 of the thermal level corresponding to this action. In this part, the weight is set to α1 according to the accuracy, and the calculation process is:

[0101]

[0102] The second part is a thermal comfort prediction module based on the temperature of multiple regions of the human face.

[0103] This part of the hardware is based on the MAG32 infrared thermal imager of JuGe Electronics, which can effectively capture thermal images of the human body in real time and store the temperature data of the temperature measured object.

[0104] 1. Set the camera position and facial acquisition area: Set the thermal imaging camera to face the person's face (-45° to 45°), at a distance of about 1.5m to 3m. Select multiple temperature acquisition areas on the human face: including forehead, left eye, right eye, left cheek, right cheek, mouth, nose and chin, a total of 8 areas (see Figure 5 );

[0105] 2. Collection of multi-region temperature data sets for the face: The prediction module in this section collects temperature data of the eight facial regions as described in 1. at different indoor ambient temperatures through designed experiments; the indoor temperature is adjusted from 31°C to 19°C, and the room temperature is adjusted successively at intervals of 2°C, so that the personnel experience heat discomfort state-comfort state-cold discomfort state in turn; after each temperature adjustment, the personnel first sit quietly for 15 to 20 minutes to achieve body homeostasis, and then use a thermal imaging camera to take 5 static thermal images of the face and save them. The experimental temperature range of this part only includes the temperature that can be reached indoors in summer, which can be achieved through climate temperature and HAVC heating and ventilation air conditioning. It should be noted that before each data collection, the personnel need to wait for 15 to 20 minutes at a constant temperature. The purpose is to allow the human body to reach a steady state, and then obtain stable facial temperature data of the human body to reduce errors;

[0106] 3. Facial temperature data preprocessing: Each time the data is collected, 5 static thermal images of the face are collected, and the temperature values ​​of the 8 areas of each image are extracted. The arithmetic mean is calculated as the temperature data value of the 8 areas of the face; the 8 data are normalized and mapped to (-1, 1) to form a feature vector T = [t1, t2, t3, t4, t5, t6, t7, t8]

[0107] 4. Thermal sensation questionnaire: Every time data is collected during the experiment, a real-time questionnaire is used to confirm the subjective thermal sensation level of the subjects. The experiment is carried out within the temperature range (31°C to 19°C) mentioned in 2. It basically covers all thermal sensation levels of the subjects, that is, -3 to 3, a total of 7 categories of thermal comfort, proving the validity of facial data. The questionnaire format for facial temperature is as follows: Figure 6 According to the questionnaire results, we can get the thermal sensation level corresponding to different facial temperatures and its voting probability (P thermal2 ).

[0108] 5. Construct a thermal comfort prediction model for multi-region temperature of the face: The prediction model adopts a support vector machine (SVM) model. The input of the model is 3. the feature vector T, and the output is 4. the thermal sensation level result. The collected facial temperature data set is preprocessed and divided: 80% as a training set and 20% as a test set, so as to train the model and adjust the model parameters. The model parameters are set as follows: the penalty factor is set to 1.3, the kernel function is set to RBF Gaussian kernel, and the gamma value is set to 2.6.

[0109] 6. Output the thermal perception prediction result and probability based on facial temperature. Input the real-time collected facial temperature data into the SVM model described in Section 3.5, and the thermal perception prediction result based on facial temperature is L2; ​​the recognition accuracy value of this result in the model is P face , and according to the probability value P corresponding to the thermal voting level in the questionnaire thremal2 , set the weight of this part to α2, and the calculation process is:

[0110]

[0111] The third part is the PMV evaluation module dominated by ambient temperature.

[0112] This part of the hardware equipment uses an ambient temperature sensor to obtain the real-time indoor temperature value through the sensor.

[0113] The predicted mean vote model PMV (Predicted Mean Vote) is an evaluation index that characterizes the thermal response of the human body. It represents the feeling of most people in the same environment, without considering individual differences or real-time measurement of personal attributes. Since the environment in which the experiment and prediction are conducted in this embodiment is indoors, most of the values ​​in the PMV index formula can be set to general values. The calculation formula of PMV is:

[0114] PMV=[0.303exp(-0.036M)+0.028]TL (5)

[0115] TL is defined as the difference between the heat generated by the human body and the heat that the human body sends to the outside world at three locations. The calculation formula for TL is:

[0116] TL=MW-3.96×10 -8 f cl [(T cl +273.15) 4 -(T r +273.15) 4 ]-f cl h c (T cl -T)-3.05×10 -3 ×[5733-6.99(MW)-P a ]-0.42(MW-58.15)-1.7×10 -5 [5876-P a ]-0.0014M(34-T) (6)

[0117] Where M is the metabolic rate, W / m 2 ; W is the external work output by humans, W / m 2 ;f cl is the ratio of clothed body surface to naked body surface, W / m 2 ; T is the ambient temperature, ℃; T cl is the temperature of the outer surface of the clothes, ℃; T r is the average radiation temperature, °C; h c is the surface heat transfer coefficient between clothing and air, W / m 2 ;P a is the partial pressure of water vapor, P a .

[0118] where f cl , T cl and h c It can be calculated according to the following formula:

[0119] f cl =1.05+0.645I cl(I cl ≥0.078) (7)

[0120] f cl =1+1.29I cl (I cl <0.078) (8)

[0121] T cl =35.7-0.028(MW)-I cl {3.96×10 -8 f cl [(T cl +273.15) 4 -(T r +273.15) 4 ]+f cl h c (T cl -T)} (9)

[0122] h c =2.38×(T cl -T) 0.25 (2.38(T cl -T) 0.25 ≥12.1u 0.5 ) (10)

[0123] In the formula, I cl is the thermal resistance of the clothing, m 2 ℃ / W; u is air velocity, m / s.

[0124] In view of the indoor conditions, this section takes the general values ​​of other data except ambient temperature in the sitting state of the experimental scene: the human metabolic rate M takes the benchmark value of 1Met in the sitting state; the human output external power takes the benchmark value of 70W / m 2 ;Clothing thermal resistance value I cl Take 0.5m 2 ℃ / W; air humidity is 50%; wind speed is the standard value of 3m / s in the office; clothing surface temperature and radiation temperature are directly taken from the ambient temperature; water vapor partial pressure is 700P a (The remaining values ​​can be obtained through the above numerical calculations.) The calculation result is dominated by the ambient temperature only. Substitute the real-time ambient temperature value into the formula to calculate the PMV result value L3; the weight of this part is set to α3, and the calculation method is:

[0125] α3=1-α1-α2 (11)

[0126] The above content has explained the three prediction modules in detail. The following is the calculation method of the fusion of the three results (i.e., the multivariate human comfort probability model):

[0127] L total =L1×α1+L2×α2+L3×α3 (12)

[0128] The calculation method of the three parts of the weights is explained as follows. In the entire system of the present invention, the first part, the behavior prediction module, and the second part, the facial temperature data-driven prediction module, have an experimental support basis, so the weights should be given higher allocations. In view of the excessive number of influencing factors in the third part, this module only uses the ambient temperature to dominate the calculation results, or errors may occur, so it is given a relatively small weight; the weight distribution adopts the total score distribution method, the first part is weighted α1, and the second part is weighted α2. According to practical considerations, it can be known that the value of α1+α2 tends to 1, so the total weight score is set to 1, and the value of α3 is calculated by the weight values ​​of the first two, that is, 1 is subtracted from the sum of the weights of the first two items, and then multiplied by the corresponding results respectively and finally added.

[0129] In the final L total After the actual predicted thermal sensation is determined, it can be known by calculation that L total ∈[-3,3], the specific interval corresponding to the thermal sensation result is shown in Table 2.

[0130] Table 2

[0131]

[0132] When there is no temperature data support, there may be certain errors in predicting thermal comfort based solely on human behavior posture. The reason is that some human movements are similar. For example, when a person rubs his eyes or strokes his hair, the prediction model will most likely associate such movements with the sweat-wiping movement that also has the tendency of raising the hand, thus leading to recognition errors.

[0133] Similarly, predicting thermal comfort status solely from human body temperature data also lacks consideration of actual accuracy. For example, the facial temperature of a person is not only affected by the ambient temperature or the person's activity status. Sometimes when a person is in an emotional state such as tension, anxiety, and excitement, the facial temperature will also fluctuate.

[0134] In summary, this embodiment integrates three methods that are of practical significance for thermal comfort prediction, including: human body posture behavior prediction method, human face real-time temperature data prediction method and PMV formula evaluation method, and reasonably allocates weights to the results of the three prediction modules. Relatively corresponding weights are assigned to the first and second parts supported by data in combination with probability values, and the weight of the third part is calculated from the first two parts. This makes the three parts complementary to each other, and all three meet the non-invasive conditions, making the overall system convenient and feasible.

[0135] The above are only embodiments of the present invention and are not intended to limit the present invention. Different considerations and changes may be made to the specific details of the present invention, including the selection of neural network models, feature extraction methods, weight allocation methods, etc. Any modification, equivalent replacement, improvement, etc. within the structure and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A prediction method for a non-invasive multivariate prediction system of human thermal comfort, It is characterized in that The prediction system includes a thermal comfort prediction module based on thermal adaptability actions, a thermal comfort prediction module based on multi-region temperature of the human face, an ambient temperature-dominated PMV evaluation module, and a multivariate human comfort probability model; The thermal comfort prediction module based on thermal adaptability actions realizes thermal comfort evaluation by collecting thermal adaptability actions of human body; The thermal comfort prediction module based on multi-region temperature of human face realizes thermal comfort evaluation by collecting thermal images of human face; The PMV evaluation module dominated by ambient temperature realizes thermal comfort evaluation by collecting the ambient temperature of the human body; The multivariate human comfort probability model integrates the above three modules to achieve thermal comfort evaluation; The prediction method comprises the following steps: collecting human thermal adaptability movements through a thermal comfort prediction module based on thermal adaptability movements to achieve thermal comfort evaluation; collecting thermal images of the human face through a thermal comfort prediction module based on multi-region temperatures of the human face to achieve thermal comfort evaluation; collecting the ambient temperature of the human body through an ambient temperature-dominated PMV evaluation module to achieve thermal comfort evaluation; and finally, multivariate human comfort probability model and the above three modules are multivariately integrated to achieve thermal comfort evaluation; The establishment process of the thermal comfort prediction module based on thermal adaptability action includes the following steps: 3.1 Define the categories of human thermal adaptation actions; 3.2 Use the depth camera to collect the skeletal trajectory data of thermal adaptive movements and establish a thermal adaptive movement database; 3.3 Establish a thermal adaptive action recognition model; 3.4 Use a questionnaire to map thermal adaptation actions to subjective thermal sensation levels. The specific content of the questionnaire is to ask the subjects about their subjective thermal sensations of thermal adaptation actions. Specifically, the thermal sensation levels are divided into seven levels: "comfortable", "a little warm", "a little cool", "warm", "cool", "hot", and "cold" according to the heat / cold adaptation actions. The questionnaire can be collected online and no less than 200 copies are collected. The corresponding thermal sensation level of each type of action and the voting probability P of each type of action corresponding to the thermal sensation are obtained through the collected questionnaire results. thermal1 ; 3.5 Output the thermal perception prediction result and probability based on thermal adaptive action; input the action sample collected in real time into the thermal adaptive action recognition model described in step 3.3 to obtain the prediction result and probability P of the action category action , and then output the predicted value L of the thermal sensation level corresponding to this action according to the thermal sensation voting level corresponding to this action in the questionnaire in step 3.4 1 and thermal prediction probability P action ×P thermal1 ; The establishment process of the thermal comfort prediction module based on multi-region temperature of human face includes the following steps: 7.1 Set the camera position and facial collection area, and set up the thermal imaging camera position: it should face the person's face (-45°~45°), with a distance of about 1.5m~3m, and select multiple temperature collection areas on the human face: including forehead, left eye, right eye, left cheek, right cheek, mouth, nose and chin, a total of 8 areas; 7.2 Sampling of multi-region facial temperature data set: Set the camera position and acquisition area according to step 7.1; adjust the room temperature in intervals of 2°C, so that the personnel experience the heat discomfort state, comfort state, and cold discomfort state in sequence; after each temperature adjustment, the personnel first sit quietly for 15 to 20 minutes to achieve a steady state of the body, and then use a thermal imaging camera to take 5 static thermal images of the face and save them. The data set requires facial temperature data of at least 20 people; 7.3 Preprocessing of facial temperature data: extract the temperature values ​​of 8 facial regions from 5 thermal images respectively, calculate their arithmetic mean as the temperature value of 8 facial regions, normalize the 8 values ​​and map them to (-1, 1) to form a feature vector T = [t 1 ,t 2 ,t 3 ,t 4 ,t 5 ,t 6 ,t 7 ,t 8 ]; 7.4 Use a questionnaire to associate facial temperature with the subjective thermal sensation level. The questionnaire is conducted simultaneously with the facial temperature data collection described in step 7.

2. The content of the questionnaire includes: the name, gender, height, weight and age of the person, the actual thermal sensation of the person at the current room temperature and the corresponding level; Based on the results of the questionnaire, the thermal sensation level corresponding to different facial temperatures and its voting probability P can be obtained. thermal2 ; 7.5 Establish a thermal comfort prediction model for multi-region temperature of the face; 7.6 Output the thermal perception prediction result and probability based on facial temperature. Input the real-time collected facial temperature data into the model in step 7.5, and the thermal perception prediction result based on facial temperature is L 2 and thermal prediction probability P face ×P thremal2 , the prediction accuracy of the model is P face ; The steps to establish the ambient temperature-dominated PMV evaluation module include: 9.1 Use the temperature sensor to obtain the indoor ambient temperature value in real time; 9.2 Use the PMV formula to estimate indoor thermal comfort. The formula is as follows: PMV=[0.303exp(-0.036M)+0.028]TL (3) Where PMV is the predicted value of thermal sensation level obtained based on the PMV evaluation module L 3 TL is defined as the difference between the heat generated by the human body and the heat that the human body sends to the outside world at three locations. The calculation formula for TL is: TL=M-W-3.96×10 -8 f cl [(T cl +273.15) 4 -(T r +273.15) 4 ]-f cl h c (T cl -T)-3.05×10 -3 ×[5733-6.99(M-W)-P a ]-0.42(M-W-58.15)-1.7×10 -5 [5876-P a ]-0.0014M(34-T) (4) Where M is the metabolic rate, W / m 2 ; W is the external work output by humans, W / m 2 ;f cl is the ratio of clothed body surface to naked body surface, W / m 2 ; T is the ambient temperature, ℃; T cl is the temperature of the outer surface of the clothes, ℃; T r is the average radiation temperature, °C; h c is the surface heat transfer coefficient between clothing and air, W / m 2 ;P a is the partial pressure of water vapor, P a ; Among them, f cl , T cl and h c It can be calculated according to the following formula: f cl =1.05+0.645I cl (I cl ≥0.078) (5) f cl =1+1.29I cl (I cl <0.078) (6) T cl =35.7-0.028(M-W)-I cl {3.96×10 -8 f cl [(T cl +273.15) 4 -(T r +273.15) 4 ]+f cl h c (T cl -T)} (7) h c =2.38×(T cl -T) 0.25 (2.38(T cl -T) 0.25 ≥12.1u 0.5 ) (8) In the formula, I cl is the thermal resistance of the clothing, m 2 ℃ / W; u is air velocity, m / s; Other parameters of the PMV formula are estimated manually based on the measured indoor ambient temperature and human basal metabolic level. The parameters that need to be estimated include: human metabolism, human external work output, clothing surface temperature, average radiation temperature, surface heat transfer coefficient between clothing and air, water vapor partial pressure, clothing thermal resistance and air velocity; The multivariate human comfort probability model establishment step comprises: The weight of the evaluation result of the thermal comfort prediction module based on thermal adaptability action is α 1 , and its calculation formula is: The weight of the evaluation result of the thermal comfort prediction module based on the multi-region temperature of the human face is α 2 , and its calculation formula is: The weight of the PMV evaluation module evaluation result based on ambient temperature is α 3 , and its calculation formula is: α 3 ＝1-α 1 -α 2 (11) Add the three different prediction results by weight to get L total , the calculation formula is: L total =L 1 ×α 1 +L 2 ×α 2 +L 3 ×α 3 (12) Where L 3 This is the PMV calculation result dominated by the ambient temperature.

2. The prediction method according to claim 1, It is characterized in that The specific categories of human body thermal adaptability actions include: action 1 wiping sweat, action 2 fanning with objects, action 3 shaking the collar, action 4 taking off the jacket, action 5 putting on the jacket, action 6 wrapping the arms around the body, action 7 rubbing the hands, and action 8 blowing on the hands.

3. The prediction method according to claim 1, It is characterized in that The action database is no less than 30 persons x 8 actions, and the skeletal data sets of the above actions are collected from three angles (0°, -45°, 45°) and three distances (1.5m, 2.5m, 3.5m) facing the person from the camera. The duration of a single action is 3 to 9 seconds, the frame number ranges from 90 to 270 frames, and each frame contains the three-dimensional coordinate values ​​of 25 skeletal points.

4. The prediction method according to claim 1, It is characterized in that The main steps of establishing a thermal adaptive action recognition model include: 1) Skeletal data preprocessing: convert the three-dimensional coordinate data of the joint points into the data centered on the base of the spine; standardize the length of the joints to unify the length of each joint in the sample; traverse the distance between the center of the clavicle and the base of the spine in each frame, and set the threshold T1 to remove abnormal frames in each sample; 2) Key frame extraction: Calculate the cumulative value sum of the distance changes of the six joints, namely, left shoulder 5, left elbow 6, left wrist 7, right shoulder 9, right elbow 10, and right wrist 11, in space as the number of frames increases, set a threshold T2, reset the sum value whenever sum>T2, and retain the number of frames accumulated to this moment, thereby extracting all key frames; 3) Spatiotemporal feature extraction: In each frame, the distance S from the six joints to the spine base as described in step 2) is calculated respectively. 5 ,s 6 ,s 7 ,s 9 ,s 10 ,s 11 ], introduce the distance s from the head to the bottom of the spine in the sample 0 To eliminate the data differences caused by height factors, the calculation formula is: Get a set of standard distance values ​​by calculation: Calculate the time series vector of the above 6 joint points changing in the previous and next two frames: Thus, a time series vector set [β 5 , β 6 β 7 , β 9 , β 10 , β 11 ]; Fusion of the above two sets can obtain the feature set of each frame Finally, the spatiotemporal feature matrix F = {f 1 ,f 2 …f n-1 ,f n }, n represents the number of key frames; 4) Constructing a thermal adaptive motion recognition model: The recognition model uses a bidirectional long short-term memory neural network (Bi-LSTM). The data set obtained in step 3.2 is divided proportionally, 80% as a training set and 20% as a test set for model training and parameter adjustment, thereby constructing a thermal adaptive motion recognition model.

5. The prediction method according to claim 1, It is characterized in that The thermal comfort prediction model of the multi-region temperature of the face adopts a support vector machine SVM model. The model input is the vector T described in step 3.3, and the output is the voting result of the thermal sensation level described in step 3.

4. The facial temperature data is pre-processed and divided into a training set: 80% as a training set and 20% as a test set. The SVM model is trained and key parameters are adjusted, including: penalty factor, kernel function, and gamma value. The test set data is input into the SVM model, and the prediction accuracy of the model is obtained as P face .

Images