Exercise thermal comfort detection and feedback adjustment method and storage medium

Abstract

The invention discloses an exercise thermal comfort detection and feedback adjustment method and a storage medium. The method comprises the following steps: acquiring multi-modal data of an exerciser in an exercise process; the multi-modal data comprises a motion image, a physiological signal and an environmental parameter; inputting the moving image into a pre-trained first evaluation model to obtain a first thermal comfort score based on the visual posture; inputting the physiological signal into a pre-trained second evaluation model to obtain a second thermal comfort score based on the physiological signal; performing weighted fusion on the first thermal comfort score and the second thermal comfort score to obtain a comprehensive thermal comfort index; and according to a comparison result of the comprehensive thermal comfort index and a preset comfort interval, in combination with the environment parameters, outputting a control instruction of the environment adjusting equipment so as to dynamically adjust local environment parameters of the region where the sporter is located. Real-time comprehensive detection of the thermal comfort state of a sporter can be achieved, and therefore environment equipment is dynamically adjusted, and closed-loop feedback is formed.

Description

Technical Field

[0001] This invention relates to a method and storage medium for detecting and adjusting thermal comfort during exercise, belonging to the field of sports health monitoring and intelligent environmental control technology. Background Technology

[0002] With increasing awareness of exercise and health, high-intensity interval training (HIIT) activities such as basketball, running, and fitness training are widely practiced in indoor venues. However, during exercise, the body's metabolic rate rises rapidly, body temperature increases, sweating intensifies, and blood flow accelerates, which can easily cause heat discomfort and, in severe cases, even lead to dehydration, heatstroke, and other health problems.

[0003] While indoor sports venues are generally equipped with air conditioning and ventilation systems, their control systems typically rely on static temperature and humidity settings, lacking real-time sensing and dynamic response capabilities to individual athletes' thermal load. This "passive response" control mechanism suffers from the following technical shortcomings:

[0004] Single-mode environmental sensors, monitoring venue conditions through parameters such as temperature, humidity, and wind speed, cannot reflect individual physiological states and true comfort levels. Single-modal physiological monitoring, such as heart rate monitors, skin temperature sensors, or infrared thermal imaging, while collecting some physiological information, suffers from poor stability during rapid movement, significant obstruction, and delayed feedback, and cannot correlate with movement actions. Single-mode visual monitoring struggles to quantify changes in thermal load. The fragmented nature of these modal data prevents coordinated analysis, leading to inaccurate assessments of thermal comfort.

[0005] Existing assessment methods are mostly single-frame or short-time analyses, which fail to capture the temporal evolution characteristics of thermal comfort, such as gradually increasing heart rate and accumulated sweating. They cannot predict thermal discomfort and miss the opportunity for early intervention.

[0006] The existing system only outputs evaluation results or alarm prompts, lacks a linkage mechanism with environmental control equipment such as air conditioning and ventilation, and cannot dynamically adjust environmental parameters according to real-time thermal comfort status, resulting in low adjustment efficiency and difficulty in guaranteeing the exercise experience. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method and storage medium for detecting and adjusting thermal comfort during exercise. This method enables real-time comprehensive detection of the thermal comfort state of athletes and dynamically adjusts environmental equipment accordingly, forming a closed-loop feedback. To achieve the above objective, this invention employs the following technical solution:

[0008] In a first aspect, the present invention provides a method for detecting and adjusting thermal comfort during exercise, comprising:

[0009] Acquire multimodal data of the athlete during exercise; the multimodal data includes motion images, physiological signals, and environmental parameters;

[0010] The motion image is input into a pre-trained first evaluation model to obtain a first thermal comfort score based on visual posture.

[0011] The physiological signals are input into a pre-trained second evaluation model to obtain a second thermal comfort score based on the physiological signals.

[0012] The first and second thermal comfort scores are weighted and fused to obtain a comprehensive thermal comfort index.

[0013] Based on the comparison results between the comprehensive thermal comfort index and the preset comfort range, and in conjunction with the environmental parameters, control commands for the environmental adjustment equipment are output to dynamically adjust the local environmental parameters of the area where the athlete is located.

[0014] In conjunction with the first aspect, the motion image may optionally include visible light video images and infrared thermal imaging images.

[0015] In conjunction with the first aspect, optionally, the first evaluation model includes a human skeletal joint detection network and a bimodal pose detection network; inputting the motion image into the pre-trained first evaluation model yields a first thermal comfort score based on visual pose, including:

[0016] The visible light video image is input into the human skeleton joint point detection network to extract the visible light skeleton key point sequence.

[0017] The visible light skeleton key point sequence, visible light video image and infrared thermal imaging image are input into the dual-modal posture detection network to obtain the first thermal comfort score.

[0018] In conjunction with the first aspect, optionally, the dual-modal attitude detection network includes an infrared branch and a visible light branch arranged in parallel;

[0019] The visible light branch is used to extract features from visible light video images and visible light skeleton key point sequences, and output a pose representation vector;

[0020] The infrared branch is used to extract features from the infrared thermal imaging image and output a thermal load characterization vector;

[0021] The thermal load representation vector and the posture representation vector are fused to output the first thermal comfort score.

[0022] In conjunction with the first aspect, optionally, the second evaluation model is a physiological data fusion network used to process time series of multiple physiological signals;

[0023] The physiological data fusion network includes:

[0024] For each physiological signal, the original time series is segmented into fragments and mapped to a token sequence;

[0025] We extract local temporal dependency features and global statistical features from the token sequence using parallel convolutional and channel compression branches, and then fuse the outputs of the two branches.

[0026] The fused sequence is input into the Transformer encoder to capture long-range dependencies within and between signals.

[0027] The global feature vectors corresponding to each physiological signal are fused, and then regressed and mapped through a fully connected layer to output the second thermal comfort score.

[0028] In conjunction with the first aspect, optionally, the first thermal comfort score and the second thermal comfort score are weighted and fused to obtain a comprehensive thermal comfort index, which is expressed by the following formula:

[0029] ,

[0030] in, To comprehensively assess thermal comfort, It received the highest thermal comfort rating. It received the second highest thermal comfort rating. The weighting parameter for the first thermal comfort score. The weighting parameter for the second thermal comfort score.

[0031] In conjunction with the first aspect, optionally, the environmental parameters include air temperature and wind speed; the environmental control equipment includes air conditioning and a ventilation system;

[0032] The step of outputting control commands for the environmental conditioning equipment based on the comparison results between the comprehensive thermal comfort index and the preset comfort range, combined with the environmental parameters, includes:

[0033] When the overall thermal comfort index exceeds the upper limit of the preset comfort range, a control command is output to reduce the temperature setting value of the area where the athlete is located and increase the air supply speed.

[0034] When the overall thermal comfort index is lower than the lower limit of the preset comfort range, a control command is output to increase the temperature setpoint and decrease the air supply speed.

[0035] In conjunction with the first aspect, optionally, it also includes:

[0036] Based on the time series of historical comprehensive thermal comfort indices and environmental parameters, a reinforcement learning model is used to predict the changing trend of thermal comfort of athletes.

[0037] When the prediction results indicate that the overall thermal comfort index will exceed the preset comfort range at a future time, the pre-control command of the environmental conditioning equipment will be output in advance.

[0038] In conjunction with the first aspect, optionally, it also includes:

[0039] Based on environmental sensor data distributed in multiple locations within the sports venue, spatial interpolation reconstruction is performed to obtain the local air temperature and wind speed at the location of the athlete during exercise.

[0040] Based on the deviation between the comprehensive thermal comfort index and the preset comfort zone, as well as the local air temperature and wind speed, differentiated control commands are output for the area where the athlete is located.

[0041] In a second aspect, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the motion thermal comfort detection and feedback adjustment method described in the first aspect.

[0042] Compared with the prior art, the beneficial effects achieved by the motion thermal comfort detection and feedback adjustment method and storage medium provided in the embodiments of the present invention include:

[0043] This invention acquires multimodal data of athletes during exercise; the multimodal data includes motion images, physiological signals and environmental parameters; this invention overcomes the technical defects of existing technologies, such as the inability of single environmental sensing to reflect individual state, the susceptibility of single-modal physiological monitoring to interference, and the difficulty of quantifying heat load by single visual monitoring;

[0044] The present invention inputs the motion image into a pre-trained first evaluation model to obtain a first thermal comfort score based on visual posture; the present invention realizes the collaborative analysis of visual posture and body surface temperature distribution;

[0045] This invention inputs the physiological signals into a pre-trained second evaluation model to obtain a second thermal comfort score based on the physiological signals. The invention uses a Transformer encoder in the second evaluation model to perform long-range dependency modeling on the time series of physiological signals, enabling the capture of the temporal evolution characteristics of thermal comfort states. Furthermore, this invention uses a reinforcement learning model to predict the changing trends of thermal comfort based on historical data, achieving proactive prediction of thermal discomfort states. This solves the problem that existing technologies can only perform single-frame or short-term analysis, missing early intervention opportunities, and provides a technical foundation for predictive environmental control.

[0046] This invention weights and fuses the first and second thermal comfort scores to obtain a comprehensive thermal comfort index. Based on the comparison between the comprehensive thermal comfort index and the preset comfort range, and combined with the environmental parameters, it outputs control commands for the environmental conditioning equipment to dynamically adjust the local environmental parameters of the area where the athlete is located. This invention achieves closed-loop feedback from perception and evaluation to control. Through bidirectional adjustment logic, directional air supply control, and spatial interpolation reconstruction, this invention can dynamically adjust the temperature setpoint and air supply speed according to the athlete's real-time position, movement direction, and local environmental differences, enabling the air conditioner to precisely target the high-sweat areas of the torso. Compared with existing technologies that only output evaluation results or alarm prompts and lack linkage control capabilities, this invention achieves proactive intervention, significantly improving the athlete's thermal comfort experience and environmental conditioning efficiency.

[0047] This invention enables real-time comprehensive detection of the thermal comfort state of athletes and dynamically adjusts environmental equipment accordingly, forming a closed-loop feedback. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a method for detecting and adjusting thermal comfort during exercise according to Embodiment 1 of the present invention.

[0049] Figure 2 This is a schematic diagram of the human skeletal joint detection network in a method for detecting and adjusting thermal comfort during exercise according to Embodiment 1 of the present invention.

[0050] Figure 3 This is a schematic diagram of the structure of the dual-modal posture detection network in a motion thermal comfort detection and feedback adjustment method according to Embodiment 1 of the present invention;

[0051] Figure 4 This is a schematic diagram of the physiological data fusion network in a method for detecting and regulating thermal comfort during exercise, as described in Embodiment 1 of the present invention. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0053] Example 1:

[0054] This embodiment provides a method for detecting and adjusting thermal comfort during exercise, including:

[0055] Acquire multimodal data of the athlete during exercise; the multimodal data includes motion images, physiological signals, and environmental parameters;

[0056] The motion image is input into a pre-trained first evaluation model to obtain a first thermal comfort score based on visual posture.

[0057] The physiological signals are input into a pre-trained second evaluation model to obtain a second thermal comfort score based on the physiological signals.

[0058] The first and second thermal comfort scores are weighted and fused to obtain a comprehensive thermal comfort index.

[0059] Based on the comparison results between the comprehensive thermal comfort index and the preset comfort range, and in conjunction with the environmental parameters, control commands for the environmental adjustment equipment are output to dynamically adjust the local environmental parameters of the area where the athlete is located.

[0060] like Figure 1 As shown, the specific steps are as follows.

[0061] Step 1: Acquire multimodal data of the athlete during the exercise process.

[0062] Multimodal data acquisition devices deployed in sports venues are used to simultaneously collect multimodal data of athletes during exercise, including motion images, physiological signals, and environmental parameters.

[0063] Moving images include visible light video images and infrared thermal imaging images.

[0064] Specifically, a visible light RGB camera is used to capture continuous motion images of athletes on the field for human skeletal key point extraction and posture analysis. An infrared thermal imaging camera is used to obtain surface temperature distribution maps of athletes, reflecting changes in thermal radiation under different exercise intensities.

[0065] Physiological signals include heart rate, body temperature, and the intensity of sweat secretion.

[0066] Specifically, athletes wear chest-strap heart rate sensors (such as the Polar H10) to collect heart rate data in real time. Wearable body temperature sensors monitor real-time body temperature in the torso and limbs. Patch-type sweat sensors quantify sweat secretion intensity as an important indicator of heat discomfort, quantifying the athlete's heat dissipation status during high-intensity exercise.

[0067] Environmental parameters include air temperature and wind speed.

[0068] Environmental monitoring nodes are set up at key locations in the sports venue (such as under the basket, sideline, and center) to collect data such as air temperature, humidity, and wind speed in real time.

[0069] All modal data were aligned using hardware synchronization triggers or timestamps. Subjective thermal sensations were asked of the exercisers every 5-10 minutes, and ASHRAE seven-level thermal sensation votes (-3 extremely cold to +3 extremely hot) were recorded. At the same time, behavioral characteristics of thermal discomfort (wiping sweat, taking off clothes, drinking water, fanning) were marked as supervision labels.

[0070] Finally, ensure that all modal data (videos, physiological signals, environmental parameters, etc.) are temporally aligned. The collected multimodal data is preprocessed into a unified format: time series (heart rate, skin temperature, sweat intensity, environmental parameters), spatial distribution map (surface temperature distribution generated by a thermal infrared camera), pose sequence (skeletal keypoints extracted by a visible light RGB camera), and annotation information. The annotation information consists of supervisory labels for subjective thermal voting and behavioral markers. This unified format facilitates subsequent feature extraction and model training, ensuring data compatibility and usability within deep learning frameworks.

[0071] This embodiment overcomes the technical defects of the prior art, such as the inability of a single environmental sensor to reflect an individual's state, the susceptibility of single-modal physiological monitoring to interference, and the difficulty of quantifying heat load with single visual monitoring.

[0072] Step 2: Input the motion image into the pre-trained first evaluation model to obtain the first thermal comfort score based on visual posture.

[0073] The first evaluation model includes a human skeletal joint detection network and a bimodal pose detection network.

[0074] Step 2.1: Input the visible light video image into the human skeleton joint detection network to extract the visible light skeleton key point sequence.

[0075] Human skeletal keypoint detection networks are used to extract skeletal keypoint sequences from visible light video images. For example... Figure 2 As shown, the human skeletal joint detection network includes four hierarchical modules: an initial feature extraction module, a dual-branch feature extraction module, a fusion module, and a feature convergence and output module.

[0076] The initial feature extraction module performs convolution and downsampling on the input visible light video image or infrared thermal imaging image, outputting a backbone feature map. Input: Visible light image. First, it goes through a 7×7 convolutional layer to compress the spatial size to... Then, through batch normalization and ReLU activation, followed by 3×3 group convolution and 1×1 convolution, the backbone feature map with a uniform number of channels C is obtained.

[0077] like Figure 2 As shown, the initial feature extraction module consists of two similar structures (including convolutional layers, batch normalization, and ReLU activation function).

[0078] Input visible light image ( Image height, Image width, The input visible light image (number of channels) first passes through a 7×7 convolutional layer with stride s=2 and padding p=3, expressed by the following formula:

[0079] ,

[0080] in, To input the pixel values ​​of a visible light image, For the input visible light image, the first Input channels, spatial location The value at that location. The first feature map output after processing by the convolutional layer Each output channel, spatial location The value at that location. The weights of the convolution kernel, with values ​​ranging from 1 to 2. ; The index of the output channel in the output feature map, i.e., the index of the output channel in the output feature map. One output channel, with a value range of [value range missing]. , This represents the number of output channels for this convolutional layer. The index of the input channel in the input visible light image, i.e., the index of the input visible light image. There are one input channel, with a value range of [value range missing]. , This represents the number of input channels for this convolutional layer. This represents the offset within the convolutional kernel, corresponding to a 7×7 convolutional layer, with a value range of [value range missing]. . For the first The bias term corresponding to each output channel. To output the spatial coordinates of the feature map, given a stride of s=2 and padding p=3, the output size is... .

[0081] The output feature map of the convolutional layer is input into batch normalization. Batch normalization is performed on the output of each channel. If each input channel has N samples, the batch normalization formula is:

[0082] ,

[0083] in, The first feature map output after batch normalization layer processing is the first feature map. Each output channel, spatial location The value at that location, For the first The average value of each output channel. For the first The variance of each output channel It is a constant used to avoid division by zero errors.

[0084] The batch-normalized feature map is then subjected to a nonlinear transformation on each element using the ReLU activation function:

[0085] ,

[0086] in, The first feature map output after processing by the ReLU activation function is the... Each output channel, spatial location The value at that location.

[0087] Next, a grouped convolutional layer is performed, with a kernel size of [size missing]. The number of input channels in each group is The grouped convolutional layers with stride s=1 and the ReLU function deepen the nonlinear representation of the network, enabling the model to fit more complex spatial patterns, such as subtle structures like edges, textures, and poses.

[0088] The resulting feature map P1 flows into the initialization layer, which mainly consists of repeatedly stacked Bottleneck structures used to adjust the number of channels in the feature map without changing its spatial size.

[0089] The dual-branch feature extraction module includes a fine-grained branch and a semantic branch; the fine-grained branch is used to extract local detail features, while the semantic branch is used to extract global skeleton features.

[0090] like Figure 2 As shown, the upper branch is a fine-grained branch, in At scale, feature map P1 is spatially captured using 3×3 convolution to extract low-level features such as edges and textures; then, image patch embedding is used to segment the convolved feature map into... Small blocks are created and each block is mapped to a vector space of fixed dimension d.

[0091] After layer normalization and linear layers, the image data is transformed from its original matrix form into a sequence of tokens that can participate in global correlation modeling. For each token p, the output is... The calculation formula is as follows:

[0092] ,

[0093] in: For the first layer after layer normalization The token in the 1st Embedded values ​​in each dimension; For the weight matrix of the linear layer, the input dimension is... Mapping to output dimension ; For the bias of the linear layer, The dimension of the embedded vector; The channel index for the output feature.

[0094] Through the linear mapping described above, each token is converted into a... 3D eigenvectors.

[0095] Next, the obtained feature vectors are input into two processing paths in parallel. The first path passes through the Global Context Modeling (GCM) module twice to enhance the global semantics, and then performs upsampling. Let the input of the GCM module be... (shape is) ,in (where the total number of tokens is), then output Calculated using the attention mechanism:

[0096] ,

[0097] ,

[0098] in, For the first The token in the 1st Input features on each channel and These are the intermediate features obtained by mapping the input features through a small convolutional layer, and are located at the th... The token and the first The position corresponding to the token The values ​​in each dimension are used to calculate spatial feature similarity; For activation functions; For the first The token and the first The normalized similarity between tokens is converted into attention weights using softmax. For the total number of tokens, This represents the total number of dimensions.

[0099] This mechanism uses global similarity aggregation to enable each token to not only retain its own information, but also to integrate features from other tokens that are semantically similar to it globally, thereby enhancing the semantic expressive power of the features.

[0100] The second path directly upsamples to preserve the original local information.

[0101] The high-resolution features obtained from the two pathways are multiplied element-wise, then channel-projected through a 1×1 convolution, and layer normalization is applied to stabilize the feature distribution. Finally, downsampling is used to restore the fused features to the input resolution, resulting in fine-grained features. .

[0102] like Figure 2 As shown, the structure of the semantic branch is basically the same as that of the fine-grained branch, the difference being that the input features are first downsampled to... To condense global skeleton information, it is then upsampled back after undergoing similar dual-path processing. semantic features are obtained. This ensures that the scale is consistent with the output of the fine-grained branches.

[0103] The fusion module is used to perform self-attention weighted fusion of the outputs of the fine-grained branch and the semantic branch. The fine-grained branch output is then fused... and semantic branch output The two sets of features are generated by applying a self-attention mechanism to reweight the two paths from a global perspective and then cross-stacking them. and .

[0104] Specifically, in the fusion module, the fine-grained branch output is first processed. and semantic branch output Perform self-attention calculations separately:

[0105] ,

[0106] ,

[0107] ,

[0108] ,

[0109] ,

[0110] in, For the first Feature maps output by each branch These correspond to the features output by the fine-grained branches. Features output by semantic branches All dimensions are The number of channels is ; , , The first A matrix of queries, keys, and values ​​for each branch; , , These are the learnable linear mapping parameter matrices used in the self-attention mechanism to generate queries, keys, and values; For the first The attention weight matrix of each branch is normalized by softmax and then used for weighted summation. Scaling factor The number of feature channels (or the dimension of the attention head) is used to prevent the softmax gradient from saturating due to an excessively large inner product. For the first The output features of each branch after self-attention weighting.

[0111] This self-attention mechanism reweights the two sets of features from a global perspective, resulting in fused features. and .

[0112] The feature aggregation and output module is used to refine the residuals of the two sets of fused results on four parallel paths.

[0113] like Figure 2 As shown, there are four parallel paths:

[0114] First route: Sampling Four basic residual blocks are concatenated, and finally max pooling is used to downsample back to the principal scale.

[0115] The second approach: perform convolution directly on the main scale and concatenate 4 residual blocks;

[0116] Third approach: First sample Perform residual refinement and then upsampling. ;

[0117] The fourth approach: Convolve first on the main scale and then refine the residuals.

[0118] Precisely align the four outputs to the same size. The elements are then added together and merged element by element to obtain a unified feature map. Finally, the feature map is grouped... Convolution maps the feature map to A heatmap of key skeletal points, in which This represents the number of keypoint categories. The heatmap stably and intuitively represents the location of each skeletal keypoint in a spatial distribution form, and can be directly input into the visible light branch of the downstream motion thermal comfort posture detection network (DualModalNet) as prior guiding information.

[0119] Step 2.2: Input the visible light skeleton key point sequence, visible light video image and infrared thermal imaging image into the dual-modal posture detection network to obtain the first thermal comfort score.

[0120] A dual-modal attitude detection network, comprising parallel infrared and visible light branches;

[0121] The visible light branch is used to extract features from visible light video images and visible light skeleton keypoint sequences, and outputs a pose representation vector;

[0122] The infrared branch is used to extract features from the infrared thermal imaging image and output a thermal load characterization vector;

[0123] The thermal load representation vector and the posture representation vector are fused to output the first thermal comfort score.

[0124] like Figure 3 As shown, the infrared branch includes:

[0125] The residual network is used to extract features from the infrared thermal imaging image, expand the number of channels while maintaining spatial resolution, and output high-dimensional features including temperature gradient and hot spot distribution.

[0126] The scale-balanced pyramid convolution module is used to apply multiple convolution kernels with different receptive fields in parallel at the same scale to process the high-dimensional features, and then fuse the processed features to output multi-scale fused features.

[0127] The dual-path feature aggregation module includes:

[0128] The first path involves performing global average pooling on the multi-scale fusion features and then mapping them through a fully connected layer to obtain the overall heat accumulation features.

[0129] The second approach involves processing the multi-scale fused features using a multilayer perceptron to obtain local thermal stress features.

[0130] The overall heat accumulation characteristics are added to the local heat stress characteristics to obtain the heat load characterization vector output by the infrared branch.

[0131] Specifically, such as Figure 3 As shown, the residual network is input to the infrared thermal imaging image. By maintaining spatial resolution through multi-layer residual blocks, the number of channels is expanded to C, and high-dimensional features including temperature gradients and hot spot distribution are extracted.

[0132] Specifically, such as Figure 3 As shown, scale-balanced pyramid convolutional modules are applied in parallel at the same scale. convolution, convolution, Convolution concatenates the outputs of each branch along the channel dimension and downsamples them to... After Convolution compression to This enables the integration of multi-scale information.

[0133] Specifically, such as Figure 3 As shown, the first path of the dual-path feature aggregation module performs batch normalization and ReLU activation on the multi-scale fused features to purify the effective signals, and then uses global average pooling to... Feature mapping to The overall heat accumulation characteristics are then obtained through fully connected layer mapping and 1×1 convolution.

[0134] The specific process is as follows: Let the input feature map be... ,in , Feature maps Height and width, For feature map The number of channels. For the input feature map Batch normalization is performed and the ReLU activation function is applied to obtain the feature map. .

[0135] For each channel Calculate the average value of the spatial dimensions to obtain the pooling vector. :

[0136] ,

[0137] in, For feature map at location The activation value on channel c.

[0138] The pooling vector m is transformed into the output vector through a fully connected layer. :

[0139] ,

[0140] in, For the weight matrix, the input dimensions are... Mapping to output dimension (usually taken) (to align with the output dimension of the other path). This is the bias vector.

[0141] Specifically, such as Figure 3 As shown, the second path of the dual-path feature aggregation module extracts deep thermal stress features from multi-scale fused features using GeLU activation and a multilayer perceptron (MLP), and is also compressed to... .

[0142] The specific process is as follows: Let the input feature map be... ,in , , For feature map The number of channels. For the input feature map Perform spatial global average pooling to obtain the pooled feature vector. .

[0143] The pooled feature vector Input a two-layer fully connected network, using the GeLU activation function in between, to obtain local thermal stress features:

[0144] ,

[0145] Wherein, H represents the characteristics of local thermal stress; The weight matrix of the first fully connected layer is the input dimension. Mapped to the output dimension; This is the bias vector of the first fully connected layer. For the hidden layer dimension; The weight matrix of the second fully connected network is the input dimension. Mapping to output dimension ; This is the bias vector for the second fully connected layer; The Gaussian error function; It is a vector consisting entirely of 1s.

[0146] The overall heat accumulation characteristics and local heat stress characteristics output from the two paths are added element-wise along the channel dimension to obtain the heat load characterization vector of the infrared branch output.

[0147] like Figure 3 As shown, the visible light branch includes:

[0148] Feature extraction network, with input size of The visible light video image and the output size of step 2.1 are as follows: The visible light skeleton keypoint sequence is spliced ​​together along the channel dimension and fused to form a size of The joint feature input is then passed through a residual network of the same depth as the infrared branch and a pyramid convolution module, which also outputs the joint feature tensor. The tensor is then used to adjust the number of tensor channels through a linear mapping. And obtained by 1×1 convolution Primary pose feature map;

[0149] The spatial selection module, since the joint features from the previous input already contain explicit skeletal keypoint location information, can accurately focus on the human skeleton region when calculating spatial attention weights through a small convolutional layer. It then generates a mask K of the same size as the feature map using the Sigmoid function. This mask is then added element-wise to the original pose features to obtain... The focus is on pose features; this feature map is then compressed using a 1×1 convolution. The vectors are used to achieve global aggregation of human posture information in the spatial dimension;

[0150] After completing the two-way feature extraction, the global convergence module concatenates the infrared thermal judgment vector and the visible light attitude vector along the feature dimension to form a... The fusion tensor is compressed to [value] through a 1×1 convolution. Batch normalization is performed on the channels to eliminate scale differences between different modes.

[0151] Finally, this fused vector is regressed and mapped through a fully connected network layer, directly outputting a continuous scalar value as the first thermal comfort score based on visual pose. This enables real-time, continuous quantitative determination of the thermal discomfort status of the moving subject in the current frame.

[0152] This embodiment achieves collaborative analysis of visual posture and body surface temperature distribution.

[0153] Step 3: Input the physiological signals into the pre-trained second evaluation model to obtain a second thermal comfort score based on the physiological signals.

[0154] like Figure 4 As shown, in this embodiment, physiological signals include heart rate signals, body temperature signals, and sweat secretion intensity.

[0155] The physiological data fusion network specifically includes the following steps.

[0156] Step 3.1: Patch Embedding stage.

[0157] For each physiological signal, the original time series is divided into equal-length segments, and each segment is mapped to a token sequence of dimension D through an embedding layer, reducing the time length to T′=⌈T / p⌉, where p is the number of segments.

[0158] This can be expressed by the following formula:

[0159] ,

[0160] in, This is the original time series; For flattening operation; The weight matrix for Patch Embedding; This is the bias vector for Patch Embedding. This is the embedded token vector.

[0161] The embedded token vector is normalized to obtain the token sequence, which is represented by the following formula:

[0162] ,

[0163] in, This is the normalized vector; To Perform normalization processing; for The mean; for Standard deviation; It is a very small constant used to prevent division by zero; and All of these are learnable scaling parameters.

[0164] Step 3.2: For each token sequence, extract local temporal dependency features and global statistical features from the token sequence through parallel convolutional branches and channel compression branches, and fuse the outputs of the two branches.

[0165] The first branch is the causal convolution branch, which extracts local temporal dependencies through causal convolution, weight normalization, ReLU activation extraction, and Dropout (regularization). The output remains as Sequence characteristics.

[0166] in, This refers to the length of the time segment after compression. The dimension of the token sequence (original dimension).

[0167] This can be expressed by the following formula:

[0168] ,

[0169] ,

[0170] ,

[0171] ,

[0172] in, For causal convolution output, The weights of the causal convolution kernel, For convolution operations, To be Transpose to a format suitable for convolution. For convolution bias;

[0173] in, This is the weighted convolution output. A learnable scaling factor. The original weight vector is learnable. For vectors L2 norm, The bias for weight normalization;

[0174] in, Features after ReLU activation To be Transpose to a format suitable for processing;

[0175] in, The features output after Dropout (regularization) are local temporal dependencies; A random mask for Dropout (regularization), with elements 0 or 1, following a Bernoulli distribution, such that each element has a probability of Set to zero, with probability reserve; This represents the dropout probability (using regularization) for Dropout. This is for element-wise multiplication.

[0176] The second branch is a channel compression branch, which extracts fragment-level global statistical information through 1×1 convolution, and the output is also... Sequence characteristics.

[0177] The outputs of the first and second branches are added together along the channel dimension to obtain a sequence feature that integrates local and global information.

[0178] Step 3.3: Input the fused sequence into the Transformer encoder. Multi-head self-attention operation and feedforward mapping are performed on the feature sequences of time length to capture long-range dependencies within and between signals.

[0179] Subsequently, after mapping the features back to the same dimensions as the original signal through an inverse normalization operation, a linear head performs global aggregation on the sequence features, and... The sequence features are compressed into a The global vector. Where, The dimension of the token sequence (the dimension after processing by the Transformer encoder). The dimension of the global vector.

[0180] Step 3.4: Fuse the global feature vectors corresponding to each physiological signal, perform regression mapping through a fully connected layer, and output the second thermal comfort score.

[0181] After repeating the above process for heart rate, sweat secretion intensity, and body temperature signals, three global feature vectors are obtained respectively. To achieve multimodal fusion, these three vectors are concatenated along their feature dimensions to obtain a fused vector. This fused vector is then processed through global average pooling, normalization, a fully connected layer, and an activation function to output the second thermal comfort score. .

[0182] This embodiment uses the Transformer encoder in the second evaluation model to perform long-range dependency modeling on the time series of physiological signals, which can capture the temporal evolution characteristics of thermal comfort state.

[0183] Step 4: Weight and fuse the first thermal comfort score and the second thermal comfort score to obtain the comprehensive thermal comfort index.

[0184] This can be expressed by the following formula:

[0185] ,

[0186] in, To comprehensively assess thermal comfort, It received the highest thermal comfort rating. It received the second highest thermal comfort rating. The weighting parameter for the first thermal comfort score. The weighting parameter for the second thermal comfort score.

[0187] Step 5: Based on the comparison results between the comprehensive thermal comfort index and the preset comfort zone, and in conjunction with the environmental parameters, output the control command of the environmental adjustment device to dynamically adjust the local environmental parameters of the area where the athlete is located.

[0188] Environmental control equipment includes air conditioning and air supply systems.

[0189] Will Thermal comfort zone recommended by ASHRAE 55 standard By comparing and combining environmental parameters (temperature, wind speed), control commands for the environmental control equipment are output, including:

[0190] When the overall thermal comfort index exceeds the upper limit of the preset comfort range, a control command is output to reduce the temperature setting value of the area where the athlete is located and increase the air supply speed.

[0191] When the overall thermal comfort index is lower than the lower limit of the preset comfort range, a control command is output to increase the temperature setpoint and decrease the air supply speed.

[0192] Specifically, When the temperature in the area where the athlete is located is reduced by 0.5~1.0℃ and the air supply speed is increased by 0.2~0.5m / s, a control command is output; When the temperature is set to increase by 0.5~1.0℃ and decrease the air supply speed by 0.2~0.5m / s, the system outputs a control command.

[0193] It should be noted that ASHRAE 55 is a thermal comfort standard developed by the American Society of Heating, Refrigeration and Air-Conditioning Engineers.

[0194] Specifically, it also includes directional air supply:

[0195] Based on the athlete's real-time position and movement direction, an instruction is generated to adjust the airflow direction, so that the cooling airflow is directed to the athlete's torso and areas with high sweating.

[0196] Specifically, this also includes differentiation control:

[0197] Based on environmental sensor data distributed in multiple locations within the sports venue, spatial interpolation reconstruction is performed to obtain the local air temperature and wind speed at the location of the athlete during exercise.

[0198] Based on the deviation between the comprehensive thermal comfort index and the preset comfort zone, as well as the local air temperature and wind speed, differentiated control commands are output for the area where the athlete is located.

[0199] Specifically, it also includes predictive control:

[0200] Based on the time series of historical comprehensive thermal comfort indices and environmental parameters, a reinforcement learning model is used to predict the changing trend of thermal comfort of athletes.

[0201] When the prediction results indicate that the overall thermal comfort index will exceed the preset comfort range at a future time, the pre-control command of the environmental conditioning equipment will be output in advance.

[0202] This embodiment uses a reinforcement learning model to predict the trend of thermal comfort changes based on historical data, thereby enabling advanced prediction of thermal discomfort. This solves the problem that existing technologies can only perform single-frame or short-term analysis and miss the opportunity for early intervention, providing a technical foundation for predictive environmental control.

[0203] In this embodiment, control commands are sent to the air conditioning and ventilation system via a message queue and evaluated every 30 seconds to avoid frequent fluctuations.

[0204] This embodiment achieves closed-loop feedback from perception and evaluation to control. Through bidirectional adjustment logic, directional air supply control, and spatial interpolation reconstruction, it can dynamically adjust the temperature setpoint and air supply speed according to the athlete's real-time position, movement direction, and local environmental differences, so that the air conditioner can accurately act on the high-sweat areas of the torso.

[0205] Compared with existing technologies that only output evaluation results or alarm prompts and lack linkage control capabilities, this embodiment achieves active intervention, significantly improving the thermal comfort experience of athletes and the efficiency of environmental regulation.

[0206] This embodiment can realize real-time comprehensive detection of the thermal comfort state of athletes and dynamically adjust environmental equipment accordingly, forming a closed-loop feedback.

[0207] Example 2:

[0208] This embodiment provides a computer-readable storage medium storing a computer program / instruction thereon. When the computer program / instruction is executed by a processor, it implements the steps of the motion thermal comfort detection and feedback adjustment method described in Embodiment 1.

[0209] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0210] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0211] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0212] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0213] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for detecting and adjusting thermal comfort during exercise, characterized in that, include: Acquire multimodal data of the athlete during exercise; the multimodal data includes motion images, physiological signals, and environmental parameters; The motion image is input into a pre-trained first evaluation model to obtain a first thermal comfort score based on visual posture. The physiological signals are input into a pre-trained second evaluation model to obtain a second thermal comfort score based on the physiological signals. The first and second thermal comfort scores are weighted and fused to obtain a comprehensive thermal comfort index. Based on the comparison results between the comprehensive thermal comfort index and the preset comfort range, and in conjunction with the environmental parameters, control commands for the environmental adjustment equipment are output to dynamically adjust the local environmental parameters of the area where the athlete is located.

2. The method for detecting and adjusting thermal comfort during exercise according to claim 1, characterized in that, The moving images include visible light video images and infrared thermal imaging images.

3. The method for detecting and adjusting thermal comfort during exercise according to claim 2, characterized in that, The first evaluation model includes a human skeletal joint detection network and a bimodal pose detection network; The motion image is input into a pre-trained first evaluation model to obtain a first thermal comfort score based on visual posture, including: The visible light video image is input into the human skeleton joint point detection network to extract the visible light skeleton key point sequence. The visible light skeleton key point sequence, visible light video image and infrared thermal imaging image are input into the dual-modal posture detection network to obtain the first thermal comfort score.

4. The method for detecting and adjusting thermal comfort during exercise according to claim 3, characterized in that, The dual-modal attitude detection network includes parallel infrared and visible light branches; The visible light branch is used to extract features from visible light video images and visible light skeleton key point sequences, and output a pose representation vector; The infrared branch is used to extract features from the infrared thermal imaging image and output a thermal load characterization vector; The thermal load representation vector and the posture representation vector are fused to output the first thermal comfort score.

5. The method for detecting and adjusting thermal comfort during exercise according to claim 1, characterized in that, The second evaluation model is a physiological data fusion network, used to process time series of various physiological signals; The physiological data fusion network includes: For each physiological signal, the original time series is segmented into fragments and mapped to a token sequence; We extract local temporal dependency features and global statistical features from the token sequence using parallel convolutional and channel compression branches, and then fuse the outputs of the two branches. The fused sequence is input into the Transformer encoder to capture long-range dependencies within and between signals. The global feature vectors corresponding to each physiological signal are fused, and then regressed and mapped through a fully connected layer to output the second thermal comfort score.

6. The method for detecting and adjusting thermal comfort during exercise according to claim 1, characterized in that, The first and second thermal comfort scores are weighted and fused to obtain a comprehensive thermal comfort index, which is expressed by the following formula: ， in, To comprehensively assess thermal comfort, It received the highest thermal comfort rating. It received the second highest thermal comfort rating. The weighting parameter for the first thermal comfort score. The weighting parameter for the second thermal comfort score.

7. The method for detecting and adjusting thermal comfort during exercise according to claim 1, characterized in that, The environmental parameters include air temperature and wind speed; the environmental control equipment includes air conditioning and air supply systems. The step of outputting control commands for the environmental conditioning equipment based on the comparison results between the comprehensive thermal comfort index and the preset comfort range, combined with the environmental parameters, includes: When the overall thermal comfort index exceeds the upper limit of the preset comfort range, a control command is output to reduce the temperature setting value of the area where the athlete is located and increase the air supply speed. When the overall thermal comfort index is lower than the lower limit of the preset comfort range, a control command is output to increase the temperature setpoint and decrease the air supply speed.

8. The method for detecting and adjusting thermal comfort during exercise according to claim 7, characterized in that, Also includes: Based on the time series of historical comprehensive thermal comfort indices and environmental parameters, a reinforcement learning model is used to predict the changing trend of thermal comfort of athletes. When the prediction results indicate that the overall thermal comfort index will exceed the preset comfort range at a future time, the pre-control command of the environmental conditioning equipment will be output in advance.

9. The method for detecting and adjusting thermal comfort during exercise according to claim 7, characterized in that, Also includes: Based on environmental sensor data distributed in multiple locations within the sports venue, spatial interpolation reconstruction is performed to obtain the local air temperature and wind speed at the location of the athlete during exercise. Based on the deviation between the comprehensive thermal comfort index and the preset comfort zone, as well as the local air temperature and wind speed, differentiated control commands are output for the area where the athlete is located.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the motion thermal comfort detection and feedback adjustment method as described in any one of claims 1-9.

Images