Non-contact athlete physiological status assessment method and system based on feature fusion and medium

Abstract

The invention discloses a non-contact athlete physiological status assessment method and system based on feature fusion and a medium, and relates to multiple technical fields of computer vision, image processing, physiological signal detection and the like, and the method comprises the following steps: S1, constructing an rPPG signal feature extraction model, extracting rPPG signal features from the preprocessed video through an rPPG signal feature extraction model; s2, constructing and training an information and action feature extraction model, extracting information features and action features of the athletes from the preprocessed video, and fusing the information features and the action features to obtain motion information features; and S3, constructing a multi-source signal fusion network based on a deep learning model, and performing continuous prediction on the fatigue degree of the athlete by taking the obtained rPPG signal features and the motion information features as input. According to the method, the rPPG signal and biomechanical parameter characteristics are effectively extracted, and the fatigue degree of an athlete is predicted in combination with a characteristic coding technology and a characteristic fusion technology.

Description

Technical Field

[0001] The present invention relates to multiple technical fields such as computer vision, image processing and physiological signal detection, and in particular to a non-contact athlete physiological status assessment method, system and medium based on feature fusion. Background Art

[0002] Remote photoplethysmography (rPPG) uses sensors such as cameras to capture the cyclical changes in skin color caused by the cardiac cycle. By capturing these subtle color changes, the blood volume pulse signal (BVP) can be extracted, allowing the measurement of cardiac cycle-related physiological indicators such as heart rate (HR), respiratory rate (RR), and heart rate variability (HRV). rPPG technology plays a crucial role in scenarios requiring contactless physiological parameter monitoring.

[0003] Heart rate, heart rate variability, and blood oxygen saturation are important indicators of autonomic nervous system activity. By analyzing the time and frequency domain characteristics of rPPG signals, we can assess trends in core indicators such as heart rate (HR), heart rate variability (HRV), and blood oxygen saturation (SpO2), thereby understanding an individual's stress response and health status. Video signals can be used to infer parameters such as exercise load (accumulated sprint distance, average speed), and movement characteristics (number of changes of direction, jump height), which help assess an athlete's fatigue level. For example, fatigued athletes exhibit signs of heavy breathing, elevated heart rate, and slower speed. Therefore, integrating these multiple indicators can yield more accurate and comprehensive information. Multimodal fusion analysis methods not only improve the accuracy and reliability of athlete activity assessments but also provide strong support for subsequent health management.

[0004] Currently, basketball teams lack tools to monitor athlete fatigue in real time during games and rely primarily on wearable devices (such as chest straps or watches) to monitor heart rate and HRV. Changes in low-frequency / high-frequency heart rate variability can reflect the fatigue state of the autonomic nervous system (such as increased sympathetic nerve activity). Wearable devices can cause discomfort in some athletes, making it impossible for every athlete to wear them accurately, resulting in the coaching staff being unable to obtain data from some athletes. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a non-contact athlete physiological status assessment method, system and medium based on feature fusion.

[0006] The object of the present invention is achieved through the following technical solutions: In a first aspect, the present invention discloses a non-contact athlete physiological state assessment method based on feature fusion, comprising the following steps: S1. Collect and preprocess videos of athletes in motion; construct an rPPG signal feature extraction model and use domain adaptation as an unsupervised learning scheme to train the rPPG signal feature extraction model; use the trained rPPG signal feature extraction model to extract rPPG signal features from the preprocessed video, including extracting local spatiotemporal features through spatiotemporal feature extraction 3D-CNN and combining it with Transform modeling to model long-term temporal dependencies to obtain rPPG signal features; S2. Build and train an information and motion feature extraction model, extract information features and motion features of athletes from the preprocessed video using the trained information and motion feature extraction model, and fuse the information features and motion features to obtain motion information features; S3. Construct a multi-source signal fusion network based on a deep learning model, use the acquired rPPG signal features and motion information features as input, and make continuous predictions on the athlete's fatigue level.

[0007] Based on the first aspect, step S1 specifically includes the following steps: S11. Video Where T represents the time step and H×W represents the spatial resolution. Specifically, the RGB color channels of the video are converted into YCbCr color channels. The mixing relationship of the rPPG signal in the YCbCr color channels is: ;in represents the blue difference change, represents the red difference change, Motion ( t ) represents the noise signal caused by motion, α and γ represent the pulse signal gain, and β and δ represent the motion interference coefficient; they are then input into the motion compensation to eliminate the interference of motion artifacts on the rPPG signal; the motion compensation formula is: , express t The skin area pixel intensity signal extracted from the RGB color channel video at each moment, Represents the motion displacement gradient signal calculated by the optical flow method, Represents the signal after eliminating motion artifacts; obtain the preprocessed video ; S12, spatiotemporal feature extraction 3D-CNN for preprocessed video Processing, that is, through the 3D convolution kernel After preprocessing, the video Processing, where represents the input channel, h×w represents the spatial resolution, Represents the output channel, and obtains the spatiotemporal feature extraction output : , represents the time step, Represents spatial resolution, RELU represents activation function, and BatchNorm represents batch normalization; S13, extract and output spatiotemporal features Flattened into a time series feature sequence And add position code , ;in Indicates the moment, , Indicates flattening of spatiotemporal features; S14. Capturing global dependencies through the encoder Transformer’s multi-head self-attention MSA and layer normalization LN: , ;in Represents the Transformer model The output features of the layer, FFN represents the feed-forward network, l is the layer index; obtain rPPG signal features .

[0008] Based on the first aspect, step S2 specifically includes: S21. Output detection box through YOLOv8s model ,in is the coordinate of the upper left corner of the detection box, is the coordinate of the lower right corner of the detection box, The confidence level of the athlete category is calculated; K-Means clustering is used to redesign the size of the anchor box to adapt to the body shape of basketball players, that is, the average aspect ratio is 0.4:1; S22. Through loss function The position sensitivity of the detection frame is enhanced, where represents the classification loss, represents the weight coefficient of classification loss, represents the bounding box regression loss, Represents the weight coefficient of the bounding box regression loss, where the bounding box regression loss The calculation formula is ,in represents the Euclidean distance, Indicates the overlap ratio between the predicted box and the true box, c represents the minimum bounding box diagonal, vrepresents the aspect ratio consistency term, represents the prediction box, represents the true frame, represents the weight coefficient; S23. Use DeepSort algorithm to track athletes, extract information features and motion features of athletes from pre-processed videos through trained information and motion feature extraction models, and predict the next frame state of athletes through Kalman filtering. , where F represents the state transfer matrix, represents the state vector, , x' represents the horizontal coordinate of the center point, y represents the vertical coordinate of the center point, w represents the width of the bounding box, and h' represents the height of the bounding box. represents the horizontal speed, Indicates vertical speed; S24, match the detection box and tracking trajectory through the Hungarian algorithm, the cost matrix C includes the Mahalanobis distance and cosine similarity of appearance features ; Mahalanobis distance The calculation formula is , where S represents the covariance matrix of the Kalman filter, represents the trajectory, T1 represents the transposition, and the cosine similarity of the appearance feature The calculation formula is ,in is the i-th appearance feature vector, is the mean of the appearance features of the j-th trajectory; S25, through comprehensive cost Tracking of athlete information, including is the weight of the motion cost, is the weight of the appearance cost; obtaining information features, wherein the athlete information includes the athlete's running distance, the athlete's foot center point, the athlete's speed and the actual speed; S26, AlphaPose joint coordinates based on the pose estimation in the tracking frame , where k is the index number of the joint point, calculate the action features: , fuse information features and action features, and output the extracted motion information features .

[0009] Based on the first aspect, step S3 specifically includes the following steps: S31, extracting the rPPG signal features and motion information features Input into the multi-source signal fusion network, first extract the local spatiotemporal information through the 3DCNN network, then input the local spatiotemporal information into the LSTM model to extract the temporal features, and then extract the temporal features through the rPPG signal features. Get the first output feature , through motion information features Get the second output feature ; S32, the first output feature Perform short-time Fourier transform (STFT) to obtain the spectrum , and then perform inverse transformation back to the time domain feature to obtain the first enhanced feature ; S33, from the second output feature Analyze joint coordinates ,in Represent feature dimensions and calculate joint importance weights , and then the second enhanced feature is obtained by weighted aggregation ; S34, the first enhancement feature and the second enhancement feature Fusion , obtain fusion features , the fusion features Input into the fatigue prediction model to obtain the relevant features of the athlete's fatigue level, including signal predicted heart rate HR, heart rate variability HRV, blood oxygen saturation SpO2, exercise load and exercise characteristics. The neuromuscular fatigue index NFI index is calculated based on the relevant features of the athlete's fatigue level, that is, ;in Indicates other, represents the weight of the signal predicting heart rate HR, Indicates the ratio of low frequency to high frequency of heart rate variability The weight of Indicates the weight of the exercise load, represents the weight of motion features, Indicates the weight of blood oxygen saturation SpO2, Indicates other The weight of the athlete's fatigue level is predicted by the NFI index.

[0010] In a second aspect, the present invention discloses a non-contact athlete physiological state assessment system based on feature fusion, which is used in the above-mentioned non-contact athlete physiological state assessment method based on feature fusion, comprising: The rPPG signal extraction module is based on the 3DCNNTransform encoder to build an rPPG signal feature extraction model. The rPPG signal feature extraction model is trained using domain adaptation as an unsupervised learning scheme. The trained rPPG signal feature extraction model is used to extract rPPG signal features from the preprocessed video. A player information extraction module based on YOLO DeepSort 3DCNN is used to extract the player's information features and action features from the pre-processed video using the trained information and action feature extraction model, and then fuse the information features and action features to obtain motion information features. The continuous body state assessment module based on fusion signals is used to build a multi-source signal fusion network based on a deep learning model. It uses the acquired rPPG signal features and motion information features as input to continuously predict the athlete's fatigue level.

[0011] In a third aspect, the present invention discloses a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in a non-contact athlete physiological state assessment method based on feature fusion as described above.

[0012] The beneficial effects of the present invention are: 1) This invention innovatively proposes a non-contact body state assessment technology that combines remote photoplethysmography (rPPG) signals to predict core indicators such as heart rate (HR), heart rate variability (HRV), and blood oxygen saturation (SpO2) with sports biomechanical parameters. By analyzing video clips of basketball games, the athlete's rPPG signal, exercise load, and exercise characteristics are extracted separately, thereby realizing feature fusion of different modal information. Then, through the designed deep learning model's multi-source signal fusion network, the athlete's fatigue level is continuously and real-time assessed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of the camera placement position according to an embodiment of the present invention; Figure 2 A schematic diagram of a process flow of an embodiment of the present invention; Figure 3 This is a schematic diagram of the process of locating athlete joint points according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the process of extracting athlete information based on YOLO DeepSort 3DCNN in an embodiment of the present invention; Figure 5 This is a schematic diagram of the process of extracting rPPG signals based on 3D-CNN Transform according to an embodiment of the present invention; Figure 6 This is a flow chart of inputting fusion signals into a multi-source signal fusion network to continuously predict the fatigue level of athletes according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0015] The present invention discloses a non-contact athlete physiological state assessment method based on feature fusion, which combines remote photoplethysmography (rPPG) signals to predict core indicators such as heart rate (HR), heart rate variability (HRV), and blood oxygen saturation (SpO2), and integrates non-contact body state assessment technology with sports biomechanical parameters. By analyzing video clips containing basketball games, the athlete's rPPG signal, exercise load, and exercise characteristics are extracted respectively, thereby realizing feature fusion of different modal information. In order to further improve the fusion efficiency and effect, a deep regression model based on multi-source signal fusion is also designed. The model can perform continuous and real-time assessment of the athlete's activity level. The schematic flow chart of its steps is shown as follows: Figure 2 As shown, the method specifically includes the following steps: S1. Collect and preprocess videos of athletes in motion; construct an rPPG signal feature extraction model and use domain adaptation as an unsupervised learning scheme to train the rPPG signal feature extraction model; use the trained rPPG signal feature extraction model to extract rPPG signal features from the preprocessed video, including extracting local spatiotemporal features through spatiotemporal feature extraction 3D-CNN and combining it with Transform modeling to model long-term temporal dependencies to obtain rPPG signal features; For example, corresponding cameras are placed at corresponding positions on the basketball court. This operation is based on the specific actual working situation and is placed in reference to the current mainstream camera placement positions. The camera placement diagram is shown in the figure below. Figure 1 As shown, the specific operations are as follows: above the center line of the audience seats, the entire court is shot from above to capture the entire game and clearly observe the running data of the athletes. Located at the edge of the audience area, at a height of about 4 meters, it focuses on shooting the details of the athletes' confrontation to capture the athletes' action information, such as breakthroughs, shots, etc. The cameras placed on the basket, behind the basketball stand, and on both sides of the center line are used to recognize the athletes' faces and neck RPPG signals in real time.

[0016] S2. Build and train an information and motion feature extraction model, extract information features and motion features of athletes from the preprocessed video using the trained information and motion feature extraction model, and fuse the information features and motion features to obtain motion information features; S3. Construct a multi-source signal fusion network based on a deep learning model, use the acquired rPPG signal features and motion information features as input, and make continuous predictions on the athlete's fatigue level.

[0017] Specifically, the process diagram of extracting rPPG signals based on 3D-CNN Transform is as follows: Figure 5 As shown, step S1 specifically includes the following steps: S11. Video Preprocessing includes color channel conversion and motion compensation. The video color channels are converted into YCbCr channels. The YCbCr space has a significantly improved signal-to-noise ratio (SNR) compared to the RGB space in motion scenes. In particular, the Cr channel is more robust to changes in illumination. T represents the time step, and H×W represents the spatial resolution. Specifically, the RGB color channels of the video are converted into YCbCr color channels. The mixing relationship of the rPPG signal in the YCbCr color channels is: ;in represents the blue difference change, represents the red difference change, Motion ( t ) represents the noise signal caused by motion, α and γ represent the pulse signal gain, and β and δ represent the motion interference coefficient; they are then input into the motion compensation to eliminate the interference of motion artifacts on the rPPG signal; the motion compensation formula is: , express t The skin area pixel intensity signal extracted from the RGB color channel video at each moment, represents the signal after removing motion artifacts, Represents the motion displacement gradient signal calculated by the optical flow method, which is used to characterize the pixel-level displacement changes of the athlete's face or neck skin during movement (such as skin deformation caused by head shaking and neck muscle stretching); obtain the preprocessed video ; S12, spatiotemporal feature extraction 3D-CNN for preprocessed video Processing, that is, through the 3D convolution kernel After preprocessing, the video Processing, where represents the input channel, h×w represents the spatial resolution, Represents the output channel, and obtains the spatiotemporal feature extraction output : , represents the time step, Represents spatial resolution, RELU represents activation function, and BatchNorm represents batch normalization; S13, extract and output spatiotemporal features Flattened into a time series feature sequence And add position code , ;in Indicates the moment, , Indicates flattening of spatiotemporal features; S14. Capturing global dependencies through the encoder Transformer’s multi-head self-attention MSA and layer normalization LN: , ;in Represents the Transformer model The output features of the layer, FFN represents the feed-forward network, l is the layer index; obtain rPPG signal features .

[0018] Specifically, the process diagram of extracting athlete information based on YOLO DeepSort 3DCNN is as follows: Figure 4 As shown, step S2 specifically includes: S21, output detection box through the You Only Look Once version 8 small (YOLOv8s) model ,in is the coordinate of the upper left corner of the detection box, is the coordinate of the lower right corner of the detection box, The confidence level of the athlete category is calculated; K-Means clustering is used to redesign the anchor box size to adapt to the body shape of basketball players, that is, the average aspect ratio is 0.4:1; S22. Through loss function The position sensitivity of the detection frame is enhanced, where represents the classification loss, represents the weight coefficient of classification loss, represents the bounding box regression loss, Represents the weight coefficient of the bounding box regression loss, where the bounding box regression loss The calculation formula is ,in represents the Euclidean distance, Indicates the overlap ratio between the predicted box and the true box, crepresents the minimum bounding box diagonal, v represents the aspect ratio consistency term, represents the prediction box, represents the true frame, Represents the weight coefficient, the detection box is the observation result of the current frame, and the prediction box is the temporal inference of the trajectory of the previous frame; S23. Use DeepSort algorithm to track athletes, extract information features and motion features of athletes from pre-processed videos through trained information and motion feature extraction models, and predict the next frame state of athletes through Kalman filtering. , where F represents the state transfer matrix, represents the state vector, , x' represents the horizontal coordinate of the center point, y represents the vertical coordinate of the center point, w represents the width of the bounding box, and h' represents the height of the bounding box. represents the horizontal speed, Indicates vertical speed; S24. Match the detection box and the tracking trajectory through the Hungarian algorithm. The cost matrix C includes the Mahalanobis distance (motion consistency) and cosine similarity of appearance features ; Mahalanobis distance The calculation formula is , where S represents the covariance matrix of the Kalman filter, represents the trajectory, T1 represents the transposition, and the cosine similarity of the appearance feature The calculation formula is ,in is the i-th appearance feature vector, is the mean of the appearance features of the j-th trajectory; S25, through comprehensive cost Tracking of athlete information, including is the weight of the motion cost, is the weight of the appearance cost, the default =0.7, that is, more trust in motion information (Mahalanobis distance); obtain information features, the athlete information includes the athlete's running distance, the athlete's foot center point, the athlete's speed and actual speed; the athlete's foot center point The calculation formula is , ; Athlete speed The calculation formula is ;Converted to actual speed through camera calibration , and its calculation formula is , unit is m / s; S26, AlphaPose joint coordinates based on the pose estimation in the tracking frame , where k is the index number of the joint point, calculate the action features: , fuse information features and action features, and output the extracted motion information features .

[0019] For example, NAS-HP²Net is a joint human parsing and pose estimation network based on neural architecture search. AlphaPose and NAS-HP²Net complement each other in function, making NAS-HP²Net more powerful in pose estimation and realizing the coordinated optimization of human parsing and pose estimation tasks. Its core design focuses on task-specific feature extraction and cross-task feature interaction, especially strengthening the precise extraction and semantic association of human joints. The architectural design of NAS-HP²Net revolves around task-specific feature extraction and cross-task feature interaction. It consists of two key parts: encoder-decoder search (Pose Encoder-Pose Decoder) and high-level feature fusion search (High-Level Feature FusionSearch). The input image is passed through two independent task-specific encoder-decoders (pose branch and parsing branch) to extract multi-scale features. Pose branch: The first output main feature ( ) and auxiliary features ( ), the auxiliary feature is the key point heat map after Gaussian blur. Parsing branch: the second output main feature ( ) and edge features ( ), edge features are used to assist in parsing tasks. High-level feature fusion: The pose cell fuses four types of high-level features from two branches: , the parsing cell input is The structure is symmetrical with the posture unit, which strengthens the fusion of analytical features and posture features. By stacking the posture unit and the analytical unit, multi-level interaction of features is achieved, and the fused prediction results are finally output. The flowchart of locating the athlete's joints is shown in the figure. Figure 3 As shown, represents the position feature of the second joint point, represents the position feature of the second joint point, represents the first player's position characteristics, Represents the second player's position characteristics.

[0020] Specifically, the fusion signal is input into the multi-source signal fusion network to continuously predict the athlete's fatigue level. Figure 6 As shown in the figure, a multi-source signal fusion network of a deep learning model is designed to continuously predict the fatigue level of athletes by integrating multi-dimensional information. The input of the model is the rPPG signal feature. and motion information features The two features are first processed through 3D Convolutional Neural Networks to obtain local spatiotemporal features, and then through Long Short-Term Memory (LSTM) networks to extract the temporal information of the features. The features are then enhanced and fused and input into the fatigue prediction model for prediction. Step S3 specifically includes the following steps: S31, extracting the rPPG signal features and motion information features The multi-source signal fusion network is input into the multi-source signal fusion network, which consists of two branches. First, the local spatiotemporal information is extracted through the 3DCNN network, and then the local spatiotemporal information is input into the LSTM model to extract the temporal features. Get the first output feature , through motion information features Get the second output feature ; S32, the first output feature Perform short-time Fourier transform (STFT) to obtain the spectrum , and then perform inverse transformation back to the time domain feature to obtain the first enhanced feature ; S33, from the second output feature Analyze joint coordinates ,in Represent feature dimensions and calculate joint importance weights The MLP (Multi-Layer Perceptron) maps the coordinates of the joints to a high-dimensional space, capturing the semantic information of their motion or posture (such as the degree of elbow bending). Softmax normalization converts the scores output by the MLP into a probability distribution, highlighting key joints (such as the wrist when shooting), and then obtains the second enhanced feature through weighted aggregation. ; S34, the first enhancement feature and the second enhancement feature Fusion , obtain fusion features , the fusion features Input into the fatigue prediction model to obtain the relevant features of the athlete's fatigue level, including signal prediction heart rate HR, heart rate variability HRV, blood oxygen saturation SpO2, exercise load and exercise characteristics and other physiological indicators related to the cardiac cycle. The neuromuscular fatigue index NFI index is calculated based on the relevant features of the athlete's fatigue level, that is, ; Indicates other, represents the weight of the signal predicting heart rate HR, Indicates the ratio of low frequency to high frequency of heart rate variability The weight of Indicates the weight of the exercise load, represents the weight of motion features, Indicates the weight of blood oxygen saturation SpO2, Indicates other The weight of each part indicates the importance of each part and is adaptively adjusted during training; the NFI index is used to predict the athlete's fatigue level.

[0021] The present invention discloses a non-contact athlete physiological state assessment system based on feature fusion, which is used in the above-mentioned non-contact athlete physiological state assessment method based on feature fusion, comprising: The rPPG signal extraction module is based on the 3DCNNTransform encoder to build an rPPG signal feature extraction model. The rPPG signal feature extraction model is trained using domain adaptation as an unsupervised learning scheme. The trained rPPG signal feature extraction model is used to extract rPPG signal features from the preprocessed video. A player information extraction module based on YOLO DeepSort 3DCNN is used to extract the player's information features and action features from the pre-processed video using the trained information and action feature extraction model, and then fuse the information features and action features to obtain motion information features. The continuous body state assessment module based on fusion signals is used to build a multi-source signal fusion network based on a deep learning model. It uses the acquired rPPG signal features and motion information features as input to continuously predict the athlete's fatigue level.

[0022] Illustratively, the present invention further provides a computer-readable storage medium having computer-readable instructions or a program stored thereon. When executed by a processor, the computer-readable instructions or program causes the computer to perform the following operations: the operations include the steps of the aforementioned method for non-contact athlete physiological status assessment based on feature fusion, which are not further described herein. The storage medium may include, for example, an optical disk, a hard disk, a floppy disk, a flash memory, a magnetic tape, and the like.

[0023] The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. A non-contact athlete physiological state assessment method based on feature fusion, characterized in that: The following steps are involved: S1. Collect and preprocess videos of athletes in motion; construct an rPPG signal feature extraction model and use domain adaptation as an unsupervised learning scheme to train the rPPG signal feature extraction model; use the trained rPPG signal feature extraction model to extract rPPG signal features from the preprocessed video, including extracting local spatiotemporal features through spatiotemporal feature extraction 3D-CNN and combining it with Transform modeling to model long-term temporal dependencies to obtain rPPG signal features; S2. Build and train an information and motion feature extraction model, extract information features and motion features of athletes from the preprocessed video using the trained information and motion feature extraction model, and fuse the information features and motion features to obtain motion information features; S3. Construct a multi-source signal fusion network based on a deep learning model, use the acquired rPPG signal features and motion information features as input, and make continuous predictions on the athlete's fatigue level.

2. A non-contact athlete physiological state assessment method based on feature fusion according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11. Video Where T represents the time step and H×W represents the spatial resolution. Specifically, the RGB color channels of the video are converted into YCbCr color channels. The mixing relationship of the rPPG signal in the YCbCr color channels is: ;in represents the blue difference change, represents the red difference change, Motion ( t ) represents the noise signal caused by motion, α and γ represent the pulse signal gain, and β and δ represent the motion interference coefficient; they are then input into the motion compensation to eliminate the interference of motion artifacts on the rPPG signal; the motion compensation formula is: , express t The skin area pixel intensity signal extracted from the RGB color channel video at each moment, Represents the motion displacement gradient signal calculated by the optical flow method, Represents the signal after eliminating motion artifacts; obtain the preprocessed video ; S12, spatiotemporal feature extraction 3D-CNN for preprocessed video Processing, that is, through 3D convolution kernel After preprocessing, the video Processing, where represents the input channel, h×w represents the spatial resolution, Represents the output channel, and obtains the spatiotemporal feature extraction output : , represents the time step, Represents spatial resolution, RELU represents activation function, and BatchNorm represents batch normalization; S13, extract and output spatiotemporal features Flattened into a time series feature sequence And add position code , ;in Indicates the moment, , Indicates flattening of spatiotemporal features; S14. Capturing global dependencies through the encoder Transformer’s multi-head self-attention MSA and layer normalization LN: , ;in Represents the Transformer model The output features of the layer, FFN represents the feed-forward network, l is the layer index; obtain rPPG signal features .

3. A non-contact athlete physiological state assessment method based on feature fusion according to claim 2, characterized in that: Step S2 specifically includes: S21. Output detection box through YOLOv8s model ,in( ) is the coordinate of the upper left corner of the detection frame, ( ) is the coordinate of the lower right corner of the detection box, The confidence level of the athlete category is calculated; K-Means clustering is used to redesign the size of the anchor box to adapt to the body shape of basketball players, that is, the average aspect ratio is 0.4:1; S22. Through loss function The position sensitivity of the detection frame is enhanced, where represents the classification loss, represents the weight coefficient of classification loss, represents the bounding box regression loss, Represents the weight coefficient of the bounding box regression loss, where the bounding box regression loss The calculation formula is ,in represents the Euclidean distance, Indicates the overlap ratio between the predicted box and the true box, c represents the minimum bounding box diagonal, v represents the aspect ratio consistency term, represents the prediction box, represents the true frame, represents the weight coefficient; S23. Use DeepSort algorithm to track athletes, extract information features and motion features of athletes from pre-processed videos through trained information and motion feature extraction models, and predict the next frame state of athletes through Kalman filtering. , where F represents the state transfer matrix, Represents the state vector, x=(x',y,w,h', , ), x' represents the horizontal coordinate of the center point, y represents the vertical coordinate of the center point, w represents the width of the bounding box, and h' represents the height of the bounding box. represents the horizontal velocity, Indicates vertical speed; S24, match the detection box and tracking trajectory through the Hungarian algorithm, the cost matrix C includes the Mahalanobis distance and cosine similarity of appearance features ; Mahalanobis distance The calculation formula is , where S represents the covariance matrix of the Kalman filter, represents the trajectory, T1 represents the transposition, and the cosine similarity of the appearance feature The calculation formula is ,in is the i-th appearance feature vector, is the mean of the appearance features of the j-th trajectory; S25, through comprehensive cost Tracking of athlete information, including is the weight of the motion cost, is the weight of the appearance cost; obtain information features , the athlete information includes the athlete's running distance, the athlete's foot center point, the athlete's speed and actual speed; S26, AlphaPose joint coordinates based on the pose estimation in the tracking frame , where k is the index number of the joint point, calculate the action features: , fuse information features and action features, and output the extracted motion information features .

4. A non-contact athlete physiological state assessment method based on feature fusion according to claim 3, characterized in that: Step S3 specifically includes the following steps: S31, extracting the rPPG signal features and motion information features Input into the multi-source signal fusion network, first extract the local spatiotemporal information through the 3DCNN network, then input the local spatiotemporal information into the LSTM model to extract the temporal features, and then extract the temporal features through the rPPG signal features. Get the first output feature , through motion information features Get the second output feature ; S32, the first output feature Perform short-time Fourier transform (STFT) to obtain the spectrum Then perform inverse transformation back to the time domain feature to obtain the first enhanced feature ; S33, from the second output feature Analyze joint coordinates ,in Represent feature dimensions and calculate joint importance weights , and then the second enhanced feature is obtained by weighted aggregation ; S34, the first enhancement feature and the second enhancement feature Fusion , obtain fusion features , the fusion features Input into the fatigue prediction model to obtain the relevant features of the athlete's fatigue level, including signal predicted heart rate HR, heart rate variability HRV, blood oxygen saturation SpO2, exercise load and exercise characteristics. The neuromuscular fatigue index NFI index is calculated based on the relevant features of the athlete's fatigue level, that is, ;in Indicates other, represents the weight of the signal predicting heart rate HR, Indicates the ratio of low frequency to high frequency of heart rate variability The weight of Indicates the weight of the exercise load, represents the weight of motion features, Indicates the weight of blood oxygen saturation SpO2, Indicates other The weight of the athlete's fatigue level is predicted by the NFI index.

5. A non-contact athlete physiological state assessment system based on feature fusion, used in the non-contact athlete physiological state assessment method based on feature fusion according to any one of claims 1 to 4, characterized in that: include: The rPPG signal extraction module is based on the 3DCNNTransform encoder to build an rPPG signal feature extraction model. The rPPG signal feature extraction model is trained using domain adaptation as an unsupervised learning scheme. The trained rPPG signal feature extraction model is used to extract rPPG signal features from the preprocessed video. A player information extraction module based on YOLO DeepSort 3DCNN is used to extract the player's information features and action features from the pre-processed video using the trained information and action feature extraction model, and then fuse the information features and action features to obtain motion information features. The continuous body state assessment module based on fusion signals is used to build a multi-source signal fusion network based on a deep learning model. It uses the acquired rPPG signal features and motion information features as input to continuously predict the athlete's fatigue level.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the non-contact athlete physiological state assessment method based on feature fusion as described in any one of claims 1-4.

Images