Abnormal physical condition attack detection method

Abstract

The invention relates to a body abnormal condition attack detection method based on adaptive weight feature fusion and a double-flow convolutional neural network, and the method comprises the following steps: constructing a data set, and adding a body condition label; performing expression analysis on the characters in the data set, and adding facial expression labels; performing data augmentation on the data set; a detection model is constructed, the body condition image serves as input, the body condition label and the facial expression label serve as output, and the detection model comprises a double-flow convolutional neural network module used for extracting overall body posture features and local facial expression features and a self-adaptive weight feature fusion module used for conducting self-adaptive weight fusion on the extracted features; and obtaining a to-be-detected image, inputting the to-be-detected image into the trained detection model, and outputting the predicted physical condition label, the facial expression label and the corresponding probabilities thereof to realize the attack detection of the abnormal physical condition. Compared with the prior art, the method has the advantages of being high in detection precision, not limited to application scenes and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method for detecting the onset of abnormal physical conditions based on adaptive weight feature fusion and a dual-stream convolutional neural network. Background Art

[0002] With the continuous advancement of science and technology, in recent years, more and more researchers have begun to pay attention to the detection methods of potential abnormal physical conditions in elderly people living alone. These methods can be roughly divided into two categories, namely, methods using sensor devices and methods based on visual images, which can also be understood as invasive methods and non-invasive methods.

[0003] For invasive methods using sensor devices, researchers usually analyze the individual's state based on the three-dimensional motion information (such as speed and acceleration) of the limbs and torso obtained by wearable sensor devices (such as accelerometers, gyroscopes, and millimeter-wave radars) fixed on the individual to identify daily behaviors, or determine whether a person has fallen based on sudden changes in data peaks in specific dimensions. There are also some methods that monitor the behavioral activities of the elderly through infrared sensors or RFID systems placed in the home environment. Traditional medical methods usually use electrocardiograms, nuclear magnetic resonance technologies, etc. to obtain patients' heart rate, systolic blood pressure and other physiological parameters and corresponding hematological results to diagnose heart disease-related diseases, which is also an invasive method.

[0004] Invasive methods using sensors require the subject to continuously wear the corresponding equipment, which causes a lot of inconvenience and affects the subject's normal life. In addition, data loss and noise caused by improper wearing are inevitable. Information redundancy between sensors will also affect the monitoring results.

[0005] Vision-based non-intrusive methods can be divided into four categories: using a single RGB camera, infrared camera, depth camera, and 3D methods based on camera array combinations, which are widely used in activity recognition and anomaly detection, especially in the task of fall detection. Since the advent of Microsoft's Kinect series of depth cameras, the technology of using its RGB-D camera for activity recognition and fall detection has been widely and deeply studied. Three-dimensional methods based on camera arrays are also widely used. This method covers a wide range of monitoring ranges from many angles, thus avoiding potential errors caused by occlusion. With the public release of OpenPose, AlphaPose and other algorithms for human pose estimation using skeleton points, applications including behavior recognition, fall detection and gait analysis have been deeply studied.

[0006] At present, some abnormal body condition detection methods based on visual images only focus on fall detection and do not have a wider range of detection tasks. At the same time, they only pay attention to the information presented by human posture and ignore the effective information provided by human facial expressions.

[0007] In summary, developing an image-based, non-invasive, fast and efficient method for detecting the onset of abnormal physical conditions that is suitable for a variety of detection tasks and fully utilizes the posture characteristics and facial expression characteristics of the subject has become an urgent problem to be solved by technicians in this research field. Summary of the invention

[0008] The purpose of the present invention is to provide a method for detecting the onset of abnormal physical conditions based on adaptive weight feature fusion and dual-stream convolutional neural network, the input source is a visual image or video frame read by a camera, and the body posture features and local facial expression features presented by the overall image are processed respectively through the structure of the dual-stream convolutional neural network, and the two are feature fused; the image data set with high precision, high character pixel ratio, and complete posture expression information is selected, and data augmentation is performed; the dual-stream convolutional neural network is trained on the augmented data set, and the network weights pre-trained on ImageNet are applied during the training process to improve the training efficiency and effect; the trained model can detect the subsequent input images, and quickly and accurately output chest pain, headache, fall, and no abnormal labels. The above method is not limited by application scenarios, specific equipment, professional operation, etc., and is fast and efficient to improve the existing shortcomings of sensor-based equipment and traditional medical methods, such as invasiveness, high cost, limited application scenarios, complex operation, and low method efficiency.

[0009] The purpose of the present invention can be achieved by the following technical solutions:

[0010] A method for detecting the onset of abnormal physical conditions based on adaptive weight feature fusion and a two-stream convolutional neural network comprises the following steps:

[0011] Build an image dataset for detecting abnormal physical condition onset and add physical condition labels;

[0012] Based on the DeepFace network framework, facial expression analysis is performed on the characters in the abnormal body condition onset detection image dataset, and facial expression labels are added;

[0013] Perform data augmentation on the body abnormality onset detection image dataset;

[0014] Constructing a detection model, taking the body condition image as input, and the body condition label and the facial expression label as output, and using the data augmented body abnormal condition onset detection image data set to train the model, wherein the detection model includes a two-stream convolutional neural network module and an adaptive weight feature fusion module, the two-stream convolutional neural network module includes two branches, respectively extracting the overall body posture features and the local facial expression features in the image, and the adaptive weight feature fusion module performs adaptive weight fusion on the features extracted by each branch;

[0015] The image to be detected is obtained from the camera, and the image to be detected is input into the trained detection model to output the predicted body condition label, facial expression label and its corresponding probability to realize the detection of abnormal body condition.

[0016] The physical condition labels included chest pain, headache, falls, and no abnormality.

[0017] The person in the image corresponding to the chest pain category physical condition label includes at least two of the following posture signs at the same time: 1) the body leans forward, 2) both hands are placed on the chest, 3) one hand is placed on the chest, 4) one hand is placed on the forehead / waist;

[0018] The person in the image corresponding to the physical condition label of the headache category includes at least one of the following posture signs: 1) both hands on the forehead, 2) one hand on the forehead, 3) both hands / one hand on one side of the head;

[0019] The body condition label of the fall category corresponds to the person in the image falling to the ground at multiple angles.

[0020] The facial expression labels include positive, negative and neutral.

[0021] The data augmentation of the abnormal body condition onset detection image dataset is specifically: using the ImageDataGenerator module in the Keras library and combining rotation, scaling, height / width movement, brightness adjustment and horizontal flipping operations to perform data augmentation on the abnormal body condition onset detection image dataset.

[0022] The first branch of the two-stream convolutional neural network module is used to extract local facial expression features, including a face extraction and alignment module based on MTCNN and a facial expression feature extraction module based on DenseNet, wherein the network structure of DenseNet includes a DenseBlock module and a Transition module, and every two DenseBlock modules are connected by a Transition module, and the last DenseBlock module is sequentially connected to a global average pooling layer, a random inactivation Dropout layer and two fully connected layers of ReLU activation functions.

[0023] The second branch of the two-stream convolutional neural network module is used to extract overall body posture features, including a size adjustment module and a body posture feature extraction module based on DenseNet, wherein the network structure of DenseNet includes a DenseBlock module and a Transition module, and every two DenseBlock modules are connected by a Transition module, and the last DenseBlock module is sequentially connected to a global average pooling layer, a random inactivation Dropout layer, and two fully connected layers of ReLU activation functions.

[0024] The DenseBlock module consists of a batch normalization layer, a fully connected layer with a ReLU activation function, and a 3×3 convolutional layer. The feature map of each layer has the same size, and dense connections are used between layers. Each layer is spliced ​​with all previous layers in the channel dimension and serves as the input of the next layer.

[0025] The Transition module is used to connect two adjacent DenseBlock modules, integrate the features output by the previous DenseBlock module and perform downsampling to reduce the size of the feature map.

[0026] The adaptive weight feature fusion module performs the following operations:

[0027] Defining a query variable, the dimension of the query variable corresponding to the dimension of the feature vector extracted by each branch of the two-stream convolutional neural network module;

[0028] Perform matrix multiplication on the feature vector extracted from each branch and the query variable to obtain the learnable attention parameters of the corresponding branch;

[0029] The learnable attention parameters are concatenated and passed through the softmax function so that the sum of the two parameters is 1;

[0030] Multiply the learnable attention parameters corresponding to each branch network by the corresponding feature vector and sum them up to obtain the fused feature.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) Not limited by application scenarios: Traditional medical identification methods require professional medical testing equipment and professional medical staff to operate. These conditions and the ground limit the application scenarios and are difficult to be installed in a home environment. This method uses a non-invasive method based on visual signals, which is not limited by application scenarios and can be deployed in home environments, nursing homes, and sanatoriums.

[0033] (2) The freedom of movement of the subject is not restricted: Traditional medical identification methods require the subject to be tested by professional medical equipment in a designated space. Some invasive methods based on sensors require the subject to wear sensor equipment all the time, which will cause inconvenience to the subject's free movement and daily life. This method is based on visual signals and is a non-invasive method. It extracts the posture features and facial expression features of the subject from the visual image to detect whether the subject has physical abnormalities.

[0034] (3) Convenience and rapid response: Compared with the diagnostic process of traditional medical methods, this method can detect signs of abnormal physical conditions from visual signals in the absence of people. It can notify emergency contacts and call 120 as soon as the abnormality occurs, thereby gaining golden time for rescue for patients. At the same time, compared with sensor-based methods, image signals can avoid information redundancy between sensors, are resource-efficient, and easy to obtain. This method only requires a camera as a signal source, does not require complex equipment installation, and is simple and convenient.

[0035] (4) Good scalability: The present invention can be widely deployed on platforms that meet the requirements of corresponding hardware devices, such as PC platforms, smart mobile devices, smart home systems, smart companion robots, etc., and has good scalability.

[0036] (5) High detection accuracy: The present invention integrates facial expression and body posture features, and can more accurately detect the onset of abnormal physical conditions, thereby improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the method of the present invention;

[0038] Figure 2 Schematic diagram of the image data set collected for the present invention;

[0039] Figure 3 It is a schematic diagram of the network framework of the detection model of the present invention;

[0040] Figure 4 This is a schematic diagram of the structure of the improved convolutional neural network DenseNet of the present invention;

[0041] Figure 5 This is a visualization diagram of the adaptive weight feature fusion proposed by the present invention;

[0042] Figure 6 The detection result of the present invention for a single image and the probability of the corresponding label;

[0043] Figure 7 A schematic diagram of the confusion matrix of the present invention on a test data set;

[0044] Figure 8 It is a schematic diagram of the implementation system framework corresponding to the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0046] This embodiment provides a method for detecting abnormal physical condition based on adaptive weight feature fusion and dual-stream convolutional neural network. Figure 1 As shown, the following steps are included:

[0047] Step 1) Build an image dataset for detecting abnormal physical condition onset and add physical condition labels.

[0048] This embodiment obtains a multi-angle and multi-posture image dataset for detecting the onset of abnormal physical conditions in multiple scenes such as home, work, and outdoors by collecting data from the Internet and taking video screenshots. There are 4 categories of physical condition labels in the dataset, and the category labels are: chest pain, headache, fall, and no abnormality. The categories of chest pain and headache are determined based on the most significant symptoms in the early stages of some life-threatening cardiovascular and cerebrovascular diseases: chest pain and chest tightness (84%), headache and dizziness (56.1%), shortness of breath (47.9%), numbness of the upper limbs (17.3%), sweating (13.8%) and back pain (5.2%). The characters in each category of images are presented in the most significant external posture of that category. For example:

[0049] ① The person in the image corresponding to the chest pain category physical condition label includes at least two of the following posture signs at the same time: 1) the body is leaning forward, 2) both hands are placed on the chest, 3) one hand is placed on the chest, 4) one hand is placed on the forehead / waist.

[0050] ② The person in the image corresponding to the physical condition label of the headache category includes at least one of the following posture signs: 1) both hands on the forehead, 2) one hand on the forehead, 3) both hands / one hand on one side of the head.

[0051] ③ The body condition label of the fall category corresponds to the person in the image falling to the ground at multiple angles.

[0052] ④ People in the images corresponding to the physical condition labels of the non-abnormal category mostly perform routine activities at home, such as standing, walking, sitting on the sofa, drinking water, eating, etc.

[0053] There is only one person in each image, and there is no overlap of multiple people. The image pixels are high, and the proportion of people in these images is between 40% and 70%, and the complete posture and facial expression information is retained, while avoiding the situation where important body parts of the people in the image are blocked. There are 1,000 images in each category, totaling 4,000 images. The collected image datasets are as follows Figure 2 Shown

[0054] Step 2) Based on the DeepFace network framework, facial expression analysis is performed on the characters in the abnormal body condition onset detection image dataset, and facial expression labels are added.

[0055] DeepFace is a lightweight facial attribute analysis tool based on the Python platform released by Facebook. It can analyze seven characteristics of facial expressions, including anger, fear, neutrality, sadness, disgust, happiness and surprise. As mentioned above, this method not only considers the posture characteristics of the character presented by the overall image, but also considers the feature information provided by the local facial expression. In this embodiment, happiness and surprise are merged into positive categories, anger, fear, sadness and disgust are merged into negative categories, and neutral categories are retained. On the basis of the above-mentioned four types of physical abnormality labels of chest pain, headache, fall and no abnormality, each image in the data set is additionally assigned a facial expression label from one of the three categories of positive, negative and neutral.

[0056] Step 3) perform data augmentation on the abnormal body condition onset detection image dataset.

[0057] Specifically, the ImageDataGenerator module in the Keras library is used in combination with rotation, scaling, height / width movement, brightness adjustment, and horizontal flipping operations to perform data augmentation on the abnormal body condition onset detection image dataset, so that 20 new images are generated for each image, bringing the total dataset size to 80,000 images, and dividing it into training set, validation set, and test set, with 56,000 images in the training set, 12,000 images in the validation set, and 12,000 images in the test set.

[0058] Data augmentation can generate a large amount of image data, which can improve the generalization ability of the model during the training process and improve the accuracy of the model in detecting the onset of abnormal physical conditions.

[0059] Step 4) Build a detection model, take the body condition image as input, and the body condition label and facial expression label as output, and train the model using the data augmented body abnormal condition onset detection image dataset.

[0060] The detection model includes a two-stream convolutional neural network module and an adaptive weight feature fusion module, such as Figure 3As shown in the figure, the two-stream convolutional neural network module includes two branches, which respectively extract the overall body posture features and local facial expression features in the image, and the adaptive weight feature fusion module performs adaptive weight fusion on the features extracted by each branch.

[0061] (I) Two-stream convolutional neural network module

[0062] like Figure 3 As shown in (a), the task of the first branch network of the two-stream convolutional neural network module is to extract and align the faces of the characters in the picture, and transmit the extracted face images to the neural network for training to extract the feature vectors of facial expressions. This embodiment uses a deeply cascaded multi-task convolutional neural network framework Multi-Task Cascaded Convolutional Neural Network (MTCNN) to achieve face image extraction and face alignment. This framework overcomes the limitations caused by various postures, lighting and occlusions, and can perform face detection and alignment without restrictions. The MTCNN algorithm uses a cascaded architecture and a three-stage built-in deep convolutional network to predict the positions of faces and landmarks in a coarse-to-fine manner, using the intrinsic connection between detection and alignment to improve performance.

[0063] After obtaining the face image through the face extraction and alignment module based on MTCNN, the face image is trained by the facial expression feature extraction module based on the convolutional neural network DenseNet, and the transfer learning method is adopted in the training process, that is, the classification detection task of the present invention is continued to be trained on the basis of the weights of the image classification pre-trained on ImageNet. The DenseNet network structure is mainly composed of the DenseBlock module and the Transition module. The DenseBlock module consists of a batch normalization layer Batch Normalization, a fully connected layer of the ReLU activation function and a 3×3 convolution layer. The feature map of each layer is of the same size, and the layers are densely connected. Each layer will be concatenated with all the previous layers in the channel dimension and serve as the input of the next layer. The function of the Denseblock module is to directly concatenate feature maps from different layers, which can realize feature reuse and improve efficiency. This is also the core mechanism of DenseNet. The Transition module is mainly used to connect two adjacent DenseBlocks, integrate the features obtained from the previous DenseBlock, reduce the width and height of the previous DenseBlock, and achieve a downsampling effect. Its function is to reduce the size of the feature map and play a role in compressing the model. There are 3 or 4 DenseBlock modules in a DenseNet network structure, which are connected by Transition modules in the middle. The last DenseBlock module is followed by a Global Average Pooling Layer, and then a fully connected layer with a softmax activation function is passed to obtain the score of each category. The present invention retains the DenseBlock module and the Transition module structure in front of the DenseNet network structure, and replaces the last fully connected layer with a softmax activation function. The specific operation is as follows: 1. Keep the structure of the first four DenseBlock modules and three Transition modules of the DenseNet network; 2. Connect a Global Average Pooling Layer after the last DenseBlock module; 3. Connect a random inactivation Dropout layer to alleviate the overfitting phenomenon, and then connect two fully connected layers of ReLU activation functions, and finally obtain a feature vector with a latitude of (1, 128). The obtained facial expression feature vector will be used for subsequent fusion with the body posture feature vector obtained by another branch network. The improved convolutional neural network DenseNet network structure is as follows Figure 4 shown.

[0064] The second branch of the two-stream convolutional neural network module is used to extract the overall body posture features, including a size adjustment module and a body posture feature extraction module based on DenseNet. The DenseNet network structure is consistent with the network structure in the first branch, and finally a feature vector with a dimension of (1, 128) extracted from the overall image is obtained.

[0065] (II) Adaptive Weight Feature Fusion Module

[0066] like Figure 3 As shown in (b), after extracting the facial expression feature vector and body posture feature vector using the two-stream convolutional neural network module, the adaptive weight feature fusion module is used to perform feature fusion to provide optimized fusion weights for the facial expression part and the entire image part. Considering that in some scenes, the human torso or face may be occluded or the presented information is incomplete. When the features provided by a certain part are noisy or blurred, the adaptive weight feature fusion will emphasize the relatively complete feature part and suppress the weaker one.

[0067] Specifically, the adaptive weight feature fusion module performs the following steps:

[0068] Define a query variable The dimension of the query variable corresponds to the dimension (1, 128) of the feature vector extracted by each branch of the two-stream convolutional neural network module;

[0069] Perform matrix multiplication on the feature vector extracted from each branch and the query variable to obtain the learnable attention parameters of the corresponding branch;

[0070] The learnable attention parameters are concatenated and passed through the softmax function so that the sum of the two parameters is 1;

[0071] Multiply the learnable attention parameters corresponding to each branch network by the corresponding feature vector and sum them up to obtain the fused feature.

[0072] The visualization diagram of the adaptive weight feature fusion mechanism is as follows: Figure 5 shown.

[0073] (III) Multi-label classification task

[0074] The fused features are passed through a fully connected layer with a sigmoid activation function to finally obtain the output label. The final output label will include one of the four physical abnormality labels: chest pain, headache, fall, and no abnormality, and one of the three facial expression labels: positive, negative, and neutral, for a total of two labels.

[0075] In this example, all training processes are performed on an Nvidia Quadro RTX 8000 GPU, with a batch size of 64 and an epoch of 100. All images are resized to 224×224 before training. The Adam optimizer is used during training, and the early stopping and learning rate adjustment mechanisms are set. The initial learning rate is 1e -3 .

[0076] Step 5) obtain the image to be detected taken by the camera, and input the image to be detected into the trained detection model, output the predicted body condition label, facial expression label and its corresponding probability, and realize the detection of abnormal body condition.

[0077] When acquiring the image to be detected, the camera reads the real-time image of each frame of the person being detected, including the complete posture outline and facial expression of the person being detected.

[0078] Figure 6 The test results of some images in the test set are shown. The fonts in the images are the output test result labels, and the corresponding probabilities of all labels are given above the images. There are four types of abnormal physical condition labels: fall, infarction, headache, and normal. There are three types of facial expression labels: positive, negative, and neutral. By inputting the images in the test set into the model, we can get the detection performance of the model on the test set. Figure 7 This is a schematic diagram of the confusion matrix of the present invention on the test data set. The accuracy of the final model is 90.26%.

[0079] A system implementation method corresponding to the method of the present invention is as follows Figure 8 As shown, in this system, the person being tested does not need to wear sensor equipment. The current state of the person is read through the camera and the picture frame is uploaded to the cloud server. The server deploys the above detection model to process the picture, and detects and judges the uploaded picture frame through the trained detection model, outputs one of the four types of abnormal physical condition labels of chest pain, headache, fall and no abnormality, and one of the three types of facial expression labels of positive, negative and neutral, a total of two labels. According to the detection result label, when the cloud server detects an abnormality and outputs chest pain / headache / fall and negative labels, it automatically sends a reminder to the contact information of the emergency contact deployed in advance through phone calls, messages, etc., or automatically dials the 120 emergency number.

[0080] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for detecting the onset of abnormal physical conditions based on adaptive weight feature fusion and two-stream convolutional neural network, characterized in that: The following steps are involved: Build an image dataset for detecting abnormal physical condition onset and add physical condition labels; Based on the DeepFace network framework, facial expression analysis is performed on the characters in the abnormal body condition onset detection image dataset, and facial expression labels are added; Perform data augmentation on the body abnormality onset detection image dataset; Constructing a detection model, taking the body condition image as input, and the body condition label and the facial expression label as output, and using the data augmented body abnormal condition onset detection image data set to train the model, wherein the detection model includes a two-stream convolutional neural network module and an adaptive weight feature fusion module, the two-stream convolutional neural network module includes two branches, respectively extracting the overall body posture features and the local facial expression features in the image, and the adaptive weight feature fusion module performs adaptive weight fusion on the features extracted by each branch; The image to be detected is obtained from the camera, and the image to be detected is input into the trained detection model to output the predicted body condition label, facial expression label and its corresponding probability to realize the detection of abnormal body condition.

2. The method for detecting the onset of abnormal physical condition based on adaptive weight feature fusion and dual-stream convolutional neural network according to claim 1, characterized in that: The physical condition labels included chest pain, headache, falls, and no abnormality.

3. The method for detecting the onset of abnormal physical condition based on adaptive weight feature fusion and two-stream convolutional neural network according to claim 2, characterized in that: The person in the image corresponding to the chest pain category physical condition label includes at least two of the following posture signs at the same time: 1) the body leans forward, 2) both hands are placed on the chest, 3) one hand is placed on the chest, 4) one hand is placed on the forehead / waist; The person in the image corresponding to the physical condition label of the headache category includes at least one of the following posture signs: 1) both hands on the forehead, 2) one hand on the forehead, 3) both hands / one hand on one side of the head; The body condition label of the fall category corresponds to the person in the image falling to the ground at multiple angles.

4. The method for detecting abnormal physical condition based on adaptive weight feature fusion and dual-stream convolutional neural network according to claim 1, characterized in that: The facial expression labels include positive, negative and neutral.

5. The method for detecting the onset of abnormal physical condition based on adaptive weight feature fusion and two-stream convolutional neural network according to claim 1, characterized in that: The data augmentation of the abnormal body condition onset detection image dataset is specifically: using the ImageDataGenerator module in the Keras library and combining rotation, scaling, height / width movement, brightness adjustment and horizontal flipping operations to perform data augmentation on the abnormal body condition onset detection image dataset.

6. The method for detecting abnormal physical condition based on adaptive weight feature fusion and dual-stream convolutional neural network according to claim 1, characterized in that: The first branch of the two-stream convolutional neural network module is used to extract local facial expression features, including a face extraction and alignment module based on MTCNN and a facial expression feature extraction module based on DenseNet, wherein the network structure of DenseNet includes a DenseBlock module and a Transition module, and every two DenseBlock modules are connected by a Transition module, and the last DenseBlock module is sequentially connected to a global average pooling layer, a random inactivation Dropout layer and two fully connected layers of ReLU activation functions.

7. The method for detecting abnormal physical condition based on adaptive weight feature fusion and dual-stream convolutional neural network according to claim 1, characterized in that: The second branch of the two-stream convolutional neural network module is used to extract overall body posture features, including a size adjustment module and a body posture feature extraction module based on DenseNet, wherein the network structure of DenseNet includes a DenseBlock module and a Transition module, and every two DenseBlock modules are connected by a Transition module, and the last DenseBlock module is sequentially connected to a global average pooling layer, a random inactivation Dropout layer, and two fully connected layers of ReLU activation functions.

8. A method for detecting abnormal physical condition based on adaptive weight feature fusion and dual-stream convolutional neural network according to claim 6 or 7, characterized in that: The DenseBlock module consists of a batch normalization layer, a fully connected layer with a ReLU activation function, and a 3×3 convolutional layer. The feature map of each layer has the same size, and dense connections are used between layers. Each layer is spliced ​​with all previous layers in the channel dimension and serves as the input of the next layer.

9. A method for detecting abnormal physical condition based on adaptive weight feature fusion and dual-stream convolutional neural network according to claim 6 or 7, characterized in that: The Transition module is used to connect two adjacent DenseBlock modules, integrate the features output by the previous DenseBlock module and perform downsampling to reduce the size of the feature map.

10. The method for detecting abnormal physical condition based on adaptive weight feature fusion and dual-stream convolutional neural network according to claim 1, characterized in that: The adaptive weight feature fusion module performs the following operations: Defining a query variable, the dimension of the query variable corresponding to the dimension of the feature vector extracted by each branch of the two-stream convolutional neural network module; Perform matrix multiplication on the feature vector extracted from each branch and the query variable to obtain the learnable attention parameters of the corresponding branch; The learnable attention parameters are concatenated and passed through the softmax function so that the sum of the two parameters is 1; Multiply the learnable attention parameters corresponding to each branch network by the corresponding feature vector and sum them up to obtain the fused feature.

Images