Patient pain degree identification method and device and pain degree identification system

Abstract

The invention relates to a patient pain degree identification method and device and a pain degree identification system. The method for identifying the pain degree of the patient comprises the following steps: judging the data type of pain data to be detected: if the data type is a physiological signal type, carrying out feature extraction by adopting a multi-scale convolution mechanism, a compression-excitation residual network and a variable-voltage encoder, and carrying out pain grading on the extracted signal features by adopting a support vector machine, obtaining the pain degree of the patient; and if the data type is an image type, carrying out multi-scale feature extraction on the facial expression image by adopting a double-branch feature extraction mechanism, alternately carrying out feature fusion by combining a residual error and an attention mechanism, and carrying out feature flattening and pain degree mapping on a fused feature map to obtain the pain degree of the patient. The method for identifying the pain degree of the patient has the advantages of high accuracy of pain grading based on physiological signals and high accuracy of pain grading based on images.

Description

Technical Field

[0001] The present invention relates to the field of medical health monitoring, and in particular to a method and device for identifying the pain level of a patient, and a pain level identification system. Background Art

[0002] Pain is an unpleasant response of the body to damaged tissue or potential damage. In traditional clinical practice, the main methods of assessing patients' pain are self-assessment and observer assessment. However, the above assessment methods are easily affected by the subjectivity of the patient or the doctor, as well as the emotions, cultural background, communication skills and other uncertain factors of the assessee. At the same time, the above methods often lack real-time performance, especially when it is necessary to respond to drastic changes in the patient's pain in a timely manner. They cannot provide rapid feedback, resulting in the inability to provide analgesic treatment in a timely manner. Early real-time pain recognition is analyzed through physiological signals, heart rate or blood pressure. Although it can reflect pain to a certain extent, it is easily interfered by external factors, emotional fluctuations and other factors, resulting in poor accuracy of pain recognition.

[0003] In order to improve the objectivity of pain degree judgment and recognition accuracy, the existing technology uses physiological signals to classify pain degrees. However, it is easy for drugs, pathological conditions and fear of pain to directly affect the characteristics of physiological signals, thereby reducing the accuracy of pain recognition. Some existing technologies capture the patient's face for pain recognition through image detection. However, the facial expression reactions of different individuals vary greatly. Different pain degrees may show different facial features for different individual patients. At the same time, the patient's emotional fluctuations, expression control, etc. will also affect the facial expression when in pain, resulting in certain limitations in the existing technology of recognizing the degree of pain through facial recognition.

[0004] Based on this, the existing technologies have defects in the classification, assessment or identification of pain levels, such as strong subjectivity, poor real-time performance, and interference from emotions and external factors. Especially when identifying pain through physiological signals and facial expressions, it is easily affected by factors such as emotional fluctuations, individual differences and external environment, which leads to the problem of insufficient accuracy in pain level identification. Summary of the invention

[0005] Based on this, an object of the present invention is to provide a method for identifying a patient's pain level.

[0006] A method for identifying the pain level of a patient comprises the following steps:

[0007] S1. Determine the data type of the pain data to be detected: if the data type is a physiological signal type, execute step S2A; if the data type is an image type, execute step S2B;

[0008] S2A, performing signal preprocessing on the pain data to be detected to obtain key pain signal data; wherein the signal preprocessing is a signal noise reduction operation;

[0009] S3A, using multi-scale convolution mechanism to extract features from key pain signal data and obtain preliminary pain signal features;

[0010] S4A, using compression-excitation residual network to extract global spatial features from preliminary pain signal features to obtain global enhanced signal features;

[0011] S5A, using a variable voltage encoder to extract sequence features from the global enhanced signal features to obtain the final pain signal features;

[0012] S6A: Support vector machine was used to grade the pain of the final signal features, obtain the patient's pain level, and complete the pain grading of physiological signal types;

[0013] S2B, performing image preprocessing on the pain data to be detected to obtain a facial pain expression image;

[0014] S3B, a dual-branch feature extraction mechanism is used to extract multi-scale features from facial pain expression images, and the residual and attention mechanisms are combined to alternately fuse features to obtain local priority feature maps and global priority feature maps;

[0015] S4B, fusing the local priority feature map and the global priority feature map to obtain the final pain feature map;

[0016] S5B. Flatten the final pain feature map into a one-dimensional vector, and map the one-dimensional vector to the pain level through a fully connected layer to obtain the patient's pain level and complete the pain grading of the image type.

[0017] Compared with the prior art, the method for identifying the degree of pain of the patient described in the present invention, by setting the pain classification based on the physiological signal type and the image type, allows the user to select the corresponding detection method according to the specific needs, thereby enhancing the convenience of pain degree identification; at the same time, the present invention is aimed at physiological signals, and effectively extracts multi-level features in the physiological signals by adopting a multi-scale convolution mechanism, so as to capture the pain information of the signal at different time scales; and by adopting a variable voltage encoder (TCN combined with a multi-head attention mechanism), it is possible to effectively capture the long-term dependency and the perception of the importance difference between different time steps in the physiological signal, ensuring that the pain features in the physiological signal can be fully extracted. Finally, the pain degree is graded by finding the optimal separation hyperplane from the feature space through SVM, so as to effectively reduce the overfitting phenomenon, thereby solving the problem that the accuracy of pain is easily unstable due to individual differences, thereby improving the accuracy of pain classification based on the type of physiological signal.

[0018] In addition, the present invention fully extracts higher-order, abstract features belonging to pain in facial images through a dual-branch feature extraction mechanism and an attention mechanism to capture subtle changes in facial pain, thereby avoiding false detection that may be caused by differences in facial expressions in the prior art, and thereby improving the accuracy of pain grading of image types.

[0019] In one embodiment, the multi-scale convolution mechanism described in step S3A includes a large-segment convolution branch and a small-segment convolution branch;

[0020] The specific steps of the large convolution branch are as follows:

[0021] Use a large kernel convolution to analyze the key pain signal data Perform large segment feature extraction to obtain large segment features Large GSR , its large kernel convolution is specifically expressed as:

[0022]

[0023] In the formula, Conv f,k,s represents the convolution operation, c, k, and s are the number of output channels, convolution kernel size, and step size, respectively. k is set to 400 and s is set to 50.

[0024] The Gaussian error linear unit activation function is used to perform nonlinear transformation, maximum pooling and feature mapping on large segments of features in sequence to obtain preliminary global signal features, which are specifically expressed as follows:

[0025] H Large =FC(MaxPool k,s (GELU(Large GSR )))

[0026] In the formula, H Large is the preliminary global signal feature; GELU represents the Gaussian error linear unit activation function; MaxPool k,s represents the maximum pooling operation, k is the window size, s is the step size; FC represents the fully connected layer;

[0027] Next, convolution-pooling is used to extract the preliminary global signal features to obtain the global priority signal features, which are specifically expressed as follows:

[0028] GlobalFeature=MaxPool k,s (GELU(Gonv c,k,s (H Large ))) R

[0029] Where GlobalFeature represents the global priority signal feature; Convc,k,s is the convolution operation; R represents the number of repeated executions;

[0030] The specific steps of the small segment convolution branch are as follows:

[0031] Use a small kernel convolution to analyze the key pain signal data Perform small segment feature extraction to obtain small segment features Small GSR , its small kernel convolution is specifically expressed as:

[0032]

[0033] In the formula, Conv f,k,s Indicates the convolution operation, k defaults to 50, and s defaults to 6;

[0034] The Gaussian error linear unit activation function is used to perform nonlinear transformation, maximum pooling and feature mapping on the small segment features in sequence to obtain preliminary local signal features, which are specifically expressed as follows:

[0035] H Small =FC(MaxPool k,s (GELU(Small GSR )))

[0036] In the formula, H Small is the preliminary local signal feature; MaxPool k,s Represents the maximum pooling operation;

[0037] Next, convolution-pooling is used to extract the preliminary local signal features to obtain the local priority signal features, which are specifically expressed as follows:

[0038] LocalFeature=MaxPool k,s (GELU(Conv c,k,s (H Small ))) R

[0039] Where LocalFeature represents the local priority signal feature; Conv c,k,s is the convolution operation; R represents the number of repeated executions;

[0040] After obtaining the global priority signal features and the local priority signal features, signal feature fusion and feature mapping are performed to obtain preliminary pain signal features. The specific calculation is as follows:

[0041]

[0042] Where PainFeature represents the preliminary pain signal feature, which integrates the features of the preliminary extracted global and local signals; is vector concatenation; FC is the fully connected layer;

[0043] The specific steps of extracting global spatial features using the compression-excitation residual network described in step S4A are as follows:

[0044] One-dimensional convolution and activation function are used to transform the features and obtain the preliminary signal feature map F trans , and its specific calculation is expressed as follows:

[0045] F trans =Conv c,1,1 (ReLU(Conv c,1,1 (PainFeature)))

[0046] Where PainFeature is the preliminary pain signal feature; Conv c,1,1 is a one-dimensional convolution; ReLU is a nonlinear activation function;

[0047] Next, global average pooling is used to spatially compress the preliminary signal feature map to obtain the compressed feature map F pool , which is specifically expressed as follows:

[0048]

[0049] In the formula, Represents the compressed feature map F pool The global average pooling value of the cth channel; represents the feature value of position (i, j) in channel c in the preliminary signal feature map; N represents the size of the feature map in the vertical dimension, that is, the height; L represents the size of the feature map in the horizontal dimension, that is, the width;

[0050] The compressed feature map is adaptively recalibrated to obtain the global enhanced signal features. The specific calculation of the adaptive recalibration is as follows:

[0051]

[0052] Where EnhanceFeature is the global enhanced signal feature; W 1 and W 2 They represent the weight matrix of the fully connected layer respectively; σ represents the Sigmoid function; is element-by-element multiplication; + is element-by-element addition.

[0053] Accordingly, the present invention uses large-segment convolution branches and small-segment convolution branches in a multi-scale convolution mechanism and convolution kernels of different scales to extract key information in the signal, thereby capturing the complex temporal relationships and high-dimensional dependencies in the pain signal, and comprehensively mining the multi-level features in the signal, thereby improving the accuracy and robustness of pain recognition.

[0054] At the same time, the present invention adopts the residual structure in the compression-excitation residual network to prevent the loss of feature information, and weights the feature map channel by channel through an adaptive calibration mechanism, so that the extracted features can adaptively focus on key signals related to pain, and suppress unimportant or noisy features, effectively capturing key changes in pain signals. Accordingly, through the synergistic effect of compression and excitation, the present invention can enhance the global information and local details in the signal during the multi-level feature extraction process, thereby making the identification of pain signals more accurate, thereby reducing the recognition errors caused by individual differences, noise or signal ambiguity, and significantly improving the accuracy of pain degree grading of physiological signal types.

[0055] Furthermore, the specific steps of the voltage transformer described in step S5A are as follows:

[0056] The global enhanced signal feature is copied to obtain a plurality of copies of the global enhanced signal feature;

[0057] Left-fill several copies of global enhanced signal features respectively, and use time-domain convolutional network to extract time-dependent features in parallel to obtain several time-domain signal feature maps;

[0058] A multi-head attention mechanism is used to extract attention features from all time domain signal feature maps to obtain the final pain signal features. The specific calculation expression of the multi-head attention mechanism is as follows:

[0059] Perform multi-head parallel attention calculation on all time domain signal feature maps to obtain several attention scores. For the i-th attention score AttScore i , and its specific calculation is expressed as:

[0060]

[0061] In the formula, is the time domain signal feature map, where V represents the feature map as a value vector; L represents the number of input feature sequences; e ij It represents the similarity score between the i-th query vector and the j-th key vector, which is obtained by dot product calculation. The specific calculation can be expressed as:

[0062]

[0063] In the formula, and Represented as the time domain signal feature map as the query vector and as the key vector respectively;

[0064] Next, all attention scores are weighted concatenated, and the weighted concatenated attention scores are fused with the global enhanced signal features to obtain the final pain signal feature FinPainFeature. The specific calculation is as follows:

[0065] FinPainFeature=Normalization(AttConcat)

[0066] AttConcat=EnhanceFeature+Concat(AttScore 1 ,AttScore 2 ,…,AttScore H )·W

[0067] In the formula, H represents the total number of attention heads; W represents the learnable weight, which is used for linear transformation and makes the dimensions of input and output the same; + represents additive fusion; Normalization represents the normalization operation; AttConcat is the attention score after weighted concatenation.

[0068] Accordingly, the present invention ensures that the convolution operation does not leak future signal information by padding the left side of the time domain convolutional network (TCN) in the transformer encoder, thereby avoiding the influence of future data on the prediction of the current state; at the same time, the receptive field of the network is comprehensively expanded in combination with the stacked one-dimensional convolution in the time domain convolutional network to learn more abstract timing features in the signal, so as to capture subtle changes in pain signals and reduce the influence of noise caused by individual differences, thereby improving the recognition accuracy of key signal features in the signal.

[0069] In addition, the present invention is based on a time-domain convolutional network and combines a multi-head attention mechanism to deeply understand from different perspectives the timing features that only depend on current and historical signals and are not interfered by future signals, so as to ensure that the time causality is strictly followed during the feature extraction process. In this way, the network can capture time dependencies and key signal areas while avoiding the leakage of future information that interferes with model predictions, thereby significantly improving the accuracy of pain recognition.

[0070] In another embodiment, the dual-branch feature extraction mechanism in step S3B includes a residual convolution local feature extraction branch and an attention-based global feature extraction branch;

[0071] The residual convolution local feature extraction branch is provided with several layers of residual convolution modules. i , which is specifically expressed as follows:

[0072] ResFeature i =Residual i (ResFeature i-1 )+ResFeature i-1

[0073] In the formula, ResFeature i Residual is the i-th layer residual convolution module i Extracted local feature map; + represents residual connection; Residual i The structure is represented as follows:

[0074]

[0075] In the formula, DownConv 1×1 (FM) represents the downsampling operation on the feature map FM; SpatialConv 3×3 (FM) represents spatial convolution of feature map FM; UpConv 1×1 (FM) represents the upsampling operation on the feature map FM; N i ResFeature represents the residual convolution module of layer i i The total number of internal convolution operations;

[0076] The attention-based global feature extraction branch is equipped with several layers of moving window attention modules SwinTrans i , which is specifically expressed as follows:

[0077]

[0078] Where AttFeature i Represented as the i-th layer moving window attention module SwinTrans i Extracted global feature map; L swin Indicates the total number of layers of the attention-based global feature extraction branch; SwinTrans i The structure is represented as follows:

[0079]

[0080] FM mid =WMSA(LN(FM))+FM

[0081] Where LN is layer normalization; WMSA is the window multi-head attention mechanism; MLP is a module containing two fully connected layers and a nonlinear activation function; FM mid Represented as an intermediate feature map;

[0082] When the residual convolution module Residual i The extracted local feature map and the i-th layer moving window attention module SwinTrans i After extracting the global feature map, the residual and attention mechanism are used alternately to perform feature fusion, which is specifically expressed as follows:

[0083]

[0084] finalFuse i =CBAM(Res(fuse i )+Conv 1×1 (fuse i ))

[0085] In the formula, reshape(AttFeature i ) represents the global feature map AttFeature i Resize to the local feature map ResFeature i Consistent; || represents a concatenation operation, which is used to form a local aggregation feature map fuse i ; Represents the aggregation operation of the i-th layer; finalFuse i is the final fused feature map of the i-th layer, which is represented by the feature map after the feature maps of different branches of the i-th layer are completely fused; Conv 1×1 Represents a convolution operation with a convolution kernel of 1x1; Res represents a residual block, and its specific calculation is as follows:

[0086] Res(fuse i )=ReLU(BN(Conv 3×3 (fuse i ))) K

[0087] In the formula, ReLU is the linear rectification activation function; BN is batch normalization; Conv 3×3 Represents a convolution operation; K represents repeated execution of ReLU (BN (Conv 4×4 (fuse i ))) operation K times; CBAM is the convolutional attention mechanism, which is specifically expressed as follows:

[0088]

[0089] Where, SpatialAtt represents the spatial attention module; ChannelAtt represents the channel attention module; Represents a point-by-point multiplication operation;

[0090] After obtaining the final fused feature map of the i-th layer, it is fused back to the feature map of the corresponding branch, which is specifically expressed as follows:

[0091] AttFeature i =AyyFeature i +finalFuse i

[0092] ResFeature i =ResFeature i +finalFuse i

[0093] Where AttFeature i and ResFeature i Represented as respectively with the final fusion feature map finalFuse i To merge.

[0094] The present invention fully extracts local feature information in facial images through the residual convolution local feature extraction branch, especially in capturing deep features of subtle changes in facial expressions and muscles. At the same time, through the attention-based global feature extraction branch, it effectively captures the overall features and contextual information in facial images, significantly improving the global representation ability of feature extraction. Based on this, the present invention further adopts the strategy of alternating fusion of residual and attention mechanisms to achieve deep fusion in the feature space extracted at each scale of the branch, thereby giving full play to the complementarity of local and global features, and improving the integrity of feature representation and the robustness of facial expression recognition.

[0095] Furthermore, the specific calculation of the final pain characteristic graph is expressed as follows:

[0096]

[0097] In the formula, ResFeature (i＝last) Represents the local priority feature map of the final layer last; AttFeature (i＝last) Represents the global priority feature map of the final layer last; Represents the addition operation;

[0098] The specific grading process of the patient's pain level based on the final pain characteristic graph is as follows:

[0099]

[0100] Where P(y=k|x) represents the probability that the input feature belongs to the kth level of pain; W k and b kRespectively represent the weight matrix and bias vector of the Softmax layer; C represents the total number of levels of pain severity; x represents a one-dimensional vector that has been nonlinearly transformed by ReLU, and its specific calculation is expressed as:

[0101] x = ReLU(W FC flatten(finalFeature (i＝last) )+b FC )

[0102] Where flatten is the flattening operation; W FC and b FC Represents the weight matrix and bias matrix of the fully connected layer.

[0103] Based on this, the present invention effectively integrates the key information of the two features by fusing the local priority features and the global priority features in the final fully connected layer, which not only retains the detailed changes in facial expressions, but also strengthens the contextual relevance of the overall expression, and uses the SoftMax layer for classification, thereby achieving a high-precision pain recognition effect.

[0104] Further, if the data type of the pain data to be detected includes both a physiological signal type and an image type, step S2A and step S2B are performed simultaneously;

[0105] After completing the pain grading of physiological signal type and image type at the same time, and obtaining the pain degree of patients of physiological signal type and image type at the same time, a fully connected layer of adaptive weight fusion is used to fuse the pain degree of all patients, obtain the pain degree of patients with high accuracy, and complete the pain grading of patients;

[0106] The specific calculation of the fully connected layer of the adaptive weight fusion is expressed as follows:

[0107] FinResult=W s SingalResult+W I ImageResult

[0108] Where FinResult is the pain level of the patient with high accuracy; SingalResult is the pain level of the patient with physiological signal type; ImageResult is the pain level of the patient with image type; W s and W I They are the learnable weight matrices of the physiological signal type and the image type in the fully connected layer, respectively.

[0109] Based on this, the present invention realizes the joint processing of physiological signal types and image types through a fully connected layer with adaptive weight fusion. By integrating the characteristics of the two modalities, namely the continuity characteristics of the physiological signals and the intuitive spatial characteristics of the facial images, the complementary advantages of the two are fully utilized, thereby overcoming the limitations of a single modality that is easily affected by noise, individual differences or specific scenarios, and significantly improving the accuracy of pain level recognition.

[0110] A device for identifying the degree of pain of a patient, comprising a signal type judgment unit, a signal preprocessing unit, a multi-scale convolution feature extraction unit, a compression-excitation residual feature extraction unit, a sequence feature extraction unit, a signal data pain grading unit, an image preprocessing unit, a dual-branch feature extraction unit, a global-local feature map fusion unit and an image data pain grading unit;

[0111] The signal type determination unit is used to determine the data type of the pain data to be detected: if the data type is a physiological signal type, the signal preprocessing unit is called; if the data type is an image type, the image preprocessing unit is called;

[0112] The signal preprocessing unit performs signal preprocessing on the pain data to be detected to obtain key pain signal data;

[0113] The multi-scale convolution feature extraction unit is used to extract features from key pain signal data using a multi-scale convolution mechanism to obtain preliminary pain signal features;

[0114] The compression-excitation residual feature extraction unit is used to extract global spatial features from the preliminary pain signal features using a compression-excitation residual network to obtain global enhanced signal features.

[0115] The sequence feature extraction unit is used to extract sequence features from the global enhanced signal features using a voltage transformer to obtain the final pain signal features;

[0116] The signal data pain grading unit is used to perform pain grading on the final signal features using a support vector machine, obtain the patient's pain level, and complete the pain grading of the physiological signal type;

[0117] The image preprocessing unit is used to perform image preprocessing on the pain data to be detected to obtain a facial pain expression image;

[0118] The dual-branch feature extraction unit is used to perform multi-scale feature extraction on the facial pain expression image by using a dual-branch feature extraction mechanism, and alternately perform feature fusion in combination with the residual and attention mechanisms to obtain a local priority feature map and a global priority feature map;

[0119] The global-local feature map fusion unit is used to fuse the local priority feature map and the global priority feature map to obtain a final pain feature map;

[0120] The image data pain grading unit is used to flatten the final pain feature map into a one-dimensional vector, and map the one-dimensional vector to the pain degree through a fully connected layer to obtain the patient's pain degree and complete the pain grading of the image type.

[0121] A pain level recognition system includes a patient end, a doctor end and a family end, wherein the patient end obtains pain data to be detected through a camera or a GSR signal real-time monitor and inputs the data into a pain level recognition device;

[0122] The pain degree recognition device extracts features and grades pain on the pain data to be detected, obtains the patient's pain degree, and inputs it to the doctor end and the family end respectively;

[0123] The doctor end is used to obtain the patient's pain level, and by obtaining the associated data in the patient database, display the associated data and the current patient's pain level for the doctor to analyze and obtain the doctor's treatment suggestions for the patient;

[0124] The family end is provided with an AI assistant module, which is used to conduct personalized analysis of the patient's pain level history records and provide preliminary medical analysis, pain care suggestions and personalized diet plans based on the analysis results;

[0125] Wherein, the pain level identification device is the pain level identification device of the above-mentioned patient.

[0126] Based on this, the present invention realizes information sharing and interaction between patients, doctors and family members through the pain level identification system, and ensures the timely transmission of patient pain data, thereby improving the transparency and interactivity of the entire medical process. Especially when the patient is in the home observation stage after discharge, family members can understand the patient's pain condition in time and obtain intelligent nursing suggestions, thereby assisting the patient's pain distress or complications caused by pain.

[0127] At the same time, through automated pain level identification, the influence of subjective factors can be eliminated, thus providing objective and timely data, effectively helping medical staff to accurately judge the patient's pain condition and adjust the medical plan in time, thereby significantly improving the efficiency of medical staff in postoperative management of patients.

[0128] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] Figure 1A simplified structural diagram of the device for identifying the pain level of a patient according to the present invention;

[0130] Figure 2 A simplified flow chart of the method for identifying the pain level of a patient according to the present invention;

[0131] Figure 3 It is a schematic diagram of the overall structure of the multi-scale convolution mechanism, compression-excitation residual network and transformer encoder described in the present invention;

[0132] Figure 4 This is a schematic diagram of a simplified structural example of the multi-scale convolution mechanism of the present invention;

[0133] Figure 5 A schematic diagram of a simplified structural example of a compression-excitation residual network according to the present invention;

[0134] Figure 6 A schematic diagram of a simplified structural example of the voltage-variable encoder of the present invention;

[0135] Figure 7 A schematic diagram showing an example of a reference image preprocessing process for facial images;

[0136] Figure 8 It is a schematic diagram of the simplified process structure of the dual-branch feature extraction mechanism and the alternating feature fusion described in the present invention;

[0137] Fig. 9 This is a schematic diagram of an example structure of a residual convolution module of a residual convolution local feature extraction branch according to the present invention;

[0138] Fig.10 A schematic diagram of an example structure of a moving window attention module of the attention-based global feature extraction branch of the present invention;

[0139] Fig.11 It is a schematic diagram of a simplified process example of the alternating feature fusion structure of the present invention;

[0140] Fig.12 A schematic diagram of a simple structural example of CBAM in the alternating feature fusion structure described in the present invention. DETAILED DESCRIPTION

[0141] In order to solve the problem of insufficient accuracy in pain degree recognition in the prior art, the present invention obtains the patient's pain data to be detected and determines the data type of the pain data: if the data type is a physiological signal type, the pain data to be detected is subjected to signal preprocessing to obtain key pain signal data, and a multi-scale convolution mechanism, a compression-excitation residual network, and a transformer encoder are sequentially used to extract features of the pain data to be detected to obtain the final pain signal features, and then, the final signal features are graded for pain by a support vector machine to obtain the patient's pain degree; if the data type is an image type, a dual-branch feature extraction mechanism is combined with a residual and attention mechanism to alternately extract and fuse features to obtain a global priority feature map and a local priority feature map, which are then fused and a fully connected layer is used to grade the fused feature map to obtain the patient's pain degree. Accordingly, the present invention adopts a dual-branch pain detection method to allow users to detect pain levels in different environments, so as to ensure the convenience of self-examination during postoperative recovery; at the same time, when grading pain based on image data, the present invention adopts a dual-branch feature extraction mechanism combined with the residual and attention mechanism to perform high-dimensional feature extraction in the image, and locally fuses global features to comprehensively consider the multi-level abstract features of the patient's expression in the image, as well as the overall structure of the face, thereby ensuring that the extracted features are sufficient to support the accuracy of pain detection.

[0142] In addition, when performing pain grading based on physiological signal data, the present invention adopts a multi-scale convolution mechanism, a compression-excitation residual network, a transformer encoder and a support vector machine to adapt to the particularity and complexity of the physiological signal data, thereby improving the accuracy of pain grading based on physiological signals.

[0143] Based on the above design, the present invention proposes a method for identifying the pain level of a patient, and based on the method, proposes a device for identifying the pain level of a patient.

[0144] Please also see Figure 1 and Figure 2 , Figure 1 is a simplified structural diagram of the device for identifying the pain level of a patient according to the present invention, Figure 2 The figure is a simplified flowchart of the method for identifying the pain level of a patient according to the present invention.

[0145] The device for identifying the patient's pain level includes a signal type judgment unit 1, a signal preprocessing unit 2A, a multi-scale convolution feature extraction unit 3A, a compression-excitation residual feature extraction unit 4A, a sequence feature extraction unit 5A, a signal data pain grading unit 6A, an image preprocessing unit 2B, a dual-branch feature extraction unit 3B, a global-local feature map fusion unit 4B and an image data pain grading unit 5B.

[0146] The signal type determination unit 1 is used to execute step S1: determine the data type of the pain data to be detected: if the data type is a physiological signal type, call the signal preprocessing unit 2A; if the data type is an image type, call the image preprocessing unit 2B.

[0147] Specifically, the pain data to be detected includes physiological signal type and image type; the pain data of the physiological signal type is skin electrical signal (GSR), which specifically obtains the skin electrical signal of a patient in pain at a preset sampling rate, and segments the skin electrical signal as the pain data to be detected; wherein the preset sampling rate defaults to 512Hz.

[0148] The image-type pain data is a facial image of a patient, which specifically involves photographing the face of a patient in pain to obtain corresponding facial expressions, which are used as the pain data to be detected.

[0149] Accordingly, the present invention flexibly switches between different types of pain data, including physiological signals (such as skin electrical signals GSR) and image data (such as facial expression images), so that users can choose appropriate data types for pain classification according to actual conditions, thereby improving the convenience and universality of the system.

[0150] It should be noted that although physiological signal data can provide more accurate and comprehensive pain grading, it usually requires dedicated testing instruments and is more complicated to operate. Therefore, it is more suitable for neurological testing in hospital environments to provide doctors with reference data on patients' responses to pain. Facial expression data may be affected by individual differences, resulting in relatively low accuracy, but it is easy to collect and can be achieved through conventional camera equipment. It has good portability and is especially suitable for postoperative or non-hospitalized patients, which can reduce the frequency of patients' visits and related expenses.

[0151] Furthermore, if the data type of the pain data to be detected includes both physiological signal type and image type, the signal preprocessing unit 2A and the image preprocessing unit 2B are called simultaneously, so as to perform pain grading in a multimodal manner and promote the complementarity or interaction of different data types, thereby improving the reliability of pain grading.

[0152] The signal preprocessing unit 2A is used to execute step S2A: performing signal preprocessing on the pain data to be detected to obtain key pain signal data.

[0153] Specifically, bandpass filtering is used to reduce the noise of the pain data to be detected to obtain the key pain signal data. The specific representation of bandpass filtering to reduce the noise of the pain data to be detected is as follows:

[0154]

[0155] In the formula, is the key pain signal data, that is, the skin electrical signal after noise reduction, which is used to retain the effective information in the specific frequency band related to pain; x[n] is the pain data to be detected, that is, the skin electrical signal collected according to the preset sampling rate; h[m] represents the impulse response function of the bandpass filter, which is used to select and retain the signal in a specific frequency band while removing the low-frequency baseline drift and high-frequency noise; m and n represent the index of the discrete signal, where the n index represents the current sampling point, and the m index represents the time delay index in the filtering operation; M represents the window length of the filter, that is, the impulse response range of the filter, which is used to determine the length of the historical data used for each sampling point in the filtering calculation, which is specifically determined by the design order of the filter.

[0156] Since bandpass filter is a commonly used basic noise reduction method, it can remove most common noises and retain the effective components of the signal. However, depending on the acquisition environment, noise type or instrument specifications, it may be necessary to select or match a suitable filter for noise reduction. Therefore, the present invention does not specifically limit the choice of filter.

[0157] The multi-scale convolution feature extraction unit 3A is used to execute step S3A: extracting features from key pain signal data using a multi-scale convolution mechanism to obtain preliminary pain signal features.

[0158] For details, please also refer to Figure 3 and Figure 4 , the multi-scale convolution mechanism includes a large-segment convolution branch and a small-segment convolution branch; the large-segment convolution branch is used to extract the global features of the signal, and its specific steps are as follows:

[0159] Use a large kernel convolution to analyze the key pain signal data Perform large segment feature extraction to capture the signal trend characteristics within a long time window and obtain large segment features. GSR , which is specifically expressed as:

[0160]

[0161] In the formula, Conv f,k,s Represents the convolution operation, c, k, s are the number of output channels (Channel), convolution kernel size (Kernel Size) and stride (Stride), where c defaults to 64, k defaults to 400, and s defaults to 50, which is used to cover the 0.8 second skin electrical signal window to obtain large-segment features.

[0162] The Gaussian error linear unit activation function is used to perform nonlinear transformation, maximum pooling and feature mapping on large segments of features in sequence to obtain preliminary global signal features, which are specifically expressed as follows:

[0163] H Large =FC(MaxPool k,s (GELU(Large GSR )))

[0164] In the formula, H Large It is a preliminary global signal feature with a global signal trend feature of a large time window; GELU represents the Gaussian error linear unit activation function, which is used to smooth the input signal to smooth the boundary data to prevent the disappearance of neurons and limit the depth of negative state activation, thereby allowing negative weights to pass through the network; MaxPool k,s represents the maximum pooling operation, which is used to compress the feature length, reduce redundancy, and retain the significant features of the signal. k is the kernel size, s is the stride, which defaults to 4 and s defaults to 2. FC represents the fully connected layer, which is used to map the pooled features to generate preliminary global signal features of fixed length.

[0165] Next, convolution-pooling is used to extract the preliminary global signal features to obtain the global priority signal features, which are specifically expressed as follows:

[0166] GlobalFeature=MaxPool k,s (GELU(Conv c,k,s (H Large ))) R

[0167] Where GlobalFeature represents the global priority signal feature; Conv c,k,s It is a convolution operation, where c defaults to 128, k defaults to 7, and s defaults to 1. R represents the number of repeated executions, which defaults to 2. Richer global features are extracted through two iterations.

[0168] The small convolution branch is used to extract the local features of the signal, and the specific steps are as follows:

[0169] Use a small kernel convolution to analyze the key pain signal data Perform small segment feature extraction to capture local change features within a short time window and obtain small segment features GSR , which is specifically expressed as:

[0170]

[0171] In the formula, Convf,k,s Represents a convolution operation, where c defaults to 64, k defaults to 50, and s defaults to 6, which is used to cover a 0.1 second skin electrical signal window to obtain small segment features.

[0172] The Gaussian error linear unit activation function is used to perform nonlinear transformation, maximum pooling and feature mapping on the small segment features in sequence to obtain preliminary local signal features, which are specifically expressed as follows:

[0173] H Small =FC(MaxPool k,s (GELU(Small GSR )))

[0174] In the formula, H Small is the preliminary local signal feature; MaxPool k,s Represents the maximum pooling operation, where k defaults to 8 and s defaults to 2.

[0175] Next, convolution-pooling is used to extract the preliminary local signal features to obtain the local priority signal features, which are specifically expressed as follows:

[0176] LocalFeature=MaxPool k,s (CELU(Conv c,k,s (H Small ))) R

[0177] Where LocalFeture represents the local priority signal feature; Conv c,k,s For the convolution operation, c defaults to 128, k defaults to 8, and s defaults to 1; R represents the number of repeated executions, which defaults to 2.

[0178] After obtaining the global priority signal features and the local priority signal features, signal feature fusion and feature mapping are performed to obtain preliminary pain signal features. The specific calculation is as follows:

[0179]

[0180] Where PainFeature represents the preliminary pain signal feature, which integrates the features of the preliminary extracted global and local signals; It is a vector concatenation layer used to fuse the global priority signal features and the local priority signal features; FC is a fully connected layer used to map the fused features.

[0181] Accordingly, the present invention extracts the global features of the GSR signal through a large convolution kernel (400x400) and a large step size to capture the trend information and global features within a long time window, thereby analyzing the slow-changing characteristics in the skin electrical signal; at the same time, the local features of the GSR signal are extracted through a small convolution kernel (50x50) and a shorter step size to capture the subtle changes and local dynamic features within a short time window, and by fusing and mapping the two features, the effective combination of global and local information is achieved, thereby improving the fine-grainedness and contextual information of the features of the pain signal, providing features with significant distinguishing ability for the subsequent classification of the degree of pain.

[0182] The compression-excitation residual feature extraction unit 4A is used to execute step S4A: using the compression-excitation residual network to perform global spatial feature extraction on the preliminary pain signal features to obtain global enhanced signal features.

[0183] Specifically, the specific steps of extracting global spatial features using the compression-excitation residual network are as follows:

[0184] One-dimensional convolution and activation function are used to transform the features and obtain the preliminary signal feature map F trans , and its specific calculation is expressed as follows:

[0185] F trans =Conv c,1,1 (ReLU(Conv c,1,1 (PainFeature)))

[0186] Where PainFeature is the preliminary pain signal feature; Conv c,1,1 It is a one-dimensional convolution, and its channel number c defaults to 30; ReLU is a nonlinear activation function.

[0187] Next, global average pooling is used to spatially compress the preliminary signal feature map to obtain the compressed feature map F pool , which is specifically expressed as follows:

[0188]

[0189] In the formula, Represents the compressed feature map F pool The global average pooling value of the cth channel; represents the feature value of position (i, j) in channel c in the preliminary signal feature map; N represents the size of the feature map in the vertical dimension, that is, the height; L represents the size of the feature map in the horizontal dimension, that is, the width; wherein the global average pooling is used to extract the global features of each channel, thereby extracting the most significant signal response value and reducing the dimension to make the features compact;

[0190] The compressed feature map is adaptively recalibrated to obtain the global enhanced signal features. The specific calculation of the adaptive recalibration is as follows:

[0191]

[0192] Where EnhanceFeature is the global enhanced signal feature, which is used to represent the recalibrated feature map; W 1 and W 2 They represent the weight matrix of the fully connected layer, which is used to reflect the relationship between channels and weight the channels with the most information, which is obtained through training and learning; σ represents the Sigmoid function, which is used to normalize the weight to the interval [0,1]; It is element-by-element multiplication, which is used for channel weighting operation, thereby suppressing irrelevant or noise features and emphasizing significant features; + is element-by-element addition, which is used for residual connection, thereby retaining the original input features and enhancing gradient transfer.

[0193] Please also see Figure 5 and Figure 6 , Figure 5 Schematic diagram of a simplified structure example of the compression-excitation residual network of the present invention, Figure 6 This is a schematic diagram of a simplified structural example of the voltage-variable encoder described in the present invention.

[0194] The sequence feature extraction unit 5A is used to execute step S5A: using a voltage transformer to extract sequence features from the global enhanced signal features to obtain the final pain signal features.

[0195] Specifically, the specific steps of the voltage transformer are as follows:

[0196] The global enhanced signal feature is copied to obtain several copies of the global enhanced signal feature; wherein the copying operation is used to perform multi-path processing on the global enhanced signal feature, thereby increasing the depth of the model or processing multiple different feature paths, thereby increasing the depth of the network or enhancing the expression ability of the network.

[0197] Then, several copies of the global enhanced signal features are padded on the left side, and the time-dependent features are extracted in parallel using a temporal convolutional network (TCN) to obtain several time-domain signal feature maps; wherein, the left-side padding is used to ensure that the output of each time step only depends on the current and past time step data, and does not involve future time step data, thereby ensuring the causality of the convolution operation; the temporal convolutional network captures the temporal dependency in the input feature sequence through causal convolution and extended convolution operations in stacked one-dimensional convolutional layers, thereby extracting the dependency between different time steps, and then modeling the long-term and short-term dependency information in the feature sequence of the time series type.

[0198] The multi-head attention mechanism (MHA) is used to extract attention features from all time domain signal feature maps to obtain the final pain signal features;

[0199] The multi-head attention mechanism obtains several attention scores by performing multi-head parallel attention calculations on all time domain signal feature maps. For the i-th attention score AttScore i , its specific calculation expression can be referred to:

[0200]

[0201] In the formula, is the time domain signal feature map, where V represents the feature map as a value vector (Value), that is, the weighted representation of the input feature; L represents the number of input feature sequences, that is, the total number of features input into the multi-head attention mechanism; e ij It represents the similarity score between the i-th query vector (Query) and the j-th key vector (Key), which is obtained by dot product calculation. The specific calculation can be expressed as:

[0202]

[0203] In the formula, and Represented as the time domain signal feature map as the query vector and as the key vector respectively;

[0204] Next, all attention scores are weighted concatenated, and the weighted concatenated attention scores are fused with the global enhanced signal features to obtain the final pain signal feature FinPainFeature. The specific calculation is as follows:

[0205] FinPainFeature=Normalization(AttConcat)

[0206] AttConcat=EnhanceFeature+Concat(AttScore 1 ,AttScore 2 ,…,AttScore H )·W

[0207] In the formula, H represents the total number of attention heads; W represents the learnable weight, which is used for linear transformation and makes the input and output dimensions the same; + represents additive fusion; Normalization represents the normalization operation, which is used to normalize the weighted concatenated attention score AttConcat.

[0208] It should be noted that the specific structural parameters (such as convolution kernel size, expansion factor, number of heads, etc.) of the time domain convolutional network (TCN) and the multi-head attention mechanism (MHA) will be flexibly selected according to the needs of actual applications. Therefore, the present invention does not specifically limit the selection of its structural parameters.

[0209] The signal data pain grading unit 6A is used to execute step S6A: using a support vector machine to perform pain grading on the final signal features, obtain the patient's pain level, and complete the pain grading of the physiological signal type.

[0210] Among them, through the support vector machine, while ensuring the overall network performance, the number of model parameters can be reduced and the relationship between features can be fully utilized.

[0211] Since support vector machines (SVMs) have strong generalization capabilities, they can effectively reduce overfitting when processing high-dimensional features. At the same time, SVMs can mine the nonlinear relationship between signal features during training and fit the grading boundaries between different pain levels. Based on this, pain grading through the interface in the feature space can reduce parameter redundancy and the interference of individual differences, thereby improving the accuracy and stability of pain grading.

[0212] In addition, since pain usually does not occur instantaneously but is a dynamic process with a temporal nature, the present invention performs temporal modeling of signal features through a time-domain convolutional network (TCN), that is, extracts the temporal dependencies between features to capture the dynamic characteristics of pain signals that change over time, thereby constructing long-term and short-term dependency information in pain signal features, thereby improving the ability to accurately classify pain.

[0213] See also Figure 7 , Figure 7 A diagram showing the image preprocessing process for facial images.

[0214] The image preprocessing unit 2B is used to execute step S2B: performing image preprocessing on the pain data to be detected to obtain a facial pain expression image.

[0215] Specifically, a patient face recognition algorithm is used to locate the patient's face in the pain data to be detected, and a patient face feature point extractor is used to obtain the patient's face key points.

[0216] Among them, the patient face recognition algorithm is used to determine the position of the patient's facial area in the image. Specifically, the patient face recognition algorithm in the Dlib library can be used to accurately locate the face in the image, so as to obtain an image of the facial area; the patient facial feature point extractor is used to locate the facial feature points in the image of the facial area, that is, the patient's facial key points, which are used for subsequent facial image alignment, size normalization, etc. The patient's facial key points have a specification of 68 points, but different numbers of key point combinations can be used according to specific circumstances, and the present invention is not specifically limited here.

[0217] Next, the patient's facial image is aligned, resized, and grayed according to the patient's facial key points to obtain a facial pain expression image.

[0218] Among them, the patient facial image alignment is used to rotate and crop according to the patient's facial key points so that the corresponding facial features are in the standard position of the image to focus on the facial expression area, thereby reducing recognition errors caused by angles or postures; the size normalization is to adjust the facial expression area to a fixed size through bilinear interpolation to ensure that the input image maintains a consistent size, thereby ensuring efficient calculation of the neural network; the grayscale is used to convert the color facial expression area image into a single-channel grayscale image through weighted averaging to reduce computational complexity and retain the core features of the facial expression area image.

[0219] It should be noted that there are many different algorithms in Dlib. Depending on the actual situation, the face recognition-related algorithms in OpenCV or MTCNN can also be used. The present invention does not specifically limit the selection of algorithms related to face recognition and positioning.

[0220] The dual-branch feature extraction unit 3B is used to execute step S3B: adopt a dual-branch feature extraction mechanism to perform multi-scale feature extraction on the facial pain expression image, and combine the residual and attention mechanisms to alternately perform feature fusion to obtain a local priority feature map and a global priority feature map.

[0221] Specifically, the dual-branch feature extraction mechanism includes a residual convolution local feature extraction branch and an attention-based global feature extraction branch;

[0222] The residual convolution local feature extraction branch is provided with several layers of residual convolution modules for performing multi-scale feature extraction on facial pain expression images. Please also refer to Figure 8 and Fig. 9 , which is the residual convolution module Residual for the i-th layer i The extracted local feature map of layer i is specifically expressed as follows:

[0223] ResFeature i =Residual i (ResFeature i-1 )+ResFeature i-1

[0224] In the formula, ResFeature i is the local feature map of the i-th layer. When i=1, it represents a facial pain expression image. The present invention defaults to a total number of i layers of 4. When i=4, it represents a local priority feature map. The + represents a residual connection. i The structure is represented as follows:

[0225]

[0226] In the formula, DownConv 1×1 (fM) represents the downsampling operation on the feature map fM, which is used to reduce the dimension of the feature map. The convolution kernel size is 1×1; SpatialConv 3×3 (FM) represents the spatial convolution of the feature map FM, and its convolution kernel size is 3×3; UpConv 1×1 (FM) represents an upsampling operation on the feature map FM, which is used to expand the feature map, and its convolution kernel size is 1×1; N i ResFeature represents the residual convolution module of layer i i The total number of internal convolution operations, the total number of convolution operations set N defaults to {3,4,6,3}.

[0227] The attention-based global feature extraction branch is equipped with several layers of moving window attention modules, which extracts features in each window by dividing the image into multiple small windows and applying the self-attention mechanism, thereby capturing local information and subtle features in the facial expression image, and improving the expression ability of the global features through the multi-level window combined with the attention mechanism; for the moving window attention module SwinTrans i The extracted i-th layer global feature map is specifically expressed as follows:

[0228]

[0229] Where AttFeature i It is represented as the global feature map of the i-th layer, which is used to represent the feature map calculated at each layer. When i=1, it represents the facial pain expression image, and the total number of layers of i is 4 by default in the present invention. When i=4, it represents the global priority feature map; L Swin represents the total number of layers of the attention-based global feature extraction branch; see Fig.10 , its SwinTrans i The structure is represented as follows:

[0230]

[0231] FM mid =WMSA(LN(FM))+FM

[0232] Where LN is layer normalization, which is used to standardize input features; WMSA is the window multi-head self-attention mechanism, which is used to capture local dependencies within the window and establish the association of global features layer by layer through the shifted windows strategy; FM mid It is represented by the intermediate feature map, that is, the feature map extracted by the window multi-head attention mechanism; MLP is a module containing two fully connected layers and a nonlinear activation function (such as GELU), which is used to enhance the nonlinear modeling ability of features; + represents the residual connection, which is used to alleviate the gradient vanishing problem, accelerate network convergence, and reduce feature loss in the feature extraction process.

[0233] Next, to obtain the local feature map ResFeature of the i-th layer i And the global feature map AttFeature i After that, the residual and attention mechanisms are used alternately to perform feature fusion. Please also refer to Fig.11 and Fig.12 , which is specifically expressed as follows:

[0234]

[0235] finalFuse i =CBAM(Res(fuse i )+Conv 1×1 (fuse i ))

[0236] In the formula, reshape(AttFeature i ) represents the global feature map AttFeature iResize to the local feature map ResFeature i Consistent; || represents concatenation, which forms a local aggregate feature map fuse by concatenating the global feature map and the local feature map i ; Represents the aggregation operation of the ReLU layer. According to the number of layers of the corresponding branch, the size of its global feature map after re is inconsistent; finalFuse i is the final fused feature map of the ReLU layer, which is represented by the feature map after the feature maps of different branches in the i-th layer are completely fused; Conv 1×1 Represents a convolution operation with a convolution kernel of 1x1; Res represents a residual block, and its specific calculation is as follows:

[0237] Res(fuse 3i )=ReLU(BN(Conv 3×3 (fuse 3i ))) K

[0238] In the formula, ReLU is the rectified linear unit, which is used to improve the nonlinear representation ability of features; BN is batch normalization; Conv 3×3 Indicates a convolution operation with a convolution kernel of 3x3; K indicates repeated execution of ReLU (BN (Conv 3×3 (fuse i ))) operation K times, the default value is 2; CBAM is the Convolutional Block Attention Module, which is used to perform channel and spatial weighting on features to enhance the key information in the features. Please also refer to it. Its specific representation is as follows:

[0239]

[0240] Where, SpatialAtt represents the spatial attention module, which is used to calculate the attention weight of each spatial position in the input feature map, so as to strengthen the features of important areas in the input feature map; ChannelAtt represents the channel attention module, which is used to calculate the attention weight of each channel in the input feature map to highlight the important channel information in the input feature map; It represents a point-by-point multiplication operation, where the attention weight is multiplied point-by-point with the input feature map, thereby realizing channel or spatial attention weighting to obtain a weighted feature map.

[0241] After obtaining the final fused feature map of the i-th layer, it is fused back to the feature map of the corresponding branch, which is specifically expressed as follows:

[0242] AttFeature i =AttFeature i +finalFuse i

[0243] ResFeature i =ResFeature i +finalFuse i

[0244] Where AttFeature i and ResFeature i Represented as respectively with the final fusion feature map finalFuse i To merge.

[0245] It should be noted that, since different branches may correspond to different feature space dimensions, and these dimensions may vary with the number of layers, when fusion is performed, if the dimensions of the feature maps of the two branches are inconsistent, it is necessary to adjust the spatial size of the feature maps by reshaping or interpolation. Accordingly, in order to adapt to different needs, the present invention does not specifically limit the actual dimensions of the feature maps.

[0246] The present invention extracts local feature maps and global feature maps respectively through a dual-branch feature extraction mechanism, and uses a residual and attention mechanism to alternately fuse the global and local feature maps of each layer. Specifically, the residual connection is used to ensure the effective transmission of local features, and the attention mechanism is used to weight the fusion of global information, so that the feature map can retain local detail information and fuse global context information in the feature extraction process of each layer of the dual branches. Accordingly, the present invention can dynamically fuse local and global features in each layer of the dual-branch feature map, thereby improving the recognition ability of facial pain expressions, and then realizing the effective capture of multi-scale information, significantly improving the accuracy of pain expression recognition.

[0247] The global-local feature map fusion unit 4B is used to execute step S4B: fusing the local priority feature map and the global priority feature map to obtain a final pain feature map.

[0248] Specifically, the specific calculation of the final pain feature map is expressed as follows:

[0249]

[0250] In the formula, ResFeature (i＝last) Represents the local priority feature map of the final layer last; AttFeature (i＝last) Represents the global priority feature map of the final layer last; Represents an addition operation, which is used to effectively combine local micro information and global macro information to form a comprehensive representation, enhance the network's understanding of the overall facial expression, and avoid possible feature loss or information error; the default value of last is 4, and the feature extraction depth of the network can be adjusted according to actual conditions. The present invention does not specifically limit the choice of the number of layers.

[0251] Accordingly, the present invention additively fuses the final hierarchical feature maps of different feature extraction branches to fully utilize the complementarity of local features and global features, especially to capture detailed features of pain severity through local priority feature maps, and to reflect the global features of overall facial emotions and pain through global priority feature maps, and to fuse the two through an addition operation to effectively fuse local detail information with global context information, thereby accurately extracting high-dimensional features of facial expressions, thereby improving the accuracy of pain severity classification, and effectively addressing individual differences in pain perception due to facial expressions.

[0252] The image data pain grading unit 5B is used to execute step S5B: using a fully connected layer to perform pain grading on the final pain feature map, obtain the patient's pain level, and complete the image type pain grading.

[0253] Specifically, the final pain feature map is flattened into a one-dimensional vector, and the one-dimensional vector is mapped to the pain degree through a fully connected layer to obtain the patient's pain degree.

[0254] The fully connected layer is used as a classifier, which includes a nonlinear activation function (ReLU) and a Softmax layer, and outputs the probability distribution of each pain level. The specific calculation of the patient's pain level is expressed as follows:

[0255]

[0256] Where P(y=k|x) represents the probability that the input feature belongs to the kth level of pain, the sum of the probabilities of all pain levels is 1, and the level with the highest probability is the pain level of the current patient; W k and b k They represent the weight matrix and bias vector of the Softmax layer, respectively, which are obtained through training; C represents the total number of levels of pain severity, and its default value is 4; x represents a one-dimensional vector that has been nonlinearly transformed by ReLU, and its specific calculation is expressed as:

[0257] x = ReLU(W FC ·flatten(FinalFeature (i＝last) )+b FC )

[0258] In the formula, flatten represents the flattening operation, which arranges each dimension of the input feature map into a one-dimensional vector in order; W FC and b FC Represents the weight matrix and bias matrix of the fully connected layer, obtained through training.

[0259] Based on this, the present invention fully integrates local and global features, uses nonlinear mapping and Softmax classification, so that the network can capture the detailed local features in facial expressions, and combines the information of the global context, thereby comprehensively improving the precision of pain grading, especially when facing individual differences, and can significantly improve the accuracy of pain grading.

[0260] Furthermore, after completing the pain grading of physiological signal type and image type at the same time, and obtaining the pain levels of patients of physiological signal type and image type at the same time, a fully connected layer with adaptive weight fusion is used to fuse the pain levels of all patients to obtain the pain levels of patients with high accuracy and complete the pain grading of patients.

[0261] Specifically, the specific calculation of the fully connected layer of the adaptive weight fusion is expressed as follows:

[0262] FinResult=W s SingalResult+W I ImageResult

[0263] Wherein, FinResult is the patient's pain level with high accuracy; SingalResult is the patient's pain level of physiological signal type, which is obtained by pain grading the final signal features by support vector machine SVM. The present invention further adopts Platt scaling to make SVM map the final signal features to the probability space to obtain the probabilities of all pain level labels; ImageResult is the patient's pain level of image type, which is obtained by classification by Softmax layer, that is, including the probabilities of all pain level labels; W s and W I They are respectively the learnable weight matrix of the physiological signal type and the learnable weight matrix of the image type in the fully connected layer, which are obtained through training.

[0264] Compared with the prior art, the present invention combines a multi-scale convolution mechanism with a compression-excitation residual network in the pain degree grading of physiological signal types to deeply mine the key information in the physiological signals, and effectively captures the complex timing characteristics and high-dimensional dependencies in the physiological signals with the cooperation of the time domain convolution network and the multi-head attention mechanism. As well as the demarcation ability of the support vector machine in the feature space, accurate grading in the context of individual differences is achieved, thereby significantly improving the accuracy, reliability and generalization of pain grading of physiological signal types.

[0265] At the same time, in the pain degree classification of facial image types, the present invention adopts a dual-branch feature extraction mechanism to fully extract the local features and global features of facial expression images, especially through the residual convolution local feature extraction branch to capture subtle changes in expression, and combined with the attention-based global feature extraction branch to focus on overall facial changes and contextual information, and on the basis of the dual branches, the alternating fusion of residual and attention mechanisms is adopted to achieve a deep integration of local and global features, making the feature extraction of facial images more robust and diverse, thereby significantly improving the accuracy and stability of image type pain classification.

[0266] In addition, the present invention further integrates the pain grading results of physiological signals and facial image types through a fully connected layer with adaptive weight fusion, fully utilizing the temporal continuity characteristics of physiological signals and the spatial intuitive characteristics of facial images, supplemented by dynamically adjusting the weight distribution of different modalities, thereby effectively compensating for the deficiencies of single-modal data due to noise, individual differences or scene limitations, and thus significantly improving the accuracy and robustness of pain recognition.

[0267] It should be noted that in order to improve the accuracy of pain degree recognition, it is also necessary to train the trainable parameters corresponding to the weight matrix and bias matrix in the pain classification based on physiological signal type and the pain classification based on facial image type respectively.

[0268] The specific steps for the training of pain grading of the physiological signal type are as follows:

[0269] DA1. Collect skin electrical signals from several patients, and set pain level labels corresponding to the skin electrical signals based on the patients' subjective scores and the evaluation of professional physicians to construct a physiological signal dataset;

[0270] DA2. Divide the physiological signal data set into a training set, a validation set, and a test set.

[0271] Among them, the division ratio of the loss training set, the validation set and the test set is 7:2:1. Since the corresponding division can be selected as 8:1:1 or others according to different needs, the present invention does not specifically limit the selection of the division ratio.

[0272] DA3. Input the skin electrical signals of the patients in the training set into the method for identifying the pain level of the patients described in the present invention, perform pain grading, and obtain the pain level corresponding to the current skin electrical signals.

[0273] Among them, the corresponding hyperparameters in the method for identifying the pain level of the patient described in the present invention, such as the optimizer, learning rate and Dropout parameter of the fully connected layer, can be selected as: Adam optimizer, 0.001 learning rate, 0.5 dropout. Due to the different actual hardware environments, different hyperparameters can be selected, such as: grid search, Bayesian optimization, etc., to improve the convergence efficiency of training, so the present invention does not specifically limit the selection of its hyperparameters.

[0274] DA4. Use the loss function to calculate the loss between the pain level corresponding to the current skin electrical signal and the corresponding pain level label in the training set to obtain the loss value.

[0275] Among them, the loss function can be selected as cross entropy loss, and the loss function can adopt a weighted combination of multiple common loss functions, which can be set according to needs. Therefore, this application does not specifically limit the choice of its loss function.

[0276] DA5. Determine whether the current training batch or loss value meets a training condition: if so, complete the training and obtain a highly accurate method for identifying the patient's pain level; if not, back-propagate the method for identifying the patient's pain level described in the present invention according to the loss value to obtain the method for identifying the patient's pain level after updating the weights, and execute step DA3.

[0277] Among them, the search training conditions also include using a test set or a training set to identify the current patient's pain level to grade pain and calculate its accuracy. When the accuracy reaches a corresponding threshold, such as 90% pain grading accuracy, the training can be terminated early.

[0278] The specific steps for training the pain classification of the image type are as follows:

[0279] DB1, taking facial photos of patients in pain, and setting pain level labels corresponding to the facial images of patients based on their subjective scores combined with the evaluation of professional physicians, to construct a facial image dataset;

[0280] DB2. Divide the facial image dataset into a training set, a validation set, and a test set.

[0281] Among them, the division ratio of the loss training set, the validation set and the test set is 7:2:1. Since the corresponding division can be selected as 8:1:1 or others according to different needs, the present invention does not specifically limit the selection of the division ratio.

[0282] DB3. Input the facial images of the patients in the training set into the method for identifying the pain level of the patients described in the present invention, perform pain grading, and obtain the pain level corresponding to the current facial image.

[0283] Among them, the corresponding hyperparameters in the method for identifying the pain level of the patient described in the present invention, such as the optimizer, the learning rate and the Dropout parameter of the fully connected layer. Since different hyperparameters can be selected according to different actual hardware environments, such as grid search, Bayesian optimization, etc., to improve the convergence efficiency of training, the present invention does not specifically limit the selection of its hyperparameters.

[0284] DB4. Use the loss function to calculate the loss between the pain level corresponding to the current facial image and the corresponding pain level label in the training set to obtain the loss value.

[0285] Among them, the loss function can be selected as cross entropy loss, and the loss function can adopt a weighted combination of multiple common loss functions, which can be set according to needs. Therefore, this application does not specifically limit the choice of its loss function.

[0286] DB5. Determine whether the current training batch or loss value meets a training condition: if so, complete the training to obtain a highly accurate method for identifying the patient's pain level; if not, back-propagate the method for identifying the patient's pain level described in the present invention according to the loss value to obtain the method for identifying the patient's pain level after updating the weights, and execute step DB3.

[0287] Among them, the search training conditions also include using a test set or a training set to identify the current patient's pain level to grade pain and calculate its accuracy. When the accuracy reaches a corresponding threshold, such as 90% pain grading accuracy, the training can be terminated early.

[0288] Furthermore, when the pain classification of physiological signal type and image type is trained, a multimodal training method can be performed, and the specific steps are as follows:

[0289] DC1. The skin electrical signals and facial images with the same pain grading labels in the physiological signal dataset and the facial image dataset are simultaneously input into the method for identifying the pain level of the patient described in the present invention, and the pain is graded to obtain the final pain level.

[0290] DC2, use the joint loss function to calculate the loss of the final pain level and the corresponding pain classification label to obtain the loss value.

[0291] DC3. Determine whether the current training batch or loss value meets a training condition: if so, complete the training and obtain a highly accurate method for identifying the patient's pain level; if not, back-propagate the method for identifying the patient's pain level described in the present invention according to the loss value to obtain the method for identifying the patient's pain level after updating the weights, and execute step DC1.

[0292] Based on the same inventive concept, the present application also provides an electronic device, which may be a terminal device such as a server, a desktop computing device or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement the method for identifying the pain level of a patient in an embodiment of the present invention; and the memory is used to store a computer program that can be executed by the processor.

[0293] Based on the same inventive concept, the present application also provides a computer-readable storage medium, corresponding to an embodiment of the aforementioned method for identifying the pain level of a patient, wherein the computer-readable storage medium stores a computer program thereon, and when the program is executed by a processor, the steps of the method for identifying the pain level of a patient recorded in any of the aforementioned embodiments are implemented.

[0294] The present application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0295] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, and the present invention is also intended to include these modifications and modifications.

Claims

1. A method for identifying the pain level of a patient, characterized in that: The following steps are involved: S1. Determine the data type of the pain data to be detected: if the data type is a physiological signal type, execute step S2A; if the data type is an image type, execute step S2B; S2A, performing signal preprocessing on the pain data to be detected to obtain key pain signal data; wherein the signal preprocessing is a signal noise reduction operation; S3A, using multi-scale convolution mechanism to extract features from key pain signal data and obtain preliminary pain signal features; S4A, using compression-excitation residual network to extract global spatial features from preliminary pain signal features to obtain global enhanced signal features; S5A, using a variable voltage encoder to extract sequence features from the global enhanced signal features to obtain the final pain signal features; S6A: Support vector machine was used to grade the pain of the final signal features, obtain the patient's pain level, and complete the pain grading of physiological signal types; S2B, performing image preprocessing on the pain data to be detected to obtain a facial pain expression image; S3B, a dual-branch feature extraction mechanism is used to extract multi-scale features from facial pain expression images, and the residual and attention mechanisms are combined to alternately fuse features to obtain local priority feature maps and global priority feature maps; S4B, fusing the local priority feature map and the global priority feature map to obtain the final pain feature map; S5B. Flatten the final pain feature map into a one-dimensional vector, and map the one-dimensional vector to the pain level through a fully connected layer to obtain the patient's pain level and complete the pain grading of the image type.

2. The method for identifying the pain level of a patient according to claim 1, characterized in that: The multi-scale convolution mechanism described in step S3A includes a large-segment convolution branch and a small-segment convolution branch; The specific steps of the large convolution branch are as follows: Use a large kernel convolution to analyze the key pain signal data Perform large segment feature extraction to obtain large segment features Large GSR , its large kernel convolution is specifically expressed as: In the formula, Conv f,k,s represents the convolution operation, c, k, and s are the number of output channels, convolution kernel size, and step size, respectively. k is set to 400 and s is set to 50. The Gaussian error linear unit activation function is used to perform nonlinear transformation, maximum pooling and feature mapping on large segments of features in sequence to obtain preliminary global signal features, which are specifically expressed as follows: H Large =FC(MaxPool k,s (GELU(Large GSR ))) In the formula, H Large is the preliminary global signal feature; GELU stands for Gaussian Error Linear Unit activation function; MaxPool k,s represents the maximum pooling operation, k is the window size, s is the step size; FC represents the fully connected layer; Next, convolution-pooling is used to extract the preliminary global signal features to obtain the global priority signal features, which are specifically expressed as follows: GlobalFeature=MaxPool k,s (GELU(Conv c,k,s (H Large ))) R Where GlobalFeature represents the global priority signal feature; Conv c,k,s is the convolution operation; R represents the number of repeated executions; The specific steps of the small segment convolution branch are as follows: Use a small kernel convolution to analyze the key pain signal data Perform small segment feature extraction to obtain small segment features Small GSR , its small kernel convolution is specifically expressed as: In the formula, Conv f,k,s Indicates the convolution operation, k defaults to 50, and s defaults to 6; The Gaussian error linear unit activation function is used to perform nonlinear transformation, maximum pooling and feature mapping on the small segment features in sequence to obtain preliminary local signal features, which are specifically expressed as follows: H Small =FC(MaxPool k,s (GELU(Small GSR ))) In the formula, H Small is the preliminary local signal feature; MaxPool k,s Represents the maximum pooling operation; Next, convolution-pooling is used to extract the preliminary local signal features to obtain the local priority signal features, which are specifically expressed as follows: LocalFeature=MaxPool k,s (GELU(Conv c,k,s (H Small ))) R Where LocalFeature represents the local priority signal feature; Conv c,k,s is the convolution operation; R represents the number of repeated executions; After obtaining the global priority signal features and the local priority signal features, signal feature fusion and feature mapping are performed to obtain preliminary pain signal features. The specific calculation is as follows: PainFeature=FC(GlobvalFeature⊕LocalFeature) Where PainFeature represents the preliminary pain signal feature, which integrates the features of the preliminary extracted global and local signals; ⊕ is vector concatenation; FC is the fully connected layer; The specific steps of extracting global spatial features using the compression-excitation residual network described in step S4A are as follows: One-dimensional convolution and activation function are used to transform the features and obtain the preliminary signal feature map F trans , and its specific calculation is expressed as follows: F trans =Conv c,1,1 (ReLU(Conv c,1,1 (PainFeature))) Where PainFeature is the preliminary pain signal feature; Conv c,1,1 is a one-dimensional convolution; ReLU is a nonlinear activation function; Next, global average pooling is used to spatially compress the preliminary signal feature map to obtain the compressed feature map F pool , which is specifically expressed as follows: In the formula, Represents the compressed feature map F pool The global average pooling value of the cth channel; represents the feature value of position (i, j) in channel c in the preliminary signal feature map; N represents the size of the feature map in the vertical dimension, that is, the height; L represents the size of the feature map in the horizontal dimension, that is, the width; The compressed feature map is adaptively recalibrated to obtain the global enhanced signal features. The specific calculation of the adaptive recalibration is as follows: Where EnhanceFeature is the global enhanced signal feature; W1 and W2 represent the weight matrices of the fully connected layer respectively; σ represents the Sigmoid function; is element-by-element multiplication; + is element-by-element addition.

3. The method for identifying the pain level of a patient according to claim 2, characterized in that: The specific steps of the voltage transformer described in step S5A are as follows: The global enhanced signal feature is copied to obtain a plurality of copies of the global enhanced signal feature; Left-fill several copies of global enhanced signal features respectively, and use time-domain convolutional network to extract time-dependent features in parallel to obtain several time-domain signal feature maps; A multi-head attention mechanism is used to extract attention features from all time domain signal feature maps to obtain the final pain signal features. The specific calculation expression of the multi-head attention mechanism is as follows: Perform multi-head parallel attention calculation on all time domain signal feature maps to obtain several attention scores. For the i-th attention score AttScore i , and its specific calculation is expressed as: In the formula, is the time domain signal feature map, where V represents the feature map as a value vector; L represents the number of input feature sequences; e ij It represents the similarity score between the i-th query vector and the j-th key vector, which is obtained by dot product calculation. The specific calculation can be expressed as: In the formula, and Represented as the time domain signal feature map as the query vector and as the key vector respectively; Next, all attention scores are weighted concatenated, and the weighted concatenated attention scores are fused with the global enhanced signal features to obtain the final pain signal feature FinPainFeature. The specific calculation is as follows: FinPainFeature=Normalization(AttConcat) AttConcat=EnhaneFeature+Concat(AttScore1,AttScore2,…,AttScore H )·W In the formula, H represents the total number of attention heads; W represents the learnable weight, which is used for linear transformation and makes the dimensions of input and output the same; + represents additive fusion; Normalization represents the normalization operation; AttConcatg is the attention score after weighted concatenation.

4. The method for identifying the pain level of a patient according to claim 1, characterized in that: The dual-branch feature extraction mechanism in step S3B includes a residual convolution local feature extraction branch and an attention-based global feature extraction branch; The residual convolution local feature extraction branch is provided with several layers of residual convolution modules. i , which is specifically expressed as follows: ResFeature i =Residual i (ResFeature i-1 )+ResFeature i-1 In the formula, ResFeature i Residual is the i-th layer residual convolution module i Extracted local feature map; + represents residual connection; Residual i The structure is represented as follows: In the formula, DownConv 1×1 (FM) represents the downsampling operation on the feature map FM; SpatialConv 3×3 (FM) represents spatial convolution of feature map FM; UpConv 1×1 (FM) represents the upsampling operation on the feature map FM; N i ResFeature represents the residual convolution module of layer i i The total number of internal convolution operations; The attention-based global feature extraction branch is equipped with several layers of moving window attention modules SwinTrans i , which is specifically expressed as follows: Where AttFeature i Represented as the i-th layer moving window attention module SwinTrans i Extracted global feature map; L Swin Indicates the total number of layers of the attention-based global feature extraction branch; SwinTrans i The structure is represented as follows: FM mid =WMSA(LN(FM))+FM Where LN is layer normalization; WMSA is the window multi-head attention mechanism; MLP is a module consisting of two fully connected layers and a nonlinear activation function; FM mid Represented as an intermediate feature map; When the residual convolution module Residual i The extracted local feature map and the i-th layer moving window attention module SwinTrans i After extracting the global feature map, the residual and attention mechanism are used alternately to perform feature fusion, which is specifically expressed as follows: finalFuse i =CBAM(Res(fuse i )+Conv 1×1 (fuse i )) In the formula, reshape(AttFeature i ) represents the global feature map AttFeature i Resize to the local feature map ResFeature i Consistent; || represents a concatenation operation, which is used to form a local aggregation feature map fuse i ; Represents the aggregation operation of the i-th layer; finalFuse i is the final fused feature map of the i-th layer, which is represented by the feature map after the feature maps of different branches of the i-th layer are completely fused; Conv 1×1 Represents a convolution operation with a convolution kernel of 1x1; Res represents a residual block, and its specific calculation is as follows: Res(fuse i )=ReLU(BN(Conv 3×3 (fuse i ))) K In the formula, ReLU is the linear rectification activation function; BN is batch normalization; Conv 3×3 Represents a convolution operation; K represents repeated execution of ReLU (BN (Conv 4×4 (fuse i ))) operation K times; CBAM is the convolutional attention mechanism, which is specifically expressed as follows: Where, SpatialAtt represents the spatial attention module; ChannelAtt represents the channel attention module; Represents a point-by-point multiplication operation; After obtaining the final fused feature map of the i-th layer, it is fused back to the feature map of the corresponding branch, which is specifically expressed as follows: AttFeature i =AttFeature i +finalFuse i ResFeature i =ResFeature i +finalFuse i Where AttFeature i and ResFeature i Represented as respectively with the final fusion feature map finalFuse i To merge.

5. The method for identifying the pain level of a patient according to claim 4, characterized in that: The specific calculation of the final pain feature map is as follows: In the formula, ResFeature (i＝last) Represents the local priority feature map of the final layer last; AttFeature (i＝last) Represents the global priority feature map of the final layer last; ⊕ represents the addition operation; The specific grading process of the patient's pain level based on the final pain characteristic graph is as follows: Where P(y=k|x) represents the probability that the input feature belongs to the kth level of pain; W k and b k Respectively represent the weight matrix and bias vector of the Softmax layer; C represents the total number of levels of pain severity; x represents a one-dimensional vector that has been nonlinearly transformed by ReLU, and its specific calculation is expressed as: x=ReLU(W FC ·flatten(finalFeature (i＝last) )+b FC ) Where flatten is the flattening operation; W FC and b FC Represents the weight matrix and bias matrix of the fully connected layer.

6. The method for identifying the pain level of a patient according to any one of claims 2 to 5, characterized in that: If the data type of the pain data to be detected includes both the physiological signal type and the image type, then the step S2A and the step S2B are performed simultaneously; After completing the pain grading of physiological signal type and image type at the same time, and obtaining the pain degree of patients of physiological signal type and image type at the same time, a fully connected layer of adaptive weight fusion is used to fuse the pain degree of all patients, obtain the pain degree of patients with high accuracy, and complete the pain grading of patients; The specific calculation of the fully connected layer of the adaptive weight fusion is expressed as follows: FinResult=W s ·SingalResult+W I ·ImageRsult Where FinResult is the pain level of the patient with high accuracy; SingalResult is the pain level of the patient with physiological signal type; ImageResult is the pain level of the patient with image type; W s and W I They are the learnable weight matrices of the physiological signal type and the image type in the fully connected layer, respectively.

7. A device for identifying the pain level of a patient, characterized in that: It includes a signal type judgment unit, a signal preprocessing unit, a multi-scale convolution feature extraction unit, a compression-excitation residual feature extraction unit, a sequence feature extraction unit, a signal data pain grading unit, an image preprocessing unit, a dual-branch feature extraction unit, a global-local feature map fusion unit and an image data pain grading unit; The signal type determination unit is used to determine the data type of the pain data to be detected: if the data type is a physiological signal type, the signal preprocessing unit is called; if the data type is an image type, the image preprocessing unit is called; The signal preprocessing unit performs signal preprocessing on the pain data to be detected to obtain key pain signal data; The multi-scale convolution feature extraction unit is used to extract features from key pain signal data using a multi-scale convolution mechanism to obtain preliminary pain signal features; The compression-excitation residual feature extraction unit is used to extract global spatial features from the preliminary pain signal features using a compression-excitation residual network to obtain global enhanced signal features. The sequence feature extraction unit is used to extract sequence features from the global enhanced signal features using a voltage transformer to obtain the final pain signal features; The signal data pain grading unit is used to perform pain grading on the final signal features using a support vector machine, obtain the patient's pain level, and complete the pain grading of the physiological signal type; The image preprocessing unit is used to perform image preprocessing on the pain data to be detected to obtain a facial pain expression image; The dual-branch feature extraction unit is used to perform multi-scale feature extraction on the facial pain expression image by using a dual-branch feature extraction mechanism, and alternately perform feature fusion in combination with the residual and attention mechanisms to obtain a local priority feature map and a global priority feature map; The global-local feature map fusion unit is used to fuse the local priority feature map and the global priority feature map to obtain a final pain feature map; The image data pain grading unit is used to flatten the final pain feature map into a one-dimensional vector, and map the one-dimensional vector to the pain degree through a fully connected layer to obtain the patient's pain degree and complete the pain grading of the image type.

8. The device for identifying the pain level of a patient according to claim 7, characterized in that: The multi-scale convolution mechanism in the multi-scale convolution feature extraction unit includes a large-segment convolution branch and a small-segment convolution branch; The specific steps of the large convolution branch are as follows: Use a large kernel convolution to analyze the key pain signal data Perform large segment feature extraction to obtain large segment features Large GSR , its large kernel convolution is specifically expressed as: In the formula, Conv f,k,s represents the convolution operation, c, k, and s are the number of output channels, convolution kernel size, and step size, respectively. k is set to 400 and s is set to 50. The Gaussian error linear unit activation function is used to perform nonlinear transformation, maximum pooling and feature mapping on large segments of features in sequence to obtain preliminary global signal features, which are specifically expressed as follows: H Large =fC(MaxPool k,s (GELU(Large GSR ))) In the formula, H Large is the preliminary global signal feature; GELU stands for Gaussian Error Linear Unit activation function; MaxPool k,s represents the maximum pooling operation, k is the window size, s is the step size; FC represents the fully connected layer; Next, convolution-pooling is used to extract the preliminary global signal features to obtain the global priority signal features, which are specifically expressed as follows: GlobalFeature=MaxPool k,s (GELU(Conv c,k,s (H Large ))) R Where GlobalFeature represents the global priority signal feature; Conv c,k,s is the convolution operation; R represents the number of repeated executions; The specific steps of the small segment convolution branch are as follows: Use a small kernel convolution to analyze the key pain signal data Perform small segment feature extraction to obtain small segment features Small GSR , its small kernel convolution is specifically expressed as: In the formula, Conv f,k,s Indicates the convolution operation, k defaults to 50, and s defaults to 6; The Gaussian error linear unit activation function is used to perform nonlinear transformation, maximum pooling and feature mapping on the small segment features in sequence to obtain preliminary local signal features, which are specifically expressed as follows: H Small =FC(MaxPool k,s (GELU(Small GSR ))) In the formula, H Small is the preliminary local signal feature; MaxPool k,s Represents the maximum pooling operation; Next, convolution-pooling is used to extract the preliminary local signal features to obtain the local priority signal features, which are specifically expressed as follows: LocalFeature=MaxPool k,s (GELU(Conv c,k,s (H Small ))) R Where LocalFeature represents the local priority signal feature; Conv c,k,s is the convolution operation; R represents the number of repeated executions; After obtaining the global priority signal features and the local priority signal features, signal feature fusion and feature mapping are performed to obtain preliminary pain signal features. The specific calculation is as follows: PainFeature=FC(GlobalFeature⊕LocalFeature) Where PainFeature represents the preliminary pain signal feature, which integrates the features of the preliminary extracted global and local signals; ⊕ is vector concatenation; FC is the fully connected layer; The specific steps of extracting global spatial features by the compression-excitation residual network in the compression-excitation residual feature extraction unit are as follows: One-dimensional convolution and activation function are used to transform the features and obtain the preliminary signal feature map F trans , and its specific calculation is expressed as follows: F trans =Conv c,1,1 (ReLU(Conv c,1,1 (PainFeature))) Where PainFeature is the preliminary pain signal feature; Conv c,1,1 is a one-dimensional convolution; ReLU is a nonlinear activation function; Next, global average pooling is used to spatially compress the preliminary signal feature map to obtain the compressed feature map F pool , which is specifically expressed as follows: In the formula, Represents the compressed feature map F pool The global average pooling value of the cth channel; F transi,j,c represents the feature value of position (i, j) in channel c in the preliminary signal feature map; N represents the size of the feature map in the vertical dimension, that is, the height; L represents the size of the feature map in the horizontal dimension, that is, the width; The compressed feature map is adaptively recalibrated to obtain the global enhanced signal features. The specific calculation of the adaptive recalibration is as follows: Where EnhanceFeature is the global enhanced signal feature; W1 and W2 represent the weight matrices of the fully connected layer respectively; σ represents the Sigmoid function; is element-by-element multiplication; + is element-by-element addition; The specific steps of the voltage transformer in the sequence feature extraction unit are as follows: The global enhanced signal feature is copied to obtain a plurality of copies of the global enhanced signal feature; Left-fill several copies of global enhanced signal features respectively, and use time-domain convolutional network to extract time-dependent features in parallel to obtain several time-domain signal feature maps; A multi-head attention mechanism is used to extract attention features from all time domain signal feature maps to obtain the final pain signal features. The specific calculation expression of the multi-head attention mechanism is as follows: Perform multi-head parallel attention calculation on all time domain signal feature maps to obtain several attention scores. For the i-th attention score AttScore i , and its specific calculation is expressed as: In the formula, is the time domain signal feature map, where V represents the feature map as a value vector; L represents the number of input feature sequences; e ij It represents the similarity score between the i-th query vector and the j-th key vector, which is obtained by dot product calculation. The specific calculation can be expressed as: In the formula, and Represented as the time domain signal feature map as the query vector and as the key vector respectively; Next, all attention scores are weighted concatenated, and the weighted concatenated attention scores are fused with the global enhanced signal features to obtain the final pain signal feature FinPainFeature. The specific calculation is as follows: FinPainFeature=Normalizatiion(AttConcat) AttConcat=EnhanceFeature+Concat(AttScore1,AttScore2,…,AttScore H )·W In the formula, H represents the total number of attention heads; W represents the learnable weight, which is used for linear transformation and makes the dimensions of input and output the same; + represents additive fusion; Normalization represents the normalization operation; AttConcat is the attention score after weighted concatenation.

9. The device for identifying the pain level of a patient according to claim 7, characterized in that: The dual-branch feature extraction mechanism in the dual-branch feature extraction unit includes a residual convolution local feature extraction branch and an attention-based global feature extraction branch; The residual convolution local feature extraction branch is provided with several layers of residual convolution modules. i , which is specifically expressed as follows: ResFeature i =Residual i (ResFeature i-1 )+ResFeature i-1 In the formula, ResFeature i Residual is the i-th layer residual convolution module i Extracted local feature map; + represents residual connection; Residual i The structure is represented as follows: In the formula, DownConv 1×1 (FM) represents the downsampling operation on the feature map FM; SpatialConv 3×3 (FM) represents spatial convolution of feature map FM; UpConv 1×1 (FM) represents the upsampling operation on the feature map FM; N i ResFeature represents the residual convolution module of layer i i The total number of internal convolution operations; The attention-based global feature extraction branch is equipped with several layers of moving window attention modules SwinTrans i , which is specifically expressed as follows: Where AttFeature i Represented as the i-th layer moving window attention module SwinTrans i Extracted global feature map; L swin Indicates the total number of layers of the attention-based global feature extraction branch; SwinTrans i The structure is represented as follows: FM mid =WMSA(LN(FM))+FM Where LN is layer normalization; WMSA is the window multi-head attention mechanism; MLP is a module consisting of two fully connected layers and a nonlinear activation function; FM mid Represented as an intermediate feature map; When the residual convolution module Residal is obtained by the i-th layer i The extracted local feature map and the i-th layer moving window attention module swinTrans i After extracting the global feature map, the residual and attention mechanism are used alternately to perform feature fusion, which is specifically expressed as follows: finalFuse i =CBAM(Res(fuse i )+Conv 1×1 (fuse i )) In the formula, reshape(AttFeature i ) represents the global feature map AttFeature i Resize to the local feature map ResFeature i Consistent; || represents a concatenation operation, which is used to form a local aggregation feature map fuse i ; Represents the aggregation operation of the i-th layer; finalFuse i is the final fused feature map of the i-th layer, which is represented by the feature map after the feature maps of different branches of the i-th layer are completely fused; Conv 1×1 Represents a convolution operation with a convolution kernel of 1x1; Res represents a residual block, and its specific calculation is as follows: Res(fuse i )=ReLU(BN(Conv 3×3 (fuse i ))) K In the formula, ReLU is the linear rectification activation function; BN is batch normalization; Conv 3×3 Represents a convolution operation; K represents repeated execution of ReLU (BN (Conv 3×3 (fuse i ))) operation K times; CBAM is the convolutional attention mechanism, which is specifically expressed as follows: Where, SpatialAtt represents the spatial attention module; ChannelAtt represents the channel attention module; Represents a point-by-point multiplication operation; After obtaining the final fused feature map of the i-th layer, it is fused back to the feature map of the corresponding branch, which is specifically expressed as follows: AttFeature i =AttFeature i +finalFuse i ResFeature i =ResFeature i +finalFuse i Where AttFeature i and ResFeature i Represented as respectively with the final fusion feature map finalFuse i to integrate; The specific fusion calculation process of the final pain feature map in the global-local feature map fusion unit is as follows: In the formula, ResFeature (i＝last) Represents the local priority feature map of the final layer last; AttFeature (i＝last) Represents the global priority feature map of the final layer last; ⊕ represents the addition operation; The specific grading process of the patient's pain level based on the final pain characteristic graph is as follows: Where P(y=k|x) represents the probability that the input feature belongs to the kth level of pain; W k and b k Respectively represent the weight matrix and bias vector of the Softmax layer; C represents the total number of levels of pain severity; x represents a one-dimensional vector that has been nonlinearly transformed by ReLU, and its specific calculation is expressed as: x=ReLU(W FC ·flatten(finalFeature (i＝last) )+b FC ) Where flatten is the flattening operation; W FC and b FC Represents the weight matrix and bias matrix of the fully connected layer.

10. A pain level recognition system, characterized in that: It includes a patient side, a doctor side and a family side, wherein the patient side obtains the pain data to be detected through a camera or a GSR signal real-time monitor, and inputs the data into a pain degree recognition device; The pain degree recognition device extracts features and grades pain on the pain data to be detected, obtains the patient's pain degree, and inputs it to the doctor end and the family end respectively; The doctor end is used to obtain the patient's pain level, and by obtaining the associated data in the patient database, display the associated data and the current patient's pain level for the doctor to analyze and obtain the doctor's treatment suggestions for the patient; The family end is provided with an AI assistant module, which is used to conduct personalized analysis of the patient's pain level history records and provide preliminary medical analysis, pain care suggestions and personalized diet plans based on the analysis results; Wherein, the pain level identification device is the patient's pain level identification device described in claims 7-9.

Images