Method, device and system for identifying the degree of pain of a patient

Abstract

The application relates to a pain degree recognition method, device and system for a patient. The pain degree recognition method for a patient comprises the following steps: judging the data type of to-be-detected pain data; if the data type is a physiological signal type, adopting a multi-scale convolution mechanism, a compression-excitation residual network and a variable pressure encoder for feature extraction, and adopting a support vector machine to perform pain grading on the extracted signal features to obtain the pain degree of the patient; if the data type is an image type, adopting a double-branch feature extraction mechanism to perform multi-scale feature extraction on a facial expression image, alternately combining residual and attention mechanisms for feature fusion, and performing feature flattening and pain degree mapping on the fused feature map to obtain the pain degree of the patient. The pain degree recognition method for a patient has the advantages of high-precision pain grading based on physiological signals and high-accuracy pain grading based on images.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical health monitoring, in particular to a method and device for identifying the degree of pain of a patient and a pain degree identification system. BACKGROUND

[0002] Pain is an unpleasant reflection of the body to the damaged tissue or potential damage. In traditional clinical practice, the pain of a patient is mainly assessed by self-assessment and observer assessment. However, the above assessment methods are easily affected by the subjectivity of the patient or the doctor, and by the emotion, cultural background, communication ability and other uncertain factors of the assessor. At the same time, the above methods often lack real-time performance, especially in the case of needing to respond to the severe changes of the pain of a patient in time, they cannot provide fast feedback, resulting in that the analgesic treatment cannot be provided in time. Early real-time pain identification is performed by analyzing physiological signals, heart rate or blood pressure, etc. Although it can reflect the pain to some extent, it is easily disturbed by external factors and emotional fluctuations, resulting in poor accuracy of pain identification.

[0003] In order to improve the objectivity and identification accuracy of the degree of pain, when the existing technology classifies the degree of pain by using physiological signals, the characteristics of the physiological signals are directly affected by drugs, pathological conditions and fear of pain, etc., thereby reducing the accuracy of pain identification. Some existing technologies capture the face of a patient by image detection to identify pain. However, the facial expression reactions of different individuals are quite different, different degrees of pain may exhibit different facial features for different patient individuals, and the emotional fluctuations and expression control of the patient also affect the facial expression when in pain, thereby resulting in that the existing technology has certain limitations in identifying the degree of pain by facial recognition.

[0004] Accordingly, the existing technology has the defects of strong subjectivity, poor real-time performance, emotional and external factor interference, etc. in the classification, evaluation or identification of the degree of pain. Especially when the physiological signals and facial expressions are used to identify pain, the emotional fluctuations, individual differences and external environment, etc. are easily affected, thereby resulting in the problem of insufficient accuracy of pain degree identification. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a method for identifying the degree of pain of a patient.

[0006] A method for identifying the degree of pain of a patient, comprising the following steps:

[0007] S1, judging the data type of the pain data to be detected: if the data type is a physiological signal type, performing step S2A; if the data type is an image type, performing step S2B;

[0008] S2A, signal pre-processing is performed on the pain data to be detected to obtain pain key signal data; wherein the signal pre-processing is a signal denoising operation;

[0009] S3A, a multi-scale convolution mechanism is used to extract features from the pain key signal data to obtain preliminary pain signal features;

[0010] S4A, a compression-excitation residual network is used to extract global spatial features from the preliminary pain signal features to obtain global enhanced signal features;

[0011] S5A, a variable pressure encoder is used to extract sequence features from the global enhanced signal features to obtain final pain signal features;

[0012] S6A: a support vector machine is used to classify the final signal features to obtain the pain degree of the patient, and the pain classification of the physiological signal type is completed;

[0013] S2B, image pre-processing is performed on the pain data to be detected to obtain a facial pain expression image;

[0014] S3B, a double-branch feature extraction mechanism is used to extract multi-scale features from the facial pain expression image, and residual and attention mechanisms are alternately combined for feature fusion to obtain a local priority feature map and a global priority feature map;

[0015] S4B, the local priority feature map and the global priority feature map are fused to obtain a final pain feature map;

[0016] S5B, the final pain feature map is flattened into a one-dimensional vector, and the one-dimensional vector is mapped to the pain degree through a fully connected layer to obtain the pain degree of the patient, and the pain classification of the image type is completed.

[0017] The pain degree recognition method of the patient according to the present application, compared with the prior art, by setting the pain classification based on the physiological signal type and the image type, the user can select the corresponding detection method according to the specific needs, thereby enhancing the convenience of pain degree recognition; at the same time, for physiological signals, the multi-scale convolution mechanism is used to effectively extract multi-level features in the physiological signals, so that the pain information of the signals at different time scales can be captured; and by using the variable pressure encoder (TCN combined with the multi-head attention mechanism), the perception of the importance difference between different time steps in the physiological signals and the long-time dependence relationship can be effectively captured, so that the pain features in the physiological signals can be fully extracted. Finally, the SVM is used to find the optimal separating hyperplane in the feature space for pain degree classification to effectively reduce the overfitting phenomenon, thereby solving the problem that pain is prone to unstable precision due to individual differences, and thereby improving the accuracy of the pain classification based on the physiological signal type.

[0018] Further, the present application fully extracts higher-order and abstract features belonging to pain in facial images through a double-branch feature extraction mechanism and an attention mechanism, so as to capture subtle changes in facial pain and avoid false detection caused by facial expression differences in the prior art, thereby improving the accuracy of image type pain grading.

[0019] In one embodiment, the multi-scale convolution mechanism described in step S3A includes a large-section convolution branch and a small-section convolution branch.

[0020] The specific steps of the large-section convolution branch are as follows:

[0021] A large-core convolution is used to extract pain key signal data Large-section feature extraction is performed to obtain large-section feature Large GSR The large-core convolution is specifically represented as:

[0022]

[0023] In the formula, Conv c,k,s represents a convolution operation, c, k, and s respectively represent the number of output channels, the size of the convolution kernel, and the step length, k is set to 400, and s is set to 50.

[0024] The large-section feature is sequentially subjected to nonlinear transformation, maximum pooling, and feature mapping using a Gaussian error linear unit activation function to obtain preliminary global signal features, which are specifically represented as:

[0025] H Large = FC(MaxPool d,s (GELU(Large GSR )))

[0026] In the formula, H Large is the preliminary global signal feature; GELU represents the Gaussian error linear unit activation function; MaxPool d,s represents the maximum pooling operation, d is the window size, and s is the step length; and FC represents the fully connected layer.

[0027] Next, the preliminary global signal feature is subjected to feature extraction using convolution-pooling to obtain global priority signal features, which are specifically represented as:

[0028] GlobalFeature = MaxPool d,s (GELU(Conv c,k,s (H Large ))) R

[0029] In the formula, GlobalFeature represents the global priority signal feature; Convc,k,s is a convolution operation; R represents the number of repeated executions;

[0030] The small convolution branch specific steps are represented as follows:

[0031] Using a small kernel convolution to pain key signal data Small segment feature extraction is performed to obtain small segment features Small GSR The small kernel convolution is specifically represented as:

[0032]

[0033] In the formula, Conv c,k,s represents a convolution operation, k is 50 by default, and s is 6 by default;

[0034] The small segment features are sequentially subjected to nonlinear transformation, maximum pooling and feature mapping using a Gaussian error linear unit activation function to obtain preliminary local signal features, which are specifically represented as:

[0035] H Small = FC(MaxPool d,s (GELU(Small GSR )))

[0036] In the formula, H Small is a preliminary local signal feature; MaxPool d,s represents a maximum pooling operation;

[0037] Then, the preliminary local signal features are subjected to feature extraction using convolution-pooling to obtain local priority signal features, which are specifically represented as follows:

[0038] LocalFeature = MaxPool d,s (GELU(Conv c,k,s (H Small ))) R

[0039] In the formula, LocalFeature represents a local priority signal feature; Conv c,k,s is a convolution operation; R represents the number of repeated executions;

[0040] After obtaining the global priority signal features and the local priority signal features, signal feature fusion and feature mapping are performed to obtain preliminary pain signal features, which are specifically calculated and represented as follows:

[0041]

[0042] In the formula, PainFeature represents the initial pain signal features, which integrates the features of the initially extracted global and local signals; ⊕ represents vector concatenation; FC represents a fully connected layer;

[0043] The specific steps for global spatial feature extraction using the compressed-excitation residual network described in step S4A are as follows:

[0044] Feature transformation is performed using one-dimensional convolution and activation functions to obtain a preliminary signal feature map F. trans The specific calculation is expressed as follows:

[0045] F trans =Conv c,1,1 (ReLU(Conv c,1,1 (PainFeature)))

[0046] In the formula, PainFeature represents the initial pain signal characteristics; Conv c,1,1 For one-dimensional convolution; ReLU is a non-linear activation function;

[0047] Next, global average pooling is used to spatially compress the preliminary signal feature map to obtain the compressed feature map F. pool Its specific representation is as follows:

[0048]

[0049] In the formula, Represents the compressed feature map F pool The global average pooling value of the c-th channel; The eigenvalue at position (i,j) in channel c is represented in the initial signal feature map; N represents the size of the feature map in the vertical dimension, i.e., the height; L represents the size of the feature map in the horizontal dimension, i.e., the width.

[0050] The compressed feature map is adaptively recalibrated to obtain globally enhanced signal features. The specific calculation method for this adaptive recalibration is as follows:

[0051]

[0052] In the formula, EnhanceFeature is the global enhancement signal feature; W1 and W2 represent the weight matrices of the fully connected layer, respectively; σ represents the Sigmoid function; `+` represents element-wise multiplication; `+` represents element-wise addition.

[0053] According to the method, the key information in the signal is extracted by the large and small convolution branches in the multi-scale convolution mechanism and the convolution kernels of different scales, so as to capture the complex time sequence relationship and high-dimensional dependence in the pain signal, fully mine the multi-level features in the signal, and then improve the accuracy and robustness of pain recognition.

[0054] Meanwhile, the residual structure in the compression-excitation residual network is adopted to prevent the loss of feature information, and the channel-by-channel weighting of the feature map is performed through the adaptive calibration mechanism, so that the extracted features are adaptively focused on the key signals related to pain and the unimportant or noise features are suppressed, and the key changes in the pain signal are effectively captured. According to the method, through the synergistic effect of compression and excitation, the global information and local details in the signal can be enhanced in the multi-level feature extraction process, so that the recognition of the pain signal is more accurate, and the recognition error caused by individual differences, noise or signal ambiguity is reduced, and the accuracy of the pain degree classification of the physiological signal type is significantly improved.

[0055] Further, the specific steps of the variable-length encoder in step S5A are as follows:

[0056] The global enhanced signal features are copied to obtain a plurality of global enhanced signal features;

[0057] The plurality of global enhanced signal features are respectively left padded, and a time domain convolution network is used to extract time-dependent features in parallel to obtain a plurality of time domain signal feature maps;

[0058] All the time domain signal feature maps are subjected to attention feature extraction by using a multi-head attention mechanism to obtain final pain signal features; wherein the specific calculation of the multi-head attention mechanism is as follows:

[0059] The multi-head parallel attention calculation is performed on all the time domain signal feature maps to obtain a plurality of attention scores, and the i-th attention score AttScore i is calculated as follows:

[0060]

[0061] In the formula, is a time domain signal feature map, V represents the value vector of the feature map, L represents the number of input feature sequences, e ij is the similarity score between the i-th query vector and the j-th key vector, which is calculated by dot product, and the specific calculation can be represented as:

[0062]

[0063] In the formula, and respectively represent the time domain signal feature map as a query vector and as a key vector;

[0064] Next, all attention scores are weighted and spliced, and the weighted and spliced attention scores are fused with the global enhanced signal features to obtain the final pain signal feature FinPainFeature, which is specifically calculated as follows:

[0065] FinPainFeature=Normalization(AttConcat)

[0066] AttConcat=EnhanceFeature+Concat(AttScore1,AttScore2,…,AttScore H )·W

[0067] In the formula, H represents the total number of attention heads; W represents a learnable weight for linear transformation, and makes the dimensions of the input and the output the same; + represents addition fusion; Normalization represents normalization operation; and AttConcat is the weighted and spliced attention score.

[0068] Accordingly, the left side padding of the time domain convolution network (TCN) in the variable pressure encoder ensures that the convolution operation does not leak future signal information, thereby avoiding the influence of future data on the prediction of the current state; at the same time, the receptive field of the one-dimensional convolution fully expanded network stacked in the time domain convolution network is combined to learn more abstract time sequence features in the signal, to capture subtle pain signal changes and reduce the noise influence caused by individual differences, thereby improving the recognition accuracy of key signal features in the signal.

[0069] In addition, based on the time domain convolution network, the multi-head attention mechanism is combined to deeply understand the time sequence features that only rely on the current and historical signals and are not disturbed by future signals from different angles, so as to ensure that the time causality is strictly followed in the feature extraction process, thereby enabling the network to capture the time dependence and key signal regions while avoiding the interference of future information leakage on model prediction, and significantly improving the accuracy of pain recognition.

[0070] In another embodiment, the double-branch feature extraction mechanism in step S3B includes a residual convolution local feature extraction branch and an attention-based global feature extraction branch;

[0071] The residual convolution local feature extraction branch is provided with a plurality of layers of residual convolution modules Residual i , which are specifically represented as follows:

[0072] ResFeaturei =Residual i (ResFeature i-1 )+ResFeature i-1

[0073] In the formula, ResFeature i For the i-th layer residual convolution module Residual i Extracted local feature map; + indicates residual connection; Residual i The structure is represented as follows:

[0074]

[0075] In the formula, DownConv 1×1 (FM) indicates a downsampling operation on the feature map FM; SpatialConv 3×3 (FM) indicates spatial convolution of the feature map FM; UpConv 1×1 (FM) indicates an upsampling operation on the feature map FM; N i ResFeature represents the residual convolutional module of the i-th layer. i The total number of internal convolution operations;

[0076] The attention-based global feature extraction branch includes several layers of moving window attention modules (SwinTrans). i Its specific representation is as follows:

[0077]

[0078] In the formula, AttFeature i Represented as the i-th layer moving window attention module, SwingTrans i Extracted global feature map; L Swin This represents the total number of layers in the attention-based global feature extraction branch; SwingTrans i The structure is represented as follows:

[0079]

[0080] FM mid =WMSA(LN(FM))+FM

[0081] In the formula, LN represents layer normalization; WMSA represents the multi-head window attention mechanism; MLP represents a module containing two fully connected layers and a non-linear activation function; FM represents... mid Represented as an intermediate feature map;

[0082] When the residual convolutional module of the i-th layer is obtainedi extracted local feature maps, and the i-th layer moving window attention module SwinTrans i After the extracted global feature maps, the residual and attention mechanism are alternately used for feature fusion, which is specifically represented as follows:

[0083]

[0084] finalFuse i = CBAM (Res (fuse i ) + Conv 1×1 (fuse i ))

[0085] In the formula, reshape (AttFeature i ) represents adjusting the size of the global feature map AttFeature i to be consistent with the local feature map ResFeature i ; || represents the splicing operation, which is used to form the local aggregation feature map fuse i ; represents the aggregation operation of the i-th layer; finalFuse i is the final fusion feature map of the i-th layer, which represents the feature map after the complete fusion of the feature maps of different branches of the i-th layer; Conv 1×1 represents the convolution operation with a 1x1 convolution kernel; Res represents the residual block, which is specifically calculated as follows:

[0086] Res (fuse i ) = ReLU (BN (Conv 3×3 (fuse i ))) K

[0087] In the formula, ReLU is a linear rectification activation function; BN is batch normalization; Conv 3×3 represents the convolution operation; K represents repeating the operation ReLU (BN (Conv 3×3 (fuse i ))) K times; CBAM is a convolution attention mechanism, which is specifically represented as follows:

[0088]

[0089] In the formula, SpatialAtt represents a spatial attention module; ChannelAtt represents a channel attention module; represents a point-by-point multiplication operation;

[0090] When the final fusion feature map of the i-th layer is obtained, it is fused back to the feature map of the corresponding branch, which is specifically represented as follows:

[0091] AttFeature i =AttFeature i +finalFuse i

[0092] ResFeature i =ResFeature i +finalFuse i

[0093] In the formula, AttFeature i and ResFeature i are fused with the final fusion feature map finalFuse i respectively.

[0094] The application fully extracts local feature information in the face image through the residual convolution local feature extraction branch, especially in capturing deep features of subtle changes of facial expressions and muscles, at the same time, effectively captures overall features and context information in the face image through the attention-based global feature extraction branch, significantly improves the global representation ability of feature extraction; and based on this, further adopts the strategy of alternating fusion of residual and attention mechanism, realizes deep fusion in the feature space extracted by each scale of the branch, so as to fully play the complementarity of local and global features, and improve the completeness of feature representation and the robustness of facial expression recognition.

[0095] Further, the specific calculation of the final pain feature map is represented as follows:

[0096]

[0097] In the formula, ResFeature (i＝last) represents the local priority feature map of the final layer last; AttFeature (i＝last) represents the global priority feature map of the final layer last; represents an addition operation;

[0098] The specific grading process of the patient's pain degree based on the final pain feature map is represented as follows:

[0099]

[0100] In the formula, P(y=k|x) represents the probability that the input feature belongs to the k-th level of pain degree; W k and b krespectively represent the weight matrix and bias vector of the Softmax layer; C represents the total number of pain degree grades; x represents a one-dimensional vector after nonlinear transformation by ReLU, and the specific calculation is represented as:

[0101] x = ReLU (W FC · flatten (finalFeature (i＝last) ) + b FC )

[0102] In the formula, flatten is a flattening operation; W FC and b FC represent the weight matrix and bias matrix of the fully connected layer.

[0103] Accordingly, the application effectively integrates the key information of the two kinds of features by fusing the local priority features and the global priority features in the final fully connected layer, retains the detail changes of facial expressions, strengthens the contextual relevance of the overall expression, adopts the SoftMax layer for classification, and thus realizes the high-precision pain recognition effect.

[0104] Further, if the data types of the pain data to be detected simultaneously include physiological signal types and image types, the step S2A and the step S2B are simultaneously performed.

[0105] When the pain grading of the physiological signal types and the image types is simultaneously completed, and the pain degrees of the patients of the physiological signal types and the image types are simultaneously obtained, an adaptive weight fusion fully connected layer is adopted to fuse the pain degrees of all the patients, the high-accuracy pain degree of the patient is obtained, and the pain grading of the patient is completed.

[0106] Wherein, the specific calculation of the adaptive weight fusion fully connected layer is as follows:

[0107] FinResult = W s · SingalResult + W I · ImageResult

[0108] In the formula, FinResult is the high-accuracy pain degree of the patient; SingalResult is the pain degree of the patient of the physiological signal type; ImageResult is the pain degree of the patient of the image type; W s and W I are respectively the learnable weight matrix of the physiological signal type and the learnable weight matrix of the image type in the fully connected layer.

[0109] Based on this, the physiological signal type and the image type are jointly processed through the full connection layer of adaptive weight fusion, the characteristics of the two modalities, i.e., the continuity feature of the physiological signal and the intuitive spatial feature of the facial image, are integrated, the complementary advantages of the two are fully given, the limitations of the single modality, such as being easily affected by noise, individual difference or specific scene, are overcome, and the accuracy of the pain degree recognition is significantly improved.

[0110] A pain degree recognition device of a patient, comprising a signal type judgment unit, a signal preprocessing unit, a multi-scale convolution feature extraction unit, a compression-excitation residual feature extraction unit, a sequence feature extraction unit, a signal data pain grading unit, an image preprocessing unit, a double-branch feature extraction unit, a global-local feature map fusion unit and an image data pain grading unit.

[0111] The signal type judgment unit is used for judging the data type of the pain data to be detected: if the data type is a physiological signal type, the signal preprocessing unit is called; if the data type is an image type, the image preprocessing unit is called.

[0112] The signal preprocessing unit is used for performing signal preprocessing on the pain data to be detected to obtain pain key signal data.

[0113] The multi-scale convolution feature extraction unit is used for performing feature extraction on the pain key signal data by using a multi-scale convolution mechanism to obtain preliminary pain signal features.

[0114] The compression-excitation residual feature extraction unit is used for performing global spatial feature extraction on the preliminary pain signal features by using a compression-excitation residual network to obtain global enhanced signal features.

[0115] The sequence feature extraction unit is used for performing sequence feature extraction on the global enhanced signal features by using a transposed encoder to obtain final pain signal features.

[0116] The signal data pain grading unit is used for performing pain grading on the final signal features by using a support vector machine to obtain the pain degree of the patient, and completing the pain grading of the physiological signal type.

[0117] The image preprocessing unit is used for performing image preprocessing on the pain data to be detected to obtain a facial pain expression image.

[0118] The double-branch feature extraction unit is used for performing multi-scale feature extraction on the facial pain expression image by using a double-branch feature extraction mechanism, and alternately performing feature fusion by combining residual and attention mechanisms to obtain a local priority feature map and a global priority feature map.

[0119] The global-local feature map fusion unit is configured to fuse the local priority feature map and the global priority feature map to obtain a final pain feature map.

[0120] The image data pain grading unit is configured to flatten the final pain feature map into a one-dimensional vector, and map the one-dimensional vector through a fully connected layer to obtain the pain degree of the patient, thereby completing the image type pain grading.

[0121] A pain degree recognition system includes a patient end, a doctor end and a family end, the patient end acquires pain data to be detected through a camera or a GSR signal real-time monitor, and inputs the pain data to a pain degree recognition device;

[0122] The pain degree recognition device extracts features from the pain data to be detected and grades the pain to obtain the pain degree of the patient, and inputs the pain degree to the doctor end and the family end respectively;

[0123] The doctor end is configured to acquire the pain degree of the patient, and display the pain degree and associated data in a patient database to the doctor for analysis and obtaining of a treatment suggestion of the doctor to the patient;

[0124] The family end is provided with an AI assistant module, the AI assistant module is configured to perform personalized analysis on a pain degree history record of the patient, and provide a preliminary medical analysis, a pain care suggestion and a personalized diet plan according to an analysis result;

[0125] The pain degree recognition device is the pain degree recognition device described above.

[0126] Based on the above, the pain degree recognition system is used to realize information sharing and interaction among the patient, the doctor and the family, and ensure timely transmission of the pain data of the patient, thereby improving transparency and interactivity of the entire medical process, especially when the patient is in a home observation stage after being discharged from hospital, the family can timely learn the pain condition of the patient and obtain intelligent care suggestions, thereby assisting the patient in pain distress or complications caused by the pain.

[0127] Meanwhile, the pain degree is recognized automatically to exclude the influence of subjective factors, thereby providing objective and timely data, effectively helping medical staff to accurately determine the pain condition of the patient and timely adjust a medical scheme, thereby significantly improving efficiency of postoperative management of the medical staff on the patient.

[0128] For better understanding and implementation, the present application is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0129] Figure 1Schematic diagram of simple structure of the pain degree recognition device for the patient of the present application;

[0130] Figure 2 Schematic diagram of simple flow of the pain degree recognition method for the patient of the present application;

[0131] Figure 3 Schematic diagram of overall structure of the multi-scale convolution mechanism, the compression-excitation residual network and the variable pressure encoder of the present application;

[0132] Figure 4 Schematic diagram of simple structure example of the multi-scale convolution mechanism of the present application;

[0133] Figure 5 Schematic diagram of simple structure example of the compression-excitation residual network of the present application;

[0134] Figure 6 Schematic diagram of simple structure example of the variable pressure encoder of the present application;

[0135] Figure 7 Schematic diagram of reference example of image preprocessing flow of the face image;

[0136] Figure 8 Schematic diagram of simple flow structure of the dual-branch feature extraction mechanism and the alternate feature fusion of the present application;

[0137] Figure 9 Schematic diagram of residual convolution module structure example of the residual convolution local feature extraction branch of the present application;

[0138] Figure 10 Schematic diagram of moving window attention module structure example of the attention-based global feature extraction branch of the present application;

[0139] Figure 11 Schematic diagram of simple flow example of the alternate feature fusion structure of the present application;

[0140] Figure 12 Schematic diagram of simple structure example of the CBAM in the alternate feature fusion structure of the present application. DETAILED DESCRIPTION

[0141] In order to solve the problem of insufficient accuracy of pain degree recognition in the prior art, the present application obtains the pain data to be detected of a patient, judges the data type of the pain data, if the data type is a physiological signal type, carries out signal preprocessing on the pain data to be detected, obtains pain key signal data, and sequentially adopts a multi-scale convolution mechanism, a compression-excitation residual network and a variable pressure encoder to extract features from the pain data to be detected, to obtain the final pain signal features, then, a support vector machine is used to classify the final signal features to obtain the pain degree of the patient, if the data type is an image type, a double-branch feature extraction mechanism combined with residual and attention mechanism is used to alternately extract features and fuse features to obtain global priority feature maps and local priority feature maps, and the global priority feature maps and the local priority feature maps are fused and a full connection layer is used to classify the fused feature maps to obtain the pain degree of the patient.

[0142] In addition, when classifying pain based on physiological signal data, the present application adopts a multi-scale convolution mechanism, a compression-excitation residual network, a variable pressure encoder and a support vector machine to adapt to the particularity and complexity of physiological signal data, thereby improving the accuracy of pain classification based on physiological signals.

[0143] Based on the above design, the present application proposes a method for recognizing the pain degree of a patient, and based on the method, a device for recognizing the pain degree of a patient is proposed.

[0144] Please refer to Figure 1 and Figure 2 , Figure 1 is a simple structure diagram of the device for recognizing the pain degree of a patient according to the present application, Figure 2 is a simple flowchart of the method for recognizing the pain degree of a patient according to the present application.

[0145] The device for recognizing the pain degree of a patient comprises a signal type judgment unit 1, a signal preprocessing unit 2A, a multi-scale convolution feature extraction unit 3A, a compression-excitation residual feature extraction unit 4A, a sequence feature extraction unit 5A, a signal data pain classification unit 6A, an image preprocessing unit 2B, a double-branch feature extraction unit 3B, a global-local feature map fusion unit 4B and an image data pain classification unit 5B.

[0146] The signal type judging unit 1 is used for executing step S1: judging the data type of the pain data to be detected: if the data type is a physiological signal type, the signal preprocessing unit 2A is called; if the data type is an image type, the image preprocessing unit 2B is called.

[0147] Specifically, the pain data to be detected includes a physiological signal type and an image type; the pain data of the physiological signal type is a galvanic skin response (GSR), which specifically acquires the galvanic skin response of a patient in pain at a preset sampling rate, and segments the galvanic skin response as the pain data to be detected; wherein the preset sampling rate is 512 Hz by default.

[0148] The pain data of the image type is a facial image of the patient, which specifically photographs the face of the patient in pain to acquire the corresponding facial expression and take it as the pain data to be detected.

[0149] Accordingly, the present application flexibly switches different types of pain data, including physiological signals (such as galvanic skin response GSR) and image data (such as facial expression images), so that users can select appropriate data types for pain classification according to actual conditions, improving the convenience and universality of the system.

[0150] It should be noted that although physiological signal data can provide more accurate and comprehensive pain classification, it usually requires a dedicated detection instrument and is more complex to operate, and is more suitable for nerve detection in a hospital environment to provide reference data for the patient's pain response for the doctor. Facial expression data may be affected by individual differences, resulting in relatively low accuracy, but it is easy to collect and can be achieved through conventional camera equipment, with good portability, especially suitable for postoperative or non-hospitalized patients, which can reduce the patient's frequency of medical treatment and related expenses.

[0151] Further, if the data type of the pain data to be detected simultaneously exists in the physiological signal type and the image type, the signal preprocessing unit 2A and the image preprocessing unit 2B are simultaneously called, so as to perform pain classification in a multi-modal manner and promote the complementation or interaction of different data types, thereby improving the reliability of pain classification.

[0152] The signal preprocessing unit 2A is used for executing step S2A: signal preprocessing of the pain data to be detected to obtain pain key signal data.

[0153] Specifically, the bandpass filtering (Bandpass Filtering) is used to reduce noise of the pain data to be detected to obtain the pain key signal data, and the bandpass filtering for reducing noise of the pain data to be detected is specifically represented as follows:

[0154]

[0155] In the formula, is the pain key signal data, that is, the skin electric signal after noise reduction, used to retain effective information in a specific frequency band related to pain; x[n] is the pain data to be detected, that is, the skin electric signal collected according to a preset sampling rate; h[m] represents a band-pass filter impulse response function, used to select signals in a specific frequency band while removing low-frequency baseline drift and high-frequency noise; m and n represent indexes of discrete signals, wherein the n index represents a current sampling point, and the m index represents a time delay index in the filtering operation; M represents the window length of the filter, that is, the impulse response range of the filter, used to determine the length of historical data used by each sampling point in filtering calculation, and is specifically determined by the design order of the filter.

[0156] Since the band-pass filter is a commonly used basic noise reduction method, it can remove most common noises and retain effective components of signals, and according to differences in collection environment, noise type or instrument specification, a suitable filter may need to be selected or matched for noise reduction. Accordingly, the selection of the filter is not specifically limited.

[0157] The multi-scale convolution feature extraction unit 3A is used to perform step S3A: adopting a multi-scale convolution mechanism to perform feature extraction on the pain key signal data to obtain preliminary pain signal features.

[0158] Specifically, please refer to Figure 3 and Figure 4 , the multi-scale convolution mechanism includes a large section convolution branch and a small section convolution branch; the large section convolution branch is used to extract global features of the signal, and the specific steps are as follows:

[0159] The large section feature extraction is performed on the pain key signal data by using a large kernel convolution, which is used to capture the signal trend characteristics in a long time window to obtain a large section feature Large GSR , which is specifically represented as:

[0160]

[0161] In the formula, Conv c,k,s represents a convolution operation, and c, k and s respectively represent the number of output channels (Channel), the size of the convolution kernel (Kernel Size) and the step (Stride), wherein c is 64 by default, k is 400 by default, and s is 50 by default, which is used to cover a skin electric signal window of 0.8 seconds, so as to obtain the large section feature.

[0162] The large section feature is sequentially subjected to nonlinear transformation, maximum pooling and feature mapping by using a Gaussian error linear unit activation function to obtain a preliminary global signal feature, which is specifically represented as:

[0163] H Large = FC(MaxPool d,s (GELU(Large GSR )))

[0164] In the formula, H Large is a preliminary global signal feature, which is a global signal trend feature with a large time window; GELU represents a Gaussian error linear unit activation function, which is used for smoothing approximation of an input signal to smooth the boundary data and prevent neuron disappearance, and limit the depth of negative state activation, thereby allowing negative weight values to pass through the network; MaxPool d,s represents a maximum pooling operation, which is used to compress feature length, reduce redundancy and retain significant features of the signal, d is a window size (Kernel Size), and s is a step (Stride), which is set to 4 by default and s is set to 2 by default; FC represents a fully connected layer, which is used to map the features after pooling to generate a preliminary global signal feature with a fixed length.

[0165] Then, the preliminary global signal feature is subjected to feature extraction by using convolution-pooling to obtain a global priority signal feature, which is specifically represented as follows:

[0166] GlobalFeature = MaxPool d,s (GELU(Conv c,k,s (H Large ))) R

[0167] In the formula, GlobalFeature represents a global priority signal feature; Conv c,k,s is a convolution operation, c is set to 128 by default, k is set to 7 by default, and s is set to 1 by default; R represents the number of repeated executions, which is set to 2 by default, and more rich global features are extracted by two iterations.

[0168] The small section convolution branch is used to extract local features of the signal, and the specific steps are represented as follows:

[0169] A small kernel convolution is used to extract small section features from the pain key signal data to capture local change features within a short time window to obtain small section features Small GSR , which is specifically represented as:

[0170]

[0171] In the formula, ConvC,k,s This indicates a convolution operation, where c defaults to 64, k defaults to 50, and s defaults to 6. It is used to cover a 0.1-second window of electrodermal signal to obtain small feature segments.

[0172] The Gaussian error linear unit activation function is used to sequentially perform nonlinear transformation, max pooling, and feature mapping on small feature segments to obtain preliminary local signal features, which are specifically expressed as follows:

[0173] H Small =FC(MaxPool) d,s (GELU(Small GSR )))

[0174] In the formula, H Small Preliminary local signal characteristics; MaxPool d,s This indicates a max pooling operation, where k defaults to 8 and s defaults to 2.

[0175] Next, convolution-pooling is used to extract features from the initial local signal features to obtain local priority signal features, which are specifically represented as follows:

[0176] LocalFeature=MaxPool d,s (GELU(Conv c,k,s (H Small ))) R

[0177] In the formula, LocalFeature represents the local priority signal feature; Conv c,k,s For convolution operations, c defaults to 128, k defaults to 8, and s defaults to 1; R represents the number of repetitions, which defaults to 2.

[0178] After obtaining the global priority signal features and the local priority signal features, signal feature fusion and feature mapping are performed to obtain preliminary pain signal features, the specific calculation of which is as follows:

[0179] PainFeature=FC(GlobalFeature⊕LocalFeature)

[0180] In the formula, PainFeature represents the initial pain signal features, which integrates the features of the initially extracted global and local signals; ⊕ is vector concatenation, used to fuse the global priority signal features and the local priority signal features; FC is a fully connected layer, used to map the fused features.

[0181] Accordingly, this invention extracts global features of GSR signals using a large convolutional kernel (400x400) and a large stride to capture trend information and global features within a long time window, thereby analyzing the slow-change characteristics in electrodermal signals. Simultaneously, it captures local features of GSR signals using a small convolutional kernel (50x50) and a shorter stride to capture subtle changes and local dynamic features within a short time window. By fusing and mapping these two types of features, it achieves an effective combination of global and local information, thereby improving the fine-grainedness and contextual information of pain signal features and providing features with significant discriminative power for subsequent pain intensity classification.

[0182] The compression-excitation residual feature extraction unit 4A is used to perform step S4A: using a compression-excitation residual network to extract global spatial features from the preliminary pain signal features to obtain global enhanced signal features.

[0183] Specifically, the steps for global spatial feature extraction using the compressed-excitation residual network are as follows:

[0184] Feature transformation is performed using one-dimensional convolution and activation functions to obtain a preliminary signal feature map F. trans The specific calculation is expressed as follows:

[0185] F trans =Conv c,1,1 (ReLU(Conv c,1,1 (PainFeature)))

[0186] In the formula, PainFeature represents the initial pain signal characteristics; Conv c,1,1 It is a one-dimensional convolution with a default channel count c of 30; ReLU is a non-linear activation function.

[0187] Next, global average pooling is used to spatially compress the preliminary signal feature map to obtain the compressed feature map F. pool Its specific representation is as follows:

[0188]

[0189] In the formula, Represents the compressed feature map F pool The global average pooling value of the c-th channel; The feature value at position (i,j) in channel c in the preliminary signal feature map is represented by N; N represents the size of the feature map in the vertical dimension, i.e., the height; L represents the size of the feature map in the horizontal dimension, i.e., the width; wherein, the global average pooling is used to extract the global features of each channel, thereby extracting the most significant signal response value and reducing the dimensionality to make the features compact;

[0190] The compressed feature map is adaptively recalibrated to obtain a global enhanced signal feature, and the adaptive recalibration is specifically calculated as follows:

[0191]

[0192] In the formula, EnhanceFeature is a global enhanced signal feature, which is used to represent the recalibrated feature map; W1 and W2 respectively represent weight matrices of the full connection layer, which are used to reflect the relationship between channels and weight the channel with the largest amount of information, and are specifically obtained through training learning; and sigma represents a Sigmoid function, which is used to normalize the weight to the interval [0, 1]; is an element-wise multiplication, which is used for channel weighting operation, so as to suppress irrelevant or noise features and emphasize significant features; and + is an element-wise addition, which is used for residual connection, so as to retain the original input feature and then enhance gradient transmission.

[0193] Please refer to Figure 5 and Figure 6 , Figure 5 is a simple structure example schematic diagram of the compression-excitation residual network described in the application, Figure 6 is a simple structure example schematic diagram of the transposed encoder described in the application.

[0194] The sequence feature extraction unit 5A is used to perform step S5A: adopting a transposed encoder to perform sequence feature extraction on the global enhanced signal feature to obtain a final pain signal feature.

[0195] Specifically, the specific steps of the transposed encoder are as follows:

[0196] The global enhanced signal feature is copied to obtain several copies of the global enhanced signal feature; wherein the copying operation is used to perform multi-path processing on the global enhanced signal feature, so as to increase the depth of the model or process multiple different feature paths, and then increase the depth of the network or enhance the expression ability of the network.

[0197] Then, the several copies of the global enhanced signal feature are respectively left padded, and a time domain convolutional network (TCN) is used to extract time-dependent features in parallel to obtain several time domain signal feature maps; wherein the left padding is used to ensure that the output of each time step only depends on the current and past time step data, without involving future time step data, so as to ensure the causality of the convolution operation; and the time domain convolutional network captures the time-dependent relationship in the input feature sequence through the causal convolution and extended convolution operations in the stacked one-dimensional convolution layer, so as to extract the dependency between different time steps, and then model the long-term and short-term dependency information in the time sequence type feature sequence.

[0198] The multi-head attention mechanism (MHA) is used to perform attention feature extraction on all time domain signal feature maps to obtain the final pain signal feature;

[0199] The multi-head attention mechanism performs multi-head parallel attention calculation on all time domain signal feature maps to obtain a plurality of attention scores. For the i-th attention score AttScore i The specific calculation expression can be referred to as follows:

[0200]

[0201] In the formula, is a time domain signal feature map. V represents that the feature map is used as a value vector (Value), i.e., a weighted representation of the input feature. L represents the number of input feature sequences, i.e., the total number of features input into the multi-head attention mechanism. e ij represents the similarity score between the i-th query vector (Query) and the j-th key vector (Key), which is obtained by dot product calculation. The specific calculation expression can be represented as follows:

[0202]

[0203] In the formula, and respectively represent the time domain signal feature map as the query vector and the key vector;

[0204] Then, all the attention scores are weighted and spliced, and the weighted and spliced attention scores are fused with the global enhancement signal feature to obtain the final pain signal feature FinPainFeature. The specific calculation expression is as follows:

[0205] FinPainFeature = Normalization (AttConcat)

[0206] AttConcat = EnhanceFeature + Concat (AttScore1, AttScore2, …, AttScore H )·W

[0207] In the formula, H represents the total number of attention heads; W represents a learnable weight used for linear transformation, and makes the dimensions of the input and the output the same; + represents addition fusion; Normalization represents a normalization operation, which is used to normalize the weighted and spliced attention scores AttConcat.

[0208] It should be noted that the specific structure parameters (such as the convolution kernel size, the expansion factor, the number of heads, etc.) of the time domain convolution network (TCN) and the multi-head attention mechanism (MHA) will be flexibly selected according to the requirements of the actual application, and therefore the selection of the structure parameters of the present application is not specifically limited.

[0209] The signal data pain grading unit 6A is configured to perform step S6A: performing pain grading on the final signal features by using a support vector machine to obtain the pain degree of the patient, and completing the pain grading of the physiological signal type.

[0210] The support vector machine can reduce the parameter amount of the model while ensuring the overall network performance, and fully utilize the relationship between the features.

[0211] Since the support vector machine (SVM) has strong generalization ability, it can effectively reduce the overfitting phenomenon when processing high-dimensional features. At the same time, the SVM can mine the nonlinear relationship between the signal features in the training process and fit the grading boundary between different pain degrees. Accordingly, the pain is graded by the dividing surface in the feature space, which can reduce the interference of individual differences while reducing parameter redundancy, thereby improving the accuracy and stability of the pain grading.

[0212] In addition, since pain usually does not occur instantaneously, but is a dynamic process with time sequence, the present application performs time sequence modeling on the signal features by using a time domain convolution network (TCN), that is, extracts the time dependence between the features to capture the dynamic characteristics of the pain signal changing over time, thereby constructing long-term and short-term dependence information in the pain signal features, and further improving the accurate grading ability of the pain.

[0213] Please refer to Figure 7 , Figure 7 For the image preprocessing flow of the face image

[0214] The image preprocessing unit 2B is configured to perform step S2B: performing image preprocessing on the pain data to be detected to obtain a facial pain expression image.

[0215] Specifically, a patient face recognition algorithm is used to locate the patient face of the pain data to be detected, and a patient face feature point extractor is used to obtain the patient face key points.

[0216] The patient face recognition algorithm is used to determine the position of the patient face region in the image, and specifically can adopt the patient face recognition algorithm in the Dlib library to accurately position the face in the image, so as to obtain the image of the face region; the patient face feature point extractor is used to locate the face feature points, i.e. the patient face key points, in the image of the face region, which are used for subsequent face image alignment, size normalization and the like, and the patient face key points are of a specification of 68 points, but different numbers of key point combinations can be adopted according to specific conditions, which are not specifically limited herein.

[0217] Then, the patient face image alignment, size normalization and graying are sequentially performed according to the patient face key points, and the face pain expression image is obtained.

[0218] The patient face image alignment is used to rotate and cut according to the patient face key points, so that the corresponding face features are in the standard position of the image, so as to focus on the face expression region and reduce the recognition error caused by the angle or posture; the size normalization is to adjust the face expression region to a fixed size through bilinear interpolation, so as to ensure that the input image maintains a consistent size, thereby ensuring the efficient calculation of the neural network; and the graying is used to convert the colorful face expression region image into a single-channel gray image through weighted average, so as to reduce the calculation complexity and retain the core features of the face expression region image.

[0219] It should be noted that there are many different algorithms in Dlib, and the face recognition algorithms related to OpenCV or MTCNN can also be adopted according to the actual situation, and the selection of the face recognition and positioning related algorithms is not specifically limited herein.

[0220] The double-branch feature extraction unit 3B is used to perform step S3B: adopting a double-branch feature extraction mechanism to perform multi-scale feature extraction on the face pain expression image, and alternately performing feature fusion in combination with a residual and an attention mechanism, to obtain a local-priority feature map and a global-priority feature map.

[0221] Specifically, the double-branch feature extraction mechanism includes a residual convolution local feature extraction branch and an attention-based global feature extraction branch.

[0222] The residual convolution local feature extraction branch is provided with a plurality of layers of residual convolution modules, which are used to perform multi-scale feature extraction on the face pain expression image, please refer to Figure 8 and Figure 9 for the residual convolution module Residual i of the i-th layer, which extracts the i-th layer of local feature map, and specifically represents as follows:

[0223] ResFeaturei = Residual i (ResFeature i-1 )+ ResFeature i-1

[0224] wherein ResFeature i is the local feature map of the i-th layer, and when i = 1, it represents the facial pain expression image, and the total number of i is 4 by default in the present application, and when i = 4, it represents the local priority feature map; and + represents the residual connection; and the structure of Residual i is shown as follows:

[0225]

[0226] wherein DownConv 1×1 (FM) represents the down-sampling operation on the feature map FM, which is used to reduce the dimension of the feature map, and the convolution kernel size is 1x1; SpatoalConv 3×3 (FM) represents the spatial convolution on the feature map FM, and the convolution kernel size is 3x3; UpConv 1×1 (FM) represents the up-sampling operation on the feature map FM, which is used to expand the feature map, and the convolution kernel size is 1x1; and N i represents the total number of convolution operations in the residual convolution module ResFeature i of the i-th layer, and the total number of convolution operations set N is {3, 4, 6, 3} by default.

[0227] The attention-based global feature extraction branch is provided with a plurality of layers of moving window attention modules, which extracts the features in each window by dividing the image into a plurality of small windows and applying a self-attention mechanism, so as to capture the local information and subtle features in the facial expression image, and improve the expression ability of the global features through the multi-level window combined with the attention mechanism; and the i-th layer of moving window attention module SwinTrans i extracts the i-th layer of global feature map, and the specific representation is as follows:

[0228]

[0229] wherein AttFeature i represents the i-th layer of global feature map, which is used to represent the feature map calculated by each layer, and when i = 1, it represents the facial pain expression image, and the total number of i is 4 by default in the present application, and when i = 4, it represents the global priority feature map; and L Swin represents the total number of layers of the attention-based global feature extraction branch; please refer to Figure 10 , wherein SwinTrans iThe structure is shown as follows:

[0230]

[0231] FM mid = WMSA(LN(FM))+FM

[0232] In the formula, LN is layer normalization, which is used to standardize the input features; WMSA is a window multi-head self-attention mechanism, which is used to capture local dependencies within a window and establish the association of global features layer by layer through a shifted window strategy; FM mid represents an intermediate feature map, that is, a feature map extracted through the window multi-head self-attention mechanism; MLP is a module containing two fully connected layers and a nonlinear activation function (such as GELU), which is used to enhance the nonlinear modeling ability of the features; + represents a residual connection, which is used to alleviate the problem of gradient disappearance, accelerate network convergence, and reduce feature loss in the feature extraction process.

[0233] Then, for the local feature map ResFeature i and the global feature map AttFeature i obtained in the i-th layer, the residual and attention mechanism are alternately used for feature fusion, please refer to Figure 11 and Figure 12 , which are specifically shown as follows:

[0234]

[0235] finalFuse i = CBAM(Res(fuse i )+Conv 1×1 (fuse i ))

[0236] In the formula, reshape(AttFeature i ) represents adjusting the size of the global feature map AttFeature i to be consistent with that of the local feature map ResFeature i ; || represents concatenation (Concat), which forms a local aggregation feature map fuse i by concatenating the global feature map and the local feature map; represents the aggregation operation in the i-th layer, and the size of the global feature map after reshaping is inconsistent according to the number of layers of the corresponding branch; finalFuse ifusei represents the final fused feature map of the i-th layer, which is the feature map after the feature maps of different branches of the i-th layer are completely fused; Conv 1×1 represents a convolution operation with a 1x1 kernel; Res represents a residual block, and its specific calculation is represented as follows:

[0237] Res(fuse i )=ReLU(BN(Conv 3×3 (fuse i ))) K

[0238] In the formula, ReLU is a rectified linear unit (Rectified Linear Unit) used to improve the nonlinear representation ability of features; BN is batch normalization (Batch Normalization); Conv 3×3 represents a convolution operation with a 3x3 kernel; K represents repeating the operation of ReLU(BN(Conv 3×3 (fuse i ))) K times, which is 2 by default; CBAM is a convolutional attention mechanism (Convolutional Block Attention Module) used to weight the channels and space of features, thereby strengthening the key information in the features. Please refer to, and its specific representation is as follows:

[0239]

[0240] In the formula, SpatialAtt represents a spatial attention module used to calculate the attention weight of each spatial position in the input feature map, thereby strengthening the features of important regions in the input feature map; ChannelAtt represents a channel attention module used to calculate the attention weight of each channel in the input feature map, thereby highlighting important channel information in the input feature map; represents a point-by-point multiplication operation, and the attention weight is multiplied point by point with the input feature map, thereby realizing channel or spatial attention weighting to obtain a weighted feature map.

[0241] After obtaining the final fused feature map of the i-th layer, it is fused back to the feature map of the corresponding branch, and its specific representation is as follows:

[0242] AttFeature i =AttFeature i +finalFuse i

[0243] ResFeature i =ResFeature i +finalFusei

[0244] In the formula, AttFeature i and ResFeature i are respectively fused with the final fusion feature map finalFuse i .

[0245] It should be noted that, since different branches may correspond to different feature space dimensions, and these dimensions may vary with the number of layers, therefore, when performing fusion, if the feature map dimensions of the two branches are inconsistent, the spatial size of the feature map needs to be adjusted by reshape or interpolation method. Accordingly, in order to adapt to different needs, the actual dimensions of the feature map are not specifically limited by the present application.

[0246] The present application extracts local feature maps and global feature maps respectively through a double-branch feature extraction mechanism, and alternately fuses the global and local feature maps of each layer using residual and attention mechanisms. Specifically, the residual connection ensures the effective transmission of local features, and the attention mechanism weightedly fuses global information, so that the feature map can not only retain local detailed information, but also fuse global context information in the feature extraction process of each layer of the double-branch. Accordingly, the present application can dynamically fuse local and global features in each layer of the double-branch, thereby improving the recognition ability of facial pain expressions, and effectively capturing multi-scale information, thereby significantly improving the accuracy of pain expression recognition.

[0247] The global-local feature map fusion unit 4B is used to perform step S4B: fuse the local priority feature map and the global priority feature map to obtain the final pain feature map.

[0248] Specifically, the specific calculation of the final pain feature map is as follows:

[0249]

[0250] In the formula, ResFeature (i＝last) denotes the local priority feature map of the final layer last; AttFeature (i＝last) denotes the global priority feature map of the final layer last; denotes an addition operation, which is used to effectively combine local microscopic information and global macroscopic information to form a comprehensive representation, enhance the understanding of the network to the overall facial expression, and at the same time avoid possible feature loss or information error; the default value of last is 4, and the feature extraction depth of the network can be adjusted according to actual conditions, and the present application does not specifically limit the selection of the number of layers.

[0251] According to the application, the final level feature maps of different feature extraction branches are added to fully utilize the complementarity of local features and global features, especially the local priority feature map is used to capture detailed features of pain degree, the global priority feature map is used to reflect the global features of the whole face emotion and pain, and the two are fused by addition operation to effectively fuse local detailed information and global context information, so as to accurately extract high-dimensional features of facial expressions, and further improve the accuracy of pain degree classification and effectively cope with individual differences of facial expressions on pain perception.

[0252] The image data pain grading unit 5B is used to perform step S5B: adopting a full connection layer to perform pain grading on the final pain feature map, obtaining the pain degree of the patient, and completing the pain grading of the image type.

[0253] Specifically, the final pain feature map is flattened into a one-dimensional vector, and the one-dimensional vector is mapped to the pain degree by the full connection layer to obtain the pain degree of the patient.

[0254] The full connection layer is used as a classifier, which includes a nonlinear activation function (ReLU) and a Softmax layer, and outputs the probability distribution of each pain degree, and the specific calculation of the pain degree of the patient is as follows:

[0255]

[0256] In the formula, P(y=k|x) represents the probability that the input feature belongs to the kth pain degree, the sum of the probabilities of all pain degrees is 1, and the highest probability level is the pain degree of the current patient; W k and b k respectively represent the weight matrix and the bias vector of the Softmax layer, which are obtained by training; C represents the total number of pain degree grades, and the default value is 4; x represents the one-dimensional vector after nonlinear transformation by ReLU, and the specific calculation is as follows:

[0257] x=ReLU(W FC ·flatten(finalFeature (i＝last) )+b FC )

[0258] In the formula, flatten represents the flattening operation, which arranges each dimension of the input feature map into a one-dimensional vector in order; W FC and b FC represent the weight matrix and the bias matrix of the full connection layer, which are obtained by training.

[0259] Accordingly, the present application can capture detailed local features in facial expressions by fully fusing local and global features, using nonlinear mapping and Softmax classification, and combining global context information, thereby comprehensively improving the fineness of pain grading, and significantly improving the accuracy of pain grading when individual differences are encountered.

[0260] Further, when the pain grading of the physiological signal type and the image type is completed at the same time, and the pain degree of the patient of the physiological signal type and the image type is obtained at the same time, an adaptive weight fusion full connection layer is used to fuse the pain degrees of all patients, obtain a high-accuracy pain degree of the patient, and complete the pain grading of the patient.

[0261] Specifically, the specific calculation of the adaptive weight fusion full connection layer is as follows:

[0262] FinResult = W s ·SingalResult + W I ·ImageResult

[0263] In the formula, FinResult is the high-accuracy pain degree of the patient; SingalResult is the pain degree of the patient of the physiological signal type, which is obtained by supporting vector machine SVM pain grading of the final signal feature, and the present application further uses Platt scaling to map the final signal feature to a probability space by SVM to obtain the probability of all pain degree labels; ImageResult is the pain degree of the patient of the image type, which is obtained by classification by the Softmax layer, that is, the probability of all pain degree labels; W s and W I are respectively the learnable weight matrix of the physiological signal type and the learnable weight matrix of the image type in the full connection layer, which are obtained by training.

[0264] Compared with the prior art, in the pain degree grading of the physiological signal type, the present application combines a multi-scale convolution mechanism and a compression-excitation residual network to deeply mine key information in the physiological signal, effectively captures complex time sequence characteristics and high-dimensional dependency in the physiological signal under the cooperation of the time domain convolution network and the multi-head attention mechanism, and realizes accurate grading under the background of individual differences through the demarcation ability of the support vector machine in the feature space, thereby significantly improving the accuracy, reliability and generalization of the physiological signal type pain grading.

[0265] Meanwhile, in the pain degree grading of the facial image type, a double-branch feature extraction mechanism is adopted to fully extract the local features and global features of the facial expression image, especially through the residual convolution local feature extraction branch to capture the subtle changes of the expression, and in combination with the attention-based global feature extraction branch to focus on the overall changes of the face and the context information, and on the basis of the double-branch, the residual and attention mechanism are alternately fused to realize the deep integration of the local and global features, so that the feature extraction of the facial image is more robust and diverse, and the accuracy and stability of the image type pain grading are significantly improved.

[0266] In addition, the physiological signal and the pain grading result of the facial image type are integrated through the adaptive weight fusion full connection layer, the time sequence continuity feature of the physiological signal and the spatial intuitiveness feature of the facial image are fully utilized, and the weight distribution of different modalities is dynamically adjusted, so that the shortcomings of the single modal data caused by noise, individual difference or scene limitation are effectively made up, and the accuracy and robustness of pain recognition are significantly improved.

[0267] It should be noted that, in order to improve the accuracy of pain degree recognition, the trainable parameters corresponding to the weight matrix and the bias matrix in the physiological signal type pain grading and the facial image type pain grading need to be trained respectively.

[0268] The specific steps of training of the physiological signal type pain grading are as follows:

[0269] DA1, collect skin electric signals of a plurality of patients, set the pain degree labels corresponding to the skin electric signals based on the subjective scores of the patients and the evaluation of professional doctors, and construct a physiological signal data set;

[0270] DA2, divide the physiological signal data set to obtain a training set, a validation set and a test set.

[0271] The division ratio of the loss training set, the validation set and the test set is 7:2:1, and according to different requirements, the corresponding division can be 8:1:1 or other, and the division ratio of the present application is not limited here.

[0272] DA3, input the skin electric signals of the patients in the training set into the pain degree recognition method of the patient of the present application to grade the pain and obtain the pain degree corresponding to the current skin electric signals.

[0273] In the method for identifying the degree of pain of the patient, the corresponding hyperparameters, such as the optimizer, the learning rate and the Dropout parameter of the full connection layer, can be selected as follows: the optimizer of Adam, the learning rate of 0.001 and the dropout of 0.5. Different hyperparameters, such as grid search and Bayesian optimization, can be selected according to different actual hardware environments, so as to improve the convergence efficiency of training, and therefore, the selection of the hyperparameters is not limited herein.

[0274] DA4, using a loss function to calculate the loss between the degree of pain corresponding to the current electrodermal signal and the degree of pain label in the training set, and obtaining a loss value.

[0275] The loss function can be selected as a cross-entropy loss, and the loss function can be a weighted combination of multiple common loss functions, which can be set according to requirements, and therefore, the selection of the loss function is not limited herein.

[0276] DA5, determining whether the current training batch or the loss value meets a training condition: if yes, completing the training and obtaining the high-accuracy method for identifying the degree of pain of the patient; if no, performing back propagation on the method for identifying the degree of pain of the patient according to the loss value, obtaining the method for identifying the degree of pain of the patient with updated weights, and performing step DA3.

[0277] The search for the training condition further includes performing pain grading on the current method for identifying the degree of pain of the patient using the test set or the training set, and calculating the accuracy rate, and when the accuracy rate reaches a corresponding threshold, such as a pain grading accuracy rate of 90%, the training can be ended in advance.

[0278] The specific steps of the training of the image type pain grading are as follows:

[0279] DB1, taking a face image of a patient in pain, and setting a pain degree label corresponding to the face image of the patient based on the subjective score of the patient and the evaluation of a professional doctor, to construct a face image data set;

[0280] DB2, dividing the face image data set to obtain a training set, a validation set and a test set.

[0281] The division ratio of the loss training set, the validation set and the test set is 7:2:1, and according to different requirements, the corresponding division can be selected as 8:1:1 or other, and the selection of the division ratio is not limited herein.

[0282] DB3, inputting the face image of the patient in the training set into the method for identifying the degree of pain of the patient to perform pain grading, and obtaining the degree of pain corresponding to the current face image.

[0283] In the method for identifying the degree of pain of the patient according to the present application, the corresponding hyperparameters such as the optimizer, the learning rate and the Dropout parameter of the fully connected layer. Since different hyperparameters can be selected according to the actual hardware environment, for example, grid search, Bayesian optimization, etc., to improve the convergence efficiency of the training, the present application does not specifically limit the selection of the hyperparameters.

[0284] DB4, using a loss function to calculate the loss between the degree of pain corresponding to the current facial image and the degree of pain label in the training set, and obtaining a loss value.

[0285] In the method for identifying the degree of pain of the patient according to the present application, the corresponding hyperparameters such as the optimizer, the learning rate and the Dropout parameter of the fully connected layer. Since different hyperparameters can be selected according to the actual hardware environment, for example, grid search, Bayesian optimization, etc., to improve the convergence efficiency of the training, the present application does not specifically limit the selection of the hyperparameters.

[0286] DB5, determining whether the current training batch or the loss value meets a training condition: if yes, completing the training and obtaining a high-accuracy method for identifying the degree of pain of the patient; if no, performing back propagation on the method for identifying the degree of pain of the patient according to the present application according to the loss value, obtaining an updated weight of the method for identifying the degree of pain of the patient, and performing step DB3.

[0287] In the method for identifying the degree of pain of the patient according to the present application, the corresponding hyperparameters such as the optimizer, the learning rate and the Dropout parameter of the fully connected layer. Since different hyperparameters can be selected according to the actual hardware environment, for example, grid search, Bayesian optimization, etc., to improve the convergence efficiency of the training, the present application does not specifically limit the selection of the hyperparameters.

[0288] Further, when the physiological signal type and the image type of the pain grading are trained, a multi-modal training method can be performed, and the specific steps are as follows:

[0289] DC1, inputting the same pain grading label of the electrodermal signal and the facial image in the physiological signal data set and the facial image data set into the method for identifying the degree of pain of the patient according to the present application, performing pain grading, and obtaining the final degree of pain.

[0290] DC2, using a joint loss function to calculate the loss between the final degree of pain and the corresponding pain grading label, and obtaining a loss value.

[0291] DC3, determining whether the current training batch or the loss value meets a training condition: if yes, completing the training and obtaining a high-accuracy method for identifying the degree of pain of the patient; if no, performing back propagation on the method for identifying the degree of pain of the patient according to the present application according to the loss value, obtaining an updated weight of the method for identifying the degree of pain of the patient, and performing step DC1.

[0292] Based on the same inventive concept, the present application also provides an electronic device, which can be a server, a desktop computing device or a mobile computing device (e.g., a laptop computer, a handheld computing device, a tablet computer, a netbook, etc.) or the like terminal device. The device includes one or more processors and a memory, wherein the processor is configured to execute a program to implement the method for identifying the degree of pain of a patient according to the embodiments of the present application; and the memory is configured to store the computer program executable by the processor.

[0293] Based on the same inventive concept, the present application also provides a computer readable storage medium, which corresponds to the method for identifying the degree of pain of a patient according to the above-mentioned embodiments, and the computer readable storage medium has a computer program stored thereon, which, when executed by a processor, implements the steps of the method for identifying the degree of pain of a patient according to any of the above-mentioned embodiments.

[0294] The present application can take the form of a computer program product implemented on one or more storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) having program code embodied in the medium. The computer usable storage medium includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0295] The above-mentioned embodiments only express several embodiments of the present application, which are described in detail and specifically, but should not be understood as limiting the scope of the patent. It should be noted that for those skilled in the art, without departing from the inventive concept, several modifications and improvements can be made, and the present application also intends to include these modifications and improvements.

Claims

1. A method of identifying the degree of pain of a patient, characterized by, The method comprises the following steps: S1, judging the data type of the pain data to be detected: if the data type is a physiological signal type, step S2A is performed; if the data type is an image type, step S2B is performed; S2A, signal preprocessing is performed on the pain data to be detected to obtain pain key signal data; wherein the signal preprocessing is a signal denoising operation; S3A, a multi-scale convolution mechanism is used to extract features from the pain key signal data to obtain preliminary pain signal features; wherein the multi-scale convolution mechanism includes a large segment convolution branch and a small segment convolution branch; The large segment convolution branch extracts global features of the pain key signal data through a large convolution kernel with a step size of 50 and 400*400; The small segment convolution branch extracts local features of the pain key signal data through a small convolution kernel with a step size of 6 and 50*50; S4A, a compression-excitation residual network is used to extract global spatial features from the preliminary pain signal features to obtain global enhanced signal features; S5A, a variable pressure encoder is used to extract sequence features from the global enhanced signal features to obtain final pain signal features; S6A: using a support vector machine to classify the final signal features to obtain the pain degree of the patient, and completing the pain classification of the physiological signal type; S2B, image preprocessing is performed on the pain data to be detected to obtain a facial pain expression image; S3B, a double-branch feature extraction mechanism is used to extract multi-scale features from the facial pain expression image, and residual and attention mechanisms are alternately used for feature fusion to obtain a local priority feature map and a global priority feature map; The double-branch feature extraction mechanism includes a residual convolution local feature extraction branch and an attention-based global feature extraction branch; The residual convolution local feature extraction branch is provided with a plurality of layers of residual convolution modules Residual i ; The attention-based global feature extraction branch is provided with a plurality of layers of moving window attention modules SwinTrans i ; When the residual convolution module Residual i extracted local feature maps, and the global feature maps extracted by the i-th layer moving window attention module SwinTrans i After the residual and attention mechanism are alternately used for feature fusion, the specific representation is as follows: finalFuse i = CBAM(Res(fuse i ) + Conv 1×1 (fuse i )) In the formula, reshape (AttFeature i ) represents resizing the global feature map AttFeature i to be consistent with the local feature map ResFeature i ; || represents a splicing operation, which is used to form the local aggregated feature map fuse i ; represents the aggregation operation of the i-th layer; finalFuse i is the final fusion feature map of the i-th layer, which represents the feature map after the complete fusion of the feature maps of different branches of the i-th layer; Conv 1×1 represents a convolution operation with a 1x1 convolution kernel; and Res represents a residual block. After obtaining the final fusion feature map of the i-th layer, it is fused back to the feature map of the corresponding branch, which is specifically represented as follows: AttFeature i = AttFeature i + finalFuse i ResFeature i = ResFeature i + finalFuse i In the formula, AttFeature i and ResFeature i are respectively fused with the final fusion feature map finalFuse i ; S4B, the local priority feature map and the global priority feature map are fused to obtain a final pain feature map; S5B, the final pain feature map is flattened into a one-dimensional vector, and the one-dimensional vector is mapped to the pain degree through a fully connected layer to obtain the pain degree of the patient, and the pain classification of the image type is completed.

2. The method of claim 1, wherein the pain level of the patient is identified by: The specific steps of the large segment convolution branch in step S3A are as follows: Using a large kernel convolution on pain key signal data Performing large segment feature extraction to obtain large segment features Large GSR The large kernel convolution is specifically represented as In the formula, Conv c,k,s represents a convolution operation, c, k, and s are respectively the number of output channels, the size of the convolution kernel, and the step length, k is set to 400, and s is set to 50; The large segment features are sequentially subjected to nonlinear transformation, max-pooling and feature mapping using a Gaussian error linear unit activation function to obtain preliminary global signal features, which are specifically represented as: H Large = FC(MaxPool d,s (GELU(Large GSR ))) In the formula, H Large is a preliminary global signal feature; GELU denotes Gaussian Error Linear Unit activation function; MaxPool d,s denotes max-pooling operation with window size d and stride s; FC denotes a fully connected layer; Then, the preliminary global signal features are extracted using convolution-pooling to obtain global priority signal features, which are specifically represented as: GlobalFeature = MaxPool d,s (GELU(Conv c,k,s (H Large )) R In the formula, GlobalFeature represents a global priority signal feature; Conv c,k,s is a convolution operation; R represents the number of repeated executions; The specific steps of the small segment convolution branch are as follows: Using a small kernel convolution on pain key signal data Small segment feature extraction is performed to obtain small segment features Small GSR The small kernel convolution is specifically represented as wherein Conv c,k,s denotes a convolution operation, k is by default 50 and s is by default 6; The small segment features are sequentially subjected to nonlinear transformation, max-pooling and feature mapping using a Gaussian error linear unit activation function to obtain preliminary local signal features, which are specifically represented as: H Small = FC(MaxPool d,s (GELU(Small GSR ))) In the formula, H Small is a preliminary local signal feature; MaxPool d,s denotes a max-pooling operation; Then, the preliminary local signal features are extracted using convolution-pooling to obtain local priority signal features, which are specifically represented as: LocalFeature = MaxPool d,s (GELU(Conv c,k,s (H Small )) R where LocalFeature represents a local priority signal feature; Conv c,k,s is a convolution operation; R represents the number of repetitions; When the global priority signal feature and the local priority signal feature are obtained, signal feature fusion and feature mapping are performed to obtain a preliminary pain signal feature, and the specific calculation thereof is shown as follows: In the formula, PainFeature represents a preliminary pain signal feature, which fuses the features of the preliminarily extracted global and local signals; is a vector concatenation; FC is a fully connected layer; The specific steps of the compression-excitation residual network for global spatial feature extraction in step S4A are shown as follows: The one-dimensional convolution and the activation function are used for feature transformation to obtain a preliminary signal feature map F trans The specific calculation is expressed as follows: F trans = Conv c,1,1 (ReLU(Conv c,1,1 (PainFeature))) where PainFeature is the preliminary pain signal feature; Conv c,1,1 is a one-dimensional convolution; ReLU is a non-linear activation function; Then, the global average pooling is used to compress the preliminary signal feature map in space, and a compressed feature map F is obtained pool which is specifically represented as follows: wherein denotes the global average pooling value of the c-th channel of the compressed feature map F pool denotes the feature value of position (u, v) in the c-th channel of the preliminary signal feature map; N denotes the size of the feature map in vertical dimension, i.e. height; L denotes the size of the feature map in horizontal dimension, i.e. width.​ The compressed feature map is adaptively recalibrated to obtain a global enhanced signal feature, and the adaptive recalibration thereof is shown as follows: In the formula, EnhanceFeature is a global enhanced signal feature; W1 and W2 respectively represent a weight matrix of a full connection layer; and σ represents a Sigmoid function. is an element-wise multiplication; and + is an element-wise addition.

3. The method of claim 2, wherein the pain level of the patient is identified by: The specific steps of the variable pressure encoder in step S5A are shown as follows: The global enhanced signal feature is copied to obtain several copies of the global enhanced signal feature; The several copies of the global enhanced signal feature are respectively left-padded, and time-dependent features are extracted in parallel by using a time domain convolution network to obtain several time domain signal feature maps; All the time domain signal feature maps are subjected to attention feature extraction by using a multi-head attention mechanism to obtain a final pain signal feature, and the specific calculation of the multi-head attention mechanism is shown as follows: The multi-head parallel attention calculation is performed on all time domain signal feature maps to obtain a plurality of attention scores. For the mth attention score AttScore m The specific calculation is represented as: wherein, is a time-domain signal feature map, V represents the feature map as a vector of values; P represents the number of feature sequences as input; e mn represents the similarity score between the mth query vector and the nth key vector, obtained by dot product calculation, which can be specifically calculated as: wherein and denote the time-domain signal feature maps as query vectors and as key vectors, respectively; Then, all the attention scores are weighted and spliced, and the weighted and spliced attention scores are fused with the global enhanced signal feature to obtain the final pain signal feature FinPainFeature, and the specific calculation thereof is shown as follows: FinPainFeature = Normalization(AttConcat) AttConcat = EnhanceFeature + Concat(AttScore1, AttScore2, …, AttScore H ) · W In the formula, H represents the total number of attention heads; W represents a learnable weight for linear transformation, and the dimensions of the input and the output are the same; + represents addition fusion; Normalization represents normalization operation; and AttConcat represents the weighted and spliced attention scores.

4. The method of claim 1, wherein the pain level of the patient is identified by: The specific calculation of the final pain feature map is shown as follows: In the formula, ResFeature (i＝last) denotes the local priority feature map of the final layer last; AttFeature (i＝last) denotes the global priority feature map of the final layer last; denotes an addition operation; The specific grading process of the pain degree of the patient based on the final pain feature map is shown as follows: where P(y=k\ x) represents the probability that the input feature belongs to the kth level of pain intensity; W k and b k respectively represent the weight matrix and bias vector of the Softmax layer; C represents the total number of levels of pain intensity grading; x represents a one-dimensional vector after nonlinear transformation by ReLU, and the specific calculation is represented as: x = ReLU(W FC + b (i＝last) ) + b FC ) where flatten is a flattening operation; W FC and b FC denote the weight matrix and bias matrix of the fully connected layer.

5. The method of claim 2-4, wherein the method further comprises: If the data type of the pain data to be detected simultaneously includes a physiological signal type and an image type, steps S2A and S2B are simultaneously performed; When the pain grading of the physiological signal type and the image type is completed, and the pain degrees of the patients of the physiological signal type and the image type are obtained, an adaptive weight fusion full connection layer is used to fuse all the pain degrees of the patients to obtain a high-accuracy pain degree of the patient, and the pain grading of the patient is completed; The specific calculation of the adaptive weight fusion full connection layer is shown as follows: FinResult = W s • SingalResult + W I • ImageResult wherein FinResult is the high-accuracy patient pain level; SingalResult is the physiological signal type patient pain level; ImageResult is the image type patient pain level; W s and W I are the learnable weight matrices for the physiological signal type and the image type in the fully connected layer, respectively.

6. An apparatus for identifying the degree of pain of a patient, characterized by The system comprises a signal type judgment unit, a signal preprocessing unit, a multi-scale convolution feature extraction unit, a compression-excitation residual feature extraction unit, a sequence feature extraction unit, a signal data pain grading unit, an image preprocessing unit, a double-branch feature extraction unit, a global-local feature map fusion unit, and an image data pain grading unit. The signal type judgment unit is configured to judge the data type of the pain data to be detected: if the data type is a physiological signal type, the signal preprocessing unit is called; and if the data type is an image type, the image preprocessing unit is called. The signal preprocessing unit is configured to perform signal preprocessing on the pain data to be detected to obtain pain key signal data. The multi-scale convolution feature extraction unit is configured to extract features of the pain key signal data by using a multi-scale convolution mechanism to obtain preliminary pain signal features, wherein the multi-scale convolution mechanism comprises a long convolution branch and a short convolution branch. The long convolution branch extracts global features of the pain key signal data by using a large convolution kernel with a step size of 50 and a size of 400*400. The short convolution branch extracts local features of the pain key signal data by using a small convolution kernel with a step size of 6 and a size of 50*50. The compression-excitation residual feature extraction unit is configured to extract global spatial features of the preliminary pain signal features by using a compression-excitation residual network to obtain global enhanced signal features. The sequence feature extraction unit is configured to extract sequence features of the global enhanced signal features by using a variable pressure encoder to obtain final pain signal features. The signal data pain grading unit is configured to grade the final signal features by using a support vector machine to obtain the pain degree of the patient, thereby completing pain grading of the physiological signal type. The image preprocessing unit is configured to perform image preprocessing on the pain data to be detected to obtain a facial pain expression image. The double-branch feature extraction unit is configured to extract multi-scale features of the facial pain expression image by using a double-branch feature extraction mechanism, and alternately perform feature fusion by combining residual and attention mechanisms to obtain a local-priority feature map and a global-priority feature map. The double-branch feature extraction mechanism in the double-branch feature extraction unit comprises a residual convolution local feature extraction branch and an attention-based global feature extraction branch. The residual convolution local feature extraction branch is provided with a plurality of layers of residual convolution modules Residual i ; The attention-based global feature extraction branch is provided with a plurality of layers of moving window attention modules SwinTrans i ; When obtaining the residual convolution module Residual i extracted local feature maps, and the global feature maps extracted by the i-th layer moving window attention module SwinTrans i After the residual and attention mechanism are alternately used for feature fusion, the specific representation is as follows: finalFuse i = CBAM(Res(fuse i )+Conv 1×1 (fuse i )) In the formula, reshape (AttFeature i ) represents resizing the global feature map AttFeature i to be consistent with the local feature map ResFeature i ; || represents a splicing operation, which is used to form the local aggregated feature map fuse i ; represents the aggregation operation of the i-th layer; finalFuse i is the final fusion feature map of the i-th layer, which represents the feature map after the complete fusion of the feature maps of different branches of the i-th layer; Conv 1×1 represents a convolution operation with a convolution kernel of 1x1; Res represents a residual block; When the final fusion feature map of the i-th layer is obtained, it is fused back to the feature map of the corresponding branch, which is specifically represented as follows: AttFeature i = AttFeature i + finalFuse i ResFeature i = ResFeature i + finalFuse i In the formula, AttFeature i and ResFeature i are respectively fused with the final fusion feature map finalFuse i ; The global-local feature map fusion unit is configured to fuse the local-priority feature map and the global-priority feature map to obtain a final pain feature map. The image data pain grading unit is configured to flatten the final pain feature map into a one-dimensional vector, and map the one-dimensional vector through a fully connected layer to obtain the pain degree of the patient, thereby completing pain grading of the image type.

7. The device for identifying the degree of pain of a patient according to claim 6, characterized in that The specific steps of the long convolution branch in the multi-scale convolution feature extraction unit are as follows: Using a large kernel convolution on pain key signal data Performing large segment feature extraction to obtain large segment features Large GSR The large kernel convolution is specifically represented as Conv c,k,s denotes a convolution operation, c, k, s are respectively the number of output channels, the size of the convolution kernel and the step, k is set to 400, and s is set to 50; The long features are sequentially subjected to nonlinear transformation, max-pooling and feature mapping by using a Gaussian error linear unit activation function to obtain preliminary global signal features, which are specifically represented as follows: H Large = FC(MaxPool d,s (GELU(Large GSR ))) In the formula, H Large is a preliminary global signal feature; GELU denotes Gaussian Error Linear Unit activation function; MaxPool d,s denotes max-pooling operation with window size d and stride s; FC denotes a fully connected layer; Then, the preliminary global signal features are extracted by using convolution-pooling to obtain global-priority signal features, which are specifically represented as follows: GlobalFeature = MaxPool d,s (GELU(Conv c,k,s (H Large )) R In the formula, GlobalFeature represents a global priority signal feature; Conv c,k,s is a convolution operation; R represents the number of repeated executions; The specific steps of the short convolution branch are as follows: Using a small kernel convolution on pain key signal data Small segment feature extraction is performed to obtain small segment features Small GSR The small kernel convolution is specifically represented as wherein Conv c,k,s denotes a convolution operation, k is by default 50 and s is by default 6; The short features are sequentially subjected to nonlinear transformation, max-pooling and feature mapping by using a Gaussian error linear unit activation function to obtain preliminary local signal features, which are specifically represented as follows: H Small = FC(MaxPool d,s (GELU(Small GSR ))) In the formula, H Small is a preliminary local signal feature; MaxPool d,s denotes a max-pooling operation; Then, the preliminary local signal features are extracted by using convolution-pooling to obtain local-priority signal features, which are specifically represented as follows: LocalFeature = MaxPool d,s (GELU(Conv c,k,s (H Smalll )) R where LocalFeature represents a local priority signal feature; Conv c,k,s is a convolution operation; R represents the number of repetitions; When the global-priority signal features and the local-priority signal features are obtained, signal feature fusion and feature mapping are performed to obtain preliminary pain signal features, which are specifically calculated as follows: In the formula, PainFeature represents a preliminary pain signal feature, which fuses the features of the preliminarily extracted global and local signals; is a vector concatenation; FC is a fully connected layer; The specific steps of the compression-excitation residual network in the compression-excitation residual feature extraction unit for global spatial feature extraction are as follows: The one-dimensional convolution and the activation function are used for feature transformation to obtain a preliminary signal feature map F trans The specific calculation is expressed as follows: F trans = Conv c,1,1 (ReLU(Conv c,1,1 (PainFeature))) where PainFeature is the preliminary pain signal feature; Conv c,1,1 is a one-dimensional convolution; ReLU is a non-linear activation function; Then, the global average pooling is used to compress the preliminary signal feature map in space, and a compressed feature map F is obtained pool which is specifically represented as follows: wherein denotes the global average pooling value of the c-th channel of the compressed feature map F pool denotes the feature value of position (u, v) in the preliminary signal feature map in channel c; N denotes the size of the feature map in vertical dimension, i.e. height; L denotes the size of the feature map in horizontal dimension, i.e. width.​ The compressed feature map is adaptively recalibrated to obtain a global enhancement signal feature, and the adaptive recalibration is specifically calculated as follows: In the formula, EnhanceFeature is a global enhanced signal feature; W1 and W2 respectively represent a weight matrix of a full connection layer; and σ represents a Sigmoid function. is an element-wise multiplication; and + is an element-wise addition. The specific steps of the variable pressure encoder in the sequence feature extraction unit are as follows: The global enhancement signal feature is copied to obtain several copies of the global enhancement signal feature; The several copies of the global enhancement signal feature are respectively left padded, and a time-dependent feature is extracted in parallel using a time domain convolution network to obtain several time domain signal feature maps; All time domain signal feature maps are extracted using a multi-head attention mechanism to obtain a final pain signal feature; wherein the specific calculation of the multi-head attention mechanism is as follows: The multi-head parallel attention calculation is performed on all time domain signal feature maps to obtain a plurality of attention scores. For the mth attention score AttScore m The specific calculation is represented as: wherein, is a time-domain signal feature map, V represents the feature map as a value vector; P represents the number of feature sequences as input; e mn represents the similarity score between the mth query vector and the nth key vector, which is obtained by dot product calculation, and the specific calculation can be represented as: wherein and denote the time-domain signal feature maps as query vectors and as key vectors, respectively; Then, all attention scores are weighted and spliced, and the weighted and spliced attention scores are fused with the global enhancement signal feature to obtain the final pain signal feature FinPainFeature, which is specifically calculated as follows: FinPainFeature=Normalization(AttConcat) AttConcat = EnhanceFeature + Concat(AttScore1, AttScore2, …, AttScore H ) · W In the formula, H represents the total number of attention heads; W represents a learnable weight for linear transformation, and makes the dimensions of the input and output the same; + represents addition fusion; Normalization represents normalization operation; AttConcat is the weighted and spliced attention score.

8. The device for recognizing the degree of pain of a patient according to claim 6, characterized in that, The specific fusion calculation process of the final pain feature map in the global-local feature map fusion unit is as follows: In the formula, ResFeature (i＝last) denotes the local priority feature map of the final layer last; AttFeature (i＝last) denotes the global priority feature map of the final layer last; denotes an addition operation; The specific grading process of the patient's pain degree based on the final pain feature map is as follows: where P(y=k\ x) represents the probability that the input feature belongs to the kth level of pain intensity; W k and b k respectively represent the weight matrix and bias vector of the Softmax layer; C represents the total number of levels of pain intensity grading; x represents a one-dimensional vector after nonlinear transformation by ReLU, and the specific calculation is represented as: x = ReLU(W FC • flatten(finalFeature (i＝last) ) + b FC ) where flatten is a flattening operation; W FC and b FC denote the weight matrix and bias matrix of the fully connected layer.

9. A pain level recognition system characterized by, It includes a patient end, a doctor end and a family end, wherein the patient end obtains the pain data to be detected through a camera or a GSR signal real-time monitor, and inputs the pain data to a pain degree recognition device; The pain degree recognition device obtains the patient's pain degree by performing feature extraction and pain grading on the pain data to be detected, and inputs the pain degree to the doctor end and the family end respectively; The doctor end is used to obtain the patient's pain degree, and by obtaining the associated data in the patient database, the associated data of the current patient is displayed for the doctor to analyze and obtain the doctor's treatment suggestion to the patient; The family end is provided with an AI assistant module, which is used to perform personalized analysis on the patient's pain degree history record, and provide preliminary medical analysis, pain care suggestion and personalized diet scheme according to the analysis result; The pain degree recognition device is the pain degree recognition device of any one of claims 6-8.

Images