Method and system for assessing pain of hospitalized neonates based on limb motion features

Abstract

The application discloses a method and system for assessing pain of hospitalized newborns based on limb action features, relates to the technical field of machine learning and pattern recognition, and comprises the following steps: acquiring a newborn video, segmenting the newborn video into a video segment sequence, performing a preprocessing operation on each frame of image in each video segment to obtain a newborn pain video segment, performing pain category labeling on the newborn pain video segment, and constructing a newborn pain video sample set; using samples in the newborn pain video sample set to train a pre-established newborn pain classification model to obtain a trained newborn pain classification model, wherein the processing procedure of the pre-established newborn pain classification model comprises video preprocessing, first branch feature extraction, second branch feature extraction, multi-scale convolution attention operation, feature fusion and classification; acquiring a test video, inputting the test video into the trained newborn pain classification model, and obtaining a newborn pain classification assessment result.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and pattern recognition technology, specifically to a method and system for assessing pain in hospitalized newborns based on limb movement features. Background Technology

[0002] Pain is considered the fifth vital sign in humans. Studies show that newborns experience an average of 7.5 to 17.3 painful stimuli per day during hospitalization, with premature infants experiencing as many as 26 times per day. Long-term, repeated pain stimulation can alter the brain structure of newborns, leading to negative effects such as changes in cognitive and motor function and weakened behavioral control. To strengthen neonatal pain management, many countries have published their own neonatal pain management guidelines, the core of which focuses on pain assessment, prevention, and treatment. The gold standard for pain assessment is self-report; however, newborns lack verbal communication skills. Therefore, researchers have developed various neonatal pain assessment scales, such as the Premature Infant Pain Profile (PIPP), the Neonatal Infant Pain Scale (NIPS), the Neonatal Acute Pain Assessment Scale (NIAPAS), and the Neonatal Postoperative Pain Measurement Tool (CRIES). Currently, neonatal pain assessment is conducted manually by healthcare professionals using these pain assessment scales. However, manual assessment is time-consuming and labor-intensive, difficult to promote and popularize, and the results depend on the experience of healthcare professionals and are influenced by individual subjective factors. Therefore, developing an automated pain assessment system for newborns to help clinicians assess pain levels more promptly and accurately, so as to take appropriate analgesic measures to alleviate the pain of newborns, has practical clinical application value.

[0003] In recent years, with the development of machine learning technology, research on neonatal pain expression recognition has attracted attention. However, pain assessment methods based on facial expressions are easily affected by occlusion and changes in lighting, resulting in low accuracy and poor robustness. Furthermore, hospitalized newborns often have their faces wrapped in gauze, making facial expressions unrecognizable. Therefore, the pain assessment method for hospitalized newborns based on limb movement characteristics proposed in this invention can open a new path for the development of automated auxiliary pain assessment systems for newborns.

[0004] Biological studies of retinal neurons in the visual system of primates have revealed that parvocellular neurons account for approximately 80% of retinal neurons, while magnocellular neurons account for approximately 15-20%. Magnocellular neurons operate with higher temporal resolution and respond to rapid changes, but they are less sensitive to spatial details or color. Parvocellular neurons are more sensitive to spatial details and color, but have lower temporal resolution and respond to stimuli more slowly.

[0005] In scenarios involving the extraction of limb action features, the spatial semantic changes of limbs typically progress relatively slowly. For example, when waving an arm, the arm remains an arm and does not transform into a leg, allowing for the extraction of spatial detail features of the limb at a lower temporal resolution. On the other hand, the changes in limb movement typically progress relatively quickly. For example, when waving an arm, the arm's movement changes rapidly, so it is necessary to effectively extract limb action features at a higher temporal resolution. Summary of the Invention

[0006] To address the shortcomings mentioned in the background art, the present invention aims to provide a method and system for assessing pain in hospitalized newborns based on limb movement features. By using a feature extraction module with two branches to extract movement features from videos at different frame rates, the ability to capture subtle changes in movement is improved. Furthermore, a multi-scale convolutional attention module is employed to focus on key features of limb movements, effectively improving the accuracy of newborn pain classification and laying the foundation for the development of an automatic assisted pain assessment system based on limb movements.

[0007] Firstly, the objective of this invention can be achieved through the following technical solution: a method for assessing pain in hospitalized newborns based on limb movement characteristics, the method comprising the following steps:

[0008] Acquire newborn videos, segment the newborn videos into video segment sequences, preprocess each frame of each video segment to obtain newborn pain video segments, label the newborn pain video segments with pain categories, and construct a newborn pain video sample set;

[0009] The pre-established neonatal pain classification model is trained using samples from a neonatal pain video sample set to obtain the trained neonatal pain classification model. The pre-established neonatal pain classification model processing includes video preprocessing, first branch feature extraction, second branch feature extraction, multi-scale convolutional attention operation, feature fusion, and classification.

[0010] Obtain the test video and input it into the trained neonatal pain classification model to obtain the neonatal pain classification assessment results.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the newborn video is acquired during clinically painful procedures and in a painless state, and the length of the video segment is T frames.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the video preprocessing operation process is as follows:

[0013] The input newborn video is segmented into a sequence of video clips. Human detection, cropping, alignment, and normalization are performed on each frame of each video clip to obtain a newborn pain video clip of length T frames. Where H and W are the height and width of the human body image, respectively, and T is the number of frames for each video segment.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: inputting the newborn pain video clip into a pre-established first branch feature extraction module and a second branch feature extraction module for feature extraction;

[0015] The first branch feature extraction module consists of a frame extraction module and a deep convolutional neural network Net1. The frame extraction module extracts one frame every M frames from the neonatal pain video clip output by the video preprocessing module, reducing the video's frame rate, where M represents the frame extraction coefficient. The deep convolutional neural network Net1 is used to extract the neonatal limb movement features from the low frame rate video clip after frame extraction. The features extracted by Net1 are divided into N levels for output, where the p-th level feature is represented as... p represents the index of the feature level, p∈{1,2,…,N}, H p W represents the height of the p-th level feature map. p C represents the width of the feature map at the p-th level. 1,p This represents the number of channels for the p-th level feature output by Net1;

[0016] The second branch feature extraction module consists of a deep convolutional neural network Net2, used to extract newborn limb movement features from high frame rate video clips output by the video preprocessing module. The features extracted by Net2 are divided into N levels of output, where the p-th level feature is represented as... p represents the index of the feature level, p∈{1,2,…,N}, H p W represents the height of the p-th level feature map. p C represents the width of the feature map at the p-th level. 2,p Let represent the number of channels for the p-th level feature output by Net2; Q represents the channel scaling ratio.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the p-th level feature F output by Net1. 1,p and the p-th level feature F output by Net2 2,p Multi-scale convolutional attention operations are introduced through the p-th multi-scale convolutional attention module among N multi-scale convolutional attention modules, outputting a multi-scale feature tensor F focusing on limb movements. p ′.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the operation process of the p-th multi-scale convolutional attention module is as follows:

[0019] The p-th level feature output by Net1 and the p-th level features output by Net2 Concatenate along the channel dimension to obtain the feature tensor. The feature tensor F is obtained by using a multi-scale convolutional attention mechanism. p Attention weight ω p for:

[0020]

[0021] Where σ represents the Sigmoid activation function, Conv 1×1×1 This represents a pointwise convolution with a 1×1×1 kernel, where L is the number of depthwise separable convolutions, and DConv j This indicates the j-th depthwise separable convolution, where each depthwise separable convolution uses a depthwise convolution with striped convolution kernels of k×1×1, 1×k×1, and 1×1×k.

[0022] The attention weight ω p Channel-by-channel and feature tensor F p The components are multiplied and then subjected to pointwise convolution with a 1×1×1 kernel to output a multi-scale feature tensor. for:

[0023] F p ′=Conv 1×1×1 (ω p ·F p )

[0024] Among them, Conv 1×1×1 This represents a pointwise convolution with a 1×1×1 kernel.

[0025] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the feature fusion operation method is: performing pooling operation on the features finally output by the first branch feature extraction module and the second branch feature extraction module, then flattening them into feature vectors, and concatenating them to obtain the fused neonatal limb movement feature vector V; the classification operation method is: inputting the neonatal limb movement feature vector V into a classifier for classification and recognition, and outputting the neonatal pain category, including calm, mild pain, moderate pain and severe pain.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the loss function for each training sample during the training of the pre-established neonatal pain classification model using samples from the neonatal pain video sample set is defined as:

[0027]

[0028] Where K is the number of pain categories, c∈{1,2,…,K}; y c This indicates that the training sample is assigned the true class label of class c. When using one-hot encoding, if the training sample is assigned to class c, then y c =1, otherwise y c =0; α represents the probability that the model will predict the training sample as class c; α is a weighting factor introduced to address the class imbalance problem; β represents the adjustment factor.

[0029] Secondly, in order to achieve the above objectives, this invention discloses a pain assessment system for hospitalized newborns based on limb movement characteristics, comprising:

[0030] The module for constructing a sample set of neonatal pain videos acquires neonatal videos, segments the videos into video segment sequences, preprocesses each frame of each video segment to obtain neonatal pain video segments, labels the neonatal pain video segments by category, and constructs a sample set of neonatal pain videos.

[0031] The model training module uses samples from the neonatal pain video sample set to train a pre-established neonatal pain classification model, resulting in a trained neonatal pain classification model. The pre-established neonatal pain classification model processing includes video preprocessing, first branch feature extraction, second branch feature extraction, multi-scale convolutional attention operation, feature fusion, and classification.

[0032] The pain classification module is used to acquire test videos, input the test videos into the trained neonatal pain classification model, and obtain neonatal pain classification assessment results.

[0033] In another aspect of the present invention, in order to achieve the above-mentioned objective, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor. When the processor loads and executes the computer program, it employs the pain assessment method for hospitalized newborns based on limb movement characteristics as described above.

[0034] The beneficial effects of this invention are:

[0035] This invention proposes a dual-branch feature extraction module that extracts spatial detail features and motion features of limbs from videos at different frame rates. The first branch mimics small-cell neurons, operates at a lower frame rate, and focuses more on the spatial detail features of limbs. The second branch mimics large-cell neurons, operates at a higher frame rate, and is used to capture rapidly changing motion features. The motion features output from the second branch are then fused into the spatial detail features of the first branch through lateral connections.

[0036] The second branch feature extraction module proposed in this invention enhances its action feature capture capability while reducing the number of model parameters through a design that reduces the number of feature channels, thereby achieving model lightweighting.

[0037] This invention employs a multi-scale convolutional attention mechanism to perform channel-wise weighted fusion of features at different scales output by the first and second branch feature extraction modules. This enables the model to focus on key features of limb movements, capture subtle differences in movement, and effectively improve the accuracy and robustness of neonatal pain classification.

[0038] The neonatal pain assessment method based on limb movement characteristics proposed in this invention can overcome the shortcomings of pain assessment methods based on facial expressions, which are easily affected by occlusion and changes in light, resulting in low accuracy and poor robustness. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0041] Figure 2 This is a structural diagram of the neonatal pain classification model according to an embodiment of the present invention;

[0042] Figure 3 This is a structural diagram of the multi-scale convolutional attention module according to an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Example 1:

[0046] like Figure 1 As shown, a method for assessing pain in hospitalized newborns based on limb movement characteristics includes the following steps:

[0047] S101: Acquire newborn videos, segment the newborn videos into video segment sequences, preprocess each frame of each video segment to obtain newborn pain video segments, label the newborn pain video segments with pain categories, and construct a newborn pain video sample set.

[0048] Specifically, videos of newborns under both clinically painful procedures and painless conditions are collected. These videos are then segmented into a sequence of video clips, each with a length of T frames. Human detection, cropping, alignment, and normalization are performed on each frame of each video clip to obtain a T-frame long video clip representing neonatal pain. Professional medical staff assess the pain level using a neonatal pain assessment scale, labeling each video clip with its corresponding neonatal pain category. This creates a neonatal pain video sample set containing K pain category labels. Here, T is the number of frames in each video clip, and K is the number of pain categories. In this embodiment, T is 64, K is 4, and the neonatal pain category set is {calm, mild pain, moderate pain, severe pain}.

[0049] S102: Use samples from the neonatal pain video sample set to train the pre-established neonatal pain classification model to obtain the trained neonatal pain classification model. The pre-established neonatal pain classification model processing includes video preprocessing, first branch feature extraction, second branch feature extraction, multi-scale convolutional attention operation, feature fusion and classification.

[0050] The system consists of a video preprocessing module, a first branch feature extraction module, a second branch feature extraction module, N multi-scale convolutional attention modules, a feature fusion module, and a classifier. In this embodiment, N is set to 3. Figure 2 As shown.

[0051] The video preprocessing module segments the input newborn video into a sequence of video segments, and performs human detection, cropping, alignment, and normalization operations on each frame of each video segment to obtain a T-frame video segment of newborn pain. Where H and W are the height and width of the human body image, respectively, and T is the number of frames for each video segment. In this embodiment, H and W are 224 and T is 64.

[0052] The first branch feature extraction module consists of a frame extraction module and a deep convolutional neural network Net1. The frame extraction module extracts one frame every M frames from the neonatal pain video segment output by the video preprocessing module, reducing the video's frame rate. M represents the frame extraction coefficient; in this embodiment, M = 16. The deep convolutional neural network Net1 is used to extract neonatal limb movement features from the low frame rate video segment after frame extraction. In this embodiment, Net1 consists of a 3D ResNet-50 backbone network, containing one 3D convolutional layer, one max-pooling layer, and 16 3D residual blocks, denoted as residual block A1 to residual block A2. 16 The features extracted by Net1 are divided into N levels of output, where the p-th level feature is represented as... p represents the index of the feature level, p∈{1,2,…,N}, H p W represents the height of the p-th level feature map. p C represents the width of the feature map at the p-th level. 1,p This represents the number of channels for the p-th level feature output by Net1. In this embodiment, N = 3, p ∈ {1, 2, 3}, and the residual blocks A5, A9, and A1 of Net1 are... 16 The output features are respectively used as the first level features. Second level features Third level features

[0053] The second branch feature extraction module consists of a deep convolutional neural network Net2, used to extract neonatal limb movement features from high frame rate video clips output by the video preprocessing module. In this embodiment, Net2 consists of a 3D ResNet-50 backbone network, containing one 3D convolutional layer, one max pooling layer, and 16 3D residual blocks, which are represented as residual block B1 to residual block B1. 16 The features extracted by Net2 are divided into N levels of output, where the p-th level feature is represented as... p represents the index of the feature level, p∈{1,2,…,N}, H p W represents the height of the p-th level feature map. p C represents the width of the feature map at the p-th level. 2,pLet represent the number of channels for the p-th level feature in the Net2 output. For the sake of model lightweighting, let Q represents the channel scaling ratio, and its values ​​are 2, 4, 8, and 16. In this embodiment, N = 3, p ∈ {1, 2, 3}, and Q = 16. The residual blocks B5, B9, and B of Net2 are... 16 The output features are respectively used as the first level features. Second level features Third level features

[0054] The structure of the p-th multi-scale convolutional attention module in the preferred embodiment of the present invention is as follows: Figure 3 As shown, in this embodiment, p∈{1,2,3}. The p-th multi-scale convolutional attention module includes depthwise separable convolution and pointwise convolution, and its operation includes the following sub-steps:

[0055] The p-th level feature output by Net1 and the p-th level features output by Net2 Concatenate along the channel dimension to obtain the feature tensor. The feature tensor F is obtained by using a multi-scale convolutional attention mechanism. p Attention weight ω p for

[0056]

[0057] Where σ represents the Sigmoid activation function, Conv 1×1×1 This indicates a pointwise convolution with a 1×1×1 kernel, where L is the number of depthwise separable convolutions, and L can take values ​​of 2, 3, 4, or 5. DConv j Let L represent the j-th depthwise separable convolution. Each depthwise separable convolution uses a depthwise convolution with a striped kernel of k×1×1, 1×k×1, and 1×1×k, where k takes values ​​of 3, 5, 7, 9, and 11. In this embodiment, L = 3. The first depthwise separable convolution uses a depthwise convolution with a striped kernel of 7×1×1, 1×7×1, and 1×1×7; the second depthwise separable convolution uses a depthwise convolution with a striped kernel of 9×1×1, 1×9×1, and 1×1×9; and the third depthwise separable convolution uses a depthwise convolution with a striped kernel of 11×1×1, 1×11×1, and 1×1×11.

[0058] The attention weight ω p Channel-by-channel and feature tensor F p The components are multiplied and then subjected to pointwise convolution with a 1×1×1 kernel to output a multi-scale feature tensor. for

[0059] Fp ′=Conv 1×1×1 (ω p ·F p )

[0060] Among them, Conv 1×1×1 This represents a pointwise convolution with a 1×1×1 kernel.

[0061] like Figure 2 As shown, the multi-scale feature tensor F output by the p-th multi-scale convolutional attention module is... p ′ and the p-th level feature F output by Net1 1,p Perform residual connections and output features In this embodiment, p∈{1,2,3}, F r,1 As input to residual block A6 of Net1, F r,2 As residual block A of Net1 10 The input, F r,3 The input is fed into the feature fusion module, where the action features output from the second branch feature extraction module are fused into the spatial detail features of the first branch feature extraction module via lateral connections.

[0062] The feature fusion module is used to perform pooling operations on the features output by the first branch feature extraction module and the second branch feature extraction module, then flatten them into feature vectors, and then concatenate them to obtain the fused neonatal limb movement feature vector V.

[0063] The classifier is used to classify and identify the action feature vector V output by the feature fusion module, and output one of the categories from the neonatal pain category set {calm, mild pain, moderate pain, severe pain}.

[0064] S102: Obtain the test video and input it into the trained neonatal pain classification model to obtain the neonatal pain classification assessment results.

[0065] Specifically, the present invention will be further illustrated below through embodiments:

[0066] A neonatal pain classification model was constructed by training samples from a neonatal pain video sample set. The loss function for each training sample during model training is defined as follows:

[0067]

[0068] Where K is the number of pain categories, c∈{1,2,…,K}; y c This indicates that the training sample is assigned the true class label of class c. When using one-hot encoding, if the training sample is assigned to class c, then y c =1, otherwise yc =0; K represents the probability that the model predicts a training sample as class c; α is a weighting factor introduced to address class imbalance, 0 < α ≤ 0.5; β represents an adjustment factor used to increase attention to samples that are difficult to classify, 0 < β ≤ 5. In this embodiment, K = 4, α = 0.3, and β = 2.

[0069] The trained neonatal pain classification model is used to classify and identify pain in newly input test videos, and output one of the categories from the neonatal pain category set {calm, mild pain, moderate pain, severe pain}.

[0070] Example 2: Second aspect, such as Figure 4 As shown, in order to achieve the above objectives, this invention discloses a pain assessment system for hospitalized newborns based on limb movement characteristics, comprising:

[0071] The neonatal pain video sample set construction module 11 acquires neonatal videos, segments the neonatal videos into video segment sequences, preprocesses each frame of each video segment to obtain neonatal pain video segments, labels the neonatal pain video segments by category, and constructs a neonatal pain video sample set.

[0072] The model training module 12 uses samples from the neonatal pain video sample set to train a pre-established neonatal pain classification model, thereby obtaining a trained neonatal pain classification model. The pre-established neonatal pain classification model processing includes video preprocessing, first branch feature extraction, second branch feature extraction, multi-scale convolutional attention operation, feature fusion, and classification.

[0073] The pain classification module 13 is used to acquire test videos and input them into the trained neonatal pain classification model to obtain neonatal pain classification assessment results.

[0074] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0075] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0076] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0077] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

Claims

1. A method for assessing pain in hospitalized newborns based on limb movement characteristics, characterized in that, The method includes the following steps: Acquire newborn videos, segment the newborn videos into video segment sequences, preprocess each frame of each video segment to obtain newborn pain video segments, label the newborn pain video segments with pain categories, and construct a newborn pain video sample set; The video preprocessing procedure is as follows: The input newborn video is segmented into a sequence of video clips. Human detection, cropping, alignment, and normalization are performed on each frame of each video clip to obtain a newborn pain video clip of length T frames. ,in, , Here, represents the height and width of the human image, respectively, and T represents the number of frames in each video segment; The pre-established neonatal pain classification model is trained using samples from a neonatal pain video sample set to obtain the trained neonatal pain classification model. The pre-established neonatal pain classification model processing includes video preprocessing, first branch feature extraction, second branch feature extraction, multi-scale convolutional attention operation, feature fusion, and classification. The video clips of newborn pain are input into the pre-established first branch feature extraction module and second branch feature extraction module for feature extraction; The first branch feature extraction module consists of a frame extraction module and a deep convolutional neural network Net1. The frame extraction module processes the neonatal pain video segments output by the video preprocessing module at intervals of... Extracting one frame reduces the video's frame rate. Represents the frame extraction coefficients; the deep convolutional neural network Net1 is used to extract the neonatal limb movement features of low frame rate video segments after frame extraction; the features extracted by Net1 are divided into N levels of output, where the first level is... Each level of feature is represented as , Indicates the sequence number of the feature level. , Indicates the first The height of each level feature map, Indicates the first The width of each level of feature map, Indicates the output of Net1 of the first The number of channels for each level of feature; The second branch feature extraction module consists of a deep convolutional neural network Net2, used to extract newborn limb movement features from high frame rate video clips output by the video preprocessing module; the features extracted by Net2 are divided into N levels for output, where the first... Each level of feature is represented as , Indicates the sequence number of the feature level. , Indicates the first The height of each level feature map, Indicates the first The width of each level of feature map, Indicates the output of Net2. The number of channels for each level of feature; let , Indicates the channel scaling ratio; The output of Net1 is the first Features at each level and the output of Net2 Features at each level Through the N multi-scale convolutional attention modules, the first one A multi-scale convolutional attention module introduces multi-scale convolutional attention operations, outputting a multi-scale feature tensor that focuses on limb movements. ; The first The operation process of the multi-scale convolutional attention module is as follows: The output of Net1 is the first Features at each level and the output of Net2 Features at each level Concatenate along the channel dimension to obtain the feature tensor. The feature tensor is obtained by using a multi-scale convolutional attention mechanism. attention weights for: in, express Activation function Indicates that the convolution kernel is Pointwise convolution, It is the number of depthwise separable convolutions. Indicates the first Each depthwise separable convolution uses a striped convolution kernel. , and Depth convolution; Attention weight Channel-by-channel and feature tensor Multiply and pass through a convolution kernel to obtain Pointwise convolution processing outputs multi-scale feature tensors. for: in, Indicates that the convolution kernel is Pointwise convolution; Obtain the test video and input it into the trained neonatal pain classification model to obtain the neonatal pain classification assessment results.

2. The method for assessing pain in hospitalized newborns based on limb movement characteristics according to claim 1, characterized in that, The neonatal videos were acquired during both painful clinical procedures and painless conditions, and the length of each video segment is T frames.

3. The method for assessing pain in hospitalized newborns based on limb movement characteristics according to claim 1, characterized in that, The feature fusion operation method is as follows: the features output by the first branch feature extraction module and the second branch feature extraction module are pooled, flattened into feature vectors, and then concatenated to obtain the fused neonatal limb movement feature vector. The classification method is as follows: The newborn's limb movement feature vector... The input is used to classify and identify the pain level in newborns, and the output is a category of pain, including calm, mild pain, moderate pain, and severe pain.

4. The method for assessing pain in hospitalized newborns based on limb movement characteristics according to claim 1, characterized in that, The loss function for each training sample in the process of training the pre-established neonatal pain classification model using samples from the neonatal pain video sample set is defined as follows: Where K is the number of pain categories, ; This indicates that the training sample is assigned the first The true class label of a class, when using one-hot encoding, is given by the training sample as the first... Class, then ,otherwise ; The model predicts the training samples as the first... The probability of a class; It is a weighting factor introduced to solve the problem of class imbalance; This indicates a regulatory factor.

5. A pain assessment system for hospitalized newborns based on limb movement characteristics, employing the pain assessment method for hospitalized newborns based on limb movement characteristics as described in any one of claims 1 to 4, characterized in that, include: The module for constructing a sample set of neonatal pain videos acquires neonatal videos, segments the videos into video segment sequences, preprocesses each frame of each video segment to obtain neonatal pain video segments, labels the neonatal pain video segments by category, and constructs a sample set of neonatal pain videos. The model training module uses samples from the neonatal pain video sample set to train a pre-established neonatal pain classification model, resulting in a trained neonatal pain classification model. The pre-established neonatal pain classification model processing includes video preprocessing, first branch feature extraction, second branch feature extraction, multi-scale convolutional attention operation, feature fusion, and classification. The pain classification module is used to acquire test videos, input the test videos into the trained neonatal pain classification model, and obtain neonatal pain classification assessment results.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on a processor. When the processor loads and executes the computer program, it employs the method for assessing pain in hospitalized newborns based on limb movement characteristics, as described in any one of claims 1 to 4.

Images