A method, system, device and medium for dynamically identifying the pain level of newborns

Abstract

The present invention belongs to the technical field of medical image recognition, and discloses a method, system, device and medium for dynamically recognizing the pain degree of newborns, including: obtaining a dataset of facial expression images of newborns in pain, constructing a dynamic recognition model for the pain degree of newborns, where the dynamic recognition model for the pain degree of newborns includes a feature extractor and a pain classifier connected in sequence; training and optimizing the dynamic recognition model for the pain degree of newborns based on the dataset of facial expression images of newborns in pain to obtain an optimized dynamic recognition model for the pain degree of newborns; and performing pain degree recognition on the facial expression image of the newborn to be recognized based on the optimized dynamic recognition model for the pain degree of newborns. The technical solution of the present invention makes full use of the key features of the extracted facial expressions, and can obtain better pain degree evaluation results on the data stream of facial images of newborns in pain.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image recognition, and particularly relates to a method, system, device and medium for dynamically recognizing the pain degree of newborns. Background Art

[0002] Existing research shows that newborns can perceive external pain stimuli just like adults. During the nursing and treatment of newborns, painful procedures are usually accompanied. Repeated pain stimuli will have a series of adverse effects on the physical and mental development of newborns in the later stage. Therefore, quickly and accurately evaluating pain and taking analgesic measures has important clinical significance.

[0003] Since newborns cannot "verbally describe" the degree of pain, pain assessment has become a challenging problem in newborn nursing. Currently, in clinical nursing, professional medical staff use evaluation tools such as the neonatal facial coding system or pain scale that take "facial expressions" as important monitoring indicators for manual evaluation. However, manual evaluation is time-consuming and laborious, and the evaluation results are affected by subjective factors such as the experience and emotions of medical staff.

[0004] In terms of automatic evaluation of newborn pain, there have been some studies. In the prior art, it is proposed to extract facial dynamic geometry and dynamic texture features from video sequences, and reduce the dimension and classify the fused features. However, it is difficult to accurately extract feature parameters automatically by this method. At the same time, there is also a study that combines deep convolutional neural network (DCNN) and transfer learning, which overcomes the problem that the optimization training of the DCNN model requires a large-scale labeled data set for support. However, the data stream of newborn facial images obtained in actual clinical practice has the characteristics of dynamics and high dimensionality, while the aforementioned research methods are only suitable for processing static data sets and are difficult to process dynamic data streams. Therefore, it has become an urgent need to develop a method for recognizing newborn pain that can process data streams in real time and dynamically. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, system, device and medium for dynamically recognizing the pain degree of newborns to solve the problems existing in the above-mentioned prior art.

[0006] To achieve the above purpose, the present invention provides a method for dynamically recognizing the pain degree of newborns, including:

[0007] Obtaining a newborn pain facial expression image data set, where the newborn pain facial expression image data set includes newborn facial expression images and corresponding pain degree labels;

[0008] Constructing a dynamic recognition model for the pain degree of newborns, where the dynamic recognition model for the pain degree of newborns includes a feature extractor and a pain classifier connected in sequence;

[0009] Optimize the dynamic recognition model of neonatal pain degree based on the neonatal pain facial expression image dataset to obtain an optimized dynamic recognition model of neonatal pain degree;

[0010] Based on the optimized dynamic recognition model of neonatal pain degree, perform pain degree recognition on the facial expression image of the neonatal to be recognized.

[0011] Optionally, obtaining the neonatal pain facial expression image dataset specifically includes:

[0012] Shoot the dynamic facial video of the neonate when experiencing different degrees of pain stimulation;

[0013] Select a number of reviewers, and each reviewer scores the expressions in the dynamic facial video according to the neonatal pain assessment scale, and determine the pain degree label based on the pain degree score; the pain degree label includes a calm label, a mild pain label, and a severe pain label;

[0014] Conduct a consistency assessment of the pain degree scores of each reviewer, and intercept the images with highly consistent scores in the dynamic facial images as typical images;

[0015] Construct the neonatal pain facial expression image dataset based on the typical images and the corresponding pain degree labels.

[0016] Optionally, optimizing the training of the dynamic recognition model of neonatal pain degree based on the neonatal pain facial expression image dataset specifically includes:

[0017] Input the neonatal pain facial expression image dataset into the feature extractor for extraction to obtain a pain feature set, and the pain feature set includes key pain expression features and corresponding pain degree labels;

[0018] Based on a preset partitioning rule, partition the pain feature set into an initial sample set and a validation set, construct a concept space based on the initial sample set, partition the validation set into several parallel data blocks, perform dynamic prediction learning on each data block, use the pain features in each data block as the concept connotation, calculate the similarity between the concept and the concept space, and output the corresponding pain degree label; during the process of dynamic prediction learning, update the concept space based on the superiority and inferiority between the new concept and each concept in the concept space to complete the dynamic learning optimization of the dynamic recognition model of neonatal pain degree.

[0019] Optionally, inputting the neonatal pain facial expression image dataset into the feature extractor for extraction specifically includes:

[0020] Input the neonatal facial expression image into the image block mapping layer for image division to obtain a number of image blocks with equal height and width. Perform convolution and flattening processing on each of the image blocks to obtain corresponding two-dimensional matrices, and perform column label quantization operations on the two-dimensional matrices to obtain mapping vectors;

[0021] After adding the position information corresponding to each of the image blocks to the corresponding mapping vectors, input vectors are obtained;

[0022] Add a Dropout layer after the MHA module and the MLP module of the Transformer encoder, and add spatial encoding and edge encoding to the attention mechanism of the Transformer encoder to obtain an improved Transformer encoder. Combine each input vector with the centrality encoding of the corresponding image block, and input the combined data into the improved Transformer encoder to output the key features of pain expressions.

[0023] Optionally, construct a concept space based on the initial sample set, specifically including:

[0024] Randomly select a preset proportion of samples from the pain feature set through the knowledge storage module as the initial sample set, and use the entire pain feature set as the validation set;

[0025] Represent the feature samples in the initial sample set in the form of triples to represent concepts. Among them, the pain feature set is the concept connotation, the neonatal facial expression image is the concept extension, and the pain degree label is the target category name;

[0026] Construct an initial concept set based on the transformed concepts. Based on the initial concept set, combine the fuzzy clustering method to construct an initial concept space on the initial concept set, and compress the initial concept space to obtain a compressed concept space.

[0027] Optionally, compress the initial concept space, specifically including:

[0028] Step 1: When the number of concepts in the initial concept space exceeds a preset threshold, perform compression of the concept space;

[0029] Step 2: Select a concept in the initial concept space. If the selected concept is a virtual concept, add it to the compressed concept space;

[0030] Step 3: For the real concepts in the initial concept space, calculate the similarity between the current real concept and the remaining real concepts in the initial concept space based on a preset neighborhood range;

[0031] If the similarity exceeds a preset similarity threshold, add it to the local concept neighborhood;

[0032] If the similarity does not exceed the preset similarity threshold, it is added to the compressed concept space;

[0033] Step 4: If the size of the local concept neighborhood reaches the preset threshold, represent the local concept neighborhood as a virtual concept and add it to the compressed concept space;

[0034] Repeat Step 2 to Step 4 until all concepts are visited, and complete the compression of the initial concept space.

[0035] Optionally, perform pain degree recognition on the facial expression image of the neonate to be recognized based on the optimized dynamic neonate pain degree recognition model, specifically including:

[0036] Input the facial expression image of the neonate to be recognized into the feature extractor for feature extraction to obtain pain features;

[0037] Convert the pain features into a concept stream and input it into the pain classifier for classification prediction to obtain the corresponding pain degree recognition result.

[0038] A dynamic neonate pain degree recognition system, including:

[0039] A data acquisition module, used to obtain a dataset of facial expression images of neonates in pain, where the dataset of facial expression images of neonates in pain includes facial expression images of neonates and corresponding pain degree labels;

[0040] A pain degree recognition module, used to construct a dynamic neonate pain degree recognition model, where the dynamic neonate pain degree recognition model includes a feature extractor and a pain classifier connected in sequence; train and optimize the dynamic neonate pain degree recognition model based on the dataset of facial expression images of neonates in pain to obtain an optimized dynamic neonate pain degree recognition model; perform pain degree recognition on the facial expression image of the neonate to be recognized based on the optimized dynamic neonate pain degree recognition model.

[0041] An electronic device, including a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the described method for dynamically recognizing the pain degree of neonates.

[0042] A computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the described method for dynamically recognizing the pain degree of neonates.

[0043] The technical effect of the present invention is:

[0044] The present invention can make full use of the key facial expression features extracted, and can obtain better pain degree evaluation results on the neonatal pain facial image data stream, providing a new method for developing a neonatal pain degree dynamic evaluation system with incremental learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0046] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0047] Figure 1 It is a schematic flow chart of the method for dynamically identifying the neonatal pain degree based on concept cognitive computing provided by the present invention;

[0048] Figure 2 It is a schematic structural diagram of the neonatal pain degree dynamic recognition model based on concept cognitive computing provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] Now, various exemplary embodiments of the present invention will be described in detail. This detailed description should not be considered as a limitation to the present invention, but rather as a more detailed description of certain aspects, features, and implementation schemes of the present invention.

[0050] It should be understood that the terms described in the present invention are only for describing specific embodiments and are not used to limit the present invention. Additionally, for the numerical ranges in the present invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Each intermediate value within any stated value or stated range, as well as each smaller range between any other stated value or intermediate value within the stated range, is also included in the present invention. The upper and lower limits of these smaller ranges can be independently included or excluded from the range.

[0051] Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention pertains. Although the present invention only describes the preferred methods, any method similar or equivalent to those described herein can also be used in the implementation or testing of the present invention. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods related to the documents. In case of conflict with any incorporated document, the content of this specification shall prevail.

[0052] Without departing from the scope or spirit of the present invention, various improvements and changes can be made to the specific embodiments of the description of the present invention, which will be obvious to those skilled in the art. Other embodiments obtained from the description of the present invention will be obvious to those skilled in the art. The description and examples of the present application are merely exemplary.

[0053] Regarding the use of "comprising", "including", "having", "containing", etc. in this article, they are all open-ended terms, meaning including but not limited to.

[0054] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will detail this application with reference to the accompanying drawings and in conjunction with the embodiments.

[0055] Embodiment 1

[0056] As Figure 1 - Figure 2 shown, in this embodiment, a method for dynamically identifying the pain level of a newborn is provided, including: obtaining a dataset of newborn pain facial expression images, where the dataset of newborn pain facial expression images includes newborn facial expression images and corresponding pain level labels; constructing a dynamic recognition model for the pain level of a newborn, where the dynamic recognition model for the pain level of a newborn includes a feature extractor and a pain classifier connected in sequence; training and optimizing the dynamic recognition model for the pain level of a newborn based on the dataset of newborn pain facial expression images to obtain an optimized dynamic recognition model for the pain level of a newborn; and performing pain level recognition on the facial expression image of the newborn to be recognized based on the optimized dynamic recognition model for the pain level of a newborn.

[0057] This embodiment discloses a method for dynamically identifying the pain level of a newborn based on concept cognitive computing. The method includes: establishing a data stream of newborn pain facial expression images, including preprocessed newborn facial images and their corresponding pain level category labels; constructing a feature extractor based on Vision Transformer to extract key features of the pain expression of the newborn's face; taking the pain feature set as the concept connotation and the image object set as the concept extension to convert the facial image instance into a concept; constructing a concept cognitive computing model (CCCM) for identifying the pain level of a newborn; and converting the new facial image (stream) into a concept (stream) and inputting it into the CCCM for pain level recognition, thereby obtaining a pain level evaluation result. This embodiment can make full use of the key features of the facial expression extracted, and can obtain better pain level evaluation results on the data stream of newborn pain facial images, providing a new method for developing an incremental learning-based dynamic evaluation system for the pain level of a newborn.

[0058] The purpose of this embodiment is to provide a dynamic recognition method for neonatal pain degree based on concept cognitive computing, which combines concept cognitive computing and vision Transformer. By using an improved Transformer encoder as a feature extractor to quickly extract key facial features, and then using a concept cognitive computing model as a classifier to identify the facial pain feature data stream, it can make full use of the parallel computing ability of the Transformer encoder and the dynamic incremental learning ability of the concept cognitive computing model, effectively improving the accuracy and time performance of neonatal pain degree assessment and meeting the actual application requirements of clinical nursing.

[0059] To achieve the above object, this embodiment provides the following solutions:

[0060] A dynamic recognition method for neonatal pain degree based on concept cognitive computing, comprising the following steps:

[0061] (1) Establish a data stream of neonatal pain facial expression images, including preprocessed neonatal facial images and their corresponding pain degree category labels;

[0062] (2) Build a feature extractor based on vision Transformer to extract key features of neonatal facial pain expressions;

[0063] (3) Take the pain feature set as the concept intension, the image object set as the concept extension, and convert the facial image instance into a concept, that is, a triple (intension, extension, pain category);

[0064] (4) Build a concept cognitive computing model (CCCM) for neonatal pain degree recognition;

[0065] (5) Convert the new facial image (stream) into a concept (stream) and input it into the CCCM for pain degree recognition, and then obtain the pain degree assessment result.

[0066] Implementable, the step (2) includes:

[0067] (2.1) Build a feature extractor by adding spatial encoding, edge encoding, and centrality encoding of block graph nodes to the encoder of vision Transformer, and adding a Dropout layer after the MHA and MLP modules of the encoder of Transformer;

[0068] (2.2) Input the neonatal facial image into the feature extractor to extract the key features of the facial pain expression, and concatenate the corresponding pain degree category label at the end of the feature.

[0069] Implementable, the step (2.2) includes:

[0070] (2.2.1) Divide the input neonatal facial 2D image into 2D sequence blocks (patches) of equal height and width;

[0071] (2.2.2) Set the vector length of each layer of the Transformer encoder to a fixed value S, and use a trainable linear mapping to flatten and map the sequence blocks into S-dimensional vectors: patch embeddings;

[0072] (2.2.3) Add the position information position embedding of the blocks to the patch embeddings, and concatenate the pain degree category labels at the end, and input them into the feature extractor to extract the key features of the facial pain expression and output.

[0073] (4) can be implemented as follows:

[0074] (4.1) Randomly select a certain proportion of instances of each class (such as 1% - 10%, depending on the actual sample size) from the output of step (2) as the initial sample set;

[0075] (4.2) Use step (3) to convert the initial sample set into an initial concept set, and use the fuzzy clustering method to construct an initial concept space (corresponding to the pain category) on the initial concept set to store knowledge;

[0076] (4.3) Take the output of step (2) as the input data stream and divide it into data blocks of equal length; (4.4) Perform dynamic concept learning on each data block of the input data stream in turn, and output the prediction result of the pain degree;

[0077] (4.5) Continuously update the original concept space during the concept learning process to achieve dynamic incremental learning of knowledge.

[0078] (4.4) can be implemented as follows:

[0079] (4.4.1) For each data block, read each instance in it in turn, use step (3) to convert it into a new concept, and use the similarity calculation method based on the concept connotation to calculate the similarity with each concept in all concept spaces, and use the maximum similarity value calculated in each concept space as the similarity value between the new concept (instance) and the concept space;

[0080] (4.4.2) Take the concept space with the greatest similarity to the new concept as the target space to which the concept belongs, and output the pain category corresponding to the target space as the pain degree evaluation result.

[0081] Implementable. Step (4.5) dynamically updates the original concept space according to the partial order relationship existing between new concepts, including the following steps:

[0082] (4.5.1) After using step (4.4) to determine the target space to which the new concept (instance) belongs, determine the partial order relationship existing between the new concept and any concept in the original concept space;

[0083] (4.5.2) If the intension of the new concept is superior to the intension of any concept C in the original concept space, then merge the instance object corresponding to the new concept with the extension of concept C, and replace the intension of concept C with the intension of the new concept;

[0084] (4.5.3) If the intension of the new concept is not superior to the intension of any concept C in the original concept space, then only merge the instance object corresponding to the new concept with the extension of concept C, and the intension of concept C remains unchanged;

[0085] (4.5.4) Otherwise, directly add the new concept to the original concept space.

[0086] Specific application examples of this embodiment include:

[0087] S1: Establish a neonatal pain facial expression image dataset, including preprocessed neonatal facial images and their corresponding pain expression category labels. The expression categories include calm, mild pain, and severe pain, corresponding to the pain degree levels.

[0088] Currently, there is no publicly available neonatal pain facial expression image dataset. For the needs of this project research, in this embodiment, a non-wide-angle camera with a resolution of 1920x1080 and a frame rate of 30fps is used to capture dynamic facial images of newborns experiencing different degrees of pain stimulation during routine pain-inducing operations (such as intramuscular injection, blood collection) in the neonatal department of a local hospital. Then, professional doctors and nurses use the internationally recognized neonatal pain assessment scale to score the pain degree of the collected neonatal facial images (1 - 10 points). Expressions with a score value between 1 - 5 are classified as mild pain expressions, and expressions with a score value between 6 - 10 are classified as severe pain expressions. Finally, images are intercepted from the captured videos, and 3 types of typical expressions (calm, mild pain, severe pain) with high scoring consistency are selected. These original color images are preprocessed such as cropping, rotating, aligning, and scale normalization (224x224 pixels), and expression category labels are marked (in this embodiment, the quiet expression is labeled 1, the mild pain expression is labeled 2, and the severe pain expression is labeled 3) to establish a neonatal facial expression image library.

[0089] S2: Construct a feature extractor based on Vision Transformer to extract the key features of neonatal facial pain expressions. As shown in Figure 2 (a) in, the constructed feature extractor includes a linear mapping layer of image patches, an embedding combination layer, and an improved Transformer encoder layer.

[0090] Step S2 includes the following sub-steps:

[0091] Sub-step S21: In the image patch mapping layer, first divide the input image into several blocks with equal width and height, then use a 16x16 convolutional kernel with a stride of 16 to convolve the [224, 224, 3] image into a [14, 14, 768] image, and then flatten the [14, 14, 768] image into a two-dimensional matrix of [196, 768]. This matrix is divided into 196 tokens, each token has a size of 768. Finally, perform column label quantization operations (such as summation) on the two-dimensional matrix to map the image patches into 768-dimensional vectors (patch embedding);

[0092] Sub-step S22: In the embedding combination layer, add the position information (position embedding) of each patch after its patch embedding to generate a 769-dimensional input vector (input embedding) for the improved encoder. Adding position embedding helps the encoder training process converge faster;

[0093] Sub-step S23: In the improved encoder layer, first combine the input vector with centrality encoding (input embedding + centrality encoding) as the input of the Transformer encoder, then sequentially add spatial encoding and edge encoding to the attention mechanism of the Transformer, and add a Dropout layer after the MHA and MLP modules of the Transformer encoder to construct the feature extractor. In the constructed feature extractor, each patch is randomly initialized with a corresponding weight matrix and finally automatically learned by the network.

[0094] Dropout can improve the robustness and generalization ability of the model by randomly discarding a part of its output, prevent overfitting to the training data, and improve its performance in processing new data. The addition of graph node structure information (such as centrality encoding, spatial encoding, and edge encoding) can better obtain the key features of facial pain expressions and improve the quality of the extracted facial features.

[0095] The feature extractor adopted in this embodiment solves the problems of slow convergence and easy overfitting of the Transformer encoder, can improve the robustness and generalization ability of the model, make full use of the advantages of parallel computing of the encoder, and realize the rapid extraction of key facial pain features.

[0096] S3: Represent the concept in the form of a triple where X represents the concept extension, represents the concept intension, and c represents the target category name. In this embodiment, the pain feature set is regarded as the concept intension, the image object set is regarded as the concept extension, and the pain category is regarded as the target category name. In the fuzzy formal scenario under the condition that and B ∈ L M , define the following operator (.)*:

[0097]

[0098] where G represents the object set, M represents the attribute set, and the fuzzy relationship between G and M is expressed as Use L M to represent the set of all fuzzy sets on M. In this embodiment of the present invention, the operator defined by formula (1) is used to convert the facial image instance into a concept, and for the convenience of notation, the concept is abbreviated as

[0099] S4: Construct a concept cognitive computing model (CCCM) for neonatal pain degree recognition. As shown in (b) of Figure 2 , the constructed CCCM is used as a pain classifier and consists of three parts: a knowledge storage module, a dynamic learning module, and a concept space update module.

[0100] Step S4 includes the following sub-steps:

[0101] Sub-step S41: In this embodiment, the knowledge storage module randomly selects 5% (which can be adjusted within the range of 1% - 10% according to the total number of each type of sample) from each type of pain sample on the extracted feature set as the initial sample set, and uses the entire feature set as the validation set;

[0102] Sub-step S42: First, use step S3 to convert the initial sample set into an initial concept set, and then use the fuzzy clustering method to construct a concept space on the initial concept set to store knowledge;

[0103] Sub-step S43: To avoid excessive number of concepts in each concept space affecting the model performance, the parameter MaxSize is used to control the size of the concept space. When the number of concepts in the concept space > MaxSize, the concept space is compressed to generate a compressed concept space. To complete the compression of the concept space, the α-concept domain and virtual concepts are introduced, and the formal representation is as follows:

[0104] Set the concept similarity threshold α ∈ [0, 1], and the α-concept domain of concept is defined as:

[0105]

[0106] where, represents an existing concept space, sim(·) is the similarity function. In this embodiment, the cosine distance similarity is adopted, and concept is regarded as an instance of M-dimensional features. Further, let be the j-th dimension of the intension of . For the real concept in the α-concept domain, is used to represent the virtual concept of the α-concept domain, and is defined as:

[0107]

[0108] where, and represent the extension and intension of the virtual concept respectively.

[0109] In this embodiment, the virtual concept, MaxSize = 50 and α = 0.6 are used to compress the concept space, which can effectively reduce the calculation time and improve the calculation performance. The compressed concept space is denoted as

[0110] The step S43 of the process of generating the compressed concept space (POCCS) further includes the following sub-steps:

[0111] Sub-step S431: Select a concept from the concept space . If it is a virtual concept, directly add it to the compressed concept space ; otherwise, execute sub-step S432;

[0112] Sub-step S432: Given the concept neighborhood range ∈ and the real concept in the concept space . For any real concept . If , then add to the local α-concept neighborhood (denoted as );Otherwise, directly add this concept to the compressed concept space

[0113] Sub-step S433: If (in this embodiment, take ∈ = 5), then represent the local α-concept neighborhood as a virtual concept and add it to the compressed concept space

[0114] Sub-step S434: Select an unvisited concept from the concept space and repeat the above sub-steps S431 - S433 until all concepts in

[0115] Sub-step S44: In this embodiment, the dynamic learning module divides the validation set (data stream) into equal-length data chunks (chunks) denoted as D1, D2,..., D t , D t+1 , and can sequentially and parallelly perform dynamic prediction learning on each data chunk

[0116] Let the concept space at the t-th stage be For any new instance (object) x r , the concept it is converted into is denoted as Then, the similarity between the concept and the concept space is defined by the following formula:

[0117]

[0118] where And

[0119] Furthermore, let where j represents the j-th concept Then, on the entire concept space, the class vectors of all maximum similarity values are represented as: And the class label with the maximum value is output as the final prediction of the model, represented by the following formula (2):

[0120]

[0121] where It represents that the instance or object x r is learned to the -th class

[0122] Furthermore, let C t+1 be the concept space of the data chunk D t+1 at the t + 1 stage, and is a partition of C t+1 , then, the concept space C t+1The similarity with is calculated by the following formula:

[0123]

[0124] In this way, the predicted value of a sample (x r , y r ) is It means that the instance x r should be classified into the th class.

[0125] Furthermore, in order to ensure the consistency of the update of the concept space in each stage during the parallel dynamic learning process, a temporary concept space E is constructed for intermediate operations. For example, the temporary concept space at stage t + 1 is denoted as E t+ 1.

[0126] Sub-step S45: In this embodiment, while performing the dynamic prediction in the previous step, the concept space update module updates the concept space of the previous step, thereby completing the entire dynamic learning process.

[0127] Specifically, the connotation of a concept is denoted as: Given a threshold λ(i), let be the concept space at the (j - 1)th stage. For any concept in the current concept space, the concept space update process (PUCS) further includes the following sub-steps:

[0128] Sub-step S451: If then That is, if the connotation of the new concept is superior to the connotation of any concept in the original concept space, then the instance object corresponding to the new concept is merged into the extension of the compared concept, and the connotation of the compared concept is replaced with the connotation of the new concept;

[0129] Sub-step S452: If then That is, if the connotation of the new concept is not superior to the connotation of any concept in the original concept space, then only the instance object corresponding to the new concept is merged into the extension of the compared concept, and the connotation of the compared concept remains unchanged;

[0130] Sub-step S453: Otherwise, That is, the new concept is directly added to the concept space at the jth stage.

[0131] In the process PUCS, if there is a partial order relationship between the new concept and any concept in the concept space at the (j - 1)th stage, the concept space will be updated.

[0132] After a certain dynamic learning process, the concept space generated by the CCCM model can be used for facial pain assessment.

[0133] S5: The new facial image (stream) is transformed into a concept (stream) through steps S2 and S3, and then input into the trained CCCM model for pain degree recognition. According to formula (2), the category corresponding to the maximum probability is the pain degree result of the neonate to be evaluated.

[0134] A dynamic recognition system for neonatal pain degree, comprising:

[0135] A data acquisition module, configured to obtain a dataset of neonatal pain facial expression images, where the dataset of neonatal pain facial expression images includes neonatal facial expression images and corresponding pain degree labels;

[0136] A pain degree recognition module, configured to construct a dynamic recognition model for neonatal pain degree, where the dynamic recognition model for neonatal pain degree includes a feature extractor and a pain classifier connected in sequence; based on the dataset of neonatal pain facial expression images, the dynamic recognition model for neonatal pain degree is trained and optimized to obtain an optimized dynamic recognition model for neonatal pain degree; based on the optimized dynamic recognition model for neonatal pain degree, the pain degree of the facial expression image of the neonate to be recognized is recognized.

[0137] An electronic device, comprising a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the described dynamic recognition method for neonatal pain degree.

[0138] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the described dynamic recognition method for neonatal pain degree.

[0139] This embodiment improves the visual Transformer encoder and the parallel computing ability of the encoder by using the spatial encoding, edge encoding, and centrality encoding of graph nodes, thereby improving the efficiency and quality of the key features of facial pain expressions;

[0140] This embodiment uses concept cognitive computing technology to construct an evaluation model CCCM for neonatal facial pain degree recognition, which can, like human beings' cognition of the objective world, comprehensively use formal concept analysis, machine learning, dynamic learning, etc. to learn new knowledge from different types of data in a dynamic environment;

[0141] The CCCM model of this embodiment can parallelly complete dynamic learning and concept space update on multiple data blocks in the data stream, reduce the time overhead for the model to accurately evaluate the neonatal pain degree, and improve the overall performance of the model;

[0142] In this embodiment, the organic integration of concept cognitive computing and vision Transformer enables the method described in this application to be well adapted to data stream application scenarios with high dimensionality and dynamics, such as clinical care.

[0143] As described above, only the preferred specific embodiments of this application are provided, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in this application should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for dynamically identifying the pain level of newborns, characterized in that, Including: Obtain a neonatal pain facial expression image dataset, where the neonatal pain facial expression image dataset includes neonatal facial expression images and corresponding pain degree labels; Construct a dynamic recognition model for neonatal pain degree, where the dynamic recognition model for neonatal pain degree includes a feature extractor and a pain classifier connected in sequence; Train and optimize the dynamic recognition model for neonatal pain degree based on the neonatal pain facial expression image dataset to obtain an optimized dynamic recognition model for neonatal pain degree; Training and optimizing the dynamic recognition model for neonatal pain degree based on the neonatal pain facial expression image dataset specifically includes: Input the neonatal pain facial expression image dataset into the feature extractor for extraction to obtain a pain feature set, where the pain feature set includes key pain expression features and corresponding pain degree labels; Divide the pain feature set into an initial sample set and a validation set based on a preset division rule, construct a concept space based on the initial sample set, divide the validation set into several parallel data blocks, perform dynamic prediction learning on each data block, use the pain features in each data block as concept intensions, calculate the similarity between the concept and the concept space, and output the corresponding pain degree label; during the dynamic prediction learning process, update the concept space based on the superiority and inferiority between the new concept and each concept in the concept space to complete the dynamic learning optimization of the dynamic recognition model for neonatal pain degree; The constructing the concept space based on the initial sample set specifically includes: Randomly select a preset proportion of samples from the pain feature set through the knowledge storage module as the initial sample set, and use the entire pain feature set as the validation set; Represent the feature samples in the initial sample set in the form of a triple to represent the concept, where the pain feature set is the concept intension, the neonatal facial expression image is the concept extension, and the pain degree label is the target category name; Construct an initial concept set based on the transformed concept, and based on the initial concept set, combine the fuzzy clustering method to construct an initial concept space on the initial concept set, and compress the initial concept space to obtain a compressed concept space; The compressing the initial concept space specifically includes: Step 1: When the number of concepts in the initial concept space exceeds a preset threshold, perform compression of the concept space; Step 2: Select a concept in the initial concept space. If the selected concept is a virtual concept, add it to the compressed concept space; Step 3: For the real concepts in the initial concept space, calculate the similarity between the current real concept and the remaining real concepts in the initial concept space based on a preset neighborhood range; If the similarity exceeds a preset similarity threshold, add it to the local concept neighborhood; If the similarity does not exceed the preset similarity threshold, add it to the compressed concept space; Step 4: If the size of the local concept neighborhood reaches a preset threshold, represent the local concept neighborhood as a virtual concept and add it to the compressed concept space; Repeat steps 2 to 4 until all concepts are visited to complete the compression of the initial concept space; Perform pain level recognition on the facial expression image of the neonate to be recognized based on the optimized dynamic neonate pain level recognition model.

2. The dynamic recognition method for neonatal pain level according to claim 1, characterized in that Obtain the neonate pain facial expression image dataset, specifically including: Shoot the dynamic facial video of the neonate when experiencing different levels of pain stimuli; Select a number of reviewers, and each reviewer scores the expressions in the dynamic facial video according to the neonate pain assessment scale, and determine the pain level label based on the pain level score; the pain level label includes a calm label, a mild pain label, and a severe pain label; Conduct a consistency assessment of the pain level scores of each reviewer, and intercept the images with highly consistent scores in the dynamic facial video as typical images; Construct the neonate pain facial expression image dataset based on the typical images and the corresponding pain level labels.

3. The dynamic recognition method for the pain level of a newborn according to claim 1, characterized in that, Input the neonate pain facial expression image dataset into the feature extractor for extraction, specifically including: Input the neonate facial expression image into the image block mapping layer for image division to obtain a number of image blocks with equal height and width, perform convolution and flattening processing on each image block respectively to obtain the corresponding two-dimensional matrix, and perform column label quantization operation on the two-dimensional matrix to obtain a mapping vector; Add the corresponding position information of each image block to the corresponding mapping vector to obtain an input vector; Add a Dropout layer after the MHA module and the MLP module of the Transformer encoder, and add spatial encoding and edge encoding to the attention mechanism of the Transformer encoder to obtain an improved Transformer encoder, combine each input vector with the centrality encoding of the corresponding image block, and input the combined data into the improved Transformer encoder to output the key pain expression features.

4. A method for dynamically identifying the pain level of a newborn according to claim 1, characterized in that, Perform pain level recognition on the facial expression image of the neonate to be recognized based on the optimized dynamic neonate pain level recognition model, specifically including: Input the facial expression image of the neonate to be recognized into the feature extractor for feature extraction to obtain pain features; Convert the pain features into a concept flow and input it into the pain classifier for classification prediction to obtain the corresponding pain level recognition result.

5. A dynamic neonatal pain level recognition system, characterized in that, Include: A data acquisition module for obtaining the neonate pain facial expression image dataset, where the neonate pain facial expression image dataset includes the neonate facial expression image and the corresponding pain level label; A pain level recognition module for constructing a dynamic neonate pain level recognition model, where the dynamic neonate pain level recognition model includes a feature extractor and a pain classifier connected in sequence; train and optimize the dynamic neonate pain level recognition model based on the neonate pain facial expression image dataset to obtain an optimized dynamic neonate pain level recognition model; Train and optimize the dynamic neonate pain level recognition model based on the neonate pain facial expression image dataset, specifically including: Input the neonatal pain facial expression image dataset into the feature extractor for extraction to obtain a pain feature set, where the pain feature set includes key pain expression features and corresponding pain level labels; Based on a preset partitioning rule, partition the pain feature set into an initial sample set and a validation set. Construct a concept space based on the initial sample set, divide the validation set into several parallel data blocks, perform dynamic prediction learning on each data block, use the pain features in each data block as the concept connotation, calculate the similarity between the concept and the concept space, and output the corresponding pain level label. During the dynamic prediction learning process, update the concept space based on the superiority and inferiority between the new concept and each concept in the concept space to complete the dynamic learning optimization of the neonatal pain level dynamic recognition model; The constructing of the concept space based on the initial sample set specifically includes: Randomly select a preset proportion of samples from the pain feature set through the knowledge storage module as the initial sample set, and use the entire pain feature set as the validation set; Represent the feature samples in the initial sample set in the form of a triple to represent a concept. Among them, the pain feature set is the concept connotation, the neonatal facial expression image is the concept extension, and the pain level label is the target category name; Construct an initial concept set based on the transformed concept. Based on the initial concept set, combine the fuzzy clustering method to construct an initial concept space on the initial concept set, and compress the initial concept space to obtain a compressed concept space; The compressing of the initial concept space specifically includes: Step 1: When the number of concepts in the initial concept space exceeds a preset threshold, perform the compression of the concept space; Step 2: Select a concept in the initial concept space. If the selected concept is a virtual concept, add it to the compressed concept space; Step 3: For the real concepts in the initial concept space, calculate the similarity between the current real concept and the remaining real concepts in the initial concept space based on a preset neighborhood range; If the similarity exceeds the preset similarity threshold, add it to the local concept neighborhood; If the similarity does not exceed the preset similarity threshold, add it to the compressed concept space; Step 4: If the size of the local concept neighborhood reaches the preset threshold, represent the local concept neighborhood as a virtual concept and add it to the compressed concept space; Repeat steps 2 to 4 until all concepts are visited to complete the compression of the initial concept space; Based on the optimized neonatal pain level dynamic recognition model, perform the pain level recognition of the neonatal facial expression image to be recognized.

6. An electronic device, characterized in that, Includes a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute a method for dynamically recognizing the pain level of a neonate according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a computer program, and when the computer program is executed by the processor, it implements a method for dynamically recognizing the pain level of a neonate according to any one of claims 1-4.

Images