Newborn pain degree dynamic identification method, system, equipment and medium

Abstract

The invention belongs to the technical field of medical image recognition, and discloses a neonatal pain degree dynamic recognition method, system and device and a medium, and the method comprises the steps: obtaining a neonatal pain facial expression image data set, constructing a neonatal pain degree dynamic recognition model, the newborn pain degree dynamic identification model comprises a feature extractor and a pain classifier which are connected in sequence; performing training optimization on the newborn pain degree dynamic identification model based on the newborn pain facial expression image data set to obtain an optimized newborn pain degree dynamic identification model; and performing pain degree identification of the to-be-identified newborn facial expression image based on the optimized newborn pain degree dynamic identification model. According to the technical scheme, the extracted facial expression key features are fully utilized, and a good pain degree evaluation result can be obtained on the newborn pain face image data stream.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image recognition, and in particular relates to a method, system, device and medium for dynamically recognizing the pain degree of a newborn. Background Art

[0002] Existing studies have shown that newborns can perceive external pain stimuli just like adults. Painful operations are often performed during the care and treatment of newborns. Repeated pain stimulation can have a series of adverse effects on the physical and mental development of newborns in the later stages, so rapid and accurate assessment of pain and taking analgesic measures are of great clinical significance.

[0003] Since newborns cannot verbally describe the degree of pain, pain assessment has become a challenging problem in neonatal care. Currently, in clinical care, professional medical staff use assessment tools such as the neonatal facial coding system or pain scale that use "facial expressions" as an important monitoring indicator for manual assessment. However, manual assessment is time-consuming and laborious, and the assessment results are affected by subjective factors such as the experience and emotions of medical staff.

[0004] There have been some studies on the automatic assessment of neonatal pain. The prior art proposes to extract facial dynamic geometry and dynamic texture features from video sequences, and to reduce the dimension and classify the fused features. However, it is difficult for this method to automatically and accurately extract feature parameters. At the same time, there are studies that combine deep convolutional neural networks (DCNNs) with transfer learning to overcome the problem that the optimization training of DCNN models requires the support of large-scale labeled data sets. However, the data streams of neonatal facial images obtained in actual clinical practice are dynamic and high-dimensional, and the aforementioned research methods are only suitable for processing static data sets and are difficult to process dynamic data streams. Therefore, it is an urgent need to develop a neonatal pain recognition method that can process data streams in real time and dynamically. Summary of the invention

[0005] The purpose of the present invention is to provide a method, system, device and medium for dynamically identifying the pain level of a newborn to solve the problems existing in the above-mentioned prior art.

[0006] To achieve the above object, the present invention provides a method for dynamically identifying the pain level of a newborn, comprising:

[0007] Acquire a dataset of neonatal facial expression images showing pain, wherein the dataset includes neonatal facial expression images and corresponding pain degree labels;

[0008] Constructing a dynamic recognition model for the pain level of newborns, wherein the dynamic recognition model for the pain level of newborns comprises a feature extractor and a pain classifier connected in sequence;

[0009] Based on the neonatal pain facial expression image data set, the neonatal pain degree dynamic recognition model is trained and optimized to obtain an optimized neonatal pain degree dynamic recognition model;

[0010] The pain level recognition of the facial expression images of the newborns to be recognized is performed based on the optimized dynamic recognition model of the pain level of the newborns.

[0011] Optionally, obtain a dataset of images of newborns with painful facial expressions, including:

[0012] Filming dynamic facial videos of newborns as they experience different levels of pain stimulation;

[0013] Selecting a number of reviewers, each reviewer scoring the pain level of the expression in the dynamic facial video according to the neonatal pain assessment scale, and determining a pain level label based on the pain level score; the pain level label includes a calm label, a mild pain label, and a severe pain label;

[0014] Conducting consistency evaluation on the pain level scores of each reviewer, and intercepting images with highly consistent scores in the dynamic facial images as typical images;

[0015] The neonatal pain facial expression image dataset is constructed based on the typical images and the corresponding pain level labels.

[0016] Optionally, training and optimizing the dynamic recognition model of neonatal pain degree based on the neonatal pain facial expression image dataset specifically includes:

[0017] Inputting the neonatal pain facial expression image data set into the feature extractor for extraction to obtain a pain feature set, wherein the pain feature set includes key features of pain expression and corresponding pain degree labels;

[0018] The pain feature set is divided into an initial sample set and a validation set based on a preset division rule, a concept space is constructed based on the initial sample set, the validation set is divided into a number of parallel data blocks, dynamic prediction learning is performed on each of the data blocks, the pain features in each of the data blocks are used as concept connotations, the similarity between the concept and the concept space is calculated, and the corresponding pain degree label is output; in the process of dynamic prediction learning, the concept space is updated based on the degree of superiority and inferiority between the new concept and the concepts in the concept space, and the dynamic learning optimization of the dynamic recognition model of the neonatal pain degree is completed.

[0019] Optionally, the neonatal pain facial expression image dataset is input into the feature extractor for extraction, specifically comprising:

[0020] Inputting the facial expression image of the newborn into the image block mapping layer for image division to obtain a number of image blocks with equal height and width, performing convolution and flattening processing on each of the image blocks to obtain a corresponding two-dimensional matrix, and performing column scalarization operation on the two-dimensional matrix to obtain a mapping vector;

[0021] After adding the position information corresponding to each of the image blocks to the corresponding mapping vector, an input vector is obtained;

[0022] A Dropout layer is added after the MHA module and MLP module of the Transformer encoder, and spatial coding and edge coding are added to the attention mechanism of the Transformer encoder to obtain an improved Transformer encoder. Each input vector is combined with the centrality code of the corresponding image block, and the combined data is input into the improved Transformer encoder to output the key features of pain expression.

[0023] Optionally, constructing a concept space based on the initial sample set specifically includes:

[0024] A preset proportion of samples are randomly selected from the pain feature set through the knowledge storage module as the initial sample set, and the entire pain feature set is used as the validation set;

[0025] The feature samples in the initial sample set are represented by triples in the form of concepts, wherein the pain feature set is the concept connotation, the newborn facial expression image is the concept extension, and the pain degree label is the target category name;

[0026] An initial concept set is constructed based on the converted concepts, an initial concept space on the initial concept set is constructed based on the initial concept set in combination with a fuzzy clustering method, and the initial concept space is compressed to obtain a compressed concept space.

[0027] Optionally, compressing the initial concept space specifically includes:

[0028] Step 1: When the number of concepts in the initial concept space exceeds a preset threshold, the concept space is compressed;

[0029] Step 2: Select a concept in the initial concept space. If the selected concept is a virtual concept, add it to the compressed concept space.

[0030] Step 3: For the real concepts in the initial concept space, the similarity between the current real concept and the remaining real concepts in the initial concept space is calculated based on a preset neighborhood range;

[0031] If the similarity exceeds the preset similarity threshold, it is added to the local concept neighborhood;

[0032] If the similarity does not exceed the preset similarity threshold, it is added to the compressed concept space;

[0033] Step 4: If the size of the local concept neighborhood reaches a preset threshold, the local concept neighborhood is represented as a virtual concept and added to the compressed concept space;

[0034] Repeat steps 2 to 4 until all concepts are visited and the initial concept space is compressed.

[0035] Optionally, performing pain level recognition of the facial expression image of the newborn to be recognized based on the optimized dynamic recognition model of pain level of the newborn, specifically includes:

[0036] Inputting the facial expression image of the newborn to be identified into the feature extractor to extract features and obtain pain features;

[0037] The pain features are converted into a concept stream and input into the pain classifier for classification prediction to obtain a corresponding pain degree recognition result.

[0038] A newborn baby pain degree dynamic identification system, comprising:

[0039] A data acquisition module, used to obtain a dataset of images of neonatal facial expressions of pain, wherein the dataset includes images of neonatal facial expressions and corresponding pain degree labels;

[0040] The pain degree recognition module is used to construct a dynamic recognition model for the pain degree of newborns, wherein the dynamic recognition model for the pain degree of newborns includes a feature extractor and a pain classifier connected in sequence; the dynamic recognition model for the pain degree of newborns is trained and optimized based on the dataset of images of painful facial expressions of newborns to obtain an optimized dynamic recognition model for the pain degree of newborns; and the pain degree recognition of the facial expression images of newborns to be recognized is performed based on the optimized dynamic recognition model for the pain degree of newborns.

[0041] An electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for dynamically identifying the pain degree of a newborn.

[0042] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for dynamically identifying the pain degree of a newborn is implemented.

[0043] The technical effects of the present invention are:

[0044] The present invention can make full use of the extracted key features of facial expressions, and can obtain better pain level assessment results on the neonatal pain facial image data stream, providing a new method for developing an incremental learning neonatal pain level dynamic assessment system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 A schematic diagram of the process of the method for dynamic identification of pain degree of newborns based on conceptual cognitive computing provided by the present invention;

[0048] Figure 2 This is a schematic diagram of the structure of a dynamic recognition model of neonatal pain degree based on conceptual cognitive computing provided by the present invention. DETAILED DESCRIPTION

[0049] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as limiting the present invention, but should be understood as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0050] It should be understood that the terms described in the present invention are only for describing special embodiments and are not intended to limit the present invention. In addition, for the numerical range in the present invention, it should be understood that each intermediate value between the upper and lower limits of the scope is also specifically disclosed. Each smaller range between the intermediate value in any stated value or stated range and any other stated value or intermediate value in the described range is also included in the present invention. The upper and lower limits of these smaller ranges can be independently included or excluded in the scope.

[0051] Unless otherwise indicated, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art generally understood by the present invention. Although the present invention describes only preferred methods, any method similar or equivalent to that described herein may also be used in the implementation or testing of the present invention. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods related to the documents. In the event of a conflict with any incorporated document, the content of this specification shall prevail.

[0052] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present invention description without departing from the scope or spirit of the present invention. Other embodiments derived from the present invention description will be apparent to those skilled in the art. The present application description and examples are exemplary only.

[0053] The words “include,” “including,” “have,” “contain,” etc. used in this article are open-ended terms, meaning including but not limited to.

[0054] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0055] Embodiment 1

[0056] like Figure 1 - Figure 2 As shown, in this embodiment, a method for dynamically identifying the pain level of a newborn baby is provided, comprising: obtaining a dataset of images of facial expression of pain of a newborn baby, wherein the dataset of images of facial expression of pain of a newborn baby comprises images of facial expression of a newborn baby and corresponding pain level labels; constructing a dynamic recognition model of the pain level of a newborn baby, wherein the dynamic recognition model of the pain level of a newborn baby comprises a feature extractor and a pain classifier connected in sequence; training and optimizing the dynamic recognition model of the pain level of a newborn baby based on the dataset of images of facial expression of pain of a newborn baby, to obtain an optimized dynamic recognition model of the pain level of a newborn baby; and performing pain level recognition of the facial expression image of a newborn baby to be recognized based on the optimized dynamic recognition model of the pain level of a newborn baby.

[0057] This embodiment discloses a method for dynamic recognition of neonatal pain level based on conceptual cognitive computing, the method comprising: establishing a neonatal pain facial expression image data stream, including a preprocessed neonatal facial image and its corresponding pain level category label; constructing a feature extractor based on a visual Transformer to extract key features of neonatal facial pain expression; using a pain feature set as a concept connotation and an image object set as a concept extension to convert facial image instances into concepts; constructing a conceptual cognitive computing model (CCCM) for neonatal pain level recognition; converting a new facial image (stream) into a concept (stream) and inputting it into the CCCM for pain level recognition, thereby obtaining a pain level assessment result. This embodiment can make full use of the extracted key features of facial expression, and can obtain a better pain level assessment result on the neonatal pain facial image data stream, providing a new method for developing an incremental learning neonatal pain level dynamic assessment system.

[0058] The purpose of this embodiment is to provide a method for dynamic recognition of neonatal pain levels based on conceptual cognitive computing, which combines conceptual cognitive computing with visual Transformer, uses an improved Transformer encoder as a feature extractor to quickly extract key facial features, and then uses the conceptual cognitive computing model as a classifier to identify facial pain feature data streams. It can fully utilize the parallel computing capability of the Transformer encoder and the dynamic incremental learning capability of the conceptual cognitive computing model, effectively improve the accuracy and time performance of neonatal pain level assessment, and meet the needs of practical clinical nursing applications.

[0059] To achieve the above purpose, this embodiment provides the following solutions:

[0060] A method for dynamically identifying the pain degree of a newborn based on conceptual cognitive computing comprises the following steps:

[0061] (1) Establishing a data stream of neonatal facial expression images of pain, including pre-processed neonatal facial images and their corresponding pain level category labels;

[0062] (2) Building a feature extractor based on visual Transformer to extract key features of newborns’ facial pain expressions;

[0063] (3) Taking the pain feature set as the concept connotation and the image object set as the concept extension, the facial image instance is converted into a concept, i.e., a triple (connotation, extension, pain category);

[0064] (4) To construct a conceptual cognitive computational model (CCCM) for neonatal pain level recognition;

[0065] (5) The new facial image (stream) is converted into a concept (stream) and input into CCCM for pain level recognition, thereby obtaining the pain level assessment result.

[0066] Optionally, step (2) includes:

[0067] (2.1) We construct a feature extractor by adding spatial encoding, edge encoding, and centrality encoding of block graph nodes to the encoder of the visual Transformer, and adding a Dropout layer after the MHA and MLP modules of the Transformer encoder;

[0068] (2.2) Input the newborn’s facial image into the feature extractor to extract the key features of facial pain expression, and concatenate the corresponding pain level category label at the end of the feature.

[0069] Optionally, the step (2.2) includes:

[0070] (2.2.1) Divide the input 2D image of the newborn's face into 2D sequence blocks (patches) of equal height and width;

[0071] (2.2.2) Set the vector length of each layer of the Transformer encoder to a fixed value S, and use a trainable linear mapping to flatten and map the sequence blocks into S-dimensional vectors: patch embeddings;

[0072] (2.2.3) Add the block position embedding to the patch embeddings, and concatenate the pain level category label at the end, input it into the feature extractor to extract the key features of facial pain expression and output it.

[0073] Optionally, step (4) includes:

[0074] (4.1) Randomly select a certain proportion of instances of each class (e.g., 1% to 10%, depending on the actual sample size) from the output of step (2) as the initial sample set;

[0075] (4.2) using the step (3) to convert the initial sample set into an initial concept set, and using the fuzzy clustering method to construct an initial concept space (corresponding to the pain category) on the initial concept set to store knowledge;

[0076] (4.3) treating the output of step (2) as an input data stream and dividing it into data blocks of equal length; (4.4) performing dynamic concept learning on each data block of the input data stream in turn and outputting a prediction result of the pain level;

[0077] (4.5) In the process of concept learning, the original concept space is continuously updated to achieve dynamic incremental learning of knowledge.

[0078] Optionally, the step (4.4) includes:

[0079] (4.4.1) For each data block, read each instance in it in turn, convert it into a new concept using the step (3), and use the similarity calculation method based on the concept connotation to calculate the similarity with each concept in all concept spaces, and use the maximum similarity value calculated in each concept space as the similarity value between the new concept (instance) and the concept space;

[0080] (4.4.2) The concept space with the greatest similarity to the new concept is taken as the target space to which the concept belongs, and the pain category corresponding to the target space is output as the pain level assessment result.

[0081] It is feasible that the step (4.5) dynamically updates the original concept space according to the partial order relationship between the new concepts, including the following steps:

[0082] (4.5.1) After determining the target space to which the new concept (instance) belongs using step (4.4), determine the partial order relationship between the new concept and any concept in the original concept space;

[0083] (4.5.2) If the connotation of the new concept is better than the connotation of any concept C in the original concept space, then merge the instance object corresponding to the new concept with the extension of concept C, and replace the connotation of concept C with the connotation of the new concept;

[0084] (4.5.3) If the connotation of the new concept is not better than the connotation of any concept C in the original concept space, then only the instance object corresponding to the new concept is merged with the extension of concept C, and the connotation of concept C remains unchanged;

[0085] (4.5.4) Otherwise, the new concept is directly added to the original concept space.

[0086] Specific application examples of this embodiment include:

[0087] S1: Establish a dataset of neonatal facial expression images of pain, including pre-processed neonatal facial images and their corresponding pain expression category labels. The expression categories include calm, mild pain, and severe pain, corresponding to the degree of pain.

[0088] At present, there is no public dataset of neonatal facial expressions of pain. In view of the needs of the project research, this embodiment uses a non-wide-angle camera with a resolution of 1920x1080 and a frame rate of 30fps to capture dynamic facial images of neonates in different degrees of pain stimulation during routine painful operations (such as intramuscular injections and blood sampling) in the neonatal department of a local hospital. Then, professional doctors and nurses use the internationally recognized neonatal pain assessment scale to score the pain level (1 to 10 points) of the collected neonatal facial images, and classify expressions with a score between 1 and 5 as mild pain expressions, and expressions with a score between 6 and 10 as severe pain expressions. Finally, images were captured from the captured video, and three types of typical expression images (calm, mild pain, and severe pain) with high score consistency were selected. These original color images were preprocessed by cropping, rotating, aligning, and scale normalization (224x224 pixels), and the expression category labels were marked (in this embodiment, the calm expression was labeled 1, the mild pain expression was labeled 2, and the severe pain expression was labeled 3) to establish a newborn facial expression image library.

[0089] S2: Build a feature extractor based on visual Transformer to extract key features of newborns’ facial pain expressions. Figure 2 As shown in (a), the constructed feature extractor includes an image patch linear mapping layer, an embedding combination layer, and an improved Transformer encoder layer.

[0090] Step S2 includes the following sub-steps:

[0091] Sub-step S21: In the image block mapping layer, the input image is first divided into several blocks of equal width and height, and then a 16x16 convolution kernel with a step size of 16 is used to convolve the [224, 224, 3] image into [14, 14, 768], and then the [14, 14, 768] image is flattened into a two-dimensional matrix of [196, 768], which is divided into 196 tags, each of which has a size of 768. Finally, a column scalarization operation (such as summation) is performed on the two-dimensional matrix to map the image block into a 768-dimensional vector (patch embedding);

[0092] Sub-step S22: In the embedding combination layer, after adding the position information (position embedding) of each patch to its patch embedding, a 769-dimensional input vector (input embedding) is generated to the improved encoder. Adding position embedding is conducive to faster convergence of the encoder training process;

[0093] Sub-step S23: In the improved encoder layer, first combine the input vector and the centrality encoding (input embedding + centrality encoding) as the input of the Transformer encoder, then add the spatial encoding (spatial encoding) and edge encoding (edge ​​encoding) to the attention mechanism of the Transformer, and add the Dropout layer after the MHA and MLP modules of the Transformer encoder to construct a feature extractor. In the constructed feature extractor, each patch is randomly initialized with the corresponding weight matrix, and finally automatically learned through the network.

[0094] Dropout can improve the robustness and generalization ability of the model by randomly discarding part of its output, prevent overfitting of the training data, and improve its performance in processing new data. The addition of graph node structure information (such as centrality coding, spatial coding, and edge coding) can better obtain the key features of facial pain expressions and improve the quality of the extracted facial features.

[0095] The feature extractor used in this embodiment solves the problems of slow convergence and easy overfitting of the Transformer encoder, improves the robustness and generalization ability of the model, makes full use of the advantages of parallel computing of the encoder, and realizes the rapid extraction of key features of facial pain.

[0096] S3: Formally using triples represents a concept, where X represents the extension of the concept. represents the concept connotation, c represents the target category name. In this embodiment, the pain feature set is regarded as the concept connotation, the image object set is regarded as the concept extension, and the pain category is regarded as the target category name. Next, for And B∈L M , define the following operator (.)*:

[0097]

[0098] Among them, G represents the object set, M represents the attribute set, and the fuzzy relationship between G and M is expressed as Use L M represents the set of all fuzzy sets on M. This embodiment uses the operator defined in formula (1) to convert the facial image instance into a concept. For convenience, the concept is abbreviated as

[0099] S4: Construct a conceptual cognitive computational model (CCCM) for neonatal pain level recognition. Figure 2 As shown in (b), the constructed CCCM as a pain classifier consists of three parts: knowledge storage module, dynamic learning module and concept space updating module.

[0100] Step S4 includes the following sub-steps:

[0101] Sub-step S41: In this embodiment, the knowledge storage module randomly selects 5% (which can be adjusted in the range of 1% to 10% according to the total number of samples of each category) from each type of pain samples on the extracted feature set as the initial sample set, and uses the entire feature set as the verification set;

[0102] Sub-step S42: first convert the initial sample set into an initial concept set using step S3, then use the fuzzy clustering method to construct a concept space on the initial concept set, and use the concept space to store knowledge;

[0103] Sub-step S43: To avoid the impact of too many concepts in each concept space on model performance, the parameter MaxSize is used to control the size of the concept space. When the number of concepts in the concept space > MaxSize, the concept space is compressed to generate a compressed concept space. To complete the compression of the concept space, α-concept domains and virtual concepts are introduced, which are expressed as follows:

[0104] Set the concept similarity threshold α∈[0,1], concept The α-concept field is defined as:

[0105]

[0106] in, represents an existing concept space, sim(·) is a similarity function, and this embodiment uses cosine distance similarity and converts the concept is considered as an instance of M-dimensional feature. Further, let for The jth dimension of connotation, for real concepts in the α-concept domain use The virtual concept representing the α-concept domain is defined as:

[0107]

[0108] in, and They respectively represent the extension and connotation of a virtual concept.

[0109] This embodiment uses virtual concepts, MaxSize=50 and α=0.6 to compress the concept space, which can effectively reduce the computing time and improve the computing performance. The compressed concept space is denoted as

[0110] The step S43 of generating a compressed concept space process (POCCS) further comprises the following sub-steps:

[0111] Sub-step S431: From the concept space Select a concept If it is a virtual concept, add it directly to the compressed concept space Otherwise, execute sub-step S432;

[0112] Sub-step S432: Given a concept neighborhood range ∈ and a concept space The real concept For any real concept if Then Add to the local α-concept neighborhood (denoted as ); otherwise, the concept is directly added to the compressed concept space

[0113] Sub-step S433: If (In this embodiment, ∈=5), the local α-concept neighborhood is represented as a virtual concept and added to the compressed concept space

[0114] Sub-step S434: From the concept space Select a concept that has not been visited yet, and repeat the above sub-steps S431-S433 until All concepts in are visited.

[0115] Sub-step S44: In this embodiment, the dynamic learning module divides the verification set (data stream) into data chunks of equal length, denoted as D1, D2, ..., D t , D t+1 , dynamic prediction learning can be performed on each data block in parallel.

[0116] Suppose the concept space of stage t is For any new instance (object) x r , the concept it transforms into is recorded as Then, the similarity between concepts and concept spaces is defined as follows:

[0117]

[0118] in, and,

[0119] Further, suppose Among them, j represents the jth concept Then, in the entire concept space, the class vectors of all maximum similarity values ​​are expressed as: And output the class label with the maximum value as the final prediction of the model, which is expressed by the following formula (2):

[0120]

[0121] in, It represents the instance or object x r Learned to A class.

[0122] Furthermore, let C t+1 For data block D t+1 In the concept space of stage t+1, It is C t+1 Then, the concept space C t+1and The similarity is calculated using the following formula:

[0123]

[0124] Thus, a sample (x r ,y r ) is predicted to be Represents instance x r Should be classified into A class.

[0125] Furthermore, in order to ensure the consistency of the concept space update in each stage of the parallel dynamic learning process, we construct The temporary concept space E is used for intermediate operations. For example, the temporary concept space of stage t+1 is recorded as E t+ 1.

[0126] Sub-step S45: In this embodiment, while the previous step of dynamic prediction is being performed, the concept space updating module updates the concept space of the previous step, thereby completing the entire dynamic learning process.

[0127] Specifically, the connotation of the concept is: Given a threshold λ(i), let is the concept space of the j-1th stage. For any concept in the current concept space The concept space update process (PUCS) further includes the following sub-steps:

[0128] Sub-step S451: If but That is, if the connotation of the new concept is better than the connotation of any concept in the original concept space, the instance object corresponding to the new concept will be merged into the extension of the compared concept, and the connotation of the new concept will be replaced by the connotation of the compared concept;

[0129] Sub-step S 452: If but That is, if the connotation of the new concept is not better than the connotation of any concept in the original concept space, then only the instance object corresponding to the new concept is merged into the extension of the compared concept, and the connotation of the compared concept remains unchanged;

[0130] Sub-step S453: Otherwise, That is, new concepts are directly added to the concept space of the jth stage.

[0131] In the process PUCS, if the new concept is in the concept space of stage j-1 There is a partial order relationship between any concepts in the concept space will all be updated.

[0132] After a certain dynamic learning process of the CCCM model, the generated concept space can be used for facial pain assessment.

[0133] S5: The new facial image (stream) is converted into a concept (stream) through steps S2 and S3, and input into the trained CCCM model for pain level recognition. According to formula (2), the category corresponding to the maximum probability is the pain level result of the newborn to be evaluated.

[0134] A newborn baby pain degree dynamic identification system, comprising:

[0135] A data acquisition module, used to obtain a dataset of images of neonatal facial expressions of pain, wherein the dataset includes images of neonatal facial expressions and corresponding pain degree labels;

[0136] The pain degree recognition module is used to construct a dynamic recognition model for the pain degree of newborns, wherein the dynamic recognition model for the pain degree of newborns includes a feature extractor and a pain classifier connected in sequence; the dynamic recognition model for the pain degree of newborns is trained and optimized based on the dataset of images of painful facial expressions of newborns to obtain an optimized dynamic recognition model for the pain degree of newborns; and the pain degree recognition of the facial expression images of newborns to be recognized is performed based on the optimized dynamic recognition model for the pain degree of newborns.

[0137] An electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for dynamically identifying the pain degree of a newborn.

[0138] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for dynamically identifying the pain degree of a newborn is implemented.

[0139] This embodiment uses the spatial coding, edge coding and centrality coding of graph nodes to improve the visual Transformer encoder and the parallel computing capability of the encoder, thereby improving the efficiency and quality of the key features of facial pain expression;

[0140] This embodiment uses conceptual cognitive computing technology to construct an evaluation model CCCM for identifying the degree of pain in the face of a newborn. It can use formal concept analysis, machine learning, dynamic learning, etc. to learn new knowledge from different types of data in a dynamic environment, just like humans recognize the objective world.

[0141] The CCCM model of this embodiment can complete dynamic learning and concept space updating in parallel on multiple data blocks on the data stream, reducing the time cost of the model to accurately assess the pain level of the newborn and improving the overall performance of the model;

[0142] The organic integration of conceptual cognitive computing and visual Transformer in this embodiment enables the method described in this application to be well adapted to application scenarios with high-dimensional and dynamic data streams, such as clinical care.

[0143] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for dynamically identifying the pain level of a newborn, characterized in that: include: Acquire a dataset of neonatal facial expression images showing pain, wherein the dataset includes neonatal facial expression images and corresponding pain degree labels; Constructing a dynamic recognition model for the pain level of newborns, wherein the dynamic recognition model for the pain level of newborns comprises a feature extractor and a pain classifier connected in sequence; Based on the neonatal pain facial expression image data set, the neonatal pain degree dynamic recognition model is trained and optimized to obtain an optimized neonatal pain degree dynamic recognition model; The pain level recognition of the facial expression images of the newborns to be recognized is performed based on the optimized dynamic recognition model of the pain level of the newborns.

2. A method for dynamically identifying the pain level of a newborn according to claim 1, characterized in that: Get the dataset of newborns’ painful facial expressions, including: Filming dynamic facial videos of newborns as they experience different levels of pain stimulation; Selecting a number of reviewers, each reviewer scoring the pain level of the expression in the dynamic facial video according to the neonatal pain assessment scale, and determining a pain level label based on the pain level score; the pain level label includes a calm label, a mild pain label, and a severe pain label; Conducting consistency evaluation on the pain level scores of each reviewer, and intercepting images with highly consistent scores in the dynamic facial images as typical images; The neonatal pain facial expression image dataset is constructed based on the typical images and the corresponding pain level labels.

3. A method for dynamically identifying the pain level of a newborn according to claim 1, characterized in that: The training and optimization of the dynamic recognition model of the neonatal pain degree is performed based on the neonatal pain facial expression image dataset, specifically including: Inputting the neonatal pain facial expression image data set into the feature extractor for extraction to obtain a pain feature set, wherein the pain feature set includes key features of pain expression and corresponding pain degree labels; The pain feature set is divided into an initial sample set and a validation set based on a preset division rule, a concept space is constructed based on the initial sample set, the validation set is divided into a number of parallel data blocks, dynamic prediction learning is performed on each of the data blocks, the pain features in each of the data blocks are used as concept connotations, the similarity between the concept and the concept space is calculated, and the corresponding pain degree label is output; in the process of dynamic prediction learning, the concept space is updated based on the degree of superiority and inferiority between the new concept and the concepts in the concept space, and the dynamic learning optimization of the dynamic recognition model of the neonatal pain degree is completed.

4. A method for dynamically identifying the pain level of a newborn according to claim 3, characterized in that: Inputting the neonatal pain facial expression image dataset into the feature extractor for extraction, specifically comprising: Inputting the facial expression image of the newborn into the image block mapping layer for image division to obtain a number of image blocks with equal height and width, performing convolution and flattening processing on each of the image blocks to obtain a corresponding two-dimensional matrix, and performing column scalarization operation on the two-dimensional matrix to obtain a mapping vector; After adding the position information corresponding to each of the image blocks to the corresponding mapping vector, an input vector is obtained; A Dropout layer is added after the MHA module and MLP module of the Transformer encoder, and spatial coding and edge coding are added to the attention mechanism of the Transformer encoder to obtain an improved Transformer encoder. Each input vector is combined with the centrality code of the corresponding image block, and the combined data is input into the improved Transformer encoder to output the key features of pain expression.

5. A method for dynamically identifying the pain level of a newborn according to claim 3, characterized in that: Constructing a concept space based on the initial sample set specifically includes: A preset proportion of samples are randomly selected from the pain feature set through the knowledge storage module as the initial sample set, and the entire pain feature set is used as the validation set; The feature samples in the initial sample set are represented by triples in the form of concepts, wherein the pain feature set is the concept connotation, the newborn facial expression image is the concept extension, and the pain degree label is the target category name; An initial concept set is constructed based on the converted concepts, an initial concept space on the initial concept set is constructed based on the initial concept set in combination with a fuzzy clustering method, and the initial concept space is compressed to obtain a compressed concept space.

6. A method for dynamically identifying the pain level of a newborn according to claim 5, characterized in that: The initial concept space is compressed, specifically including: Step 1: When the number of concepts in the initial concept space exceeds a preset threshold, compression of the concept space is performed; Step 2: Select a concept in the initial concept space. If the selected concept is a virtual concept, add it to the compressed concept space. Step 3: For the real concepts in the initial concept space, the similarity between the current real concept and the remaining real concepts in the initial concept space is calculated based on a preset neighborhood range; If the similarity exceeds the preset similarity threshold, it is added to the local concept neighborhood; If the similarity does not exceed the preset similarity threshold, it is added to the compressed concept space; Step 4: If the size of the local concept neighborhood reaches a preset threshold, the local concept neighborhood is represented as a virtual concept and added to the compressed concept space; Repeat steps 2 to 4 until all concepts are visited and the initial concept space is compressed.

7. A method for dynamically identifying the pain level of a newborn according to claim 1, characterized in that: Based on the optimized dynamic recognition model of pain level of newborns, the recognition of pain level of the facial expression image of the newborn to be recognized is performed, specifically including: Inputting the facial expression image of the newborn to be identified into the feature extractor to extract features and obtain pain features; The pain features are converted into a concept stream and input into the pain classifier for classification prediction to obtain a corresponding pain degree recognition result.

8. A dynamic recognition system for neonatal pain level, characterized in that: include: A data acquisition module, used to obtain a dataset of neonatal facial expression images showing pain, wherein the dataset includes neonatal facial expression images and corresponding pain degree labels; The pain degree recognition module is used to construct a dynamic recognition model for the pain degree of newborns, wherein the dynamic recognition model for the pain degree of newborns includes a feature extractor and a pain classifier connected in sequence; the dynamic recognition model for the pain degree of newborns is trained and optimized based on the dataset of images of painful facial expressions of newborns to obtain an optimized dynamic recognition model for the pain degree of newborns; and the pain degree recognition of the facial expression images of newborns to be recognized is performed based on the optimized dynamic recognition model for the pain degree of newborns.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute a method for dynamically identifying the pain degree of a neonate according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The device stores a computer program, which, when executed by a processor, implements a method for dynamically identifying the pain degree of a newborn as described in any one of claims 1 to 7.

Images