Delirium phenomenon determination device

Abstract

The present disclosure relates to a device for determining delirium phenomenon, which processes and judges patient information collected by an information collection module through an information processing module, and outputs the judgment result of the information processing module through an information output module, thereby improving the efficiency and accuracy of determining delirium phenomenon. The device comprises: an information collection module, an information processing module connected to the information collection module, and an information output module connected to the information processing module; the information processing module comprises a mental power level recognition submodule, an attention assessment submodule, a sedation level assessment submodule, and a thinking ability recognition submodule.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition and the field of sound recognition, and in particular, to a device for determining delirium. Background Art

[0002] The ICU (Intensive Care Unit) department needs to conduct multiple status assessments on patients, among which delirium assessment is a very important assessment. Delirium refers to a group of syndromes, also known as acute cerebral syndrome, which is manifested by impaired consciousness, disorganized behavior, lack of purpose, and inability to concentrate. Doctors will formulate subsequent treatment plans based on whether critically ill patients have delirium, but most patients' delirium is irregular and non-periodic. Therefore, the process of judging whether critically ill patients suffer from delirium is relatively cumbersome, and the judgment of whether critically ill patients suffer from delirium can only be made manually by medical staff.

[0003] This manual method of judging delirium, on the one hand, is inefficient and consumes more medical resources; on the other hand, some inexperienced medical staff may make wrong judgments or misjudgments, thus affecting the accuracy of the judgment of delirium. Summary of the invention

[0004] The purpose of the present disclosure is to provide a device for determining delirium to solve the technical problems of low efficiency and low accuracy in the prior art in manually determining delirium.

[0005] In order to achieve the above object, the present disclosure provides a delirium phenomenon determination device, comprising:

[0006] An information acquisition module, an information processing module connected to the information acquisition module, and an information output module connected to the information processing module;

[0007] The information processing module includes a mental strength level identification submodule, an attention assessment submodule, a calmness level assessment submodule, and a thinking ability identification submodule;

[0008] The mental strength level identification submodule is used to output a first delirium judgment result according to the first monitoring image of the patient collected by the information collection module;

[0009] The attention assessment submodule is used to output a second delirium judgment result according to the action image of the patient to the preset action instruction collected by the information collection module;

[0010] The sedation level assessment submodule is used to output a third delirium judgment result according to the second monitoring image of the patient collected by the information collection module;

[0011] The thinking ability identification submodule is used to output a fourth delirium judgment result according to the patient's voice response to the preset question collected by the information collection module;

[0012] The information output module is used to determine whether the patient has delirium according to one or more of the first delirium judgment result, the second delirium judgment result, the third delirium judgment result and the fourth delirium judgment result, and output prompt information indicating whether the patient has delirium.

[0013] Optionally, the mental strength level recognition submodule is used to process the first monitoring image as follows according to a pre-trained first mathematical model:

[0014] The first monitoring image is processed as follows based on the attention mechanism:

[0015] Determining a high-level feature map and a shallow-level feature map of the first monitoring image;

[0016] Downsampling the shallow feature map;

[0017] Fusing the down-sampled feature map with the high-level feature map, and obtaining a fused feature map of the first monitoring image based on weight parameters learned in a pre-training process;

[0018] Image classification is performed according to the fused feature map to obtain and output the first delirium judgment result.

[0019] Optionally, the first mathematical model includes a Resnet34 network, and the training of the Resnet34 network includes:

[0020] Divide sample images pre-labeled with mental state classification labels into training set, validation set and test set;

[0021] According to the training set, the Resnet34 network is iteratively trained using a small batch gradient descent method and a learning rate adaptive decay method;

[0022] Verify whether the Resnet34 network after each round of training is overfitted according to the verification set;

[0023] The generalization performance of the Resnet34 network after training is tested according to the test set.

[0024] Optionally, the device also includes a sound playing module for playing the preset action instructions to the patient via voice, and the information acquisition module is used to acquire action images performed by the patient in response to the preset action instructions, and send the action images to the attention assessment submodule.

[0025] Optionally, the attention assessment submodule processes the action image as follows using a second mathematical model:

[0026] performing an identity transformation on the action image according to a spatial identity function, wherein parameters of the spatial identity function are learned during the training process of the second mathematical model;

[0027] Perform action recognition on the image obtained through the identity transformation to obtain and output the second delirium judgment result.

[0028] Optionally, the second mathematical model includes an adversarial recognition unit, the adversarial recognition unit includes a convolutional layer, a residual structure, a feature fusion layer, and a softmax layer, and the training of the adversarial recognition unit includes:

[0029] According to the sample images pre-marked with the attention level classification labels, the adversarial recognition unit is adversarially trained based on the Yolov3 target detection algorithm to obtain a trained adversarial recognition unit.

[0030] Optionally, the sedation level assessment submodule is used to classify the second monitoring image through a pre-trained binary classification model, and the third delirium judgment result includes whether the sedation level of the patient belongs to the alert but quiet category in the sedation scoring standard.

[0031] Optionally, the binary classification model includes a Densenet neural network model, the loss function of the Densenet neural network model is a cross entropy loss function, and the cross entropy loss function includes a hyperparameter corresponding to a target classification label of the binary classification model;

[0032] The training process of the Densenet neural network model includes:

[0033] According to the sample images pre-labeled with binary classification labels, the Densenet neural network model is iteratively trained, and after multiple rounds of iterative training are completed, the hyperparameters in the cross entropy loss function are adjusted.

[0034] Optionally, the mental strength level identification submodule is used to process the first monitoring image according to a first mathematical model to obtain and output a first delirium judgment result;

[0035] The attention assessment submodule is used to process the action image according to the second mathematical model to obtain and output a second delirium judgment result;

[0036] The sedation level assessment submodule is used to process the second monitoring image according to the third mathematical model to obtain and output a third delirium judgment result;

[0037] Wherein, any two of the first mathematical model, the second mathematical model and the third mathematical model are trained by transfer learning.

[0038] Optionally, the third mathematical model is a binary classification model, and the transfer learning between the second mathematical model and the third mathematical model includes:

[0039] After the training of the first mathematical model is completed, calculating the entropy value of each model layer parameter in the model parameters of the second mathematical model;

[0040] According to the entropy value of each model layer parameter, the model layer parameters that do not meet the first preset condition are deleted, and the remaining parameters are retained as the initial parameters of the binary classification model.

[0041] Optionally, the thinking ability identification submodule is used to process the reply speech as follows using a fourth mathematical model:

[0042] Extracting first sound feature information of the reply voice and second sound feature information of the standard reply voice corresponding to the preset question;

[0043] The fourth delirium judgment result is obtained and outputted based on the first sound feature information and the second sound feature information.

[0044] Optionally, the fourth mathematical model includes an LSTM network and a false voice generation unit;

[0045] The LSTM network is used to extract the initial feature information of the reply speech;

[0046] The false voice generation unit is used to perform signal confrontation simulation on the initial feature information to obtain the first sound feature information that is enhanced relative to the initial feature information.

[0047] Optionally, the fourth mathematical model also includes a sound binary classification sub-model, which is used to classify the first sound feature information and the second sound feature information, and output the fourth delirium judgment result indicating whether the first sound feature information and the second sound feature information belong to the same classification.

[0048] Optionally, it further comprises a trigger, wherein the trigger is used to trigger the device to perform a delirium determination operation on the patient in response to a user's operation; or,

[0049] The trigger is also used to trigger the device to periodically perform a delirium determination operation on the patient according to a preset timing cycle.

[0050] The technical solution provided by the embodiment of the present disclosure is based on the judgment standard of delirium phenomenon, and a delirium phenomenon determination device including a mental strength level recognition submodule, an attention assessment submodule, a sedation level assessment submodule, and a thinking ability recognition submodule is set. Each submodule judges whether the patient has delirium phenomenon from multiple dimensions used to judge delirium phenomenon, and the information output module in the delirium phenomenon determination device can finally determine whether the patient has delirium phenomenon according to the judgment results of each submodule in different dimensions, and output prompt information indicating whether the patient has delirium phenomenon. This relieves the pressure of medical staff to a great extent, saves medical resources, and improves the efficiency and accuracy of determining delirium phenomenon.

[0051] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0053] Figure 1 is a block diagram of a device for determining delirium according to an exemplary embodiment;

[0054] Figure 2 is a block diagram of a device for determining delirium according to another exemplary embodiment;

[0055] Figure 3 is a schematic diagram of a model training process according to an exemplary embodiment of the present disclosure;

[0056] Figure 4 is a flowchart of multi-scale feature fusion according to an exemplary embodiment of the present disclosure;

[0057] Figure 5 is a block diagram of a device for determining delirium according to another exemplary embodiment.

[0058] Description of Reference Numerals

[0059] 101-Information Collection Module 102-Information Processing Module

[0060] 103-Information output module 1021-Mental strength level identification submodule

[0061] 1022-Attention Assessment Submodule 1023-Calmness Level Assessment Submodule

[0062] 1024-Thinking Ability Identification Submodule 104-Trigger

[0063] 105-Sound playback module DETAILED DESCRIPTION

[0064] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0065] As mentioned in the background technology, the process of judging whether a patient suffers from delirium is rather cumbersome, and manual judgment of delirium by medical staff consumes more medical resources and is inefficient. At the same time, some inexperienced medical staff may make wrong judgments or misjudgments, thereby affecting the accuracy of the judgment of delirium.

[0066] In view of this, the present disclosure provides a delirium phenomenon determination device, which processes and judges the patient information collected by the information collection module through the information processing module, and the information output module outputs the judgment result of the information processing module, thereby improving the efficiency and accuracy of determining the delirium phenomenon.

[0067] The technical solution of the present disclosure is described in detail below with reference to embodiments.

[0068] Figure 1 is a block diagram of a device for determining delirium according to an exemplary embodiment. Figure 1 , the delirium phenomenon determination device comprises:

[0069] An information acquisition module 101, an information processing module 102 connected to the information acquisition module 101, and an information output module 103 connected to the information processing module 102;

[0070] The information processing module 102 includes a mental strength level identification submodule 1021, an attention assessment submodule 1022, a calmness level assessment submodule 1023, and a thinking ability identification submodule 1024;

[0071] The mental strength level identification submodule 1021 is used to output a first delirium judgment result according to the first monitoring image of the patient collected by the information collection module 101;

[0072] The attention assessment submodule 1022 is used to output a second delirium judgment result according to the action image of the patient to the preset action instruction collected by the information collection module 101;

[0073] The sedation level assessment submodule 1023 is used to output a third delirium judgment result according to the second monitoring image of the patient collected by the information collection module 101;

[0074] The thinking ability identification submodule 1024 is used to output a fourth delirium judgment result according to the patient's voice answer to the preset question collected by the information collection module 101;

[0075] The information output module 103 is used to determine whether the patient has delirium according to one or more of the first delirium judgment result, the second delirium judgment result, the third delirium judgment result and the fourth delirium judgment result, and output prompt information indicating whether the patient has delirium.

[0076] It should be understood that the information acquisition module 101 is used to collect patient information and send the information to the information processing module 102 for processing. The information acquisition module 101 can collect images and sounds of critically ill patients at all times, or trigger the information acquisition module 101 to collect images and / or sounds of patients through event triggering.

[0077] Specifically, the information acquisition module 101 may include an image acquisition device, and the acquisition of the patient's image may be acquisition of video frames in a patient's status video captured by the image acquisition device, or the patient may be periodically photographed by the image acquisition device. The image acquisition device may be a camera, or may be an electronic device provided with a camera, and the embodiments of the present disclosure are not limited thereto.

[0078] The information collection module 101 may further include a sound collection device, which may be a microphone, or may be an electronic device provided with a microphone, to collect the patient's voice, and the embodiments of the present disclosure are not limited to this.

[0079] It should also be understood that the determination of delirium can be carried out in a preset order, in which the mental power level is determined, the attention level is determined, the sedation level is determined, and the thinking ability level is determined. That is to say, in the present disclosure, the information processing module 102 can also process the patient information collected by the information collection module 101 in the order of the mental power level identification submodule 1021, the attention assessment submodule 1022, the sedation level assessment submodule 1023, and the thinking ability identification submodule 1024. When the first delirium judgment result shows that the patient does not have delirium, the information output module 103 outputs that the patient does not have delirium, that is, the final judgment result is that the patient does not have delirium, and the processing of the subsequent modules is stopped, otherwise the next module is entered. By analogy, when the fourth delirium judgment result shows that the patient does not have delirium, the information output module 103 outputs that the patient does not have delirium. That is, the final judgment result is that the patient does not have delirium, otherwise the final judgment result is that the patient has delirium. Furthermore, when the device determines that the patient is experiencing delirium, an alarm may be issued to notify medical staff to perform subsequent related processing. The embodiments of the present disclosure do not limit the specific form of the alarm and the content of the prompt.

[0080] At the same time, the mental strength level identification submodule 1021, the attention assessment submodule 1022, the sedation level assessment submodule 1023, and the thinking ability identification submodule 1024 can all be used separately, that is, to determine a certain level separately, such as determining the patient's thinking ability level separately, and the embodiments of the present disclosure are not limited to this.

[0081] Reference Figure 2 The delirium phenomenon determination device also includes a trigger 104, which is used to respond to the user's operation and trigger the delirium phenomenon determination device to perform a delirium phenomenon determination operation on the patient. Alternatively, the trigger 104 is also used to trigger the delirium phenomenon determination device to periodically perform a delirium phenomenon determination operation on the patient according to a preset timing cycle.

[0082] For example, the trigger 104 may be a button switch or a function option set in the terminal. When the medical staff needs to perform an additional test on the patient's delirium at a certain moment, the trigger 104 may be used to manually trigger the delirium phenomenon determination device to perform a delirium phenomenon determination operation on the patient. At the same time, the trigger 104 may be set to trigger the delirium phenomenon determination device according to a preset timing cycle, so that the patient's delirium phenomenon is determined once every certain interval, for example, the patient's delirium phenomenon is determined once every two hours, so as to monitor the patient in real time. The embodiments of the present disclosure do not limit the specific form and triggering conditions of the trigger 104.

[0083] In one embodiment, the mental strength level recognition submodule 1021 is used to process the first monitoring image according to the pre-trained first mathematical model as follows:

[0084] The first monitoring image is processed as follows based on the attention mechanism:

[0085] Determine a high-level feature map and a shallow-level feature map of the first monitoring image, downsample the shallow-level feature map, and then fuse the downsampled feature map with the high-level feature map, and based on the weight parameters learned in the pre-training process, obtain a fused feature map of the first monitoring image, and then perform image classification based on the fused feature map to obtain and output a first delirium judgment result.

[0086] It should be understood that the mental strength level identification submodule 1021 identifies the patient's mental strength level based on the first monitoring image of the patient collected by the information collection module 101 to determine whether the patient's mental strength level has an acute change or fluctuates within a preset time period. Considering that in an image, since the information of each area contributes to the target to be predicted to a different extent, learnable weights can be set for different areas of the image. A weight distribution for image features is obtained through the training process, and positions with high weights are more important to the target to be predicted. Applying this weight distribution to the features of the image can provide different feature influences for tasks such as image classification or image recognition for the image, so that the task focuses on some important features, ignores unimportant features, and improves task efficiency.

[0087] For example, the shallow features of an image refer to the contour, edge, color, texture and shape features, and the high-level features of an image refer to what can be seen by the naked eye. For example, when extracting shallow features of a face, the contour, nose, eyes, etc. of the face can be extracted. The size of the high-level feature map is smaller than the shallow-level feature map, so the shallow-level feature map can be downsampled, and then the feature map obtained by downsampling is fused with the high-level feature map to obtain the first fused feature map.

[0088] The first fused feature map is processed by the sigmoid function to obtain the first weight map, wherein the size of the first weight map is smaller than the shallow feature map, and each pixel in the first weight map has a different weight value. Then the first weight map can be upsampled to obtain the second weight map, and then the second weight map is fused with the shallow feature map to obtain the final fused feature map, so as to fuse the pre-trained weight parameters to the image features of the first monitoring image, thereby realizing that the model pays more attention to important features (features with high weight values) when performing image classification.

[0089] Reference Figure 4, feature map A represents the shallow feature map, feature map B represents the high-level feature map, and the size of feature map A is twice the size of feature map B. After downsampling, the size of feature map A will become half of the original size, that is, feature map A becomes feature map A1 of the same size as feature map B. Then feature map A1 and feature map B are spliced ​​according to the channel, and feature map C is the feature map after feature fusion. Feature map C is processed by the sigmoid function to obtain weight map C1, and then weight map C1 is upsampled to make the size of weight map C1 the same as that of feature map A, and then weight map C1 is fused with feature map A to obtain feature map A2, and then image classification is performed based on feature map A2 to obtain and output the first delirium judgment result.

[0090] In one embodiment, the first mathematical model includes a Resnet34 network, and the training of the Resnet34 network includes dividing the sample images pre-labeled with mental state classification labels into a training set, a validation set, and a test set. Then, according to the training set, the Resnet34 network is iteratively trained using a small batch gradient descent method and a learning rate adaptive decay method. Then, the Resnet34 network after each round of iterative training is verified based on the validation set to see whether overfitting occurs, and finally the generalization performance of the Resnet34 network after training is tested based on the test set.

[0091] It should be understood that the last layer of the Resnet34 network is the FC layer, which is used to classify sample images with classification labels. The other layers are all convolutional layers, which are mainly used to extract features from sample images. The Resnet34 network can also include 16 residual units. Among them, the residual unit refers to a skip connection structure, which can solve the problem of network degradation caused by the increase of network depth and make the model close to the identity mapping.

[0092] When calculating the loss function in the Resnet34 network, the classification labels of the sample images can be uniquely encoded first, that is, the N categories corresponding to the sample images are encoded using an N-bit state register so that each category corresponding to the sample image has an independent register bit. When classifying a sample image, only one register bit is valid, that is, each sample image corresponds to only one category.

[0093] For example, the sample image can be normalized by the following calculation formula so that the sum of the classification probabilities is exactly 1:

[0094]

[0095] Among them, e represents the natural base, a i represents the i-th element in vector a, k represents the k-th category, a krepresents the elements in vector a that belong to the kth category, T represents the total number of categories, S j Represents the value after normalization of the j-th vector.

[0096] For example, the cross entropy loss can be calculated by the following formula:

[0097]

[0098] Among them, T represents the total number of categories, y i Indicates the actual classification result.

[0099] For example, refer to Figure 3 , to train the first mathematical model, first, sample images can be collected to construct a data set. The facial position in the sample image used for model training is marked with a box, and the mental state category corresponding to the sample image is annotated. Before training the first mathematical model, the width and height of the image can be uniformly adjusted to the same size. For example, uniformly adjusted to 416×416, it is convenient for subsequent convolution and classification operations. Then the sample images in the data set are divided into three parts: training set, validation set and test set. Among them, the training set is used to train the parameters of the first mathematical model. The validation set is used to test the performance of the first mathematical model after each round of iterative training to prevent overfitting, and is used to guide the adjustment of hyperparameters such as the learning rate. The test set is used to test the performance of the first mathematical model and does not participate in the training process of the model. In practical applications, the ratio of the number of images in the training set to the number of images in the validation set can be 9 to 1.

[0100] For example, during the model training process, the model parameters can also be optimized by the small batch gradient descent method, that is, the sample images are divided into multiple batches of images, such as 8 batches, and then the sample images of each batch are input into the Resnet34 network in turn for model training. At the same time, the learning rate can also be automatically adjusted by the learning rate adaptive decay method, that is, after a preset number of rounds of iterative training, if the loss value calculated by the loss function of the model has not decreased significantly, the learning rate of the next round of iterative training is reduced. Moreover, after reducing the learning rate for multiple times, if the loss value calculated by the loss function has not decreased, an early stopping strategy can be executed, that is, the training process of the Resnet34 network is stopped in advance.

[0101] For example, a total of 500 rounds of iterative training can be designed during the training process, and the learning rate of the first 50 rounds of iterative training is uniformly set to 10 -3If the loss value calculated by the loss function during the first 50 rounds of iterative training is less than the preset threshold (i.e. the model accuracy meets the requirement), the training process of the Resnet34 network can be stopped. If the loss value calculated by the loss function has not decreased significantly after 50 rounds of iterative training, the learning rate for the 51st round of iterative training can be set to 10 -4 In this way, with 50 rounds of iterative training as one training stage, if the loss function still does not decrease after 4 training stages, the early stopping strategy can be implemented, that is, the training process of the Resnet34 network can be stopped in advance.

[0102] In one embodiment, referring to Figure 5 The delirium phenomenon determination device further includes a sound playing module 105 for playing the preset action instruction to the patient through voice. The information collection module 101 is used to collect the action image of the patient performing the preset action instruction and send the action image to the attention evaluation submodule 1022.

[0103] It should be understood that the function of the sound playing module 105 is to play the preset action instructions to the patient through voice, so that the patient can make the action corresponding to the preset action instructions according to the voice heard. The information acquisition module 101 can collect the action image of the patient after hearing the voice (for example, the sound playing module 105 can instruct the information acquisition module 101 to collect the action image when playing the voice), and send the action image to the attention assessment submodule 1022. The preset action instructions can store the sound information in the delirium phenomenon determination device in the form of recording. The preset action instructions can be, for example, actions such as smiling or blinking, opening the mouth, closing the eyes, etc. that do not require the participation of medical staff and are easy for patients to complete. The embodiments of the present disclosure do not limit the preset actions.

[0104] In one embodiment, the attention assessment submodule 1022 processes the action image through the second mathematical model as follows: an identity transformation is performed on the action image according to a spatial identity function, the parameters of the spatial identity function are learned during the training of the second mathematical model, action recognition is performed on the image obtained after the identity transformation, and a second delirium judgment result is obtained and output.

[0105] It should be understood that the purpose of performing identity transformation on action images is to diversify the images, thereby making the resulting model more robust, that is, the model can output results stably and accurately under abnormal conditions such as input errors or network overloads.

[0106] For example, the motion image can be transformed identically by the following calculation formula:

[0107]

[0108] Among them, a, b, c, d, e, and f are learnable parameters, all of which are learned during the training process of the second mathematical model. By learning the parameters of the spatial identity function, operations such as image rotation, translation, and affine transformation can be realized.

[0109] In one embodiment, the second mathematical model includes an adversarial recognition unit, which includes a convolutional layer, a residual structure, a feature fusion layer, and a softmax layer. The training of the adversarial recognition unit includes: performing adversarial training on the adversarial recognition unit based on a sample image pre-labeled with an attention level classification label based on a Yolov3 target detection algorithm to obtain a trained adversarial recognition unit.

[0110] For example, the adversarial recognition unit includes an action analysis module and a discrimination module, and an adversarial relationship is formed between the two modules, that is, the action analysis module analyzes the action of the image and tries to make the discrimination module believe that the analyzed action is correct, while the discrimination module tries to determine that the action analyzed by the action analysis module is wrong. In this way, in order to achieve their own goals, the two modules continuously promote the model to improve the recognition accuracy, and finally a model with better performance can be obtained.

[0111] In one embodiment, the sedation level assessment submodule 1023 is used to classify the second monitoring image through a pre-trained binary classification model, and the third delirium judgment result includes whether the patient's sedation level belongs to the alert but quiet category in the sedation scoring standard.

[0112] For example, the sedation level may include 10 types: aggressive, very agitated, restless, restless, alert but quiet, sleepy, mildly sedated, moderately sedated, deeply sedated, and not easily alert, and the scoring standards are assigned values ​​of +4, +3, +2, +1, 0, -1, -2, -3, -4, and -5. The sample images used for binary classification model training are only labeled with two types of alert but quiet and other sedation levels. Therefore, if the third delirium judgment result is alert but quiet, the final delirium determination result output by the information output module 103 is that the patient does not have delirium.

[0113] In one embodiment, the binary classification model includes a Densenet neural network model, and the loss function of the Densenet neural network model is a cross entropy loss function. The cross entropy loss function includes a hyperparameter corresponding to a target classification label of the binary classification model, and the training process of the Densenet neural network model includes: iteratively training the Densenet neural network model according to sample images pre-labeled with binary classification labels, and adjusting the hyperparameters in the cross entropy loss function after multiple rounds of iterative training are completed.

[0114] For example, the structure of the Densenet neural network includes convolutional layers, pooling layers, and dense blocks. The function of the dense block is to define the connection relationship between input and output. The structure between the two dense blocks is also called the transition layer. The main function of the transition layer is to control the number of channels, and finally the classification work is completed in the classification layer.

[0115] The cross entropy loss value L can be calculated using the following formula:

[0116]

[0117] Among them, N is the total number of sample images used in one round of iterative training, y i Represents the binary classification label of the i-th sample image, y i It takes 0 or 1. When the sample image is alert but quiet, it takes 1. Otherwise, it takes 0. i Represents the predicted classification label of the corresponding sample image, p i The value is calculated by the Densenet network and ranges from 0 to 1.

[0118] Optionally, considering dividing the sample images into two categories labeled alert but quiet and other calmness levels, the data categories of the sample images may be unbalanced, that is, the number of sample images in the alert but quiet category is small, and the number of sample images in other categories is large. During the training process, if the data categories are unbalanced, the model will predict the category of the data as the category corresponding to the large number of samples to achieve a good accuracy rate. As a result, samples with fewer samples will be mistakenly predicted as the category corresponding to the samples with more sample data. Therefore, in order to alleviate the problems caused by the imbalance of data categories and improve the recognition accuracy of the trained model, the embodiment of the present disclosure can also set a hyperparameter corresponding to the target classification label in the loss function. The target classification label is, for example, a classification label that characterizes the classification of the sample image as alert but quiet. In this way, the training process of the Densenet neural network model may include: iteratively training the Densenet neural network model according to the sample images pre-labeled with binary classification labels, and after multiple rounds of iterative training, adjusting the hyperparameters in the cross entropy loss function to balance the sample images with a small amount of data and the sample images with a large amount of data.

[0119] Still using the above cross entropy loss function, the improved cross entropy loss function can be:

[0120]

[0121] Among them, α is a hyperparameter corresponding to the target classification label. In this way, after multiple rounds of iterative training are completed, α can be adjusted to balance the data (when the model tends to predict the classification category of the image as the category corresponding to a large number of samples for accuracy, the existence of hyperparameters will increase the loss value in this case), thereby improving the recognition accuracy of the model after training. It should be understood that whether the hyperparameter needs to be adjusted is judged based on the output of the validation set. After the preset number of rounds of training are completed, the training can be stopped and the hyperparameter α can be adjusted according to the output of the validation set, and then the iterative training can be restarted. It is also possible to customize the process of iterative training to stop training and adjust the hyperparameter α after multiple rounds of training are completed, and then restart the iterative training. The disclosed embodiment is not limited to this. In addition, if after multiple rounds of iterative training are completed, it is judged based on the output of the validation set that the hyperparameter α does not need to be adjusted, then the hyperparameter α is not adjusted and the iterative training continues, and after multiple rounds of iterative training are completed, it is continued to judge whether the hyperparameter α needs to be adjusted based on the output of the validation set.

[0122] In one embodiment, the mental strength level recognition submodule 1021 is used to process the first monitoring image according to the first mathematical model to obtain and output a first delirium judgment result, the attention assessment submodule 1022 is used to process the action image according to the second mathematical model to obtain and output a second delirium judgment result, and the sedation level assessment submodule 1023 is used to process the second monitoring image according to the third mathematical model to obtain and output a third delirium judgment result, wherein any two of the first mathematical model, the second mathematical model and the third mathematical model are trained by transfer learning.

[0123] It should be understood that since the sample data collected in the first mathematical model, the second mathematical model and the third mathematical model are all image samples, the data used are substantially the same. Therefore, in order to improve the utilization rate of data and reduce the difficulty of model learning, the method of transfer learning can be used to transfer the parameters of the trained first mathematical model to the second mathematical model, and then transfer the parameters of the second mathematical model to the third mathematical model after the training of the second mathematical model is completed. Migrating the parameters learned during the training process to the new model can greatly reduce the difficulty of model training and improve the efficiency of model training.

[0124] In one embodiment, the third mathematical model is a binary classification model, and the transfer learning between the second mathematical model and the third mathematical model includes: after the training of the second mathematical model is completed, the entropy value of each model layer parameter in the model parameters of the second mathematical model is calculated, and based on the entropy value of each model layer parameter, the model layer parameters that do not meet the first preset conditions are deleted, and the remaining parameters are retained as the initial parameters of the binary classification model.

[0125] For example, the data used by the second mathematical model and the third mathematical model are substantially the same, but the parameters used are not completely the same, so the model parameters of the second mathematical model can be pruned, that is, some model layer parameters can be deleted. For example, the entropy value of each model layer parameter in the model parameters of the second mathematical model can be calculated first, and the model layers can be sorted according to the entropy value. The first preset condition can be a preset hyperparameter, and the model layers corresponding to some of the hyperparameters are deleted and sorted, and the remaining parameters are retained as the initial parameters of the binary classification model.

[0126] In one embodiment, the thinking ability identification submodule 1024 is used to process the reply speech through a fourth mathematical model as follows: extract the first sound feature information of the reply speech and the second sound feature information of the standard reply speech corresponding to the preset question, and then obtain and output the fourth delirium judgment result based on the first sound feature information and the second sound feature information.

[0127] It should be understood that the thinking ability identification submodule 1024 is used to determine the patient's thinking ability level, and the preset questions for determining the thinking ability level are played through the sound playback device, and the patient answers the preset questions. At the same time, the sound collection device extracts the sound features of the patient's reply voice, and extracts the sound features of the standard reply voice corresponding to the preset question, and then determines whether the patient's answer to the preset question is correct based on the two sound feature information. If the number of errors is less than the preset number, the fourth delirium judgment result is that the patient has not experienced delirium. The preset questions can be: Can stones float on water? Are there fish in the sea? Is 1 larger than 2? The present disclosed embodiment does not limit the number, specific content and preset number of preset questions.

[0128] In one embodiment, the fourth mathematical model includes an LSTM network and a false voice generation unit. The LSTM network is used to extract initial feature information of the reply speech, and the false voice generation unit is used to perform signal adversarial simulation on the initial feature information to obtain first sound feature information that is enhanced relative to the initial feature information.

[0129] It should be understood that the LSTM network (Long Short-Term Memory) has three gates, namely the forget gate, input gate and output gate. The forget gate determines how much of the previous moment can participate in the update of the memory unit, the input gate determines how much of the current moment can participate in the update of the memory unit, and the output gate determines the relationship between the output at the current moment and the memory unit at the current moment.

[0130] The forget gate at time t can be calculated by the following formula:

[0131] f t =σ(W f x t+U f h t-1 +b f )

[0132] Among them, σ represents the sigmoid activation function, W f , U f represents the weight of the forget gate part, b f Represents the bias of the forget gate. t represents the input at time t, h t-1 Represents the output of the hidden layer at time t-1.

[0133] The input gate at time t can be calculated by the following formula:

[0134] i t =σ(W i x t +U i h t-1 +b i )

[0135] Among them, σ represents the sigmoid activation function, W i , U i represents the weight of the input gate, b i Represents the bias of the input gate part. t represents the input at time t, h t-1 Represents the output of the hidden layer at time t-1.

[0136] The output gate at time t can be calculated by the following formula:

[0137] o t =σ(W σ x t +U O h t-1 +b o )

[0138] Among them, σ represents the sigmoid activation function, W o , U o represents the weight of the output gate, b o Represents the bias of the output gate part. t represents the input at time t, h t-1 Represents the output of the hidden layer at time t-1.

[0139] The cell state at time t can be calculated by the following formula:

[0140]

[0141]

[0142] in, represents the candidate cell state, Tanh represents the Tanh activation function, W c , U c Table 2. Weights of candidate cell states, b c represents the bias of the candidate cell state part. t represents the input at time t, h t-1 represents the output of the hidden layer at time t-1. t Represents the cell state. The calculation formula for the value of the hidden layer at time t is as follows:

[0143] h t =o t ×Tanh(c t )

[0144] Among them, h t represents the value of the hidden layer at time t, o t represents the output value of the output gate, Tanh represents the tanh activation function, c t Represents the value of the cell state at time t.

[0145] It should also be understood that since the patient's voice in answering questions may be relatively weak, in order to prevent misidentification caused by the patient's signal being too weak, the present disclosure uses a signal confrontation simulation method to process the patient's reply voice and generate a stronger sound signal to facilitate the recognition of the thinking ability recognition submodule 1024. For example, the CP-GAN algorithm can be used, which consists of a generator and an adversary. The generator is a structure based on encoding-decoding. The encoder part uses multi-layer convolution with a stride of 2 to generate features of different levels, and the decoder part restores features of different levels by upsampling, thereby obtaining an enhanced sound signal.

[0146] In one embodiment, the fourth mathematical model also includes a sound binary classification sub-model, which is used to classify the first sound feature information and the second sound feature information, and output a fourth delirium judgment result representing whether the first sound feature information and the second sound feature information belong to the same classification.

[0147] For example, the first sound feature information and the second sound feature information are input into the sound binary classification sub-model. If the classification results are consistent, it means that the patient's answer to the preset question is correct, so the fourth delirium judgment result is that the patient does not have delirium. If the classification results are inconsistent, it means that the patient's answer to the preset question is wrong, so the fourth delirium judgment result is that the patient has delirium.

[0148] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0149] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0150] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A device for determining delirium, characterized in that: include: An information acquisition module, an information processing module connected to the information acquisition module, and an information output module connected to the information processing module; The information processing module includes a mental strength level identification submodule, an attention assessment submodule, a calmness level assessment submodule, and a thinking ability identification submodule; The mental strength level identification submodule is used to process the first monitoring image according to the first mathematical model to obtain and output a first delirium judgment result, wherein the first monitoring image is a monitoring image of the patient collected by the information collection module; The mental strength level recognition submodule is used to process the first monitoring image as follows according to the first mathematical model that has been pre-trained: Determining a high-level feature map and a shallow-level feature map of the first monitoring image; Downsampling the shallow feature map; Fusing the down-sampled feature map with the high-level feature map, and obtaining a fused feature map of the first monitoring image based on weight parameters learned in a pre-training process; Performing image classification according to the fused feature map to obtain and output the first delirium judgment result; The attention assessment submodule is used to process the patient's action image according to the second mathematical model to obtain and output a second delirium judgment result, wherein the action image is an action image of the patient to a preset action instruction collected by the information collection module; The sedation level assessment submodule is used to process the second monitoring image according to the third mathematical model to obtain and output a third delirium judgment result, wherein the second monitoring image is a monitoring image of the patient acquired by the information acquisition module, and any two of the first mathematical model, the second mathematical model and the third mathematical model are trained by transfer learning; The third mathematical model is a binary classification model, and the transfer learning between the second mathematical model and the third mathematical model includes: After the training of the first mathematical model is completed, calculating the entropy value of each model layer parameter in the model parameters of the second mathematical model; According to the entropy value of each model layer parameter, the model layer parameters that do not meet the first preset condition are deleted, and the remaining parameters are retained as the initial parameters of the binary classification model, wherein the first preset condition is that the entropy value is less than a preset hyperparameter; The thinking ability identification submodule is used to output a fourth delirium judgment result according to the patient's voice response to the preset question collected by the information collection module; The information output module is used to determine whether the patient has delirium according to one or more of the first delirium judgment result, the second delirium judgment result, the third delirium judgment result and the fourth delirium judgment result, and output prompt information indicating whether the patient has delirium.

2. The device according to claim 1, characterized in that The first mathematical model includes a Resnet34 network, and the training of the Resnet34 network includes: Divide sample images pre-labeled with mental state classification labels into training set, validation set and test set; According to the training set, the Resnet34 network is iteratively trained using a small batch gradient descent method and a learning rate adaptive decay method; Verify whether the Resnet34 network after each round of training is overfitted according to the verification set; The generalization performance of the Resnet34 network after training is tested according to the test set.

3. The device according to claim 1, characterized in that The device also includes a sound playing module for playing the preset action instructions to the patient through voice, and the information collection module is used to collect action images performed by the patient in response to the preset action instructions, and send the action images to the attention assessment submodule.

4. The device according to claim 3, characterized in that The attention evaluation submodule processes the action image as follows using a second mathematical model: performing an identity transformation on the action image according to a spatial identity function, wherein parameters of the spatial identity function are learned during the training process of the second mathematical model; Perform action recognition on the image obtained through the identity transformation to obtain and output the second delirium judgment result.

5. The device according to claim 4, characterized in that The second mathematical model includes an adversarial recognition unit, the adversarial recognition unit includes a convolutional layer, a residual structure, a feature fusion layer, and a softmax layer, and the training of the adversarial recognition unit includes: According to the sample images pre-marked with attention level classification labels, the adversarial recognition unit is adversarially trained based on the Yolov3 target detection algorithm to obtain a trained adversarial recognition unit.

6. The device according to claim 1, characterized in that The sedation level assessment submodule is used to classify the second monitoring image through a pre-trained binary classification model, and the third delirium judgment result includes whether the sedation level of the patient belongs to the alert but quiet category in the sedation scoring standard.

7. The device according to claim 6, characterized in that The binary classification model includes a Densenet neural network model, the loss function of the Densenet neural network model is a cross entropy loss function, and the cross entropy loss function includes a hyperparameter corresponding to a target classification label of the binary classification model; The training process of the Densenet neural network model includes: According to the sample images pre-labeled with binary classification labels, the Densenet neural network model is iteratively trained, and after multiple rounds of iterative training are completed, the hyperparameters in the cross entropy loss function are adjusted.

8. The device according to claim 1, characterized in that The thinking ability identification submodule is used to process the reply speech as follows through the fourth mathematical model: Extracting first sound feature information of the reply voice and second sound feature information of the standard reply voice corresponding to the preset question; The fourth delirium judgment result is obtained and outputted based on the first sound feature information and the second sound feature information.

9. The device according to claim 8, characterized in that The fourth mathematical model includes an LSTM network and a false voice generation unit; The LSTM network is used to extract the initial feature information of the reply speech; The false voice generation unit is used to perform signal confrontation simulation on the initial feature information to obtain the first sound feature information that is enhanced relative to the initial feature information.

10. The device according to claim 8, characterized in that The fourth mathematical model also includes a sound binary classification sub-model, which is used to classify the first sound feature information and the second sound feature information, and output the fourth delirium judgment result indicating whether the first sound feature information and the second sound feature information belong to the same classification.

11. The device according to any one of claims 1 to 10, characterized in that Also included is a trigger, the trigger being used to trigger the device to perform a delirium determination operation on the patient in response to a user's operation; or, The trigger is also used to trigger the device to periodically perform a delirium determination operation on the patient according to a preset timing cycle.

Images