Nursing process control method and system based on face recognition and storage medium

Abstract

The invention relates to the technical field of face recognition, in particular to a nursing process control method and system based on face recognition and a storage medium. Comprising the following steps: firstly, performing identity recognition on a person in a hospital monitoring video through an identity recognition model; the model identifies clothing features, action features, carried article features and activity background features of a person to distinguish nursing objects; secondly, the nursing object is analyzed during monitoring, if the current monitoring time is the nursing time, corresponding operation is carried out, and if the current monitoring time is not the nursing time, state recognition is carried out on the nursing object in all aspects, and reminding is carried out in time; finally, a nursing effect evaluation method is provided, and according to the method, effect evaluation is carried out by collecting various data of a nursing object in the monitoring period and combining physical examination data of the nursing object after nursing is completed. According to the method, the efficiency and quality of the nursing control process are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a nursing process control method, system and storage medium based on face recognition. Background Art

[0002] As an advanced biometric technology, face recognition technology uses a camera to capture the facial image of the target person and can perform identity recognition, emotion analysis, and behavior analysis by analyzing the features of the facial image.

[0003] Facial recognition technology is widely used in many fields due to its powerful analytical capabilities, such as security monitoring, identity authentication, smart home and medical fields. In the medical field, facial recognition technology is often used to control the patient care process to improve the efficiency and safety of care. By automatically identifying the patient's identity and analyzing the patient's expression, caregivers can provide personalized services more accurately.

[0004] The traditional medical care process is that nurses or doctors communicate with patients face to face to understand their condition, conduct physical examinations, and collect information on patients' vital signs and symptoms. Then, they manually record medical records and care plans. With the improvement of medical equipment, most of the care for patients nowadays is done by monitoring medical equipment to improve the quality and efficiency of care. However, this method requires patients to wear monitoring equipment for a long time, and caregivers rely too much on equipment data and ignore patients' feelings and symptoms. Compared with the above methods, nursing process control based on face recognition can effectively identify patients' facial conditions and analyze patients' reactions.

[0005] In order to improve the efficiency and quality of nursing process control, the present invention proposes a nursing process control method, system and storage medium based on face recognition. Summary of the invention

[0006] The purpose of the present invention is to provide a nursing process control method, system and storage medium based on face recognition for effective control of the patient's nursing process; it mainly starts from the following points: first, an identity recognition model is used to identify the nursing object in the hospital monitoring video; the model mainly analyzes the clothing, actions, items carried and activity space of the person photographed; in this way, the identity of the person in the video can be more comprehensively identified, and the nursing object can be locked at a faster speed; secondly, the present invention analyzes the nursing object during the monitoring process, mainly including matching whether the current monitoring time is a nursing time point; if it is a nursing time point, the corresponding nursing content is obtained and communicated in a timely manner to inform the corresponding nursing staff; if the monitoring time point is not the nursing time point, the space, facial expressions and behavioral actions of the nursing object are monitored to timely remind relevant personnel; finally, the present invention proposes an evaluation method for evaluating the effect of the nursing object after completing the nursing task; this method collects various data of the nursing object in the nursing process as benchmark data; and combines the physical examination report data of the nursing object after completing the nursing task as the nursing effect; by combining the benchmark data and the physical examination report data, a machine learning model is used to analyze the nursing evaluation result; the result can be fed back to relevant personnel and combined with the electronic medical record of the nursing object as analysis data.

[0007] In a first aspect, the present invention provides a nursing process control method for face recognition, comprising:

[0008] Obtain surveillance videos within the hospital's permitted surveillance range;

[0009] Further, extracting the human area in the surveillance video;

[0010] Further, segmenting the character image and the moving background image in the character area;

[0011] Further, a plurality of the person images and the activity background image are input into an identity recognition model to obtain a care object set and a non-care object set;

[0012] Wherein, the identity recognition model includes:

[0013] An input layer, used for receiving the character image and the activity background image;

[0014] An image preprocessing layer, used for preprocessing the character image and the activity background image to obtain a standard character image and a standard activity background image;

[0015] The image preprocessing layer includes: resizing the character image and the activity background image to obtain a first character image and a first activity background image;

[0016] Perform denoising processing to obtain a second character image and a second active background image; perform data enhancement on the second character image and the second active background image to obtain a third character image and a third active background image; remove static interference of the third character background image and the third active background image to obtain the standard character image and the standard active background image.

[0017] The identity feature extraction layer is used to extract features from the standard person image and the standard activity background image to obtain an initial feature set for identity recognition;

[0018] Among them, the identity feature extraction layer includes: using a convolutional neural network to extract clothing features of the standard character image; using a temporal convolutional network to extract character action features of multiple continuous time periods of the standard character images; using an object detection model to extract object features in the standard character image; using a target detection model to extract background features of the standard activity background image; integrating the clothing features, the character action features, the object features and the background features to obtain the initial feature set for identity recognition.

[0019] A multi-identity feature weighting layer, used to perform item-wise weighting on the features in the initial identity recognition feature set to obtain an identity recognition weighted feature set;

[0020] The weighting process of the multi-identity feature weighted layer specifically includes: collecting the performance of each identity recognition feature in the standard person image and the standard activity background image changing over time to obtain an identity recognition feature change data set; using a time series analysis method to perform importance analysis on each identity recognition feature in the identity recognition feature change data set to obtain an identity recognition feature importance weight; weighting the corresponding features in the identity recognition initial feature set according to the identity recognition feature importance weight to obtain the identity recognition weighted feature set;

[0021] An identity recognition layer, used to perform identity recognition on each feature in the identity recognition weighted feature set to obtain multiple identity recognition probabilities;

[0022] The identity recognition output layer is used to cross-check the multiple identity recognition probabilities and output the identity recognition results; wherein the identity recognition results include: care objects and non-care objects.

[0023] Further, continuously monitoring the care objects in the care object set and acquiring facial images of the care objects;

[0024] Further, extracting facial features of the facial image using a facial recognition model;

[0025] Furthermore, the facial features are compared with real facial features of the nursing object pre-stored in a facial database to obtain the identity information of the nursing object;

[0026] Further, obtaining the electronic medical record, nursing plan and nursing staff of the nursing object according to the identity information;

[0027] Further, performing nursing analysis on the nursing object to obtain analysis results;

[0028] The analysis results include:

[0029] Extracting the key nursing fields of the nursing plan; the key nursing fields include: nursing time point, nursing content and nursing equipment; wherein the nursing time point and the nursing content are in a one-to-one correspondence; the nursing content and the nursing equipment are in a one-to-one correspondence;

[0030] Matching the current monitoring time of the nursing object with the nursing time point; if the current monitoring time is any nursing time point in the nursing plan; obtaining nursing content and nursing equipment according to the nursing plan, and reminding nursing staff to provide nursing care;

[0031] If the current monitoring time is not any nursing time point in the nursing plan; performing state recognition on the nursing object; the state recognition includes: spatial state recognition, facial state recognition and behavioral state recognition;

[0032] Wherein, the spatial state recognition includes: acquiring the activity background image of the nursing object;

[0033] Performing layout recognition on the active background image; if the result of the layout recognition does not meet the nursing place requirement of the nursing object, obtaining a place abnormality;

[0034] The facial state recognition comprises: obtaining the facial image of the nursing object; inputting the facial image into a facial state recognition model to obtain a facial recognition result; if the facial recognition result is abnormal, obtaining a facial abnormality;

[0035] The behavior state recognition includes: obtaining the character images of the care object in multiple continuous time periods; pre-processing the character images and inputting them into a temporal convolutional network to obtain the action recognition results of the care object; when the action recognition results are abnormal, obtaining an action abnormality.

[0036] Further, reminding the nursing staff according to the analysis result;

[0037] Furthermore, various data of the nursing object during the monitoring process are recorded; wherein the various data include: nursing object behavior, vital signs, nursing staff operations and nursing equipment;

[0038] Furthermore, after the nursing is completed, the nursing effect of the nursing object is evaluated according to the physical examination report of the nursing object; wherein the specific process of evaluating the nursing object includes: preprocessing the data to obtain standard data; quantifying the physical examination report to obtain a physical examination data set; integrating the standard data with the physical examination data set to obtain a comprehensive data set; inputting the comprehensive data set into a machine learning model to obtain a nursing evaluation result;

[0039] Furthermore, the nursing assessment results are fed back to relevant personnel.

[0040] In a second aspect, the present invention proposes a nursing process control system based on face recognition, the system comprising: a monitoring unit, an identity recognition unit, a face recognition unit, a data management unit, an abnormality recognition unit, an early warning unit, an evaluation unit and a feedback unit;

[0041] Wherein, the monitoring unit is used to monitor within the permitted monitoring range in the hospital;

[0042] The identity recognition unit is used to identify nursing objects and non-nursing objects in the hospital;

[0043] The identity recognition unit uses an identity recognition model to identify the nursing object and the non-nursing object, and the specific process includes:

[0044] The monitoring unit obtains monitoring video within the monitoring range allowed by the hospital;

[0045] Further, extracting the human area in the surveillance video;

[0046] Further, segmenting the character image and the moving background image in the character area;

[0047] Furthermore, the character image and the activity background image are received by an input layer;

[0048] Further, the character image and the activity background image are input into an image preprocessing layer to obtain a standard character image and a standard activity background image;

[0049] Furthermore, the standard person image and the standard activity background image are input into the identity feature extraction layer to obtain an initial feature set for identity recognition;

[0050] Further, the initial identity recognition feature set is input into a multi-identity feature weighted layer to obtain an identity recognition weighted feature set;

[0051] Further, the identity recognition weighted feature set is input into the identity recognition layer to obtain a plurality of identity recognition probabilities;

[0052] Furthermore, the multiple identity recognition probabilities are input into an identity recognition output layer to obtain an identity recognition result.

[0053] The face recognition unit obtains the identity information of the nursing object through face recognition technology;

[0054] The data management unit is used to store and manage the electronic medical records, nursing plans and related information of the nursing object;

[0055] The abnormality recognition unit is used to recognize various abnormal information of the nursing object; wherein the abnormality recognition unit includes: spatial state recognition, facial state recognition and behavioral state recognition;

[0056] The spatial state recognition includes: acquiring the activity background image of the nursing object;

[0057] Performing layout recognition on the active background image; if the result of the layout recognition does not meet the nursing place requirement of the nursing object, obtaining a place abnormality;

[0058] The facial state recognition comprises: obtaining a facial image of the nursing object; inputting the facial image into a facial state recognition model to obtain a facial recognition result; if the facial recognition result is abnormal, obtaining a facial abnormality;

[0059] The behavior state recognition includes: obtaining character images of the care object in multiple continuous time periods; pre-processing the character images and inputting them into a temporal convolutional network to obtain action recognition results of the care object; when the action recognition results are abnormal, obtaining an action abnormality.

[0060] The early warning unit is used to issue a reminder to the nursing staff corresponding to the nursing object;

[0061] The evaluation unit is used to evaluate the nursing effect of the nursing object;

[0062] The evaluation unit includes: obtaining various data in nursing; wherein the various data include: nursing object behavior, vital signs, nursing staff operations and nursing equipment;

[0063] Further, after the nursing of the nursing object is completed, a physical examination report of the nursing object is obtained;

[0064] Further, preprocessing is performed on the data to obtain standard data;

[0065] Further, the physical examination report is quantified to obtain a physical examination data set;

[0066] Further, the standard data are integrated with the physical examination data set to obtain a comprehensive data set;

[0067] Furthermore, the comprehensive data set is input into a machine learning model to obtain nursing assessment results.

[0068] The feedback unit is used to feed back the nursing assessment results to relevant personnel.

[0069] In the third aspect, the present invention proposes a nursing process control storage medium based on face recognition, on which a nursing process control program based on face recognition is stored. When the nursing process control program based on face recognition is executed by a processor, any one of the nursing process control steps based on face recognition in the first aspect or the second aspect is implemented.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] 1. The present invention proposes an identity recognition model for identifying nursing objects in surveillance videos; the model uses an identity feature extraction layer to obtain the clothing features, action features, item features and activity background features of the characters in the surveillance videos; and uses a multi-identity feature weighting layer to weight each feature item by item through analysis of the video stream; in order to improve the accuracy of recognition, a cross-check method is used in the identity recognition output layer to calculate the output probability; the proposed model solves the problem of quickly locking the nursing objects in complex backgrounds and improves the efficiency of nursing process control.

[0072] 2. The present invention performs analysis during the monitoring process of the nursing object, mainly including: determining whether the monitoring time point is a nursing time point; if it is a nursing time point, obtaining the corresponding nursing content, and promptly notifying the corresponding nursing staff; if the monitoring time point is not a nursing time point, the space, facial expressions and behavioral actions of the nursing object are monitored, so as to promptly remind relevant personnel when abnormalities occur; this not only improves the efficiency and quality of the nursing control process, but also provides a safer and more personalized care experience for the nursing object through real-time information feedback and early warning mechanisms for abnormal conditions.

[0073] 3. This paper proposes a nursing assessment method for evaluating the nursing effect of nursing objects; this method collects data such as nursing object behavior, vital signs, nursing staff operations and nursing equipment during the nursing period, and processes the data to obtain standard data; and combines the quantitative data of physical examination of nursing objects after the completion of nursing tasks to evaluate the effect; this promotes the quality and effectiveness of the nursing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A flowchart of a nursing process control method based on face recognition provided by an embodiment of the present invention;

[0075] Figure 2 A framework diagram of a nursing process control system based on face recognition provided by an embodiment of the present invention;

[0076] Figure 3 A structural diagram of an identity recognition model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] Medical care refers to a series of comprehensive nursing service processes for patients, aiming to promote the physical health, mental health and social health of patients; the traditional nursing process is that doctors formulate corresponding nursing plans according to the patient's condition and are implemented by nursing staff; in this process, some medical equipment is needed to monitor the patient's physical condition; however, this method has defects, such as patients wearing monitoring equipment for a long time, so nursing staff rely too much on equipment data and ignore the patient's feelings and symptoms; in order to further optimize the traditional nursing control process, the present invention proposes a nursing process control method, system and storage medium based on face recognition; compared with traditional methods, nursing process control based on face recognition can effectively identify patients and monitor the patient's condition in real time. In order to illustrate the effectiveness of the present invention, it will be described in detail from the following embodiments.

[0079] Embodiment 1:

[0080] In the embodiment of the present application, the nursing process of the nursing object in Hospital A is controlled by combining the method and system proposed in the present invention; Figure 1 and Figure 2 content, Figure 1The content of the present invention is about the step flow of the implementation of the present invention, including: S10. Obtaining the monitoring video within the monitoring range allowed by the hospital; S20. Extracting the human area in the monitoring video; S30. Segmenting the human area; S40. Identifying the human identity; S50. Continuously monitoring the nursing object according to the human identity recognition result; S60. Performing face recognition on the nursing object; S70. Obtaining various relevant information according to the face recognition result; S80. Monitoring and analyzing the nursing object, and recording data; S90. Evaluating the nursing effect after the nursing task is completed; S100. Feedback the evaluation effect to relevant personnel. Figure 2 The structure diagram of the system of the present invention includes: a monitoring unit, an identity recognition unit, a face recognition unit, a data management unit, an abnormality recognition unit, an early warning unit, an evaluation unit and a feedback unit. The method steps of the present invention have a corresponding relationship with each unit of the system. Figure 1 and Figure 2 Provide explanation;

[0081] Obtain the monitoring video of Hospital A from the monitoring unit of the system, corresponding to Figure 1 Step S10; wherein the monitoring video refers to the video within the monitoring range of Hospital A; for example, wards, corridors, public areas and outpatient clinics all belong to the definition of the area within the monitoring range; the above examples are only used for illustration, and the specific areas included in the monitoring range can be explained according to the authority of the hospital to open the monitoring area under actual circumstances;

[0082] Furthermore, according to the above step S20, the area where the person appears in the surveillance video is extracted; this function can use neural networks such as YOLO, SDD and Faster-RCNN in computer vision to detect the person in the video frame and frame the position where the person appears in the video;

[0083] Furthermore, an image segmentation algorithm is used to segment the person image and the active background image of the detected person area, corresponding to step S30; wherein the specific segmentation process for the person image and the active background image is:

[0084] Performing standardization processing on the image of the human figure region to obtain a standard human figure region image;

[0085] Furthermore, the standard person area image is input into the segmentation network for forward propagation to generate a segmentation mask; the segmentation model will output a classification containing the prediction of each pixel, usually a binary mask, indicating whether the pixel belongs to "person" or "background".

[0086] Furthermore, a binary mask is extracted from the segmentation model output, where the background part is usually 0 and the person part is 1.

[0087] Furthermore, the original input person region image and the generated mask are subjected to a bitwise operation (usually multiplication) to retain only the person region, thereby forming an independent person image;

[0088] Further, the original input character region image and the inverse (negation) of the mask are subjected to a bitwise operation to extract the active background image;

[0089] In the implementation of this application, according to the above segmentation process, a person image and an activity background image can be obtained;

[0090] Furthermore, the segmented person image and the active background image are input into the identity recognition unit to obtain the identity of the person in the surveillance video, corresponding to the above-mentioned step S40; wherein the identity recognition unit is integrated by the identity recognition model; the identity recognition model structure is as follows Figure 3 As shown, it includes: an input layer, an image preprocessing layer, an identity feature extraction layer, an identity feature extraction layer, a multi-identity feature weighting layer, an identity recognition layer, and an identity recognition output layer;

[0091] The specific process of character identification includes:

[0092] The character image and the activity background image are received by an input layer;

[0093] Further, the received image is input into the image preprocessing layer for preprocessing to obtain a standard character image and a standard activity background image;

[0094] Among them, the preprocessing process of the image preprocessing layer includes: resizing the character image and the active background image to obtain a first character image and a first active background image; denoising the first character image and the first active background image to obtain a second character image and a second active background image; data enhancing the second character image and the second active background image to obtain a third character image and a third active background image; removing static interference of the third character background image and the third active background image to obtain the standard character image and the standard active background image.

[0095] In the embodiment of the present application, by resizing, denoising, data enhancement and removing static interference on the character image and the active background image, the preprocessing process can significantly improve the accuracy of identity recognition and the effectiveness of nursing process control.

[0096] Furthermore, the preprocessed person image and the standard activity background image are input into the identity feature extraction layer to obtain an initial feature set for identity recognition;

[0097] Wherein, in the identity feature extraction layer, the character's clothing features, action features, carried item features and background features are respectively obtained;

[0098] (1) About clothing features;

[0099] By analyzing the color, style and logo of clothing (such as nurse uniforms, patient uniforms, etc.), the system can quickly identify nursing staff and nursing recipients; non-medical uniforms can promptly detect visitors who are not hospital staff;

[0100] (2) About action characteristics

[0101] Analyze the person's behavior, such as how they walk, how they use equipment, etc., to determine whether they are a non-care target or a care target;

[0102] (3) Characteristics of the items you carry

[0103] Analyze the items carried by the person, such as medical equipment, medical record folders, etc., and obtain them to determine their identity;

[0104] (4) Background characteristics

[0105] Regarding the activity areas of the characters in the hospital, determine whether they appear in specific work areas (such as nurse stations and wards) and analyze the frequency of appearance in specific areas. Nursing staff usually move frequently in specific areas.

[0106] The identity feature extraction layer mainly uses multiple pre-trained network models to extract different recognition features, including:

[0107] Using a convolutional neural network to extract clothing features of the standard person image;

[0108] A temporal convolutional network is used to extract character action features of a plurality of standard character images in consecutive time periods;

[0109] Using an object detection model to extract object features in the standard person image;

[0110] Using a target detection model to extract background features of the standard activity background image;

[0111] The clothing features, the character action features, the object features and the background features are integrated to obtain the initial feature set for identity recognition.

[0112] In the embodiments of the present application, using multiple pre-trained network models for feature extraction can significantly enhance the performance of the identity recognition system and improve the efficiency and accuracy of nursing process control.

[0113] Furthermore, the initial identity recognition feature set is input into the multi-identity feature weighting layer for item-by-item weighting; the specific weighting process includes:

[0114] Collecting the performance of each identity recognition feature in the standard person image and the standard activity background image changing over time to obtain an identity recognition feature change data set;

[0115] The process of generating the identity recognition feature change data set includes:

[0116] Get the video stream within the monitoring period;

[0117] Extract key frames from the video stream; each of the key frames contains a person and an active background;

[0118] Extracting relevant data of the person and the activity background in the key frame; wherein the relevant data refers to the change data of each identification feature in the standard person image and the standard activity background image;

[0119] The degree of change of the relevant data is calculated according to the corresponding time interval; for example, the calculation formula of the degree of change can be expressed as: Wherein, F(t1) represents the feature vector of the relevant data at time point t1; F(t2) represents the feature vector of the relevant data at time point t2; ε represents a constant;

[0120] The degree of change is used as the identity recognition feature change data set;

[0121] Furthermore, a time series analysis method is used to perform importance analysis on each identity recognition feature in the identity recognition feature change data set to obtain the identity recognition feature importance weight; wherein the time series analysis method can be performed using an autoregressive model, a moving average model, etc.;

[0122] Weighting the corresponding features in the initial identity recognition feature set according to the identity recognition feature importance weights to obtain the identity recognition weighted feature set;

[0123] Further, the identity recognition weighted feature set is input into the identity recognition layer to obtain multiple identity recognition probabilities; refer to Table 1, which gives the results of identifying people in the surveillance video from different angles;

[0124] Table 1 Results of identifying some people in surveillance videos

[0125]

[0126] Table 1 lists the probability data of identifying person 1 and person 2 in multiple aspects in the surveillance video; for example, the probability value of person 1 being identified as a care object when the feature type is clothing feature is 0.85; the probability of being a non-care object is 0.15; from the perspective of clothing, it is highly likely that person 1 is a care object;

[0127] Furthermore, the identity recognition weighted feature set is input into the identity recognition output layer to obtain the identity recognition result; wherein the calculation formula of the identity recognition output layer for identity recognition can be expressed as: Where IR represents the output probability of the identity recognition output layer; argmax represents the function for finding the maximum value; P(y i |F) is expressed as the probability of identity i; y i It is represented as the corresponding label of identity i; F is represented as the comprehensive probability of identity recognition;

[0128] For example, in order to calculate the output result of person 1, it is necessary to obtain a comprehensive recognition result based on the data provided in Table 1 and the feature weights; specifically:

[0129] (1) Obtaining feature weights based on the model

[0130] Clothing feature weight: w1 = 0.4; action feature weight: w2 = 0.3; item feature weight: w3 = 0.3; background feature weight: w4 = 0.1;

[0131] (2) Insert the probability value of each feature recognition

[0132] Probability of identification of nursing object:

[0133] Clothing features: P 护理 (Clothing) = 0.85; Action characteristics: P 护理 (Action) = 0.70; Item Features: P 护理 (Item) = 0.75; Background Features: P 护理 (background) = 0.90;

[0134] Probability of non-nursing object identification:

[0135] Clothing features: P 非护理 (Clothing) = 0.15; Action characteristics: P 非护理 (Action) = 0.30; Item Features: P 非护理 (Item) = 0.25; Background Features: P 非护理 (background) = 0.10;

[0136] (3) Calculate weighted features

[0137] F 护理 =w1*P 护理 (clothing)+w2*P护理 (Action)+w3*P 护理 (Item)+w4*P 护理 (background);

[0138] F 护理 =0.4*0.85+0.3*0.70+0.2*0.75+0.1*0.90=0.79;

[0139] F 非护理 =w1*P 非护理 (clothing)+w2*P 非护理 (Action)+w3*P 非护理 (Item)+w4*P 非护理 (background);

[0140] F 非护理 =0.4*0.15+0.3*0.30+0.2*0.25+0.1*0.10=0.21;

[0141] (4) Final recognition result: IR = argmaxP(F 护理 ,F 非护理 )=argmax(0.79,0.21); Recognition result: Person 1 is identified as a care object because its recognition probability of a care object (0.79) is higher than that of a non-care object (0.21). Therefore, the final recognition result of person 1 is a care object.

[0142] In the example of the present application, an identity recognition model is used to identify the personnel in the monitorable area of ​​Hospital A; the model monitors the clothing features, action features, item features and activity background features of the characters in the video; and uses a multi-identity feature weighting layer to weight each feature item by item through analysis of the video stream; thereby improving the accuracy of identifying the nursing objects; the proposed identity recognition model solves the identification process of non-nursing objects based on face recognition, speeds up the search speed for nursing objects, and is conducive to improving the efficiency and quality of nursing process control.

[0143] Further, the identification unit identifies the nursing object and continuously monitors it using the monitoring unit, corresponding to the above step S50;

[0144] Furthermore, during the monitoring process, face recognition is performed on the care object, and this function is implemented by the face recognition unit of the system, corresponding to the above step S60; in addition, during the monitoring process, in case the camera in the monitoring area cannot accurately capture the face image of the care object, the medical staff in the neighboring area can be reminded to check the situation of the care object;

[0145] When the face image of the nursing object can be captured, the face features of the face image are extracted; the face features are input into the face recognition model, and the identity information of the nursing object is obtained by comparing with the real face features of the nursing object pre-stored in the face database; wherein the identity information mainly includes: name, age, gender and medical record number; corresponding to the above step S70;

[0146] Furthermore, the corresponding electronic medical records, nursing plans and corresponding nursing staff in the patient archive database can be retrieved from the data management unit of the system according to the identity information of the nursing object;

[0147] Among them, the electronic medical records, nursing plans and other information stored in the patient archive database involve the privacy and medical safety of patients, so it is crucial to ensure their security. Therefore, some protective measures are adopted for the security of the patient archive database; for example: storage encryption: all patient data is encrypted using advanced encryption standards (such as AES-256) when stored to ensure that even if the data is accessed without authorization, the content cannot be read; transmission encryption: when the data is transmitted in the network, the data is encrypted using a secure communication protocol (such as TLS / SSL) to prevent middleman attacks and information leakage; permission management: a role-based access control (RBAC) model is used to ensure that only authorized personnel (such as doctors and nurses) can access the data of specific patients. Different user roles have different access levels to prevent irrelevant personnel from obtaining sensitive information; access logs: the system will record all operations of accessing and modifying patient files, including the identity, time, and operation content of the visitor, to ensure that they are well documented; regular audits: medical institutions will regularly conduct security audits on the access to patient files to ensure that there are no violations and prevent abuse and data leakage.

[0148] Furthermore, while recognizing the face of the nursing object, the abnormality recognition unit of the system analyzes the nursing object under monitoring and records the data of the process, corresponding to the above step S80; the abnormality recognition unit includes: analyzing the nursing object during the nursing object period and the non-nursing object period; the specific process includes:

[0149] According to the nursing plan of the nursing object identified by face recognition, the corresponding nursing key fields are extracted; wherein the nursing key fields include: nursing time point, nursing content and nursing equipment; the nursing time point and the nursing content are in a one-to-one correspondence; the nursing content and the nursing equipment are in a one-to-one correspondence; refer to Table 2, in which the nursing plan table for person 1 as the nursing object is shown;

[0150] Table 2 Nursing plan for person 1

[0151] Nursing time point Care content Nursing equipment 08:00 Temperature monitoring Electronic thermometer 08:30 Oxygen Oxygen cylinder, mask 07:00 Taking medication Medicine box, drinking cup 10:00 Intravenous fluids Infusion pump, infusion tube 12:00 Dietary guidance Diet Record 14:00 Wound dressing Dressing bag, disinfectant 16:00 Rehabilitation Rehabilitation equipment (such as elastic bands) 18:00 Psychological counseling Relaxing Music Player 20:00 Nighttime care preparation Nursing Record Book

[0152] Further, matching is performed according to the current monitoring time of the nursing object and the nursing time point in the nursing plan; and corresponding operations are performed according to the matching result;

[0153] When the matching result is a nursing time point, the nursing staff corresponding to the nursing object is notified to perform nursing care on the object;

[0154] When the matching result is not a nursing time point, the nursing object is subjected to state recognition; wherein the state recognition includes: spatial state recognition, facial state recognition and behavioral state recognition;

[0155] Wherein, the space state identification includes:

[0156] Acquire the active background image of the nursing object;

[0157] Furthermore, if the result of the layout recognition does not meet the nursing place requirement of the nursing object, a place abnormality is obtained; wherein, the nursing place requirement means that the monitored nursing object should rest in bed; therefore, there should be elements such as a bed, a nursing table and an infusion stand in the layout recognition result; if there are no elements in the nursing place requirement in the active background image of the nursing object, it is proved that the nursing object meets the nursing requirements;

[0158] Facial state recognition includes: obtaining the facial image of the nursing object; inputting the facial image into a facial state recognition model to obtain a facial recognition result; if the facial recognition result is abnormal, obtaining a facial abnormality;

[0159] The facial recognition process includes:

[0160] Use TensorFlow's deep learning framework to train a convolutional neural network-based facial state recognition model;

[0161] Further, the face image of the nursing object is input into the facial state recognition model;

[0162] Recognition result output: The model will output the probability value or category of each expression, such as "happy", "sad", "angry", and "surprised".

[0163] Furthermore, define abnormal criteria: set criteria for facial abnormalities, such as: facial expressions that are inconsistent with nursing activities (such as angry or painful expressions when resting). Failure to recognize valid facial expressions (such as output of "unrecognizable"). The intensity of the detected emotion exceeds a certain threshold (for example, the probability of anger or fear exceeds 70%).

[0164] Further, in the judgment process: if the recognized expression does not match the expectation (for example, the care recipient should be in a relaxed state), it is recorded as "facial abnormality". If no expression can be recognized, it is also recorded as "facial abnormality".

[0165] The behavioral state recognition: The behavioral state recognition includes: obtaining the character images of the care object in multiple continuous time periods; pre-processing the character images and inputting them into a temporal convolutional network to obtain the action recognition results of the care object; when the action recognition results are abnormal, obtaining an action abnormality.

[0166] Among them, when the behavior state recognition is performed on the monitored person in the identity recognition unit; when the identity is recognized as a nursing object, the corresponding behavior state is recorded and analyzed at the same time; the specific process includes:

[0167] The preprocessed continuous image sequence is input into the Temporal Convolutional Network (TCN). TCN has the ability to capture the characteristics of time series data and is suitable for analyzing continuous behavior patterns.

[0168] Among them, CN usually consists of multiple convolutional layers, which can automatically extract spatial and temporal features in image sequences. The network recognizes specific behavior patterns by learning the associations between each time step.

[0169] Furthermore, the action recognition results are output: the TCN model outputs the analysis results as the behavior category of the care object (for example: sitting, standing, walking, falling, etc.), as well as the probability value of each behavior.

[0170] Furthermore, the behavior status is recorded: in the identity recognition unit, when the monitoring system recognizes the identity of the care recipient, the system records the corresponding behavior status.

[0171] Further, define abnormal criteria: criteria for determining abnormal behavior, such as: behavior inconsistent with expected care activities (such as rapid walking or falling while at rest). Failure of behavior recognition (such as failure to recognize any behavior). Abnormal intensity of behavior (such as long periods of inactivity, which may indicate an unexpected situation).

[0172] Furthermore, in the abnormality judgment process, when the recognized behavior results are significantly different from normal activities, the system marks it as "abnormal action". For example, if the behavior detected by the system is "falling", it is immediately marked as abnormal.

[0173] In the embodiment of the present application, the nursing object is analyzed during the monitoring period, and the main contents are: whether the monitoring time point is a nursing time point; if it is a nursing time point, the corresponding nursing content is obtained, and the corresponding nursing staff is notified in time; if the monitoring time point is not a nursing time point, the nursing object's space, facial expressions and behavioral actions are monitored, and when abnormalities occur, relevant personnel are reminded in time; this not only improves the efficiency and quality of the nursing control process.

[0174] Furthermore, the early warning unit of the system promptly reminds the corresponding nursing staff of the analysis results of the abnormality identification unit;

[0175] Furthermore, during the nursing process, the system will automatically record various data to support comprehensive monitoring and analysis of the nursing object. The various data can be divided into two categories: data during nursing and data outside nursing.

[0176] 1. Data recording during nursing care

[0177] (1) Operation information of nursing staff: Operation timestamp: records the exact time when the nursing staff performs each operation to facilitate subsequent time series analysis;

[0178] Operation type, including but not limited to:

[0179] Medication management: such as drug name, dosage and administration method. Physical care: such as bathing, dressing changes and body position changes. Nursing records: Detailed records of each nursing session, including nursing goals, implementation steps and nursing staff observations. Abnormality reports: Record any abnormalities found during the nursing process (such as abnormal vital signs) and attach corresponding treatment measures.

[0180] (2) Vital sign data of the nursing object

[0181] Heart rate: monitor the changes in the heart rate of the care recipient and record the results of each measurement. Blood pressure: including the recording of systolic and diastolic blood pressure. Body temperature: regularly measure and record the body temperature of the care recipient. Respiratory rate: record the number of breaths per minute.

[0182] 2. Data recording during non-nursing period

[0183] During non-nursing periods, the system will also record relevant data of the care recipients in order to comprehensively analyze their behavior and physiological status.

[0184] (1) Behavior records of the nursing recipients

[0185] Behavior patterns: Capture the behavior patterns of the care recipients during non-care time through surveillance cameras, and record the types of behaviors, such as stillness: the state of staying in the same position for a long time. Walking: the frequency and duration of activities. Sitting: the time spent in a seat. Fall events: Once a fall is detected, the time and duration of the event are immediately recorded and marked. Emotional state: Analyze the facial expression recognition results of the care recipients, record emotional changes, and identify emotional states such as anxiety, frustration, or anger.

[0186] Furthermore, the evaluation unit of the system is used to evaluate the nursing effect of the nursing object, corresponding to the above step S90; the specific process includes:

[0187] Data preprocessing is performed on various data of the nursing object to obtain standard data; the data preprocessing process is as follows: preprocessing is performed on various collected data to ensure that the data are in the same dimension. Common standardization methods include: minimum-maximum standardization: scaling the data to the range of [0,1]. Z-score standardization: calculating the deviation of each data point from the mean and scaling it according to the standard deviation;

[0188] Further, after the nursing of the nursing object is completed, a physical examination report of the nursing object is obtained;

[0189] Furthermore, the physical examination report is quantified to obtain a physical examination data set; wherein, quantified physical examination data: the physiological indicators contained in the physical examination report are quantified to form a numerical data set. Common quantification methods include: converting classification indicators (such as health, sub-health, disease) into numerical values ​​(such as 0, 1, 2). Processing continuous data (such as blood pressure, blood sugar, etc.), directly using the measured value.

[0190] Further, the standard data are integrated with the physical examination data set to obtain a comprehensive data set;

[0191] Further, the comprehensive data set is input into the machine learning model to obtain the nursing assessment result; wherein, the decision tree model is selected as the machine learning model for nursing assessment in the embodiment of the present application; the specific process includes:

[0192] The decision tree model is fed with a comprehensive dataset that has been collected and organized. The dataset should include features relevant to the nursing assessment, such as vital signs, nursing notes, medication use, and physical examination results.

[0193] Furthermore, relevant features are extracted from the input data and the data format is ensured to meet the model requirements;

[0194] Furthermore, the input data is evaluated using a decision tree model, which will make predictions based on the extracted features and output nursing effect evaluation results. This result may be a classification result, such as "effective", "average" or "ineffective";

[0195] Furthermore, the output of the model is organized into a nursing evaluation report. The report should include the following: Evaluation results: describe the nursing effect of the nursing object (for example, the evaluation result is "effective"). Influencing factors: point out the main characteristics that affect the evaluation results to help nursing staff understand the decision-making basis of the model.

[0196] Furthermore, the feedback unit of the system feeds back the evaluation result of the evaluation unit to relevant personnel, corresponding to the above step S100.

[0197] In the embodiment of the present application, the nursing process of the patients in need of nursing in Hospital A is controlled mainly from the following aspects: first, an identity recognition model is used to identify the personnel in the monitoring area of ​​Hospital A; the model mainly identifies the personnel by analyzing the clothing features, action features, items carried by the personnel and the background features of the personnel's activities in the video, so as to accurately and quickly locate the nursing object; secondly, in the process of monitoring the nursing object, the nursing object is analyzed; including: whether the monitoring time point is the nursing time point; if it is the nursing time point, the corresponding nursing content is obtained, and the corresponding nursing person is notified in time staff; if the monitoring time point is not the nursing time point, the space, facial expressions and behavioral actions of the nursing object are monitored, and when an abnormality occurs, relevant personnel can be reminded in time; finally, a nursing effect evaluation method is used to evaluate the effect of the nursing object after completing the nursing task; this method collects various data of the nursing object in the nursing process as benchmark data; and combines the physical examination report data of the nursing object after completing the nursing task as the nursing effect; by combining the benchmark data and the physical examination report data, a machine learning model is used to analyze the nursing evaluation results; the results can be fed back to relevant personnel and combined with the electronic medical records of the nursing object as analysis data.

[0198] Embodiment 2:

[0199] In Example 1, the nursing process control of Hospital A is realized by the method and system proposed by the present invention; in order to further illustrate the effectiveness of the present invention, in Example 2, the nursing process control of the patient of Hospital B is carried out by the present invention, and the specific process includes:

[0200] Obtain surveillance videos within the permitted surveillance range of Hospital B;

[0201] Further, extracting the human area in the surveillance video;

[0202] Further, segmenting the character image and the moving background image in the character area;

[0203] Further, a plurality of the person images and the activity background image are input into an identity recognition model to obtain a care object set and a non-care object set;

[0204] Further, continuously monitoring the care objects in the care object set and acquiring facial images of the care objects;

[0205] Further, extracting facial features of the facial image using a facial recognition model;

[0206] Furthermore, the facial features are compared with real facial features of the nursing object pre-stored in a facial database to obtain the identity information of the nursing object;

[0207] Further, obtaining the electronic medical record, nursing plan and nursing staff of the nursing object according to the identity information;

[0208] Further, performing nursing analysis on the nursing object to obtain analysis results;

[0209] The analysis result includes: extracting the key nursing fields of the nursing plan; the key nursing fields include: nursing time point, nursing content and nursing equipment; the nursing time point and the nursing content are in a one-to-one correspondence; the nursing content and the nursing equipment are in a one-to-one correspondence;

[0210] Matching the current monitoring time of the nursing object with the nursing time point; if the current monitoring time is any nursing time point in the nursing plan; obtaining nursing content and nursing equipment according to the nursing plan;

[0211] If the current monitoring time is not any nursing time point in the nursing plan; performing state recognition on the nursing object; the state recognition includes: spatial state recognition, facial state recognition and behavioral state recognition;

[0212] Wherein, the spatial state recognition includes: acquiring the activity background image of the nursing object;

[0213] Performing layout recognition on the active background image; if the result of the layout recognition does not meet the nursing place requirement of the nursing object, obtaining a place abnormality;

[0214] The facial state recognition comprises: obtaining the facial image of the nursing object; inputting the facial image into a facial state recognition model to obtain a facial recognition result; if the facial recognition result is abnormal, obtaining a facial abnormality;

[0215] The behavior state recognition includes: obtaining the character images of the care object in multiple continuous time periods; pre-processing the character images and inputting them into a temporal convolutional network to obtain the action recognition results of the care object; when the action recognition results are abnormal, obtaining an action abnormality.

[0216] Further, reminding the nursing staff according to the analysis result;

[0217] Furthermore, various data of the nursing object during the monitoring process are recorded; wherein the various data include: nursing object behavior, vital signs, nursing staff operations and nursing equipment;

[0218] Furthermore, after the nursing is completed, the nursing effect of the nursing object is evaluated according to the physical examination report of the nursing object; wherein the specific process of evaluating the nursing object includes: preprocessing the data to obtain standard data; quantifying the physical examination report to obtain a physical examination data set; integrating the standard data with the physical examination data set to obtain a comprehensive data set; inputting the comprehensive data set into a machine learning model to obtain a nursing evaluation result;

[0219] Furthermore, the nursing assessment results are fed back to relevant personnel.

[0220] Embodiment three:

[0221] A nursing process control storage medium based on face recognition, wherein a nursing process control program based on face recognition is stored on the control storage medium, and when the nursing process control program based on face recognition is executed by a processor, all the nursing process control processes based on face recognition disclosed in embodiment one are implemented.

[0222] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A nursing process control method based on face recognition, characterized in that: include: Obtain surveillance videos within the hospital's permitted surveillance range; Extracting a person area in the surveillance video; Segmenting the character image and the moving background image in the character area; Input the person image and the activity background image into the identity recognition model to obtain a nursing object set and a non-nursing object set; continuously monitor the nursing objects in the nursing object set and obtain the facial images of the nursing objects; use the facial recognition model to extract the facial features of the facial images; compare the facial features with the real facial features of the nursing objects to obtain the identity information of the nursing objects; obtain the electronic medical records, nursing plans and nursing staff of the nursing objects according to the identity information; Performing nursing analysis on the nursing object to obtain analysis results; and reminding the nursing staff according to the analysis results; Record various data of the nursing object during the monitoring process; After the nursing is completed, the nursing effect is evaluated according to the physical examination report of the nursing object; The specific process of evaluating the nursing object includes: preprocessing the data to obtain standard data; quantifying the physical examination report to obtain a physical examination data set; integrating the standard data with the physical examination data set to obtain a comprehensive data set; inputting the comprehensive data set into a machine learning model to obtain a nursing evaluation result; Feedback the nursing assessment results to relevant personnel.

2. A nursing process control method based on face recognition according to claim 1, characterized in that: The identity recognition model includes: An input layer, used for receiving the character image and the activity background image; An image preprocessing layer, used for preprocessing the character image and the activity background image to obtain a standard character image and a standard activity background image; The identity feature extraction layer is used to extract features from the standard person image and the standard activity background image to obtain an initial feature set for identity recognition; A multi-identity feature weighting layer, used to perform item-wise weighting on the features in the initial identity recognition feature set to obtain an identity recognition weighted feature set; The weighting process of the multi-identity feature weighted layer specifically includes: collecting the performance of each identity recognition feature in the standard person image and the standard activity background image changing over time to obtain an identity recognition feature change data set; using a time series analysis method to perform importance analysis on each identity recognition feature in the identity recognition feature change data set to obtain an identity recognition feature importance weight; weighting the corresponding features in the identity recognition initial feature set according to the identity recognition feature importance weight to obtain the identity recognition weighted feature set; An identity recognition layer, used to perform identity recognition on each feature in the identity recognition weighted feature set to obtain multiple identity recognition probabilities; The identity recognition output layer is used to cross-check the multiple identity recognition probabilities and output the identity recognition results; wherein the identity recognition results include: care objects and non-care objects.

3. A nursing process control method based on face recognition according to claim 2, characterized in that: The image preprocessing layer comprises: The character image and the active background image are resized to obtain a first character image and a first active background image; the first character image and the first active background image are denoised to obtain a second character image and a second active background image; the second character image and the second active background image are data enhanced to obtain a third character image and a third active background image; static interference of the third character background image and the third active background image is removed to obtain the standard character image and the standard active background image.

4. The nursing process control method based on face recognition according to claim 2 is characterized in that: The identity feature extraction layer includes: using a convolutional neural network to extract clothing features of the standard character image; using a temporal convolutional network to extract character action features of multiple continuous time periods of the standard character images; using an object detection model to extract object features in the standard character image; using a target detection model to extract background features of the standard activity background image; integrating the clothing features, the character action features, the object features and the background features to obtain the initial feature set for identity recognition.

5. The nursing process control method based on face recognition according to claim 1 is characterized in that: The analysis results include: Extracting the key nursing fields of the nursing plan; the key nursing fields include: nursing time point, nursing content and nursing equipment; wherein the nursing time point and the nursing content are in a one-to-one correspondence; the nursing content and the nursing equipment are in a one-to-one correspondence; Matching the current monitoring time of the nursing object with the nursing time point; if the current monitoring time is any nursing time point in the nursing plan; obtaining nursing content and nursing equipment according to the nursing plan; If the current monitoring time is not any nursing time point in the nursing plan; performing state recognition on the nursing object; the state recognition includes: spatial state recognition, facial state recognition and behavioral state recognition; Wherein, the space state recognition includes: Performing layout recognition on the active background image; if the result of the layout recognition does not meet the nursing place requirement of the nursing object, obtaining a place abnormality; The facial state recognition comprises: obtaining the facial image of the nursing object; inputting the facial image into a facial state recognition model to obtain a facial recognition result; if the facial recognition result is abnormal, obtaining a facial abnormality; The behavior state recognition includes: obtaining the character images of the care object in multiple continuous time periods; pre-processing the character images and inputting them into a temporal convolutional network to obtain the action recognition results of the care object; when the action recognition results are abnormal, obtaining an action abnormality.

6. A nursing process control system based on face recognition, characterized in that: The system includes: a monitoring unit, an identity recognition unit, a face recognition unit, a data management unit, an abnormality recognition unit, an early warning unit, an evaluation unit and a feedback unit; wherein the monitoring unit is used to monitor within the permitted monitoring range in the hospital; the identity recognition unit is used to identify the nursing objects and non-nursing objects in the hospital; the face recognition unit is used to obtain the identity information of the nursing objects; the data management unit is used to store and manage the electronic medical records, nursing plans and related information of the nursing objects; the early warning unit is used to issue reminders and status identification to the nursing staff corresponding to the nursing objects; the abnormality recognition unit is used to identify various abnormal information of the nursing objects; the early warning unit is used to issue reminders to the nursing staff corresponding to the nursing objects; the evaluation unit is used to evaluate the nursing effect of the nursing objects; and the feedback unit is used to feedback the nursing evaluation results to relevant personnel.

7. A nursing process control system based on face recognition according to claim 6, characterized in that: The identity recognition unit adopts an identity recognition model to identify nursing objects and non-nursing objects, and the specific process includes: the monitoring unit obtains the monitoring video within the monitoring range allowed by the hospital; extracts the human area in the monitoring video; divides the human image and the activity background image in the human area; receives the human image and the activity background image by the input layer; inputs the human image and the activity background image into the image preprocessing layer to obtain a standard human image and a standard activity background image; inputs the standard human image and the standard activity background image into the identity feature extraction layer to obtain an initial feature set for identity recognition; inputs the initial feature set for identity recognition into a multi-identity feature weighted layer to obtain an identity recognition weighted feature set; inputs the identity recognition weighted feature set into the identity recognition layer to obtain multiple identity recognition probabilities; and inputs the multiple identity recognition probabilities into the identity recognition output layer to obtain an identity recognition result.

8. A nursing process control system based on face recognition according to claim 6, characterized in that: The abnormality recognition unit includes: spatial state recognition, facial state recognition and behavioral state recognition; Wherein, the spatial state recognition includes: acquiring an activity background image of the nursing object; Performing layout recognition on the active background image; if the result of the layout recognition does not meet the nursing place requirement of the nursing object, obtaining a place abnormality; The facial state recognition comprises: obtaining a facial image of the nursing object; inputting the facial image into a facial state recognition model to obtain a facial recognition result; if the facial recognition result is abnormal, obtaining a facial abnormality; The behavior state recognition includes: obtaining character images of the care object in multiple continuous time periods; pre-processing the character images and inputting them into a temporal convolutional network to obtain action recognition results of the care object; when the action recognition results are abnormal, obtaining an action abnormality.

9. A nursing process control system based on face recognition according to claim 6, characterized in that: The evaluation unit includes: obtaining various data in nursing; wherein the various data include: nursing object behavior, vital signs, nursing staff operations and nursing equipment; obtaining the physical examination report of the nursing object after the nursing of the nursing object is completed; performing data preprocessing on the various data to obtain standard data; quantifying the physical examination report to obtain a physical examination data set; integrating the standard data with the physical examination data set to obtain a comprehensive data set; inputting the comprehensive data set into a machine learning model to obtain a nursing evaluation result.

10. A nursing process control storage medium based on face recognition, characterized in that: The control storage medium stores a nursing process control program based on face recognition, and when the nursing process control program based on face recognition is executed by the processor, the nursing process control method based on face recognition as claimed in any one of claims 1 to 5 is implemented.

Images