Method and device for guiding operation of diagnosed patient based on medical indicator

Abstract

The invention discloses a method and device for conducting operation guiding on a diagnosed patient based on a medical indicator, the medical indicator comprises a camera and a display screen, the camera and the display screen are rotatably arranged, and the method comprises the steps that the patient serves as a monitoring target, and the camera is used for continuously collecting face images of the patient; performing sentiment analysis on the face image to obtain the emotional state of the patient; and performing man-machine interaction with the patient through a display screen according to the emotion state of the patient. The angle of the display screen can be dynamically adjusted through cooperation of the camera and the display screen which are rotatably arranged, so that the display screen always faces the face of the patient, the patient can conveniently check information on the display screen, the information transmission effect and accuracy are improved, meanwhile, emotion analysis is carried out on the face image of the patient, and the accuracy of information transmission is improved. Prompts and guidance can be given in time according to the emotion change of the patient or when operation problems occur, the patient is fully concerned and cared, the fear and unease of the patient are reduced, and the diagnosis efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and more specifically to a method and device for guiding the operation of a diagnosed patient based on a medical indicator. Background Art

[0002] With the continuous progress of technology, the medical field is also actively introducing advanced technologies to improve the quality and efficiency of medical services. In a medical environment, such as in scenarios like CT scans and radioactive chemotherapy, patients often have to face complex medical equipment and operation procedures alone, which may bring about emotions of tension, fear, and uneasiness to the patients. At the same time, medical staff cannot always accompany the patients to provide psychological counseling and operation guidance. Therefore, a solution that can replace medical staff, pay attention to the patients' emotions in real time, and provide medical operation instructions is needed.

[0003] Currently, in the medical field, there are already some similar technologies and products that have solved some problems to a certain extent. For example, the current traditional medical display signs guide the patients in diagnosis to perform corresponding operations by displaying fixed text, pictures, or video information. However, the position of the traditional medical display signs is fixed and cannot be adjusted according to the facial angle of the patient, resulting in patients with physical disabilities, unable to turn around or turn their heads, having difficulty seeing the information on the medical display signs, and also unable to give timely prompts and guidance according to the patients' emotional changes or when problems occur during the operation.

[0004] Although there are also some medical devices equipped with voice prompt systems that can provide simple voice guidance to patients during the operation process. However, this kind of voice prompt system is usually one-way and cannot be adjusted according to the patients' feedback. Moreover, the content of the voice prompt system is often relatively simple and cannot provide detailed operation guidance and psychological counseling.

[0005] In addition, the remote medical systems on the market allow medical staff to remotely monitor the situation of patients through the network and provide real-time guidance and suggestions. However, the remote medical systems need to rely on network connections and professional medical equipment, with high costs, and may not be able to be used normally in some areas with poor network conditions. Summary of the Invention

[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method and device for guiding the operation of a diagnosed patient based on a medical indicator, aiming to achieve personalized guidance and indication for patients during the diagnosis process, so as to improve the effect of information transmission and the accuracy of diagnosis guidance.

[0007] To achieve the above purpose, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a method for operating and guiding a diagnostic patient based on a medical indicator, the medical indicator including a camera and a display screen, the camera and the display screen being rotatably arranged, the method comprising:

[0009] Taking the patient as a monitoring target and continuously collecting facial images of the patient by using the camera;

[0010] Performing emotion analysis on the facial images to obtain the emotional state of the patient;

[0011] Performing human-computer interaction with the patient by using the display screen according to the emotional state of the patient.

[0012] Further, the taking the patient as a monitoring target and continuously collecting facial images of the patient by using the camera includes:

[0013] Obtaining the eye coordinates where the patient's eye position is located;

[0014] Setting the eye coordinates as the tracking target of the camera;

[0015] The display screen adjusts its angle according to the tracking situation of the camera for the tracking target, so that the display area of the display screen is always facing the patient's face directly.

[0016] Further, the obtaining the eye coordinates where the patient's eye position is located includes:

[0017] Performing human eye detection on the facial images to obtain the bounding box of the human eye;

[0018] Extracting human eye features from the bounding box of the human eye, the human eye features including the shape of the human eye, the texture of the human eye, and the eyeball color of the human eye;

[0019] Predicting the position of the human eye in subsequent images to obtain the eye coordinates.

[0020] Further, the display screen adjusts its angle according to the tracking situation of the camera for the tracking target, so that the display area of the display screen is always facing the patient's face directly, includes:

[0021] Obtaining the spatial relationship between the eye coordinates and the center of the display screen;

[0022] Calculating the target deflection angle required for the display screen to face the patient's face directly according to the spatial relationship, the target deflection angle including a horizontal deflection angle and a vertical deflection angle;

[0023] Driving the display to adjust its angular position according to the target deflection angle.

[0024] Further, the display screen adjusts its angle according to the tracking situation of the camera for the tracking target, so that the display area of the display screen is always facing the patient's face directly, further includes:

[0025] When the patient's face leaves the field of view that the camera can track, the angle of the display screen remains at the boundary position of the field of view.

[0026] Furthermore, performing emotion analysis on the facial image to obtain the patient's emotional state, including:

[0027] Annotating multiple key points of the facial image, where the key points include eyes, eyebrows, and head;

[0028] Extracting facial geometric features and facial dynamic features from the annotated key points, where the facial geometric features include the distance between the eyebrows, the direction of the eye gaze, and the head posture, and the facial dynamic features include the blink frequency and the intensity of facial expressions;

[0029] Concatenating the facial geometric features and the facial dynamic features into a multi-dimensional vector, and using a classification algorithm to output an emotion label to obtain the patient's emotional state.

[0030] Furthermore, performing human-computer interaction with the patient using the display screen according to the patient's emotional state, including:

[0031] Displaying the relevant operations of the diagnosis process on the display screen;

[0032] When the patient makes an operation error, the display screen immediately shows an error reminder and the method of correct operation;

[0033] When the patient's emotional state changes, playing voice and images that match the emotional state using the display screen according to the result of the emotional state.

[0034] In a second aspect, the present invention also provides a device for guiding the operation of diagnosing a patient based on a medical indicator, where the medical indicator includes a camera and a display screen, and the camera and the display screen are rotatably arranged. The device includes:

[0035] An acquisition unit, configured to use the camera to continuously acquire the facial image of the patient with the patient as the monitoring target;

[0036] An emotion analysis unit, configured to perform emotion analysis on the facial image to obtain the patient's emotional state;

[0037] A human-computer interaction unit, configured to perform human-computer interaction with the patient using the display screen according to the patient's emotional state.

[0038] In a third aspect, the present invention also provides a medical indicator, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the method for guiding the operation of diagnosing a patient based on the medical indicator as described above.

[0039] In a fourth aspect, the present invention further provides a computer-readable storage medium storing a computer program, the computer program including program instructions which, when executed by a processor, cause the processor to execute the method for guiding the operation of a diagnosed patient based on a medical indicator as described above.

[0040] The beneficial effects of the present invention compared with the prior art are as follows: For the method of guiding the operation of a diagnosed patient based on a medical indicator, the medical indicator includes a camera and a display screen, and the camera and the display screen are rotatably arranged. The method includes: taking the patient as a monitoring target and continuously collecting the facial images of the patient by using the camera; performing emotional analysis on the facial images to obtain the emotional state of the patient; and performing human-computer interaction with the patient by using the display screen according to the emotional state of the patient. The present invention utilizes the cooperation of the rotatably arranged camera and the display screen, enabling the angle of the display screen to be dynamically adjusted, so that it always faces the patient's face directly, facilitating special patients with inconvenient bodies, unable to turn around or twist their heads, etc. to view the information on the display screen, improving the effect and accuracy of information transmission. At the same time, by performing emotional analysis on the patient's facial images, it is possible to give timely prompts and guidance according to the patient's emotional changes or when problems occur during the operation, fully giving attention and care to the patient, reducing the patient's fear and uneasiness, and improving the diagnosis efficiency.

[0041] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other objects, features and advantages of the present invention more obvious and understandable, the following preferred embodiments are specifically given and described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 The structural schematic diagram of the medical indicator provided by the specific embodiment of the present invention;

[0044] Figure 2 The flowchart of the method for guiding the operation of a diagnosed patient based on the medical indicator provided by the specific embodiment of the present invention;

[0045] Figure 3 The schematic block diagram of the device for guiding the operation of a diagnosed patient based on the medical indicator provided by the specific embodiment of the present invention.

[0046] Figure 4Schematic block diagram of the medical indicator provided by the specific embodiment of the present invention.

[0047] Reference numeral

[0048] 1. Camera; 2. Display screen; 3. Connecting member; 4. Support column; 5. Base; 6. Steering gear. Detailed implementation manners

[0049] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0050] It should be understood that when used in this specification and the appended claims, the terms "comprises" and "comprising" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0051] It should also be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0052] It should be further understood that the term " / and / " used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] The embodiments of the present invention provide a method for operating and guiding a diagnosed patient based on a medical indicator. The medical indicator of the present application is mainly used in a medical diagnosis scenario to guide the patient. The diagnosis scenario can be a scenario such as CT scanning or radioactive chemotherapy using corresponding medical equipment.

[0054] The medical indicator can be independently placed in the diagnosis scenario or installed on the corresponding diagnosis device, such as installed on a CT scanning device or a radioactive chemotherapy device. The medical indicator includes a camera and a display screen, and the camera and the display screen can be rotatably arranged. The rotatable manner can be driven by a power component such as a motor or a cylinder.

[0055] In one embodiment, as Figure 1As shown, the medical indicator further includes a base, a support column, a connecting member, and a servo motor. The camera is integrated in the display screen, and the display screen is connected to the servo motor. Among them, the base, as a basic support component, has a flat plate structure with a large contact area. For example, it is made of a rectangular or circular metal plate material to increase the friction with the placement plane and ensure the stability of the medical indicator during use. A plurality of mounting holes are provided on the base, and they are connected to fixed surfaces such as the ground and the tabletop through bolts, or connected by magnetic attraction.

[0056] The support column is vertically installed on the base, and its material can be selected as high-strength aluminum alloy pipe, which has the characteristics of light weight and high strength. The height of the support column can be designed according to actual use requirements. For example, it is designed as a telescopic structure, and the height adjustment is achieved through the internally nested sleeve and the locking bolt.

[0057] The connecting member is used to connect the support column and the servo motor. One end of it is fixed to the top of the support column by welding or threaded connection, and the other end is closely connected to the servo motor housing. The shape and size of the connecting member are customized according to the specific structures of the support column and the servo motor. For example, an L-shaped or U-shaped metal connecting member is used to ensure the firmness and stability of the connection.

[0058] The servo motor, as the core component for controlling the rotation of the display screen, selects a model with high-precision angle control ability. For example, a micro servo motor that can achieve precise rotation of 0 - 180 degrees is selected. The display screen is fixedly connected to the output shaft of the servo motor through a bushing and a screw, so that the servo motor can drive the display screen to rotate in the horizontal or vertical direction. The camera is integrated inside the display screen. For example, a camera mounting groove is set at the border position of the display screen, and a small high-definition camera is embedded in it, so that the camera can rotate with the display screen and synchronously collect the facial images of the patient.

[0059] As Figure 2 shown, the method for operating and guiding the diagnosis of a patient based on the medical indicator includes the following steps: S10 - S30.

[0060] S10. Take the patient as the monitoring target and continuously collect the facial images of the patient using the camera.

[0061] Different types of cameras can be selected according to different needs. For example, a camera with high resolution, low noise, and good color restoration can be selected according to the needs to ensure that the facial features of the patient can be accurately captured.

[0062] In one embodiment, step S10 specifically includes the following steps: S101 - S103.

[0063] S101. Obtain the eye coordinates where the patient's eye position is located.

[0064] In one embodiment, S101 specifically includes the following steps: S1011 - S1013.

[0065] S1011. Perform human eye detection on the facial image to obtain the bounding box of the human eye.

[0066] After the medical indicator is turned on, the camera will start continuously collecting the facial images of the patient. Specifically, an eye detection algorithm can be used to preprocess the collected facial images, including operations such as image denoising, grayscale conversion, and contrast enhancement to improve the accuracy of human eye detection. Use the human eye detection algorithm to detect the position of the human eye in the image. The algorithm will search for areas in the image where the human eye may exist and determine the bounding box of the human eye. Specifically, the human eye detection algorithm can be a cascade classifier algorithm based on Haar features, a HOG+SVM classifier algorithm, or a deep learning-based algorithm such as MTCNN, YOLOv5, etc.

[0067] Taking the cascade classifier algorithm based on Haar features as an example, use Haar features to describe the features of objects in the facial image, and judge whether it is a human eye by calculating the Haar feature values of different regions. If the feature values conform to the feature pattern of the human eye, the window area may be determined as the human eye area, and then its bounding box is determined. This algorithm has a relatively fast calculation speed and can better meet the human eye detection requirements in some medical scenarios with high real-time requirements and relatively simple image backgrounds.

[0068] S1012. Extract human eye features from the bounding box of the human eye. The human eye features include the shape of the human eye, the texture of the human eye, and the color of the eyeball of the human eye.

[0069] Extract features from the detected bounding box of the human eye. The extracted features should be discriminative and robust, and can remain stable under different lighting, poses, and expressions.

[0070] For the shape of the human eye, it can be determined whether the human eye is approximately circular or elliptical by calculating the aspect ratio of the bounding box, ellipse fitting, etc. The texture extraction of the human eye can use algorithms such as local binary pattern (LBP), which can effectively describe the texture information of the human eye area, such as the folds of the eyelids. For the color of the eyeball of the human eye, the color of the human eye area can be statistically analyzed, such as calculating the mean and variance of each channel in the RGB color space to determine the approximate characteristics of the eyeball color.

[0071] S1013. Predict the position of the human eye in subsequent images to obtain the eye coordinates.

[0072] Select a suitable tracking algorithm. Predict the position of the human eye in subsequent facial images. After detecting the human eye, use the features and position information of the human eye to initialize the tracker and track the human eye in real time. In subsequent images, use the tracking algorithm to predict the position of the human eye according to the features and motion model of the human eye, and continuously update the state of the tracker.

[0073] The tracking algorithm can specifically be the Kalman filter algorithm, the particle filter tracking algorithm, the CSRT tracking algorithm, or the mean shift tracking algorithm, etc. Taking the Kalman filter algorithm as an example, predict the position of the human eye in subsequent images according to the position of the human eye in the current image and its previous motion state.

[0074] S102. Set the eye coordinates as the tracking target of the camera.

[0075] After obtaining the eye coordinates, transfer the coordinate information to the camera. The camera will adjust the shooting angle and focal length of the camera according to the eye coordinates so that the camera always aims at the position of the human eye.

[0076] S103. The display screen adjusts its angle according to the tracking situation of the camera for the tracking target, so that the display area of the display screen is always facing the patient's face directly.

[0077] In one embodiment, step S103 specifically includes the following steps: S1031 - S1033.

[0078] S1031. Obtain the spatial relationship between the eye coordinates and the center of the display screen.

[0079] During the process of the camera tracking the human eye, the spatial relationship between the eye coordinates and the center of the display screen is obtained in real time through the angle sensor and position sensor installed at the connection structure between the display screen and the camera. These sensors can accurately measure information such as the horizontal distance, vertical distance, and angular difference between the two.

[0080] S1032. Calculate the target deflection angle required for the display screen to face the patient's face directly according to the spatial relationship. The target deflection angle includes the horizontal deflection angle and the vertical deflection angle.

[0081] According to the obtained spatial relationship, use mathematical methods such as trigonometric functions to calculate the horizontal angle and vertical angle that the display screen needs to deflect. For example, through the known eye coordinates and the coordinates of the center of the display screen, the included angle between the line connecting the two and the initial position of the display screen can be calculated, so as to determine the target deflection angles in the horizontal and vertical directions.

[0082] S1033. Drive the display to adjust its angular position according to the target deflection angle.

[0083] According to the calculated target deflection angle, drive the servo installed between the display screen and the support structure to rotate, so as to realize the angle adjustment of the display screen. The servo has high precision and response speed, and can accurately adjust the display screen to the required angle. For example, when it is calculated that the display screen needs to deflect 15 degrees to the left and 10 degrees upward, the servo can quickly and accurately complete the corresponding angle adjustment to make the display screen face the patient's face directly.

[0084] In one embodiment, step S103 specifically includes the following steps: S1034.

[0085] S1034. When the patient's face leaves the field of view that the camera can track, the angle of the display screen remains at the boundary position of the field of view.

[0086] When the patient leaves the tracking field of view of the camera due to certain reasons (such as getting up, turning the head, etc.) during the examination, the angle position of the display screen at the moment when the patient's face leaves the field of view will be recorded and the angle will be kept unchanged. In this way, when the patient returns to the field of view of the camera again, the display screen can quickly return to the appropriate angle and continue to have effective human-computer interaction with the patient.

[0087] S20. Perform emotion analysis on the facial image to obtain the patient's emotional state.

[0088] In one embodiment, step S20 specifically includes the following steps: S201 - S203.

[0089] S201. Mark multiple key points of the facial image. The key points include eyes, eyebrows and head.

[0090] Use MTCNN to detect multiple key points of the face, and focus on marking the key points of eyes, eyebrows and head. Among them, the eyes are marked at the positions of the upper and lower eyelids and the corners of the eyes; the eyebrows are marked at the positions of the brow tips, the peaks of the eyebrows and the ends of the eyebrows; the head pose is marked at the positions of the tip of the nose, the chin and the temples. In addition, use the KCF tracker or the optical flow method to continuously track the key points to ensure real-time position update.

[0091] S202. Extract the facial geometric features and facial dynamic features from the marked key points. The facial geometric features include the distance between the two eyebrows, the direction of the eyesight and the head pose. The facial dynamic features include the blinking frequency and the intensity of facial expressions.

[0092] The extraction of facial geometric features is as follows:

[0093] Frowning detection: Calculate the vertical distance between the key points on the inner sides of the two eyebrows (such as point positions 19 and 24). If the distance shortens and the duration exceeds the threshold (such as 2 seconds), it is determined as frowning.

[0094] Eye direction: The gaze direction (horizontal deflection angle θ, vertical pitch angle φ) is calculated by the angle between the pupil center and the eye corner.

[0095] Head posture: Based on the PnP algorithm, the 3D facial model and 2D key points are used to calculate the head deflection angle (Yaw, Pitch, Roll).

[0096] Facial dynamic feature extraction is as follows:

[0097] Blinking frequency: Count the number of times the upper and lower eyelids are closed (distance < 5 pixels) per unit time. The normal range is 10-20 times / minute. A high frequency may indicate nervousness.

[0098] Micro-expression analysis: Extract facial muscle movement units (such as AU4: corrugator supercilii activity) and use FACS (Facial Action Coding System) to quantify the intensity of micro-expressions.

[0099] S203, concatenating facial geometric features and facial dynamic features into a multi-dimensional vector, and using a classification algorithm to output an emotion label to obtain the patient's emotional state.

[0100] The geometric features and dynamic features are spliced into a multidimensional vector. SVM or random forest is used to input the feature vector and output the emotional label (tension, relaxation, confusion, etc.) to determine the patient's emotional state.

[0101] S30. Performing human-computer interaction with the patient using a display screen according to the patient's emotional state.

[0102] In one embodiment, step S30 specifically includes the following steps: S301 - S303 .

[0103] S301, displaying the relevant operations of the diagnosis process on a display screen.

[0104] Before the diagnosis begins, the detailed operation steps and precautions of the diagnosis process are displayed on the display screen in the form of pictures and texts. For example, during a CT scan, the display screen will show the patient's body position and breathing requirements, and provide corresponding animation demonstrations to help patients better understand and cooperate with the diagnosis operation.

[0105] S302. When the patient makes an operation error, the display screen immediately displays an error reminder and the correct operation method.

[0106] When the patient operates according to the prompts on the display screen, the operation situation of the patient will be monitored in real time. If it is found that the patient makes an operation error, for example, during a liver MR scan, due to nervousness, the patient's breathing rhythm is disordered, and after the system detects three consecutive breath-holding interruptions, the display will automatically switch to the "breathing training mode": guiding the patient to practice synchronously through the dynamic breathing ball in the center of the screen, and at the same time superimposing voice and text prompts: "Please slowly inhale following the breathing ball → hold your breath until the red countdown ends → exhale evenly", to help the patient master the correct breath-holding method in a short time.

[0107] S303. When the emotional state of the patient changes, use the display screen to play voices and images that match the emotional state according to the result of the emotional state.

[0108] According to the emotional state of the patient obtained in step S20, the display screen will automatically adjust the displayed content. If the patient is in an anxious state, the display screen will play soothing music and images of warm natural scenery, and at the same time play soothing voices, such as "Please relax, the examination process is very safe, don't be nervous"; if the patient's emotion is relatively calm, the diagnostic-related information and operation guidance content will be normally displayed. For example, in pediatric examinations, when it is detected that the child is nervous, the display screen will play cartoon animations and lively music to relieve the child's fear.

[0109] Generally speaking, by accurately obtaining the eye coordinates of the patient and realizing the tracking of the camera and the angle adjustment of the display screen, it can ensure that the camera continuously captures clear facial images, the display screen is always facing the patient's face directly, improving the efficiency and quality of human-computer interaction, and enabling the patient to obtain diagnostic information more conveniently. By performing emotional analysis on the facial images, the emotional state of the patient can be understood in a timely manner, and personalized human-computer interaction can be carried out according to the emotional state, such as playing voices and images that match the emotion, which helps to relieve the patient's tension and anxiety, and enhance the comfort and cooperation of the patient during the diagnosis process. In addition, displaying diagnostic operation information on the display screen and timely reminding and correcting when the patient makes an operation error can reduce diagnostic errors caused by improper operations and improve the accuracy and efficiency of diagnosis.

[0110] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0111] The embodiment of the present invention also provides a device for guiding the operation of a diagnostic patient based on a medical indicator. The device for guiding the operation of a diagnostic patient based on a medical indicator is used to execute the steps in any one of the foregoing embodiments of the method for guiding the operation of a diagnostic patient based on a medical indicator. Specifically, please refer to Figure 3 , Figure 3FIG. 0 shows a schematic block diagram of a device 100 for operating guidance of a diagnostic patient based on a medical indicator. The device 100 for operating guidance of a diagnostic patient based on a medical indicator specifically includes:

[0112] An acquisition unit 110, configured to use a camera to continuously acquire facial images of a patient with the patient as a monitoring target. An emotion analysis unit 120, configured to perform emotion analysis on the facial images to obtain the emotional state of the patient. A human-computer interaction unit 130, configured to perform human-computer interaction with the patient using a display screen according to the emotional state of the patient.

[0113] In one embodiment, the acquisition unit 110 is specifically configured to: obtain the eye coordinates where the patient's eyes are located; set the eye coordinates as the tracking target of the camera; and the display screen adjusts its angle according to the tracking situation of the camera on the tracking target, so that the display area of the display screen is always facing the patient's face directly.

[0114] In one embodiment, the acquisition unit 110 is further specifically configured to: perform human eye detection on the facial images to obtain the bounding box of the human eyes; extract human eye features from the bounding box of the human eyes, where the human eye features include the shape of the human eyes, the texture of the human eyes, and the eyeball color of the human eyes; and predict the position of the human eyes in subsequent images to obtain the eye coordinates.

[0115] In one embodiment, the acquisition unit 110 is further specifically configured to: obtain the spatial relationship between the eye coordinates and the center of the display screen; calculate the target deflection angle required for the display screen to face the patient's face directly according to the spatial relationship, where the target deflection angle includes a horizontal deflection angle and a vertical deflection angle; and drive the display to adjust its angular position according to the target deflection angle.

[0116] In one embodiment, in one embodiment, the acquisition unit 110 is further specifically configured to: when the patient's face leaves the field of view that the camera can track, the angle of the display screen remains at the boundary position of the field of view.

[0117] In one embodiment, the emotion analysis unit 120 is specifically configured to: label multiple key points of the facial images, where the key points include eyes, eyebrows, and head; extract facial geometric features and facial dynamic features from the labeled key points, where the facial geometric features include the distance between the two eyebrows, the direction of the eye gaze, and the head pose, and the facial dynamic features include the blinking frequency and the intensity of facial expressions; splice the facial geometric features and the facial dynamic features into a multi-dimensional vector, and use a classification algorithm to output an emotion label to obtain the emotional state of the patient.

[0118] In one embodiment, the human-computer interaction unit 130 is specifically configured to: display relevant operations of the diagnosis process on the display screen; when the patient makes an operation error, immediately display an error reminder and the correct operation method on the display screen; when the patient's emotional state changes, play voice and images that match the emotional state according to the result of the emotional state by using the display screen.

[0119] It should be noted that those skilled in the art can clearly understand that the specific implementation processes of the above-mentioned device 100 for guiding the operation of diagnosing patients based on a medical indicator and each unit can refer to the corresponding descriptions in the foregoing method embodiments. For the convenience and brevity of description, they will not be elaborated here.

[0120] The above-mentioned device for guiding the operation of diagnosing patients based on a medical indicator can be implemented in the form of a computer program, and this computer program can run on a medical indicator as Figure 4 shown.

[0121] Please refer to Figure 4 , Figure 4 which is a schematic block diagram of a medical indicator provided by an embodiment of the present application.

[0122] As Figure 4 shown, the medical indicator includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the method steps of guiding the operation of diagnosing patients based on the medical indicator as described above.

[0123] The medical indicator 700 includes a processor 720, a memory, and a network interface 750 connected through a system bus 710. Among them, the memory may include a non-volatile storage medium 730 and an internal memory 740.

[0124] The non-volatile storage medium 730 can store an operating system 731 and a computer program 732. When the computer program 732 is executed, it can cause the processor 720 to execute the method of guiding the operation of diagnosing patients based on the medical indicator.

[0125] The processor 720 is used to provide computing and control capabilities to support the operation of the entire medical indicator 700.

[0126] The internal memory 740 provides an environment for the operation of the computer program 732 in the non-volatile storage medium 730. When the computer program 732 is executed by the processor 720, it can cause the processor 720 to execute the method of guiding the operation of diagnosing patients based on the medical indicator.

[0127] The network interface 750 is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that Figure 4The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the medical indicator 700 to which the solution of this application is applied. Specifically, the medical indicator 700 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements. Among them, the processor 720 is used to run the program code stored in the memory to implement a method for guiding the operation of diagnosing a patient based on the medical indicator.

[0128] Those skilled in the art can understand that Figure 4 The embodiments of the medical indicator shown do not constitute a limitation on the specific composition of the medical indicator. In other embodiments, the medical indicator may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements. For example, in some embodiments, the medical indicator may only include a memory and a processor. In such an embodiment, the structures and functions of the memory and the processor are the same as those of Figure 4 the embodiments shown and will not be elaborated here.

[0129] It should be understood that in the embodiments of this application, the processor 720 may be a central processing unit (CPU), and this processor 720 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0130] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method for guiding the operation of diagnosing a patient based on the medical indicator disclosed in the embodiments of the present invention.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the devices, apparatuses, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein. Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0132] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, or units with the same function can be aggregated into one unit. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the displayed or discussed couplings, direct couplings, or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices, or units, and can also be electrical, mechanical, or other forms of connection.

[0133] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present invention.

[0134] In addition, the functional units in each embodiment of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0135] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), magnetic disks, or optical discs that can store program codes.

[0136] As described above, the foregoing is only a specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for operating guidance of a diagnostic patient based on a medical indicator, characterized in that, The medical indicator includes a camera and a display screen, and the camera and the display screen are rotatably arranged. The method includes: Taking the patient as the monitoring target and continuously collecting the patient's facial images using the camera; Performing emotion analysis on the facial images to obtain the patient's emotional state; Performing human-computer interaction with the patient using the display screen according to the patient's emotional state.

2. The method for operating and guiding a diagnosed patient based on a medical indicator according to claim 1, wherein The step of taking the patient as the monitoring target and continuously collecting the patient's facial images using the camera includes: Obtaining the eye coordinates where the patient's eye position is located; Setting the eye coordinates as the tracking target of the camera; The display screen adjusts the angle according to the tracking situation of the camera on the tracking target, so that the display area of the display screen is always facing the patient's face directly.

3. The method for operating and guiding a diagnostic patient based on a medical indicator according to claim 2, characterized in that, The step of obtaining the eye coordinates where the patient's eye position is located includes: Performing human eye detection on the facial images to obtain the bounding box of the human eye; Extracting human eye features from the bounding box of the human eye, and the human eye features include the shape of the human eye, the texture of the human eye, and the eyeball color of the human eye; Predicting the position of the human eye in subsequent images to obtain the eye coordinates.

4. The method for operating and guiding a diagnosed patient based on a medical indicator according to claim 2, wherein The step that the display screen adjusts the angle according to the tracking situation of the camera on the tracking target, so that the display area of the display screen is always facing the patient's face directly includes: Obtaining the spatial relationship between the eye coordinates and the center of the display screen; Calculating the target deflection angles required for the display screen to face the patient's face directly according to the spatial relationship, and the target deflection angles include the horizontal deflection angle and the vertical deflection angle; Driving the display to adjust the angular position according to the target deflection angles.

5. The method for operating and guiding a diagnostic patient based on a medical indicator according to claim 2, wherein The step that the display screen adjusts the angle according to the tracking situation of the camera on the tracking target, so that the display area of the display screen is always facing the patient's face directly further includes: When the patient's face leaves the field of view that the camera can track, the angle of the display screen remains at the boundary position of the field of view.

6. The method for operating and guiding a diagnosed patient based on a medical indicator according to claim 1, characterized in that, The step of performing emotion analysis on the facial images to obtain the patient's emotional state includes: Annotating multiple key points of the facial images, and the key points include the eyes, eyebrows, and head; Extracting facial geometric features and facial dynamic features from the annotated key points, and the facial geometric features include the distance between the two eyebrows, the eye direction, and the head posture, and the facial dynamic features include the blink frequency and the facial expression intensity; Splicing the facial geometric features and the facial dynamic features into a multi-dimensional vector and using a classification algorithm to output an emotion label to obtain the patient's emotional state.

7. The method for operating and guiding a diagnostic patient based on a medical indicator according to claim 1, characterized in that, The step of performing human-computer interaction with the patient using the display screen according to the patient's emotional state includes: Displaying the relevant operations of the diagnosis process on the display screen; When the patient makes an operation error, the display screen immediately displays an error reminder and the method of correct operation; When the patient's emotional state changes, playing voice and images that match the emotional state using the display screen according to the result of the emotional state.

8. A device for guiding the operation of a diagnostic patient based on a medical indicator, characterized in that, The medical indicator includes a camera and a display screen, and the camera and the display screen are rotatably arranged. The device includes: An acquisition unit for taking the patient as the monitoring target and continuously collecting the patient's facial images using the camera; An emotion analysis unit for performing emotion analysis on the facial images to obtain the patient's emotional state; A human-computer interaction unit for performing human-computer interaction with a patient using a display screen according to the patient's emotional state.

9. A medical indicator, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the method for guiding the operation of diagnosing a patient based on a medical indicator as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method for guiding the operation of diagnosing a patient based on a medical indicator as described in any one of claims 1 to 7.

Images