Patient state auxiliary judgment and posture adjustment method, device and system

Abstract

The invention provides a patient state auxiliary judgment and posture adjustment method, device and system, and relates to the technical field of medical equipment. The method comprises the following steps: acquiring real-time state data of a patient; generating a three-dimensional posture model and emotion parameters of the current body posture of the patient based on the real-time state data; according to the difference degree of the preset target posture and the three-dimensional posture model, a posture adjustment instruction is generated in combination with a manual adjustment instruction of a doctor; and based on the posture adjustment instruction, prompting the patient to adjust the posture until the difference degree is smaller than a preset threshold value and the emotion parameters are in a target emotion range and lasts for a preset duration, and confirming that the patient completes examination preparation. Through the technical scheme provided by the invention, the problems that accurate state judgment is difficult to obtain and posture auxiliary adjustment cannot be performed on the patient before the patient is examined are solved, the accuracy of state perception of the patient and the efficiency of posture adjustment are improved, and the examination experience of the patient is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of medical devices, and particularly to a method, device and system for assisting in judging the state of a patient and adjusting the posture thereof. Background Art

[0002] When a patient is examined by a medical device, for example, when radiotherapy devices such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), and Single Photon Emission Computed Tomography (SPCT) are used for examination. The operator communicates with the person being examined and guides them to adjust their body position or posture to ensure that the scanning conditions are met and facilitate the normal progress of the scanning.

[0003] However, before the examination, the existing operators cannot fully and clearly explain all the matters that the patient should pay attention to during the examination, and further communication is required during the examination; during the examination, the operator or the system of the examination device issues instructions, such as holding the breath, but it cannot be observed whether the patient understands and cooperates correctly, nor is there any feedback; when the patient enters a large instrument for examination or treatment, especially when there is no one around, it is easy to cause claustrophobia, resulting in palpitation, mental tension, and involuntary body movement, which are not conducive to the imaging of the scanning device or the accurate treatment of the treatment device; when the patient moves during the examination or treatment in a large instrument, in addition to affecting the image quality or treatment accuracy, re-scanning or radiotherapy is required, which will greatly increase the radiation received by the patient.

[0004] In summary, in the related art, it is difficult to obtain an accurate state judgment before the patient undergoes the examination, and it is also impossible to assist in adjusting the posture of the patient. Summary of the Invention

[0005] The present disclosure aims to solve at least one of the technical problems in the related art to some extent.

[0006] The present disclosure provides a method, device, and system for assisting in judging a patient's state and adjusting the posture, so as to solve the problem that it is difficult to obtain an accurate state judgment before the patient undergoes an examination and it is also impossible to efficiently guide the patient to adjust the display posture. By collecting the patient's facial image data, body image data, bioelectrical signals, and audio data, analyzing the patient's current body posture and mood, combining the doctor's manual adjustment instructions, and using the preset target examination posture and mood to generate the patient's posture adjustment instructions, prompting the patient to adjust the examination posture until it is confirmed that the examination preparation is completed.

[0007] In a first aspect embodiment of the present disclosure, a method for assisting in judging a patient's state and adjusting the posture is proposed. The method includes:

[0008] Obtain the patient's real-time state data, where the real-time state data includes facial image data, body image data, bioelectrical signals, and audio data;

[0009] Based on the real-time state data, generate a three-dimensional body posture model and emotion parameters of the patient's current body posture;

[0010] According to the difference degree between the preset target posture and the three-dimensional body posture model, and combining the doctor's manual adjustment instructions, generate a posture adjustment instruction;

[0011] Based on the posture adjustment instruction, prompt the patient to adjust the posture until the difference degree is less than the preset threshold, and the emotion parameters are within the target emotion range and last for a preset duration, and confirm that the patient has completed the examination pre-preparation.

[0012] In an embodiment of the present disclosure, based on the real-time state data, generating a three-dimensional body posture model and emotion parameters of the patient's current body posture includes:

[0013] According to the body image data, detect the patient's body joint points and generate a three-dimensional model;

[0014] According to the facial image data, identify the facial expression parameters through a convolutional neural network;

[0015] According to the bioelectrical signals and audio data, determine the patient's tension parameter;

[0016] Based on the facial expression parameters and the tension parameter, generate emotion parameters.

[0017] In an embodiment of the present disclosure, according to the difference degree between the preset target posture and the three-dimensional body posture model, and combining the doctor's manual adjustment instructions, generating a posture adjustment instruction includes:

[0018] According to the difference degree of the spatial transformation between the preset target posture and the three-dimensional body posture model, generate a reference adjustment instruction, where the difference degree includes a translation component and a rotation component;

[0019] Analyze the priorities of manual adjustment instructions and reference adjustment instructions using a large language model;

[0020] Based on the priorities, fuse the manual adjustment instructions and the reference adjustment instructions into pose adjustment instructions according to the large language model.

[0021] In one embodiment of the present disclosure, based on the pose adjustment instructions, prompt the patient to adjust the pose, including:

[0022] Based on the pose adjustment instructions, disassemble them into multiple pose adjustment sub-steps through the large language model;

[0023] According to the multiple pose adjustment sub-steps, generate a picture sequence or video through a multi-modal large model and play it to the patient to prompt for pose adjustment;

[0024] In response to the patient's adjustment amplitude being greater than the upper threshold or less than the lower threshold, feedback whether the current pose is correct through a prompt message, and the prompt message includes sound, AR, and vibration signals.

[0025] In one embodiment of the present disclosure, after confirming that the patient has completed the pre-examination preparation, it further includes:

[0026] In response to the doctor's examination instructions, convey them to the patient through audio or video.

[0027] In one embodiment of the present disclosure, based on the real-time status data, generate a three-dimensional body posture model and emotional parameters of the patient's current body posture, and further include:

[0028] Use the real-time status data as the input of the multi-modal large model, and combine the patient's medical record data to generate a spatial description matrix of the patient's current body posture;

[0029] Use the spatial description matrix to construct a three-dimensional body posture model of the patient;

[0030] Generate the patient's emotional parameters according to the facial image data, bioelectrical signals, and audio data.

[0031] In a second aspect embodiment of the present disclosure, a device for assisting in judging the patient's state and adjusting the pose is proposed, and the device includes:

[0032] An acquisition module for acquiring the patient's real-time status data, and the real-time status data includes facial image data, body image data, bioelectrical signals, and audio data;

[0033] A first generation module for generating a three-dimensional body posture model and emotional parameters of the patient's current body posture based on the real-time status data;

[0034] A second generation module for generating pose adjustment instructions according to the difference between the preset target pose and the three-dimensional body posture model, in combination with the doctor's manual adjustment instructions;

[0035] An adjustment module, configured to prompt the patient to adjust the posture based on a posture adjustment instruction until the difference degree is less than a preset threshold, and the emotion parameter is within a target emotion range and lasts for a preset duration, so as to confirm that the patient has completed the pre - preparation for the examination.

[0036] The third - aspect embodiment of the present disclosure provides a patient state auxiliary judgment and posture adjustment system, which includes a bearing device, a support, a rotating device, an audio device, and a video device. Among them,

[0037] A head support device is installed on the bearing device. The head support device is used to support the patient's head and collect the patient's bio - electrical signals. The support is installed on the bearing device, and the installation method of the support is detachable installation or fixed installation. The rotating device is hinged to the support. The support is a rod - shaped structure, perpendicular to the plane of the bearing device. There is a column on the support, and the column is slidably installed on the support. An adjustment knob is provided on the support. When the column slides to a suitable position, the adjustment knob fixes the column on the support. The rotating device includes a rotating arm, an upper rotating shaft, and a lower rotating shaft. The support is slidably installed on the bearing device. There is a sliding groove on the bearing device, and the lower end of the support is clamped in the sliding groove;

[0038] The audio device is installed on the support. There are two audio devices, symmetrically arranged on the support. When the patient lies on the head support device, the audio devices are located on both sides of the patient's head and are used to collect the patient's audio data;

[0039] The video device is installed on the rotating device. The rotating arm is rotatably connected to the support through the lower rotating shaft; the video device is rotatably connected to the rotating arm through the upper rotating shaft. The rotating arm is a telescopic member, including a first sleeve and a second sleeve. There is a card slot on the first sleeve and a pressing protrusion on the second sleeve. After the first sleeve and the second sleeve are sleeved to a predetermined position, the pressing protrusion is stuck in the predetermined card slot. The video device is used to collect the patient's facial image data and body image data, generate a three - dimensional body posture model and an emotion parameter of the patient's current body posture based on the real - time state data. The real - time state data includes facial image data, body image data, bio - electrical signals, and audio data. According to the difference degree between the preset target posture and the three - dimensional body posture model, combined with the doctor's manual adjustment instruction, a posture adjustment instruction is generated. Based on the posture adjustment instruction, the patient is prompted to adjust the posture until the difference degree is less than a preset threshold, and the emotion parameter is within a target emotion range and lasts for a preset duration, so as to confirm that the patient has completed the pre - preparation for the examination.

[0040] In an embodiment of the present disclosure, the system further includes a control terminal, and the video device and the audio device are connected to the control terminal through a wireless connection method.

[0041] A fourth aspect embodiment of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of the first aspect embodiments of the present disclosure.

[0042] A fifth aspect embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to the first aspect embodiments of the present disclosure.

[0043] A sixth aspect embodiment of the present disclosure provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of the first aspect embodiments of the present disclosure.

[0044] A seventh aspect embodiment of the present disclosure provides a chip, including at least one processor and a communication interface; the communication interface is used to receive signals input to the chip or signals output from the chip, and the processor communicates with the communication interface and implements the method according to any one of the first aspect embodiments of the present disclosure through logic circuits or by executing code instructions.

[0045] In summary, according to the patient state assisted judgment and posture adjustment method proposed by the present disclosure, real-time state data of the patient is obtained. The real-time state data includes facial image data, body image data, bioelectrical signals, and audio data, providing a data source for the evaluation of the patient's body posture and emotional state; based on the real-time state data, a three-dimensional body posture model and emotional parameters of the patient's current body posture are generated, obtaining the patient's current body posture and emotional state; according to the difference degree between the preset target posture and the three-dimensional body posture model, combined with the doctor's manual adjustment instructions, a posture adjustment instruction is generated, providing an adjustment instruction for the patient's posture adjustment; based on the posture adjustment instruction, the patient is prompted to adjust the posture until the difference degree is less than the preset threshold, and the emotional parameters are within the target emotional range and last for a preset duration, confirming that the patient has completed the pre-examination preparation, improving the accuracy of patient state perception and the efficiency of posture adjustment, and enhancing the patient's examination experience.

[0046] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure and do not constitute an improper limitation to the present disclosure.

[0048] Figure 1Flowchart of a method for assisting in judging patient status and adjusting posture according to an embodiment of the present disclosure;

[0049] Figure 2 Flowchart of generating a three-dimensional body posture model and emotional parameters of a patient's current body posture based on real-time status data according to an embodiment of the present disclosure;

[0050] Figure 3 Flowchart of generating a posture adjustment instruction according to the difference between a preset target posture and a three-dimensional body posture model and combining with a doctor's manual adjustment instruction according to an embodiment of the present disclosure;

[0051] Figure 4 Flowchart of prompting a patient to adjust their posture based on a posture adjustment instruction according to an embodiment of the present disclosure;

[0052] Figure 5 Flowchart of generating a three-dimensional body posture model and emotional parameters of a patient's current body posture based on real-time status data according to an embodiment of the present disclosure;

[0053] Figure 6 Structural schematic diagram of a device for assisting in judging patient status and adjusting posture according to an embodiment of the present disclosure;

[0054] Figure 7 Schematic diagram of a system for assisting in judging patient status and adjusting posture according to an embodiment of the present disclosure;

[0055] Figure 8 Schematic diagram of the telescoping of an audio device and the rotation of a video device of a system for assisting in judging patient status and adjusting posture according to an embodiment of the present disclosure;

[0056] Figure 9 An application schematic diagram of a system for assisting in judging patient status and adjusting posture according to an embodiment of the present disclosure;

[0057] Figure 10 Block diagram of an electronic device for implementing the method for assisting in judging patient status and adjusting posture according to the present disclosure shown according to an exemplary embodiment;

[0058] Figure 11 Structural schematic diagram of a chip according to an embodiment of the present disclosure. Detailed implementation manners

[0059] The embodiments of the present disclosure are described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present disclosure, and should not be construed as a limitation of the present disclosure.

[0060] The present disclosure aims to solve the problems that it is difficult to obtain an accurate status judgment before a patient undergoes an examination and it is also impossible to assist in adjusting the posture of the patient. When a patient undergoes some physical examinations, especially radiological examinations, in order to obtain accurate examination results, the emotional state of the patient before the examination and the body posture during the examination will be adjusted until the requirements of the examination item are met. For example, when performing a CT scan examination on a patient's lungs, the doctor will require the patient to adopt a flat supine position, face up, and raise both arms above the head. However, the patient will be in a highly tense mood before the examination and it is difficult to quickly and intuitively understand and implement the doctor's requirements.

[0061] The method proposed in this disclosure is applied to the tasks of patient status assisted judgment and posture adjustment, and its application has rich scenarios. This technical solution realizes the precise assessment and adjustment of the patient's posture and emotion through multi-modal data fusion and intelligent analysis, and can be widely applied to the following scenarios and fields: In the field of medical and health, it can be used in scenarios such as imaging examination assistance (MRI / CT / X-ray, etc.), and can guide patients to adjust to the standard scanning position in real time (such as spinal alignment, joint angle), reducing image blurring or repeated scanning caused by improper posture. It can also combine emotion monitoring to relieve patients' anxiety (such as patients with claustrophobia), and improve the cooperation degree through voice prompts or adjustment suggestions. In the scenarios of preoperative preparation and intraoperative monitoring of surgery, automatically detect the patient's body position stability on the operating table (such as specific joint angles required in orthopedic surgery), and assist medical staff to optimize the fixation plan. Monitor the physiological signals and micro-expressions of anesthetized patients, and warn of sudden pain or abnormal physiological reactions. In the field of rehabilitation therapy and physical therapy, correct incorrect postures in exercise rehabilitation (such as traction movements for patients with lumbar disc herniation), dynamically adjust the training intensity in combination with emotion parameters, or provide real-time posture feedback for stroke patients to prevent secondary injuries caused by abnormal muscle tension. It can also be applied to the scenario of mental and psychological assessment, comprehensively evaluate the depressive and anxious states through facial expressions, intonation, and bioelectrical signals (such as skin conductance), and assist in formulating personalized intervention plans. In the fields of telemedicine and smart elderly care, especially in the scenarios of remote consultation and chronic disease management, it can guide home patients to complete standardized self-check actions (such as deep breathing postures during lung auscultation), ensuring the reliability of remote diagnosis data. It can also monitor the body position and emotion state of solitary elderly people after falling, automatically trigger first aid notifications and prompt adjustment to a safe position. In the intelligent monitoring scenario of nursing homes / nursing institutions, it can be used for the prevention of pressure sores in long-term bedridden patients, analyze the pressure distribution through body posture models, and regularly prompt nursing staff to assist in turning over. It can also be used for early warning of abnormal behaviors in dementia patients, and trigger interventions in combination with emotional fluctuations and abnormal actions (such as wandering, stiffness). It can also be applied to scenarios of special populations. For example, in the field of pediatric medicine, guide children to correct their examination postures (such as oral examinations) through animated interactions, and the emotion parameters trigger an appeasement mechanism (such as playing music). In the treatment of autistic children, quantify the body postures and emotional responses during social interactions (such as eye avoidance, limb tension). In the fields of sports medicine and fitness guidance, it can be used to correct the biomechanical deviations of fitness movements (such as knee valgus during squats), and avoid excessive load in combination with biological signals such as heart rate. It can also be used for the standardized management of postures in the rehabilitation training of athletes after injury, reducing the risk of compensatory movements. This technical solution can also be used in scenarios of the integration of cutting-edge technologies. For example, in virtual reality (VR) medical training, in virtual surgery teaching, it can evaluate the compliance of trainees' operation postures in real time and provide three-dimensional visual feedback on body posture deviations. In robot-assisted diagnosis and treatment, provide patient posture adjustment instructions for nursing robots (such as assisting in transferring the bed), and combine emotion recognition to avoid resistance caused by mechanical operations.It can also be used in the fields of digital therapy and metaverse health, reconstructing the patient's digital twin in the virtual diagnosis and treatment space, and optimizing the remote collaborative diagnosis and treatment process through multi-modal data. In addition, the technical solution of the present disclosure can also extend the application field to industrial safety, monitoring the fatigue postures (such as the body tilt during high-altitude operations) and stress emotions of high-risk operators, and giving early warnings in a timely manner. It can also be extended to education and training to optimize the sitting postures and attention states of students in the online education scenario (such as the spinal health management in remote classrooms). It can also be extended to the intelligent cockpit field of automobiles, adjusting the seat parameters in combination with the passenger's body posture and emotion, and preventing health risks during long-distance driving.

[0062] Through the closed-loop of precise perception (data layer), intelligent decision-making (algorithm layer), and user-friendly interaction (execution layer), this technology solves the inefficiency problem of relying on manual observation in traditional medicine, and is particularly suitable for scenarios that require high-precision body position control, are emotion-sensitive, or resource-constrained (such as remote / primary care), with both clinical value and commercial potential. In the embodiments of the present disclosure, the application scenarios are not limited.

[0063] The following will introduce in detail the patient state assisted judgment and posture adjustment method provided by the present disclosure with reference to the accompanying drawings.

[0064] Figure 1 It is a flowchart of a patient state assisted judgment and posture adjustment method according to an embodiment of the present disclosure. As Figure 1 shown in the embodiment, the patient state assisted judgment and posture adjustment method includes:

[0065] Step 101, obtaining the real-time state data of the patient, where the real-time state data includes facial image data, body image data, bioelectrical signals, and audio data.

[0066] In this embodiment, the real-time status data refers to the physiological and behavioral data of the patient collected instantaneously through sensors or devices. The facial image data refers to the visual information such as facial expressions and pupil changes captured by a camera. The body image data refers to the spatial information such as body contours and joint positions obtained through a depth camera or a 3D scanner. The bioelectrical signals refer to the signals reflecting the physiological status such as Electrocardiogram (ECG), Electromyogram (EMG), and Electrodermal Activity (EDA). The audio data refers to the acoustic features such as the pitch, speech rate, and volume of the patient's voice. Multimodal sensors (such as RGB-D cameras, wearable devices, and microphones) are used to synchronously collect the patient's facial expressions, body postures, physiological signals, and voice data, and are integrated into a unified time-series data set through data fusion technology. Through the method of this step, by combining visual, physiological, and voice data, the limitations of a single modality are avoided, providing continuous input for subsequent dynamic analysis and supporting timely feedback.

[0067] Step 102: Based on the real-time status data, generate a three-dimensional body posture model and emotion parameters of the patient's current body posture.

[0068] In this embodiment, the three-dimensional body posture model refers to a digital 3D human model reconstructed based on the body image data, with key bone points and joint angles marked. The emotion parameters refer to the emotion indicators (such as anxiety index, pain level) obtained through algorithmic quantitative analysis. After obtaining the real-time status data, a computer vision algorithm is used to convert the body image data into a three-dimensional model containing joint coordinates and limb angles. Optionally, the computer vision algorithm can select a skeleton tracking or point cloud registration algorithm. Or the emotion parameters of the patient are generated through facial expression recognition. Or, by combining facial expression recognition, speech emotion analysis, and bioelectrical signals (such as EDA and heart rate variability), an emotion score is output through a multimodal fusion model. The subjective body posture and emotion are converted into measurable objective parameters (such as "scoliosis 15°", "anxiety value 75 / 100"). The individual judgment deviation of traditional manual observation is avoided.

[0069] Step 103: Generate a posture adjustment instruction according to the difference between the preset target posture and the three-dimensional body posture model, in combination with the manual adjustment instruction of the doctor.

[0070] In this embodiment, the preset target posture refers to a standard body position predefined according to the inspection or treatment requirements (such as "lying on the back with both arms flat" for MRI examination). The degree of difference refers to the quantified value of the spatial deviation between the current body posture model and the target posture (such as Euclidean distance, joint angle difference). The manual adjustment instruction refers to additional correction suggestions input by the doctor based on clinical experience (such as "raise the left shoulder by 2 cm"). By comparing the three-dimensional body posture model with the preset target posture, the degree of difference (such as the offset of key points) is calculated. Combining the adjustment suggestions input by the doctor (such as priority setting, avoidance of prohibited actions), specific posture adjustment instructions (such as "rotate the head 5° to the left") are generated using a rule engine or a reinforcement learning algorithm. Preliminary instructions can be automatically generated through the above method, reducing the operation load of medical staff. And by combining doctor experience with algorithm calculation, the reliability and personalization of the adjustment plan are improved.

[0071] Step 104: Based on the posture adjustment instruction, prompt the patient to adjust the posture until the degree of difference is less than the preset threshold, and the emotion parameter is within the target emotion range and lasts for the preset duration, and confirm that the patient has completed the pre-examination preparation.

[0072] In this embodiment, the preset threshold refers to the upper limit of allowable body posture difference, such as the joint angle error ≤ 3°. The target emotion range refers to the interval of the emotion state required for the examination, such as the anxiety value ≤ 30 / 100. The preset duration refers to the time during which the emotion parameter needs to be stably maintained (such as meeting the standard continuously for 5 minutes). The patient is guided to gradually adjust the posture through vision, such as augmented reality (AR) projection; hearing (voice prompt) or touch (vibration feedback). The degree of difference and the emotion parameter are monitored in real time. If the following conditions are met simultaneously, it is determined that the adjustment is completed: the degree of difference < the preset threshold, and the emotion parameter is stable within the target range ≥ the preset duration. In this step, an automated loop of "detection → adjustment → verification" is realized, ensuring the reliability of the adjustment result and achieving dynamic closed-loop control. Through the requirement of emotional stability, it is avoided that the examination is interrupted or the data is distorted due to nervousness, and the examination experience of the patient is improved.

[0073] In summary, according to the patient status assisted judgment and posture adjustment method proposed by the present disclosure, real-time status data of the patient is obtained. The real-time status data includes facial image data, body image data, bioelectrical signals, and audio data, providing a data source for the evaluation of the patient's body posture and emotional state. Based on the real-time status data, a three-dimensional body posture model and emotional parameters of the patient's current body posture are generated, obtaining the patient's current body posture and emotional state. According to the difference degree between the preset target posture and the three-dimensional body posture model, combined with the manual adjustment instructions of the doctor, a posture adjustment instruction is generated, providing an adjustment instruction for the patient's posture adjustment. Based on the posture adjustment instruction, the patient is prompted to adjust the posture until the difference degree is less than the preset threshold, and the emotional parameters are within the target emotional range and continue for a preset duration, confirming that the patient has completed the pre-examination preparation, improving the accuracy of patient status perception and the efficiency of posture adjustment, and enhancing the patient's examination experience.

[0074] Figure 2 It is a flowchart for generating a three-dimensional body posture model and emotional parameters of the patient's current body posture based on real-time status data in an embodiment of the present disclosure. Figure 2 It is a further explanation of Figure 1 Step 102 of, based on Figure 2 The shown embodiment includes the following steps:

[0075] Step 201, according to the body image data, detect the patient's body joint points and generate a three-dimensional model.

[0076] The body joint points refer to the key connection points in the human skeleton, such as shoulders, elbows, hips, knees, etc., which are used to describe the limb positions and postures. The three-dimensional model refers to a digital human skeleton model reconstructed through joint point coordinates, including spatial information such as limb lengths and joint angles. According to the body image data, using computer vision algorithms, such as OpenPose, MediaPipe, or deep learning models, extract the joint point coordinates from the body image data. Combining multi-view images or depth sensor data, such as RGB-D cameras, generate a three-dimensional skeleton model containing joint rotations and displacements through the Inverse Kinematics (IK) algorithm. Convert the patient's posture into computable geometric parameters, such as "knee joint flexion angle 30°", providing an objective basis for subsequent adjustments. Through this step, the model can be updated in real time to adapt to the patient's minute movement changes and avoid the subjective errors of traditional manual measurements.

[0077] Step 202, according to the facial image data, identify the facial expression parameters through a convolutional neural network.

[0078] A Convolutional Neural Network (CNN) refers to a deep learning model that is good at extracting local features of images, such as edges and textures, and is commonly used in image classification and recognition. Facial expression parameters refer to metrics that quantify expression features, such as "the upward amplitude of the corners of the mouth" and "the intensity of frowning", and usually correspond to basic emotion types, such as joy and fear. According to the facial image data, data preprocessing is performed, including normalizing, denoising, and cropping key regions of the facial image, such as the eyes and mouth. Secondly, feature extraction and classification are carried out. Expression features are extracted through a pre-trained CNN model, such as VGG and ResNet, and probability distributions or parameter values are output, such as the intensity of Action Units (AU) based on the Facial Action Coding System (FACS). Through the method of the embodiment, interference such as light and occlusion can be overcome, and micro-expressions can be accurately captured, such as brief pain expressions. In addition, the parallel computing ability of CNN supports millisecond-level response, which is suitable for dynamic monitoring of scenarios and has high real-time performance.

[0079] Step 203: Determine the tension parameter of the patient according to the bioelectrical signal and audio data.

[0080] The tension parameter refers to a quantitative index that synthesizes physiological and speech characteristics, such as a 0-100 scale, which reflects the patient's psychological stress or physiological stress level. For bioelectrical signals, features are extracted after filtering and denoising, such as the rising slope of EDA and the low-frequency / high-frequency power ratio of Heart Rate Variability (HRV). For audio data, emotion-related acoustic features are extracted through Mel-scale Frequency Cepstral Coefficients (MFCC) or pitch analysis. Optionally, the two types of features are input into a regression model, such as a support vector machine or neural network, to output a tension score. Further, physiological signals and speech features are mutually complementary, such as the combination of increased skin conductance and voice tremor, to improve the reliability of tension assessment. The influence of single-signal noise is reduced through multi-source data, such as the interference of environmental noise on speech analysis.

[0081] Step 204: Generate an emotion parameter based on the facial expression parameter and the tension parameter.

[0082] The emotion parameter refers to a quantitative index that comprehensively combines facial expressions and tension levels, such as "anxiety index 60 / 100", which is used to describe the overall emotional state of a patient. Facial expression parameters, such as the intensity of frowning, and tension parameters, such as HRV values, are weighted and fused to achieve feature fusion. And through calibration experiments (such as comparison with clinical psychological scales), the mapping relationship of emotion parameters is established, such as "anxiety index > 70 requires intervention". In this step, by combining the explicit facial expressions with the internal physiological reactions, that is, tension, the one-sidedness of a single modality is avoided. The clinical interpretability is enhanced, and parameter calibration enables it to be directly used in medical decision-making, such as adjusting the examination time or giving soothing measures.

[0083] In this embodiment, based on the real-time state data of the patient, a three-dimensional body posture model and emotion parameters of the patient's current body posture are generated, obtaining the patient's current body posture and emotional state, which provides key data for posture adjustment and examination preparation.

[0084] Figure 3 It is a flowchart for generating a posture adjustment instruction according to the difference degree between a preset target posture and a three-dimensional body posture model, in combination with the manual adjustment instruction of a doctor. Figure 3 It is a further explanation of Figure 1 Step 103 of Figure 3 Shown in the embodiment, it includes the following steps:

[0085] Step 301, according to the difference degree of the spatial transformation between the preset target posture and the three-dimensional body posture model, generate a reference adjustment instruction, and the difference degree includes a translation component and a rotation component.

[0086] The translation component refers to the linear displacement deviation between the patient's current posture and the target posture in three-dimensional space, such as "shift 2 cm to the right". The rotation component refers to the joint or overall rotation angle deviation between the patient's current posture and the target posture, such as "the torso rotates 5° around the Y axis". The spatial transformation difference between the three-dimensional body posture model and the preset target posture is disassembled into translation (X / Y / Z axis displacements) and rotation (represented by Euler angles or quaternions) components. According to the direction and amplitude of the components, a preliminary adjustment instruction is generated, such as "translate 3 cm to the left" or "reduce the pelvic anterior tilt angle by 10°". In this step, the translation and rotation deviations are analyzed independently to avoid the adjustment ambiguity caused by compound errors. At the same time, the component-based output is convenient for medical staff to understand the adjustment logic. For example, the rotation deviation is preferably resolved first.

[0087] Step 302, use a large language model to analyze the priority of the manual adjustment instruction and the reference adjustment instruction.

[0088] A large language model (LLM) refers to a natural language processing model based on deep learning that can understand text semantics and infer logical relationships, such as GPT, DeepSeek, and PaLM. Priority refers to the order or weight of instruction execution, which is determined by factors such as clinical rules and patient safety. Convert manual adjustment instructions (such as "adjust the head position first" dictated by a doctor) and reference adjustment instructions (such as "translate the left arm 5 cm" generated by the system) into structured text. Combine the LLM with a medical knowledge base, such as surgical specifications and a library of prohibited actions, to judge the urgency and rationality of instructions. For example, "avoid excessive neck rotation" takes precedence over "adjust the leg angle" for priority evaluation. Convert doctor experience into computable priority weights to avoid conflicts between algorithms and manual decisions. At the same time, support instruction sorting in complex scenarios, such as prioritizing postures related to stabilizing vital signs during first aid.

[0089] Step 303: Based on the priority, fuse the manual adjustment instruction and the reference adjustment instruction into a posture adjustment instruction according to the large language model.

[0090] Perform weighted fusion of the manual adjustment instruction and the reference adjustment instruction according to the priority to obtain a posture adjustment instruction. For example, a high-priority instruction overrides a conflicting low-priority item. Generate natural language or visual instructions through the LLM, such as "Please slowly raise the right arm to 30° while keeping the head centered". Human-machine collaborative optimization: Retain the advantages of algorithm accuracy and doctor experience to generate a safe and feasible adjustment plan. And reduce repeated adjustments caused by conflicting instructions, such as prioritizing the handling of key deviations and then optimizing the details.

[0091] In this embodiment, through the process of spatial difference decomposition → semantic priority analysis → multi-source instruction fusion, the geometric deviation of the target posture is decomposed into translation and rotation components to generate reference instructions, and the clinical intention of the manual instruction is parsed using the large language model, and finally an adjustment instruction that takes into account both algorithm accuracy and medical experience is output.

[0092] Figure 4 This is a flowchart of a posture adjustment instruction based on an embodiment of the present disclosure, which prompts the patient to adjust the posture. Figure 4 It is a specific description of Figure 1 Step 104 of Figure 4 The shown embodiment includes the following steps:

[0093] Step 401: Based on the posture adjustment instruction, disassemble it into multiple posture adjustment sub-steps through the large language model.

[0094] Posture adjustment in steps means decomposing complex adjustment instructions into sub - actions to be executed sequentially. For example, "first raise the right arm to 45° → then rotate the torso 10° to the left". By using the LLM to analyze the semantic logic of the posture adjustment instructions and combining with the patient's physical ability model, such as the joint range of motion limit library, the instructions are disassembled into an ergonomic step - by - step sequence that can be independently executed. Through posture adjustment in steps, the execution difficulty is reduced, complex actions are decomposed into simple steps, and it avoids the failure of adjustment caused by the patient's misunderstanding. Dynamically adjust the step - by - step sequence or amplitude according to the individual differences of patients. For example, the elderly have poor flexibility, so their posture adjustment steps are more, thus realizing adaptive optimization.

[0095] Step 402: According to multiple posture adjustment steps, generate a picture sequence or video through a multimodal large - model and play it to the patient to prompt for posture adjustment.

[0096] A multimodal large - model refers to an AI model that supports cross - modal (text → image / video) generation and reasoning, such as GPT - 4V, Sora. The picture sequence / video is used to display the visual guidance content of the step - by - step actions, such as dynamically demonstrating "the transition process from sitting position to supine position". For multimodal generation, the step - by - step text description can be input, and the corresponding action demonstration images or videos (such as 3D animations) can be generated through the multimodal large - model. Thus, the playback content can be switched in real - time according to the patient's adjustment progress (such as looping the current step until completion). This visual demonstration reduces the understanding threshold of language or text, especially suitable for children or patients with language disorders. Moreover, the video can show continuous action details (such as the way of muscle exertion), improving the adjustment accuracy.

[0097] Step 403: In response to the patient's adjustment amplitude being greater than the upper threshold or less than the lower threshold, feedback whether the current posture is correct through prompt information, and the prompt information includes sound, AR, and vibration signals.

[0098] The upper threshold / lower threshold refers to the allowable adjustment amplitude range, such as "the knee joint bending angle needs to be between 20° - 40°". Optionally, AR is used to overlay virtual guides (such as arrows, highlighted areas) on the real - world scene through a headset or screen. The vibration signal refers to the tactile feedback triggered by a wearable device (such as a smart bracelet) (such as continuous vibration indicating an error).

[0099] Compare the patient's current posture with the step - by - step target posture in real - time and calculate the deviation value. If the deviation exceeds the threshold, prompt through sound "Please stop, the right arm is raised too high", mark the wrong part with AR (red highlight) or give a vibration warning. If the deviation is within the threshold, encourage the patient to continue through positive feedback (such as a green AR mark). Through the feedback method of this step, it prevents the patient from being injured due to excessive adjustment (such as joint hyperextension). It can also adapt to the perception preferences of different patients (such as hearing - impaired patients relying on AR / vibration).

[0100] In this embodiment, through a closed-loop process of instruction decomposition, visual guidance, and real-time feedback, complex posture adjustment instructions are transformed into step-by-step actions. With the help of a multimodal large model, intuitive pictures or videos are generated to guide the patient to execute step by step, and the adjustment amplitude is monitored in real time through sound, AR, or vibration signals to ensure that the action is completed within the safety threshold. Improve patient compliance and operation accuracy (especially for non-professionals), reduce the risk of secondary injuries caused by incorrect actions, and enhance the flexibility and inclusiveness of human-computer interaction.

[0101] In one embodiment of the present disclosure, after Figure 1 step 104, it further includes: in response to the patient's examination preparation being completed, conveying the doctor's examination instructions to the patient through audio or video.

[0102] In this embodiment, after the patient's examination preparation is completed, the doctor's examination instructions can be sent to the patient in the form of audio / video to continue with other matters of the examination. Thus, the interaction between the patient and the doctor through audio / video data is realized. It is convenient for the doctor to master the patient's situation and thus issue appropriate examination instructions.

[0103] Figure 5 It is a flowchart for generating a three-dimensional body posture model and emotion parameters of the patient's current body posture based on real-time state data in an embodiment of the present disclosure. Figure 5 It is a specific description of Figure 1 step 102, based on the Figure 5 embodiment shown, including the following steps:

[0104] Step 501, taking the real-time state data as the input of the multimodal large model, and combining the patient's medical record data to generate a spatial description matrix of the patient's current body posture.

[0105] The spatial description matrix refers to a multi-dimensional mathematical matrix that contains spatial parameters such as the coordinates of the patient's joint points, the orientation of the limbs, and the joint angles, and is used to quantitatively describe the body posture. Multimodal input fusion means inputting real-time state data, such as body images and bioelectrical signals, and medical record data, such as previous surgical history and joint movement limitation records, into a multimodal large model, such as CLIP and UniT. Through the model, spatio-temporal features are extracted, and a spatial description matrix that integrates the patient's individual characteristics (such as the impact of scar tissue on joint movement) is output. In this step, the general model deviation is corrected by combining medical record data (such as considering the impact of the patient's joint degenerative disease on the posture) for personalized modeling. Among them, the spatial description matrix is compatible with the multi-dimensional analysis requirements of subsequent algorithms (such as deep learning and physics engines).

[0106] Step 502, using the spatial description matrix to construct a three-dimensional body posture model of the patient.

[0107] Based on the parameters in the spatial description matrix, such as joint coordinates and bone lengths, a three-dimensional visualization model including surface meshes and skeletons is generated through computer graphics algorithms, such as skinning technology and inverse kinematics. Ensure that the model can render real-time changes in the patient's posture (such as the movement trajectory during rehabilitation training). Optionally, the model supports importing medical imaging systems, such as DICOM or AR / VR devices, for multi-scenario applications.

[0108] Step 503, generate the patient's emotional parameters based on the facial image data, bioelectrical signals, and audio data.

[0109] For the facial image data, micro-expressions (such as downturned corners of the mouth and frowning) are recognized through CNN. For the bioelectrical signals, the stress responses of skin conductance and heart rate variability are analyzed for correlation. For the audio data, acoustic features such as voice tremor and intonation changes are extracted. By weighted integration of the three types of data features, comprehensive emotional parameters are output, such as "pain index: 65 / 100". The multi-modal data complement each other, reducing misjudgment of single-modal data. For example, a patient smiles to conceal pain, providing a dynamic emotional baseline for medical decision-making, such as monitoring anxiety fluctuations during the anesthesia recovery period.

[0110] In this embodiment, through the process of generating a spatial description matrix, three-dimensional body posture modeling, and multi-source emotional parameter calculation by multi-modal data fusion, the patient's real-time physiological data is combined with the historical medical records to construct a high-precision and personalized digital twin body posture model, and its emotional state is quantified synchronously. It improves the individual adaptability of complex posture analysis, such as considering the limitations of the patient's medical history on movements, realizing the correlation analysis of emotion-body posture, such as involuntary body position offset caused by pain, and enhancing the comprehensiveness and predictability of clinical diagnosis and treatment.

[0111] Corresponding to the methods provided in the above several embodiments, the present disclosure also provides a device for assisting in judging the patient's state and adjusting the posture. Since the device provided in the embodiments of the present disclosure corresponds to the methods provided in the above several embodiments, the implementation manners of the methods are also applicable to the device provided in this embodiment and will not be described in detail in this embodiment.

[0112] Figure 6 It is a schematic structural diagram of a device 600 for assisting in judging the patient's state and adjusting the posture according to an embodiment of the present disclosure. As Figure 6 shown, the device for assisting in judging the patient's state and adjusting the posture includes:

[0113] An acquisition module 610, configured to acquire the patient's real-time state data, where the real-time state data includes facial image data, body image data, bioelectrical signals, and audio data;

[0114] A first generation module 620, configured to generate a three-dimensional body posture model and emotional parameters of the patient's current body posture based on the real-time state data;

[0115] A second generation module 630, configured to generate a posture adjustment instruction according to the difference degree between a preset target posture and a three-dimensional body posture model, in combination with the manual adjustment instruction of a doctor;

[0116] An adjustment module 640, configured to prompt a patient to adjust the posture based on the posture adjustment instruction until the difference degree is less than a preset threshold, and the emotion parameter is within a target emotion range and lasts for a preset duration, and confirm that the patient has completed the pre-examination preparation.

[0117] In some embodiments, the first generation module 620 is configured to:

[0118] Detect the body joint points of the patient and generate a three-dimensional model according to the body image data;

[0119] Identify the facial expression parameters through a convolutional neural network according to the facial image data;

[0120] Determine the tension parameter of the patient according to the bioelectrical signal and the audio data;

[0121] Generate an emotion parameter based on the facial expression parameter and the tension parameter.

[0122] In some embodiments, the second generation module 630 is configured to:

[0123] Generate a reference adjustment instruction according to the difference degree of the spatial transformation between the preset target posture and the three-dimensional body posture model, and the difference degree includes a translation component and a rotation component;

[0124] Analyze the priority of the manual adjustment instruction and the reference adjustment instruction by using a large language model;

[0125] Based on the priority, fuse the manual adjustment instruction and the reference adjustment instruction into a posture adjustment instruction according to the large language model.

[0126] In some embodiments, the adjustment module 640 prompts the patient to adjust the posture based on the posture adjustment instruction in the following manner:

[0127] Decompose the posture adjustment instruction into multiple posture adjustment sub-steps by using a large language model based on the posture adjustment instruction;

[0128] Generate a picture sequence or a video according to the multiple posture adjustment sub-steps through a multi-modal large model and play it to the patient to prompt the patient to adjust the posture;

[0129] In response to the patient's adjustment amplitude being greater than an upper limit threshold or less than a lower limit threshold, feedback whether the current posture is correct through a prompt message, and the prompt message includes sound, AR, and vibration signals.

[0130] In some embodiments, after confirming that the patient has completed the pre-examination preparation, the adjustment module 640 is further configured to:

[0131] In response to the doctor's examination instruction, it is conveyed to the patient through audio or video.

[0132] In some embodiments, the first generation module 620 is configured to:

[0133] Take the real-time status data as the input of the multimodal large model, and combine the medical record data of the patient to generate a spatial description matrix of the patient's current body posture;

[0134] Use the spatial description matrix to construct a three-dimensional body posture model of the patient;

[0135] Generate the emotional parameters of the patient according to the facial image data, bioelectrical signals and audio data.

[0136] In summary, through the patient status assisted judgment and posture adjustment device, the real-time status data of the patient is obtained, and the real-time status data includes facial image data, body image data, bioelectrical signals and audio data; based on the real-time status data, a three-dimensional body posture model and emotional parameters of the patient's current body posture are generated; according to the difference degree between the preset target posture and the three-dimensional body posture model, combined with the doctor's manual adjustment instruction, a posture adjustment instruction is generated; based on the posture adjustment instruction, the patient is prompted to adjust the posture until the difference degree is less than the preset threshold, and the emotional parameters are within the target emotional range and continue for the preset duration, and it is confirmed that the patient has completed the pre-examination preparation.

[0137] This device solves the problems that it is difficult to obtain an accurate status judgment before the patient undergoes an examination and it is impossible to assist the patient in posture adjustment, improves the accuracy of patient status perception and the efficiency of posture adjustment, and enhances the patient's examination experience.

[0138] Figure 7 It is a schematic diagram of a patient status assisted judgment and posture adjustment system according to an embodiment of the present disclosure. As Figure 7 shown, the system includes a carrying device 24, a support 23, a rotating device, an audio device 20, and a video device 10. Among them,

[0139] A head support device 25 is installed on the carrying device 24. The head support device 25 is used to carry the patient's head and collect the bioelectrical signals of the patient. The support 23 is installed on the carrying device 24, and the installation method of the support is detachable installation or fixed installation. The rotating device is hinged on the support 23. The support 23 is a rod-shaped structure, perpendicular to the plane of the carrying device 24. There is a column 22 on the support 23. The column 22 is slidably installed on the support 23. There is an adjustment knob 21 on the support 23. When the column 22 slides to the appropriate position, the adjustment knob 21 fixes the column 22 on the support 23. The rotating device includes a rotating arm 12, an upper rotating shaft 11, and a lower rotating shaft 13. The support 23 is slidably installed on the carrying device. There is a sliding groove on the carrying device. The lower end of the support 23 is clamped in the sliding groove;

[0140] The audio device 20 is installed on the support 23. There are two audio devices 20, which are symmetrically arranged on the support 23. When the patient lies on the head support device 25, the audio devices 20 are located on both sides of the patient's head and are used to collect the patient's audio data.

[0141] The video device 10 is installed on the rotating device. The rotating arm 12 is rotatably connected to the support 23 through the lower rotating shaft 13; the video device 10 is rotatably connected to the rotating arm 12 through the upper rotating shaft 11. The rotating arm 12 is a telescopic member. The rotating arm 12 includes a first sleeve and a second sleeve. There is a card slot on the first sleeve, and a pressing protrusion is provided on the second sleeve. After the first sleeve and the second sleeve are sleeved to a predetermined position, the pressing protrusion is stuck in the predetermined card slot. The video device 10 is used to collect the patient's facial image data and body image data, generate a three-dimensional body posture model and emotion parameters of the patient's current body posture based on the real-time state data. The real-time state data includes facial image data, body image data, bioelectrical signals and audio data. According to the difference between the preset target posture and the three-dimensional body posture model, combined with the doctor's manual adjustment instruction, a posture adjustment instruction is generated. Based on the posture adjustment instruction, the patient is prompted to adjust the posture until the difference is less than the preset threshold, and the emotion parameter is within the target emotion range and lasts for a preset duration, and it is confirmed that the patient has completed the pre-examination preparation.

[0142] In an embodiment of the present disclosure, the patient state auxiliary judgment and posture adjustment system further includes a control terminal. The video device and the audio device are connected to the control terminal through a wireless connection method.

[0143] In an embodiment of the present disclosure, the patient state judgment and posture adjustment system further includes a control terminal. The video device 10 and the audio device 20 are connected to the control terminal through a wireless connection method.

[0144] In the present disclosure, the audio device 20 and the video device 10 are arranged on the device that can move and carry the patient, and are close to the patient's head position, so that the patient and the audio device and the video device move together, ensuring that the relative position with the patient remains stable during the entire examination process and ensuring the consistency of the sound transmission effect.

[0145] Both the audio device 20 and the video device 10 can be connected to the control terminal. The doctor can visually see the patient's state through the video and explain the examination process, requirements and precautions to the patient in detail. The patient can also clearly see the doctor's instruction actions and expressions, cooperate with the examination better, and improve the communication efficiency and the accuracy of the examination.

[0146] Figure 8 It is a schematic diagram of the telescoping of the audio device and the rotation of the video device of a patient state auxiliary judgment and posture adjustment system according to an embodiment of the present disclosure. As Figure 8As shown, the audio device 20 can be telescopically extended and retracted in the vertical direction of the carrying device 24, facilitating the adjustment of the height of the microphone for collecting sounds according to the head heights of different patients. The image acquisition device provided in the video device 10 can acquire the facial and body image data of the patient. By rotating the screen, it can be adjusted to a position where the user is comfortable and can clearly view.

[0147] Figure 9 This is an application schematic diagram of the patient status auxiliary judgment and posture adjustment system according to an embodiment of the present disclosure. As Figure 8 shown, the doctor in the operating room interacts with the patient in the treatment room or examination room through audio and video via the software control system. The audio and video are processed and transmitted through the interaction system, where the audio and video support wireless transmission.

[0148] In the above embodiments provided by the present disclosure, the methods and devices provided by the embodiments of the present disclosure are introduced. To implement each function in the methods provided by the above embodiments of the present disclosure, the electronic device may include a hardware structure, software modules, and implement the above functions in the form of a hardware structure, software modules, or a combination of a hardware structure and software modules. A certain function among the above functions can be executed in the manner of a hardware structure, software module, or a combination of a hardware structure and software module.

[0149] Figure 10 This is a block diagram of an electronic device 1000 for implementing the above patient status auxiliary judgment and posture adjustment method shown according to an exemplary embodiment.

[0150] For example, the electronic device 1000 can be a mobile phone, computer, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0151] Referring to Figure 10 , the electronic device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0152] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0153] The memory 1004 is configured to store various types of data to support the operation of the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, videos, and the like. The memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0154] The power supply component 1006 provides power to various components of the electronic device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 1000.

[0155] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0156] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 1000 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 1004 or transmitted via the communication component 1016. In some embodiments, the audio component 1010 further includes a speaker for outputting audio signals.

[0157] The I / O interface 1012 provides an interface between the processing component 1002 and the peripheral interface module, and the peripheral interface module may be a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a power button, and a lock button.

[0158] The sensor component 1014 includes one or more sensors for providing an assessment of the status of various aspects of the electronic device 1000. For example, the sensor component 1014 can detect the on / off state of the electronic device 1000, the relative positioning of components, such as the display and keypad of the electronic device 1000. The sensor component 1014 can also detect a change in the position of the electronic device 1000 or a component of the electronic device 1000, the presence or absence of user contact with the electronic device 1000, the orientation or acceleration / deceleration of the electronic device 1000, and a change in the temperature of the electronic device 1000. The sensor component 1014 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 1014 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 1014 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0159] The communication component 1016 is configured to facilitate communication between the electronic device 1000 and other devices in a wired or wireless manner. The electronic device 1000 can access a wireless network based on communication standards, such as WiFi, 2G or 3G, 4G LTE, 5G NR (New Radio), or a combination thereof. In an exemplary embodiment, the communication component 1016 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0160] In an exemplary embodiment, the electronic device 1000 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0161] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as the memory 1004 including instructions, and the above instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method. For example, the non-transitory computer-readable storage medium can be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0162] An embodiment of the present disclosure also proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the patient state assisted judgment and posture adjustment method described in the above embodiments of the present disclosure.

[0163] An embodiment of the present disclosure also proposes a computer program product, including a computer program, and the computer program executes the patient state assisted judgment and posture adjustment method described in the above embodiments of the present disclosure when being executed by a processor.

[0164] Figure 11 FIG. is a schematic structural diagram of a chip 1100 for implementing the above patient state assisted judgment and posture adjustment method according to an exemplary embodiment.

[0165] Referring to Figure 11 , the chip 1100 includes at least one communication interface 1101 and a processor 1102; the communication interface 1101 is used to receive signals input to the chip 1100 or signals output from the chip 1100, and the processor 1102 communicates with the communication interface 1101 and implements the patient state assisted judgment and posture adjustment method described in the above embodiments through logic circuits or by executing code instructions.

[0166] It should be noted that the terms "first", "second", etc. in the description and claims of the present disclosure and the above drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0167] In the description of this specification, the descriptions with reference to terms such as "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0168] Any process or method description represented in a flowchart or described otherwise herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations, where functions may be executed in a manner not shown or discussed, including in a substantially simultaneous manner according to the functions involved or in a reverse order, which should be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0169] The logic and / or steps represented in a flowchart or described otherwise herein, for example, may be considered as a sequenced list of executable instructions for implementing a logical function and may be specifically implemented in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processing module, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion having one or more wirings (control method), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which a program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing as necessary, and then stored in a computer memory.

[0170] It should be understood that each part of the embodiments of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0171] Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0172] In addition, each functional unit in the various embodiments of the present disclosure can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage media mentioned above can be read-only memories, magnetic disks, optical discs, etc.

[0173] Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limitations on the present disclosure. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for assisting in judging a patient's state and adjusting the posture, characterized in that, The method includes: Obtaining real-time status data of the patient, where the real-time status data includes facial image data, body image data, bioelectrical signals, and audio data; Generating a three-dimensional body posture model and emotion parameters of the patient's current body posture based on the real-time status data; Generating a posture adjustment instruction according to the difference degree between a preset target posture and the three-dimensional body posture model, in combination with the doctor's manual adjustment instruction; Based on the posture adjustment instruction, prompting the patient to adjust the posture until the difference degree is less than a preset threshold, and the emotion parameters are within a target emotion range and last for a preset duration, and confirming that the patient has completed the pre-examination preparation.

2. The method according to claim 1, characterized in that The generating a three-dimensional body posture model and emotion parameters of the patient's current body posture based on the real-time status data includes: Detecting the patient's body joint points and generating a three-dimensional model according to the body image data; Identifying facial expression parameters through a convolutional neural network according to the facial image data; Determining the tension parameter of the patient according to the bioelectrical signal and audio data; Generating the emotion parameters based on the facial expression parameters and the tension parameter.

3. The method according to claim 1, wherein The generating a posture adjustment instruction according to the difference degree between a preset target posture and the three-dimensional body posture model, in combination with the doctor's manual adjustment instruction, includes: Generating a reference adjustment instruction according to the difference degree of the spatial transformation between the preset target posture and the three-dimensional body posture model, where the difference degree includes a translation component and a rotation component; Analyzing the priority of the manual adjustment instruction and the reference adjustment instruction using a large language model; Based on the priority, fusing the manual adjustment instruction and the reference adjustment instruction into a posture adjustment instruction according to the large language model.

4. The method according to claim 1, wherein The prompting the patient to adjust the posture based on the posture adjustment instruction includes: Decomposing the posture adjustment instruction into multiple posture adjustment sub-steps through a large language model based on the posture adjustment instruction; Generating a picture sequence or video according to the multiple posture adjustment sub-steps through a multimodal large model and playing it to the patient to prompt for posture adjustment; In response to the patient's adjustment amplitude being greater than an upper limit threshold or less than a lower limit threshold, feedback whether the current posture is correct through a prompt message, where the prompt message includes sound, AR, and vibration signals.

5. The method according to claim 1, wherein After the confirming that the patient has completed the pre-examination preparation, it further includes: In response to the patient's examination preparation being completed, conveying the doctor's examination instruction to the patient through audio or video.

6. The method according to claim 1, wherein The generating a three-dimensional body posture model and emotion parameters of the patient's current body posture based on the real-time status data further includes: Taking the real-time status data as the input of a multimodal large model, and combining the patient's medical record data to generate a spatial description matrix of the patient's current body posture; Using the spatial description matrix to construct the three-dimensional body posture model of the patient; Generating the emotion parameters of the patient according to the facial image data, bioelectrical signals, and audio data.

7. A patient state auxiliary judgment and posture adjustment device, characterized in that The device includes: An acquisition module for acquiring real-time status data of the patient, where the real-time status data includes facial image data, body image data, bioelectrical signals, and audio data; The first generation module is configured to generate a three-dimensional body posture model and emotional parameters of the patient's current body posture based on the real-time status data; The second generation module is configured to generate a posture adjustment instruction according to the difference degree between the preset target posture and the three-dimensional body posture model, in combination with the doctor's manual adjustment instruction; The adjustment module is configured to, based on the posture adjustment instruction, prompt the patient to adjust the posture until the difference degree is less than a preset threshold, and the emotional parameters are within the target emotional range and last for a preset duration, and confirm that the patient has completed the pre-examination preparation.

8. A patient status assisted judgment and posture adjustment system, characterized in that, The system includes a bearing device, a support, a rotating device, an audio device, and a video device, where, A head support device is installed on the bearing device. The head support device is configured to support the patient's head and collect the patient's bioelectrical signals. The support is installed on the bearing device. The installation method of the support is detachable installation or fixed installation. The rotating device is hinged to the support. The support is a rod-shaped structure. The support is perpendicular to the plane of the bearing device. A column is provided on the support. The column is slidably installed on the support. An adjustment knob is provided on the support. When the column slides to a suitable position, the adjustment knob fixes the column on the support. The rotating device includes a rotating arm, an upper rotating shaft, and a lower rotating shaft. The support is slidably installed on the bearing device. A sliding groove is provided on the bearing device. The lower end of the support is clamped in the sliding groove; An audio device, the audio device is installed on the support. There are two audio devices, symmetrically arranged on the support. When the patient lies on the head support device, the audio devices are located on both sides of the patient's head and are configured to collect the patient's audio data; A video device, the video device is installed on the rotating device. The rotating arm is rotationally connected to the support through the lower rotating shaft; the video device is rotationally connected to the rotating arm through the upper rotating shaft. The rotating arm is a telescopic member. The rotating arm includes a first sleeve and a second sleeve. A card slot is provided on the first sleeve, and a pressing protrusion is provided on the second sleeve. After the first sleeve and the second sleeve are sleeved to a predetermined position, the pressing protrusion is stuck in the predetermined card slot. The video device is configured to collect the patient's facial image data and body image data, generate a three-dimensional body posture model and emotional parameters of the patient's current body posture based on the real-time status data. The real-time status data includes facial image data, body image data, bioelectrical signals, and audio data. Generate a posture adjustment instruction according to the difference degree between the preset target posture and the three-dimensional body posture model, in combination with the doctor's manual adjustment instruction, and based on the posture adjustment instruction, prompt the patient to adjust the posture until the difference degree is less than a preset threshold, and the emotional parameters are within the target emotional range and last for a preset duration, and confirm that the patient has completed the pre-examination preparation.

9. The patient status assisted judgment and posture adjustment system according to claim 8, characterized in that, It further includes a control terminal. The video device and the audio device are connected to the control terminal through a wireless connection method.

10. A computer program product, characterized in that, Comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.

11. An electronic device, characterized in that, Comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, enable the at least one processor to execute the method according to any one of claims 1-6.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

Images