An intelligent nursing bed control method based on eye movement tracking and speech recognition

Abstract

This invention discloses a control method for an intelligent nursing bed based on eye tracking and voice recognition, belonging to the technical field of intelligent nursing equipment. The method includes the following steps: 1. Initializing the intelligent nursing bed control system, and performing calibration and parameter setting; 2. Collecting data and transmitting it in real time; 3. Processing the collected data to obtain eye tracking commands and voice commands; 4. Obtaining control commands and sending them to the execution unit; 5. The control unit controls the intelligent nursing bed to rise and fall, while the central processing unit monitors the movement angle of the intelligent nursing bed in real time; 6. After the action is completed, data is recorded and stored; 7. Providing nursing reminders based on historical data and real-time monitored body status data. This invention uses the above method to solve the problems of narrow applicability, high error rate, weak anti-interference ability, and lack of proactive nursing capabilities of existing control methods, thereby enhancing the user's autonomy and nursing experience, and reducing the nursing burden on caregivers.

Description

Technical Field

[0001] This invention belongs to the field of intelligent nursing equipment technology, and in particular relates to an intelligent nursing bed control method based on eye tracking and voice recognition. Background Technology

[0002] As an important nursing device, the convenience and intelligence level of intelligent nursing beds directly affect the quality of care and the user's autonomy in daily life. Currently, the control methods of existing intelligent nursing beds are mainly divided into manual control and single voice or eye-tracking control. Manual control requires users to have a certain degree of limb mobility. For people who are completely disabled or quadriplegic, they cannot operate independently and rely entirely on caregivers, thus losing their autonomy in daily life. Moreover, manual operation by caregivers is inefficient and cannot respond to users' immediate needs in a timely manner. Single voice or eye-tracking control has the problems of weak anti-interference ability, narrow applicability, and high error rate. Summary of the Invention

[0003] The purpose of this invention is to provide an intelligent nursing bed control method based on eye tracking and voice recognition. Targeting disabled / semi-disabled individuals with limb movement disorders and speech impairments, as well as elderly patients, this method achieves contactless, precise, and humanized control of the nursing bed through deep integration of eye tracking and voice recognition, combined with user physical status monitoring. This solves the problems of existing control methods, such as narrow applicability, high error rate, weak anti-interference ability, and lack of proactive care capabilities, thereby enhancing users' autonomy in daily life and nursing experience, and reducing the caregiving burden on caregivers.

[0004] To achieve the above objectives, the present invention provides a smart nursing bed control method based on eye tracking and voice recognition, comprising the following steps: S1. Initialize the intelligent nursing bed control system, and calibrate and set the parameters of the system; S2. Start the system. Each unit in the system starts collecting data and transmitting it in real time at the frequency preset in S1. S3. The central processing unit in the system processes the collected data to obtain eye-tracking commands and voice commands. S4. The eye-tracking command and the voice command are fused to obtain the control command, and the control command is sent to the execution unit; S5. The control unit controls the intelligent nursing bed to rise and fall according to the received control commands, while the central processing unit monitors the movement angle of the intelligent nursing bed in real time. S6. After the action is completed, record and store the data; S7. Provide care reminders based on historical data and real-time monitored body status data.

[0005] Preferably, the intelligent nursing bed control system mentioned in S1 includes an eye-tracking unit for capturing the user's eye movement signals; A speech recognition unit used to acquire users' initial voice command signals in real time; A body status monitoring unit used to collect users' physical status data in real time; Central processing unit used to process various signals and output control commands; An execution unit used to perform various actions on the intelligent nursing bed; An interactive feedback unit used to provide feedback on operation results to users and send signals to the host computer.

[0006] Preferably, the process of S1 is as follows: S11. Install the intelligent nursing bed control system on the intelligent nursing bed; S12. Instruct the user to gaze sequentially at five preset calibration points on the screen of the bedside interactive feedback unit, with a gazing time for each calibration point. The eye-tracking unit uses an infrared camera to collect pupil images and gaze data from different gaze angles to build a personalized eye-tracking model for the user. S13. The user reads aloud several preset standard instructions several times in sequence, and the user's pronunciation characteristics are collected and recorded by the microphone array in the speech recognition unit. S14. Have the user maintain several standard body positions for a number of seconds each, and then collect the pressure distribution, heart rate and respiratory rate under each body position as baseline data through the body condition monitoring unit. S15. Based on the needs of the nursing scenario and the user's physical condition, four types of core thresholds are set: eye movement command threshold, voice recognition confidence threshold, physical condition warning threshold, and maximum angle threshold of bed movement. The thresholds can be adjusted after authorization from the host terminal and the user.

[0007] Preferably, the process of S2 is as follows: S21. After completing S1, the eye-tracking unit, voice recognition unit and body status monitoring unit are activated and put into continuous working state; when the user does not operate, it is in low power mode, and after detecting the user's command, it switches to normal working mode. S22. When in working mode, the eye-tracking unit uses an infrared camera to capture several frames of the user's eye images per second, focusing on capturing the pupil position, eyelid state, and eye movement trajectory to obtain eye movement signals; the voice recognition unit continuously collects ambient sound through a microphone array and simultaneously filters ambient noise to obtain voice signals; and the body status monitoring unit collects the user's body status data at a preset frequency. S23. Encrypt the data collected in S22 using the AES encryption algorithm; S24. The encrypted data is transmitted to the data buffer in the central processing unit. If the transmission is interrupted, the untransmitted data is automatically buffered and automatically retransmitted after the connection is restored to avoid data loss.

[0008] Preferably, the process of S3 is as follows: S31. The eye movement signals in the data buffer are processed by the central processing unit to obtain eye movement instructions; S32. Filter and differentiate the speech signal to obtain explicit speech commands and ambiguous speech commands. The process is as follows: S321. Use the endpoint detection algorithm to distinguish non-speech noise in the speech signal and filter out non-speech noise; S322. Employs beamforming technology with microphone arrays to suppress echoes; S323. If the specific angle and amplitude are not clearly defined in the processed speech signal, it is marked as an ambiguous speech signal and proceeds to S33. If the specific angle and amplitude are clearly defined, it is marked as a clear speech signal and proceeds to S34. S33. Process the ambiguous speech signal to obtain speech commands. The process is as follows: S331. Identify ambiguous words in the instruction using natural language processing technology, determine the ambiguity level of the signal, and set a default adjustment range for each ambiguity level. S332. Generate adapted default action parameters by combining the user's historical operation habits and current physical state data with the default amplitude corresponding to the fuzziness level. S333: The default action parameters are broadcast to the user through the interactive feedback unit. After the user confirms, the voice command is obtained. If the user does not agree, the user is reminded to reissue the explicit command. S34. Process the explicit speech signal to obtain speech instructions. The process is as follows: S341. Extract Mel-frequency cepstral coefficients from a clear speech signal, and combine natural language processing technology to match a preset instruction library and identify the control intent. S342. Calculate the confidence level of the control intention through a probability model, that is, the degree of matching between the control intention and the user's actual instructions. S343. When the confidence level reaches the preset threshold, it is determined to be a valid voice command and the voice command is obtained. If the confidence level does not reach the threshold, it is determined to be an invalid command and the process returns to S341 for reprocessing. If all three commands are invalid, it is determined to be an ambiguous voice signal and S33 is performed.

[0009] Preferably, the process of S31 is as follows: S311. Image preprocessing of eye movement signals: Grayscale conversion, Gaussian blur denoising and histogram equalization are performed on the eye images acquired by the eye tracking unit to enhance the contrast between the pupil and iris and eliminate the influence of ambient light on recognition. S312. Extract the eye image features processed in S311 using the iTracker deep learning model, and detect the pupil center coordinates based on the ellipse fitting algorithm. S313. Combine the head posture compensation algorithm with the user's head posture data to correct the line of sight deviation caused by head movement and accurately locate the point where the line of sight falls on the bedside screen. S314. Statistically measure fixation duration and blink frequency, while filtering out unconscious movements; S315. When the eye movement meets the preset eye movement command threshold, it is determined to be a valid eye movement command. At the same time, the control intention is extracted, the eye movement command is obtained, and the validity of the eye movement command is calculated.

[0010] Preferably, the process of S4 is as follows: S41. Extract the control intent of the eye-tracking and voice commands obtained in S3 through the central processing unit, establish a command fusion verification table, and record the command type, control intent, and command reception time. S42. Perform fusion verification on eye-tracking commands and voice commands; When only a valid eye movement command is detected but no valid voice command is detected, the central control unit re-verifies the validity of the eye movement command. If the verification is successful, a control command is obtained and the corresponding control command is output to the execution unit. When only valid voice commands are detected and no valid eye-tracking commands are detected, the confidence level of the voice commands is checked to see if it is stable. If the check is passed, the control commands are obtained and the corresponding control commands are output to the execution unit. When both valid eye-tracking commands and valid voice commands are detected simultaneously, the control intentions of the two commands are first compared. If they are consistent, the command with the higher confidence score and validity score is selected as the control command and output to the execution unit. If the control intentions of the two commands are inconsistent, the conflict resolution mechanism is activated. S43. When the detected eye movement signal or voice signal does not reach the valid threshold, it is determined to be an invalid command. The central control unit does not output control commands, but only issues a voice prompt to the user that the command is invalid and to repeat the operation through the interactive feedback unit. At the same time, the reason for invalidity is displayed on the bedside screen.

[0011] Preferably, the process of S5 is as follows: S51. The execution unit receives the control command output by the central control unit, parses the command type and action parameters, and after parsing, sends a signal to the central control unit that the command has been successfully received and is ready to be executed. S52. The execution unit executes the corresponding action according to the parsed parameters, and at the same time, it adaptively adjusts the action speed according to the user's physical condition. S53. During the execution of the intelligent nursing bed's movements, the central control unit monitors the movement angle in real time through an angle sensor; If the maximum threshold of the motion angle distance is detected The execution unit is immediately controlled to slow down the action speed. The interactive feedback unit issues a voice prompt to the user. At the same time, the screen displays the current angle and the maximum threshold. The execution continues after the user confirms through either eye movement or voice. If no confirmation is made, the action stops immediately and the current angle is maintained. If the detected motion angle exceeds the maximum threshold, the central control unit immediately controls the execution unit to stop the current action, issues a red warning through the interactive feedback unit, and sends a reminder message to the upper terminal, including the user ID, current action, exceeded angle, and timestamp, so that the caregiver can view and handle it in a timely manner. After the caregiver and user confirm the abnormality, they adjust the bed angle to the maximum threshold range through the upper terminal, eye tracking, and voice commands. After the adjustment is completed, the warning is lifted, and the system resumes normal monitoring. S54. During the execution of the action, the body status monitoring unit continuously collects the user's body status data, and the central control unit compares the data with the preset warning threshold in real time.

[0012] Preferably, the process of S6 is as follows: S61. After the execution unit completes the action, it immediately sends a signal to the central control unit that the action has been completed and there is no abnormality. The central control unit controls the interactive feedback unit to provide feedback synchronously through multiple methods such as voice broadcast, light reminder and screen display. S62. The central control unit automatically records all data of this operation to form an operation record, which includes: instruction type, instruction content, executed action, action parameters, execution time, user physical status data, instruction confidence and validity score, whether any abnormality occurred and the processing result; S63. Store and back up the data recorded in S62 using a dual storage mode of local and cloud storage.

[0013] Preferably, the process of S7 is as follows: S71. The central control unit periodically mines and analyzes the operation data stored locally and in the cloud at set times to extract user usage habit characteristics. S72. Combining user habit profiles and physical condition monitoring data, when the trigger conditions of postural maintenance exceeding the time limit, common time points, abnormality prediction, and personalized reminders are met, proactive care reminders are activated. S73. The central control unit automatically adjusts the reminder time and method based on the user's response to the reminder, and optimizes the user habit profile.

[0014] Therefore, the intelligent nursing bed control method based on eye tracking and voice recognition described above has the following advantages: 1. Combining eye tracking and voice recognition control methods, it solves the problem that people with aphasia or unclear pronunciation cannot use voice control, as well as the problem that people with complete physical disability cannot use manual control. It also supports personalized command training to adapt to the usage habits of different users. It is suitable for various disabled, semi-disabled patients, the elderly and other people with limited mobility. Compared with the existing single control method, the scope of application is greatly expanded. 2. Users can complete various operations of the nursing bed without physical contact by simply using eye movements or voice commands, reducing dependence on caregivers. Caregivers' daily working hours can be reduced by more than 2 hours, effectively reducing the nursing burden and improving nursing efficiency.

[0015] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0016] Figure 1 This is an overall flowchart of an intelligent nursing bed control method based on eye tracking and voice recognition according to the present invention; Figure 2 This is an exported image of the data records stored in Embodiment 2 of the intelligent nursing bed control method based on eye tracking and voice recognition of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Specific model specifications need to be selected and determined according to the actual specifications of the device, etc. The specific selection calculation method adopts existing technology in the art, and therefore will not be described in detail.

[0018] Example 1; like Figure 1 As shown, this invention provides a smart nursing bed control method based on eye tracking and voice recognition, comprising the following steps: S1. Initialize the intelligent nursing bed control system, and calibrate and set the parameters of the system; S11. Install the intelligent nursing bed control system on the intelligent nursing bed; S12. Have the user gaze sequentially at five preset calibration points on the screen of the bedside interactive feedback unit, located at the four corners and center of the screen. The gazing time for each calibration point is... By using an infrared camera in the eye-tracking unit to collect pupil images and gaze data at different gaze angles, a personalized eye-tracking model for the user is established, eliminating errors caused by individual eye differences (such as cataracts, ptosis). S13. Have the user read aloud 10 preset standard instructions twice in sequence, such as "raise back", "turn to left side" and "emergency help". The microphone array in the speech recognition unit collects and records the user's pronunciation characteristics, including speech rate, tone and accent, to optimize the adaptability of the recognition model. S14. Have the user maintain one of four standard body positions: lying flat, lying on the left side, lying on the right side, or sitting up. Hold each position for 30 seconds. Then, use the body condition monitoring unit to collect pressure distribution, heart rate, and respiratory rate under each position as baseline data for subsequent body position recognition and abnormal judgment. S15. Based on the needs of the nursing scenario and the user's physical condition, four types of core thresholds are set: eye movement command threshold, voice recognition confidence threshold, physical condition warning threshold, and maximum angle threshold of bed movement. The thresholds can be adjusted after authorization from the host terminal and the user.

[0019] Eye movement command thresholds: gaze duration threshold 500ms-1500ms, default is 1000ms, can be adjusted according to the user's eye fatigue tolerance; blink count threshold 1-3 times, while the duration of a single blink is between 200ms and 500ms to avoid misjudging it as an unconscious blink; The confidence threshold for speech recognition is 85%-95%, with a default of 90%. It can be increased to 95% in noisy environments and reduced to 85% in quiet environments. Body condition warning thresholds: Position holding time threshold 1h-2h, default 1.5h, which can be adjusted to 1h for high-risk groups of pressure ulcers; Vital sign abnormality thresholds: heart rate less than 60 beats / min or greater than 100 beats / min, respiratory rate less than 12 breaths / min or greater than 20 breaths / min, which can be adjusted according to different age groups of users; Maximum bed movement angle thresholds: maximum back lifting angle 70°, maximum leg lifting angle 45°, maximum turning angle 30°. Exceeding these thresholds will trigger a higher-level reminder mechanism, and the system will lock the actions exceeding the thresholds. The user must confirm again or the caregiver must authorize before the actions can continue.

[0020] S2. Start the system. Each unit in the system starts collecting data and transmitting it in real time at the frequency preset in S1. S21. After completing S1, the eye-tracking unit, voice recognition unit, and body status monitoring unit are activated and kept in continuous working state. When the user does not operate, it is in low power mode, at which time the acquisition frequency of the infrared camera and sensor is reduced. After detecting the user's command, it switches to normal working mode within 100ms. S22. When in working mode, the eye-tracking unit uses an infrared camera to capture 20 frames of the user's eye images per second, focusing on capturing pupil position, eyelid state, and eye movement trajectory to obtain eye movement signals; the voice recognition unit continuously collects ambient sound through a microphone array at a sampling frequency of 16kHz, and simultaneously filters ambient noise to obtain voice signals; the body status monitoring unit collects the user's body status data at a preset frequency, with the pressure sensor collecting pressure distribution data once per second, and the heart rate sensor and respiratory sensor collecting heart rate and respiratory rate data once every 10 seconds, simultaneously recording the collection timestamp to ensure data traceability; S23. Encrypt the data collected in S22 using the AES encryption algorithm; S24. The encrypted data is transmitted to the data buffer in the central processing unit via wired transmission. If the transmission is interrupted (for no more than 1 second), the untransmitted data is automatically buffered and automatically retransmitted after the connection is restored to avoid data loss.

[0021] S3. The central processing unit in the system processes the collected data to obtain eye-tracking commands and voice commands. S31. The eye movement signals in the data buffer are processed by the central processing unit to obtain eye movement instructions; S311. Preprocess the eye movement signal: Perform grayscale conversion, Gaussian blur denoising (Gaussian kernel size 3×3) and histogram equalization on the eye images acquired by the eye tracking unit to enhance the contrast between the pupil and iris and eliminate the influence of ambient light on recognition. S312. Extract the eye image features processed in S311 using the iTracker deep learning model, and detect the pupil center coordinates based on the ellipse fitting algorithm. The specific structure of the iTracker deep learning model consists of an input layer, a lightweight feature extraction network, a multi-branch fusion layer, and an output layer. 1. Input layer; The input is an image of the user's eye area captured by an infrared camera, with the size uniformly normalized to 224×224×3 (RGB three channels). Image preprocessing (grayscale conversion, Gaussian blur denoising, histogram equalization) is used to reduce ambient light interference and ensure that the model is adaptable to the low light and reflective environment of the nursing bed scene.

[0022] Simultaneously, the user's initial head posture detection data (pitch angle and yaw angle collected by a simple posture sensor) is input to provide a basic input for subsequent line-of-sight compensation.

[0023] 2. Feature extraction network; A lightweight CNN architecture (based on an improvement of MobileNetV2) is adopted, balancing feature extraction capabilities with the operating efficiency of edge computing devices, specifically including: Initial convolutional layer: 1 3×3 convolutional kernel with a stride of 2 and 32 output channels. The activation function is ReLU6, which is used to initially extract shallow features of the eye image (such as pupil outline and eyelid edge).

[0024] Depthwise separable convolutional blocks (6): Each block consists of "depthwise convolution + pointwise convolution", with the number of channels gradually increasing from 32 to 128. The receptive field is expanded through dilated convolution to capture subtle eye movement features (such as pupil contraction and blink amplitude). At the same time, the BatchNormalization layer accelerates training convergence and reduces overfitting.

[0025] Global average pooling layer: Compresses the feature map into a 1×1×128 feature vector, retaining core features while reducing computational load, adapting to the edge computing requirements of the central control unit (ensuring a response within 200ms).

[0026] 3. Multi-branch fusion layer; The design incorporates three branches to achieve feature fusion, addressing the issue of gaze displacement caused by user head movement in nursing bed scenarios: Branch 1 (Pupil Feature Branch): Perform a fully connected operation (128→64→32) on the feature vector after global pooling, and output the predicted value of the pupil center coordinates (x1, y1), focusing on capturing the pupil position features.

[0027] Branch 2 (Gaze Direction Branch): Introduces head movement compensation features, fuses head pose data with feature vectors, and outputs gaze direction vectors (dx, dy) through a fully connected layer (128+3→64→32) to correct gaze deviation caused by head movement.

[0028] Branch 3 (Action Classification Branch): By combining a fully connected layer (128→64→2) with the Softmax activation function, eye movements are classified into two categories (effective actions / unconscious actions), filtering out interfering actions such as random eye movements and slight blinking, thereby improving the accuracy of command recognition.

[0029] Fusion strategy: A weighted summation method (with weights of 0.5, 0.3, and 0.2) is used to fuse the outputs of the three branches to obtain the final prediction results of the pupil center coordinates and the gaze point.

[0030] 4. Output layer; Main outputs: Pupil center coordinates (x, y) with an accuracy of 1 pixel, supporting real-time tracking; gaze point coordinates (X, Y), combined with a 3D gaze vector model to achieve precise positioning of the bedside screen area (adapted to the size of the nursing bed headboard screen).

[0031] Auxiliary output: The result of the validity judgment of eye movements (0 indicates invalid movement, 1 indicates valid movement), which provides a direct basis for the eye movement command judgment of the central control unit.

[0032] S313. Combine the head posture compensation algorithm with the user's head posture data to correct the line of sight deviation caused by head movement and accurately locate the point where the line of sight falls on the bedside screen. S314. The motion filtering unit counts fixation duration and blink frequency to filter out unconscious movements such as random eye movements (rotation speed > 50° / s) and slight blinks (duration < 200ms). S315. When the eye movement meets the preset eye movement command threshold, it is determined to be a valid eye movement command. At the same time, the control intention is extracted, the eye movement command is obtained, and the validity of the eye movement command is calculated.

[0033] Eye movement instruction effectiveness score = gaze duration / preset threshold × 100; S32. Filter and differentiate the speech signal to obtain explicit speech commands and ambiguous speech commands. The process is as follows: S321. Use the endpoint detection algorithm to distinguish non-speech noise in the speech signal and filter out non-speech noise; S322. Employs beamforming technology with microphone arrays to suppress echoes; S323. If the specific angle and amplitude are not clearly defined in the processed speech signal, it is marked as an ambiguous speech signal and proceeds to S33. If the specific angle and amplitude are clearly defined, it is marked as a clear speech signal and proceeds to S34. S33. Process the ambiguous speech signal to obtain speech commands. The process is as follows: S331. Identify ambiguous words in the instruction using natural language processing technology, determine the ambiguity level of the signal, and set a default adjustment range for each ambiguity level. S332. Generate adapted default action parameters by combining the user's historical operation habits and current physical state data with the default amplitude corresponding to the fuzziness level. S333: The default action parameters are broadcast to the user through the interactive feedback unit. After the user confirms, the voice command is obtained. If the user does not agree, the user is reminded to reissue the explicit command. S34. Process the explicit speech signal to obtain speech instructions. The process is as follows: S341. Extract Mel-frequency cepstral coefficients from a clear speech signal, and combine natural language processing technology to match a preset instruction library and identify the control intent. S342. Calculate the confidence level of the control intention through a probability model, that is, the degree of matching between the control intention and the user's actual instructions. S343. When the confidence level reaches the preset threshold, it is determined to be a valid voice command and the voice command is obtained. If the confidence level does not reach the threshold, it is determined to be an invalid command and the process returns to S341 for reprocessing. If all three commands are invalid, it is determined to be an ambiguous voice signal and S33 is performed.

[0034] S4. The eye-tracking command and the voice command are fused to obtain the control command, and the control command is sent to the execution unit; S41. Extract the control intent of the eye-tracking and voice commands obtained in S3 through the central processing unit, establish a command fusion verification table, and record the command type, control intent, and command reception time. S42. Perform fusion verification on eye-tracking commands and voice commands; When only a valid eye movement command is detected but no valid voice command is detected, the central control unit re-verifies the validity of the eye movement command. If the verification is successful, a control command is obtained and the corresponding control command is output to the execution unit. When only valid voice commands are detected and no valid eye-tracking commands are detected, the confidence level of the voice commands is checked to see if it is stable. If the check is passed, the control commands are obtained and the corresponding control commands are output to the execution unit. When both valid eye-tracking commands and valid voice commands are detected simultaneously, the control intentions of the two commands are first compared. If they are consistent, the command with the higher confidence score and validity score is selected as the control command and output to the execution unit. If the control intentions of the two commands are inconsistent, the conflict resolution mechanism is activated. Conflict resolution mechanism: A dual decision-making process of "priority + weighted score" is adopted. ① Priority division: Emergency commands (emergency help, stop action) have the highest priority, followed by ordinary commands (back raising / lowering, turning over, knee bending, etc.). ② Weighted calculation of ordinary commands: The confidence score of voice commands (weight 0.6) and the effectiveness score of eye-tracking commands (weight 0.4) are weighted and summed. The formula is: Final score = Voice confidence score × 0.6 + Eye-tracking effectiveness score × 0.4. ③ Decision output: The command with the higher priority is selected. If the priorities are the same, the command with the higher final score is selected as the final control command. At the same time, a confirmation prompt is issued to the user through the interactive feedback unit (such as "Command conflict detected, it is recommended to perform back raising, do you confirm?"). The user can confirm through eye-tracking or voice commands to avoid accidental operation.

[0035] S43. When the detected eye movement signal or voice signal does not reach the valid threshold, it is determined to be an invalid command. The central control unit does not output control commands, but only issues a voice prompt to the user that the command is invalid and to repeat the operation through the interactive feedback unit. At the same time, the reason for invalidity is displayed on the bedside screen.

[0036] If a user issues two valid commands within 1 second (e.g., first "raise back" and then "stop"), the central control unit caches the first command and executes the second command first. After execution, the interactive feedback unit prompts the user "The previous command has been cached, do you want to continue?". If the user confirms, the command will be executed; otherwise, the cached command will be cleared.

[0037] (1) When only a valid eye movement command is detected and no valid voice command is detected, the central control unit confirms the control intent of the eye movement command and outputs the corresponding control command to the execution unit; (2) When only a valid voice command is detected and no valid eye movement command is detected, the central control unit confirms the control intent of the voice command and outputs the corresponding control command to the execution unit; (3) When both valid eye-tracking commands and valid speech commands are detected simultaneously, if both control the intention Figure 1 If the control commands are consistent, the control command is confirmed and output to the execution unit. If the control intentions are inconsistent, a conflict resolution mechanism is activated, using a priority mechanism (emergency commands > normal commands) and a confidence weighted algorithm for decision-making. Emergency commands (such as emergency help or stop action) have the highest priority. For normal commands, the confidence score of voice commands and the effectiveness score of eye-tracking commands are weighted and calculated, with weights set to 0.6 and 0.4 respectively. The command with the higher weighted score is taken as the final control command. At the same time, a confirmation prompt is issued to the user through the interactive feedback unit. The user can confirm through eye-tracking or voice commands to avoid accidental operation. (4) When the detected eye movement signal or voice signal does not reach the effective threshold, it is determined to be an invalid command. The central control unit does not output control commands, but only sends a prompt to the user through the interactive feedback unit that "the command is invalid and you should try again".

[0038] S5. The control unit controls the intelligent nursing bed to rise and fall according to the received control commands, while the central processing unit monitors the movement angle of the intelligent nursing bed in real time. S51. The execution unit receives the control command output by the central control unit, parses the command type and action parameters, and after parsing, sends a signal to the central control unit that the command has been successfully received and is ready to be executed. S52. The execution unit executes the corresponding action according to the parsed parameters, and at the same time, it adaptively adjusts the action speed according to the user's physical condition. S53. During the execution of the intelligent nursing bed's movements, the central control unit monitors the movement angle in real time through an angle sensor; If the maximum threshold of the motion angle distance is detected The execution unit is immediately controlled to slow down the action speed. The interactive feedback unit issues a voice prompt to the user. At the same time, the screen displays the current angle and the maximum threshold. The execution continues after the user confirms through either eye movement or voice. If no confirmation is made, the action stops immediately and the current angle is maintained. If the detected motion angle exceeds the maximum threshold, the central control unit immediately controls the execution unit to stop the current action and issues a red warning through the interactive feedback unit (voice announcement "Motion has exceeded the maximum angle, stopped" + flashing red LED light). At the same time, it sends a reminder message to the upper-level terminal (caregiver workstation, family member's mobile APP), which includes the user ID, current action, exceeded angle, and timestamp, so that the caregiver can check and handle it in time. After the caregiver and user confirm the abnormality, they adjust the bed angle to the maximum threshold range through the upper-level terminal, eye tracking, and voice commands. After the adjustment is completed, the warning is lifted, and the system resumes normal monitoring. S54. During the execution of the action, the body status monitoring unit continuously collects the user's body status data, and the central control unit compares the data with the preset warning threshold in real time.

[0039] S6. After the action is completed, record and store the data; S61. After the execution unit completes the action, it immediately sends a signal to the central control unit that the action has been completed and there is no abnormality. The central control unit controls the interactive feedback unit to provide feedback synchronously through multiple methods such as voice broadcast, light reminder and screen display. S62. The central control unit automatically records all data of this operation to form an operation record, which includes: instruction type, instruction content, executed action, action parameters, execution time, user physical status data, instruction confidence and validity score, whether any abnormality occurred and the processing result; S63. Store and back up the data recorded in S62 using a dual storage mode of local and cloud storage.

[0040] S7. Provide care reminders based on historical data and real-time monitored body status data.

[0041] S71. The central control unit periodically mines and analyzes the operation data stored locally and in the cloud at set times to extract user usage habit characteristics. S72. Combining user habit profiles and physical condition monitoring data, when the trigger conditions of postural maintenance exceeding the time limit, common time points, abnormality prediction, and personalized reminders are met, proactive care reminders are activated. S73. The central control unit automatically adjusts the reminder time and method based on the user's response to the reminder, and optimizes the user habit profile.

[0042] The intelligent nursing bed control system includes an eye-tracking unit for capturing signals of the user's eye movements; A speech recognition unit used to acquire users' initial voice command signals in real time; A body status monitoring unit used to collect users' physical status data in real time; Central processing unit used to process various signals and output control commands; An execution unit used to perform various actions on the intelligent nursing bed; An interactive feedback unit used to provide feedback on operation results to users and send signals to the host computer.

[0043] Example 2: Turning control for completely disabled and aphasic users; The user is a quadriplegic and aphasic patient who is unable to express themselves verbally or perform limb movements. This invention's method controls the turning of the nursing bed, such as... Figure 2 As shown, the specific steps are as follows: 1. System initialization: Calibrate the eye-tracking unit and body status monitoring unit, set the gaze duration threshold to 1000ms, the posture holding duration threshold to 1.5h, and the abnormal heart rate threshold to <60 beats / min or >100 beats / min.

[0044] 2. The eye-tracking unit captures the user's eye movements in real time, while the body status monitoring unit continuously collects the user's posture data (lying flat) and heart rate data (75 beats / min) and transmits them to the central control unit.

[0045] 3. The user stares at the "turn to the left" icon on the bedside screen for 1000ms. The eye-tracking unit captures the eye movement, processes it, and determines it as a valid eye movement command, with the control intention being "turn to the left". At this time, the voice recognition unit does not detect a voice signal and determines that there is no valid voice command.

[0046] 4. After the central control unit performs fusion verification, it confirms that the eye-tracking command is valid and outputs the "turn to the left" control command to the execution unit.

[0047] 5. The execution unit receives the instruction and controls the nursing bed to slowly turn to the left. During the turning process, the body status monitoring unit continuously collects the user's body position data (left lateral decubitus) and heart rate data (78 beats / min). If the heart rate does not exceed the warning threshold, the turning action is executed normally.

[0048] 6. After the rolling over is completed, the interactive feedback unit announces "Left rolling over completed" via voice, the LED light turns green, and the central control unit records the operation data (command type: eye movement command, executed action: left rolling over, execution time: 14:30, body position: left lateral decubitus, heart rate: 78 beats / min).

[0049] 7. When the user has maintained the left lateral decubitus position for 1.5 hours, the central control unit will issue a voice prompt "Please turn over to prevent pressure sores" through the interactive feedback unit. At the same time, "Turn over reminder" will be displayed on the bedside screen. After the user stares at the "Turn over to the right" icon for 1000ms, the execution unit will complete the right turn action.

[0050] Therefore, this invention adopts an intelligent nursing bed control method based on eye tracking and voice recognition, which solves the problems of narrow applicability, high error rate, weak anti-interference ability and lack of proactive nursing ability of existing control methods, enhances the user's autonomy in life and nursing experience, and reduces the nursing burden of caregivers.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for controlling an intelligent nursing bed based on eye tracking and voice recognition, characterized in that, Includes the following steps: S1. Initialize the intelligent nursing bed control system, and calibrate and set the parameters of the system; S2. Start the system. Each unit in the system starts collecting data and transmitting it in real time at the frequency preset in S1. S3. The central processing unit in the system processes the collected data to obtain eye-tracking commands and voice commands. S4. The eye-tracking command and the voice command are fused to obtain the control command, and the control command is sent to the execution unit; S5. The control unit controls the intelligent nursing bed to rise and fall according to the received control commands, while the central processing unit monitors the movement angle of the intelligent nursing bed in real time. S6. After the action is completed, record and store the data; S7. Provide care reminders based on historical data and real-time monitored body status data.

2. The intelligent nursing bed control method based on eye tracking and voice recognition according to claim 1, characterized in that: The intelligent nursing bed control system mentioned in S1 includes an eye-tracking unit for capturing the user's eye movement signals; A speech recognition unit used to acquire users' initial voice command signals in real time; A body status monitoring unit used to collect users' physical status data in real time; Central processing unit used to process various signals and output control commands; An execution unit used to perform various actions on the intelligent nursing bed; An interactive feedback unit used to provide feedback on operation results to users and send signals to the host computer.

3. The intelligent nursing bed control method based on eye tracking and voice recognition according to claim 2, characterized in that, The process of S1 is as follows: S11. Install the intelligent nursing bed control system on the intelligent nursing bed; S12. Instruct the user to gaze sequentially at five preset calibration points on the screen of the bedside interactive feedback unit, with a gazing time for each calibration point. The eye-tracking unit uses an infrared camera to collect pupil images and gaze data from different gaze angles to build a personalized eye-tracking model for the user. S13. The user reads aloud several preset standard instructions several times in sequence, and the user's pronunciation characteristics are collected and recorded by the microphone array in the speech recognition unit. S14. Have the user maintain several standard body positions for a number of seconds each, and then collect the pressure distribution, heart rate and respiratory rate under each body position as baseline data through the body condition monitoring unit. S15. Based on the needs of the nursing scenario and the user's physical condition, four types of core thresholds are set: eye movement command threshold, voice recognition confidence threshold, physical condition warning threshold, and maximum angle threshold of bed movement. The thresholds can be adjusted after authorization from the host terminal and the user.

4. The intelligent nursing bed control method based on eye tracking and voice recognition according to claim 3, characterized in that, The process of S2 is as follows: S21. After completing S1, the eye-tracking unit, voice recognition unit and body status monitoring unit are activated and put into continuous working state; when the user does not operate, it is in low power mode, and after detecting the user's command, it switches to normal working mode. S22. When in working mode, the eye-tracking unit uses an infrared camera to capture several frames of the user's eye images per second, focusing on capturing the pupil position, eyelid state, and eye movement trajectory to obtain eye movement signals; the voice recognition unit continuously collects ambient sound through a microphone array and simultaneously filters ambient noise to obtain voice signals; and the body status monitoring unit collects the user's body status data at a preset frequency. S23. Encrypt the data collected in S22 using the AES encryption algorithm; S24. The encrypted data is transmitted to the data buffer in the central processing unit. If the transmission is interrupted, the untransmitted data is automatically buffered and automatically retransmitted after the connection is restored to avoid data loss.

5. The intelligent nursing bed control method based on eye tracking and voice recognition according to claim 4, characterized in that, The process of S3 is as follows: S31. The eye movement signals in the data buffer are processed by the central processing unit to obtain eye movement instructions; S32. Filter and differentiate the speech signal to obtain explicit speech commands and ambiguous speech commands. The process is as follows: S321. Use the endpoint detection algorithm to distinguish non-speech noise in the speech signal and filter out non-speech noise; S322. Employs beamforming technology with microphone arrays to suppress echoes; S323. If the specific angle and amplitude are not clearly defined in the processed speech signal, it is marked as an ambiguous speech signal and proceeds to S33. If the specific angle and amplitude are clearly defined, it is marked as a clear speech signal and proceeds to S34. S33. Process the ambiguous speech signal to obtain speech commands. The process is as follows: S331. Identify ambiguous words in the instruction using natural language processing technology, determine the ambiguity level of the signal, and set a default adjustment range for each ambiguity level. S332. Generate adapted default action parameters by combining the user's historical operation habits and current physical state data with the default amplitude corresponding to the fuzziness level. S333: The default action parameters are broadcast to the user through the interactive feedback unit. After the user confirms, the voice command is obtained. If the user does not agree, the user is reminded to reissue the explicit command. S34. Process the explicit speech signal to obtain speech instructions. The process is as follows: S341. Extract Mel-frequency cepstral coefficients from a clear speech signal, and combine natural language processing technology to match a preset instruction library and identify the control intent. S342. Calculate the confidence level of the control intention through a probability model, that is, the degree of matching between the control intention and the user's actual instructions. S343. When the confidence level reaches the preset threshold, it is determined to be a valid voice command and the voice command is obtained. If the confidence level does not reach the threshold, it is determined to be an invalid command and the process returns to S341 for reprocessing. If all three commands are invalid, it is determined to be an ambiguous voice signal and S33 is performed.

6. The intelligent nursing bed control method based on eye tracking and voice recognition according to claim 5, characterized in that, The process of S31 is as follows: S311. Image preprocessing of eye movement signals: Grayscale conversion, Gaussian blur denoising and histogram equalization are performed on the eye images acquired by the eye tracking unit to enhance the contrast between the pupil and iris and eliminate the influence of ambient light on recognition. S312. Extract the eye image features processed in S311 using the iTracker deep learning model, and detect the pupil center coordinates based on the ellipse fitting algorithm. S313. Combine the head posture compensation algorithm with the user's head posture data to correct the line of sight deviation caused by head movement and accurately locate the point where the line of sight falls on the bedside screen. S314. Statistically measure fixation duration and blink frequency, while filtering out unconscious movements; S315. When the eye movement meets the preset eye movement command threshold, it is determined to be a valid eye movement command. At the same time, the control intention is extracted, the eye movement command is obtained, and the validity of the eye movement command is calculated.

7. The intelligent nursing bed control method based on eye tracking and voice recognition according to claim 6, characterized in that, The process of S4 is as follows: S41. Extract the control intent of the eye-tracking and voice commands obtained in S3 through the central processing unit, establish a command fusion verification table, and record the command type, control intent, and command reception time. S42. Perform fusion verification on eye-tracking commands and voice commands; When only a valid eye movement command is detected but no valid voice command is detected, the central control unit re-verifies the validity of the eye movement command. If the verification is successful, a control command is obtained and the corresponding control command is output to the execution unit. When only valid voice commands are detected and no valid eye-tracking commands are detected, the confidence level of the voice commands is checked to see if it is stable. If the check is passed, the control commands are obtained and the corresponding control commands are output to the execution unit. When both valid eye-tracking commands and valid voice commands are detected simultaneously, the control intentions of the two commands are first compared. If they are consistent, the command with the higher confidence score and validity score is selected as the control command and output to the execution unit. If the control intentions of the two commands are inconsistent, the conflict resolution mechanism is activated. S43. When the detected eye movement signal or voice signal does not reach the valid threshold, it is determined to be an invalid command. The central control unit does not output control commands, but only issues a voice prompt to the user that the command is invalid and to repeat the operation through the interactive feedback unit. At the same time, the reason for invalidity is displayed on the bedside screen.

8. The intelligent nursing bed control method based on eye tracking and voice recognition according to claim 7, characterized in that, The process of S5 is as follows: S51. The execution unit receives the control command output by the central control unit, parses the command type and action parameters, and after parsing, sends a signal to the central control unit that the command has been successfully received and is ready to be executed. S52. The execution unit executes the corresponding action according to the parsed parameters, and at the same time, it adaptively adjusts the action speed according to the user's physical condition. S53. During the execution of the intelligent nursing bed's movements, the central control unit monitors the movement angle in real time through an angle sensor; If the maximum threshold of the motion angle distance is detected The execution unit is immediately controlled to slow down the action speed. The interactive feedback unit issues a voice prompt to the user. At the same time, the screen displays the current angle and the maximum threshold. The user can continue to execute after confirming through either eye movement or voice. If no confirmation is made, the action is stopped immediately and the current angle is maintained. If the detected motion angle exceeds the maximum threshold, the central control unit immediately controls the execution unit to stop the current action, issues a red warning through the interactive feedback unit, and sends a reminder message to the upper terminal, including the user ID, current action, exceeded angle, and timestamp, so that the caregiver can view and handle it in a timely manner. After the caregiver and user confirm the abnormality, they adjust the bed angle to the maximum threshold range through the upper terminal, eye tracking, and voice commands. After the adjustment is completed, the warning is lifted, and the system resumes normal monitoring. S54. During the execution of the action, the body status monitoring unit continuously collects the user's body status data, and the central control unit compares the data with the preset warning threshold in real time.

9. A method for controlling an intelligent nursing bed based on eye tracking and voice recognition according to claim 8, characterized in that, The process of S6 is as follows: S61. After the execution unit completes the action, it immediately sends a signal to the central control unit that the action has been completed and there is no abnormality. The central control unit controls the interactive feedback unit to provide feedback synchronously through multiple methods such as voice broadcast, light reminder and screen display. S62. The central control unit automatically records all data of this operation to form an operation record, which includes: instruction type, instruction content, executed action, action parameters, execution time, user physical status data, instruction confidence and validity score, whether any abnormality occurred and the processing result; S63. Store and back up the data recorded in S62 using a dual storage mode of local and cloud storage.

10. A method for controlling an intelligent nursing bed based on eye tracking and voice recognition according to claim 9, characterized in that, The process for S7 is as follows: S71. The central control unit periodically mines and analyzes the operation data stored locally and in the cloud at set times to extract user usage habit characteristics. S72. Combining user habit profiles and physical condition monitoring data, when the trigger conditions of postural maintenance exceeding the time limit, common time points, abnormality prediction, and personalized reminders are met, proactive care reminders are activated. S73. The central control unit automatically adjusts the reminder time and method based on the user's response to the reminder, and optimizes the user habit profile.

Images