Platform and method for non-language communication between medical care and patient

Abstract

The invention relates to the field of language communication, and discloses a medical care and patient non-language communication platform which comprises a communication platform device, and the communication platform device comprises an icon library module, a user interface module, a direct pointing interaction module, an action recognition module, a touch screen and touchpad interaction module and a result integration and feedback module. Diversified graphic cards and gestures are imported on the platform, interfaces of daily demands, common discomfort and emotion expression icons are designed, each interface represents different specific meanings, and a patient selects different interfaces to express own demands through simple pointing actions; a high-sensitivity sensor is adopted, tiny eye and head actions of a patient are accurately captured, a computer carries out image processing and calculation in real time, and conscious or unconscious actions of the patient are recognized and distinguished, so that effective communication media between medical staff and the patient are provided, and an effective channel is increased for the patient to express own requirements.

Description

Technical Field

[0001] The present invention relates to the field of language communication technology, and in particular to a platform and method for non-verbal communication between medical staff and patients. Background Art

[0002] Patients undergoing mechanical ventilation, tracheostomy, or voice disorders in clinical practice are unable to effectively communicate verbally during treatment. This lacks effective communication media with medical staff and channels for expressing their needs, which can negatively impact both the patient's physical and psychological well-being, leading to confusion, anxiety, irritability, fear, or depression, which directly impacts recovery. Currently, some patients can express their needs through writing, but this requires a high level of cognitive and knowledge, and there are currently no simple, effective non-verbal communication platforms or methods available in clinical practice. Summary of the Invention

[0003] In response to the deficiencies of the existing technology, the present invention provides a platform and method for non-verbal communication between medical staff and patients, which solves the problem that medical staff and patients lack effective media for communication and patients lack effective channels to express their needs.

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions: a platform for non-verbal communication between medical staff and patients, including a communication platform device, the communication platform device including an icon library module, a user interface module, a direct pointing interaction module, an action recognition module, a touch screen and touchpad interaction module and a result integration and feedback module.

[0005] A method for non-verbal communication between a medical professional and a patient, comprising the following steps:

[0006] Step 1: Create an icon library. Use an icon classification method to design and import a variety of image cards, including daily needs, common discomforts, and emotional expressions. Use visual design tools to ensure that each icon is concise and easy to understand. Generate an icon library.

[0007] Step 2: User Interface Design: Build the user interface based on user experience design principles, design icons of different colors and sizes to enhance recognizability, and adopt responsive design to ensure that the interface can be displayed well on different devices; generate the user interface;

[0008] Step 3: Direct pointing interaction: Using a pointing selection algorithm, patients are allowed to select icons by simply pointing their fingers. A click feedback mechanism is designed to ensure that the patient's selection is correctly recognized. The pointing selection result is generated.

[0009] Step 4: Eye and head movement recognition: Using highly sensitive sensor technology, the system captures the patient's eye and head movements in real time, uses image processing algorithms to identify the patient's intentions, and designs an interface to convert the movements into cursor movements or selections, generating the movement recognition results.

[0010] Step 5: Interact with the touch screen and touchpad; Based on touch technology, design an information platform to provide audio feedback for key operations, use gesture recognition algorithms, set simple gestures to help patients express their needs, and generate touch interaction results;

[0011] Step 6: Result integration and feedback mechanism; combining the pointing selection result of step 3, the action recognition result of step 4, and the touch interaction result of step 5, the data fusion algorithm is used to integrate the information, and visual or auditory feedback is provided to confirm the correctness of the system's interpretation; generating an integrated feedback result.

[0012] Preferably, step 1: establishing an icon library includes the following specific steps:

[0013] Sub-step 1.1: Needs analysis: Through research and interviews, collect information on the non-verbal communication needs of doctors, nurses, and patients, determine the necessary icon types and content, and generate a needs list.

[0014] Sub-step 1.2: Icon design: Use visual design tools to design various icons based on the requirements list, ensuring they are concise and easy to understand; generate a draft of the icons;

[0015] Sub-step 1.3: Classify icons; classify the designed icons according to daily needs, common discomforts, and emotional expressions for ease of use; and generate a classified icon library;

[0016] Sub-step 1.4: User feedback: Collect feedback from medical staff and patients on the icons through user testing, make necessary modifications, and generate the final icon library.

[0017] Preferably, step 2: user interface design; the specific steps are as follows:

[0018] Sub-step 2.1: Interface prototype design: Based on user experience principles, build a preliminary prototype of the interface, set icons of different colors and sizes, and generate the interface prototype;

[0019] Sub-step 2.2: Responsive design: Use responsive design methods to ensure the interface displays well on different devices, test its compatibility, and generate an adaptive interface.

[0020] Sub-step 2.3: User testing: Conduct user experience testing, observe user interactions on the interface, collect feedback for optimization, and generate a user feedback report.

[0021] Sub-step 2.4: Interface optimization; adjust and optimize the interface based on user feedback reports to ensure smooth user operations; generate the final user interface.

[0022] Preferably, step 3: directly pointing to the interaction mode; the specific steps are as follows:

[0023] Sub-step 3.1: Pointing selection algorithm design; Design a simple pointing selection algorithm that allows patients to select icons by pointing their fingers; Generate a selection algorithm;

[0024] Sub-step 3.2: Click feedback mechanism; add a click feedback mechanism to the selection algorithm to ensure that the patient's choice is correctly identified; generate a feedback mechanism;

[0025] Sub-step 3.3: Interaction testing: verify the accuracy of the pointing selection through user testing, collect data for analysis, and generate test results;

[0026] Sub-step 3.4: Algorithm adjustment: Based on the test results, make necessary adjustments to the pointing selection algorithm to improve accuracy; and generate optimized selection results.

[0027] Preferably, step 4: eye expression and head movement recognition; the specific steps are as follows:

[0028] Sub-step 4.1: Sensor technology selection: Select a high-sensitivity sensor that can capture the patient's eye and head movements in real time; generate a sensor configuration;

[0029] Sub-step 4.2: Image processing algorithm design; using image processing algorithms, analyze the data captured by the sensor to identify the patient's intention; generate an intention recognition algorithm;

[0030] Sub-step 4.3: Action conversion mechanism: Design a mechanism to convert the recognized actions into cursor movement or selection to facilitate patient interaction; generate action conversion results;

[0031] Sub-step 4.4: System testing; perform system testing to verify the accuracy of eye gaze and head movement recognition, make necessary adjustments, and generate the final action recognition results.

[0032] Preferably, step 5: touching the screen and interacting with the touchpad; the specific steps are as follows:

[0033] Sub-step 5.1: Touch technology selection: Select appropriate touch technology to support touch and gesture interaction and ensure user experience; generate touch technology configuration;

[0034] Sub-step 5.2: Audio feedback design; Design audio feedback for touch interactions to enhance the user's sense of confirmation; Create an audio feedback mechanism;

[0035] Sub-step 5.3: Gesture recognition algorithm development; Develop a simple gesture recognition algorithm to enable patients to express their needs through gestures; Generate a gesture recognition algorithm;

[0036] Sub-step 5.4: Interaction test; perform interaction test to verify the accuracy and smoothness of touch screen and touchpad operation; generate touch interaction results.

[0037] Preferably, step 6: result integration and feedback mechanism; the specific steps are as follows:

[0038] Sub-step 6.1: Data fusion algorithm design; Design a data fusion algorithm to integrate the results of steps 3, 4, and 5 to form a comprehensive output; Generate a fusion algorithm;

[0039] Sub-step 6.2: Feedback mechanism establishment; Establish visual and auditory feedback mechanisms to confirm the correctness of the system's interpretation; Generate feedback mechanisms;

[0040] Sub-step 6.3: System testing: Conduct overall system testing to ensure that all modules function properly after integration and that feedback is effective; generate a system test report;

[0041] Sub-step 6.4: Optimize the integration results; according to the test report, optimize the integration results and feedback mechanism as necessary; generate the final integration feedback results.

[0042] The present invention provides a platform and method for non-verbal communication between medical staff and patients. It has the following beneficial effects:

[0043] The present invention introduces a variety of picture cards and gestures on the platform, designs interfaces for daily needs, common discomforts and emotional expression icons, each interface represents a different specific meaning, and patients express their needs by simple pointing movements to select different interfaces; uses highly sensitive sensors to accurately capture the patient's subtle eye and head movements, and the computer performs image processing and calculations in real time to identify and distinguish whether the patient's movements are conscious or unconscious; the touch screen and touchpad information platform can provide audio feedback for key operations, and simple gestures can be set to allow patients to express their needs, thereby realizing an effective communication medium between medical staff and patients, and increasing effective channels for patients to express their needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a diagram showing the platform of the present invention;

[0045] Figure 2 It is a module framework diagram of the present invention;

[0046] Figure 3 This is a framework diagram of the method of the present invention.

[0047] Among them, 1. Communication platform device; 11. Icon library module; 12. User interface module; 13. Direct pointing interaction module; 14. Action recognition module; 15. Touch screen and touchpad interaction module; 16. Result integration and feedback module. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] Example:

[0050] Please see the attached Figure 1 - Attachment Figure 3 An embodiment of the present invention provides a platform for non-verbal communication between medical staff and patients, including a communication platform device 1, which includes an icon library module 11, a user interface module 12, a direct pointing interaction module 13, an action recognition module 14, a touch screen and touchpad interaction module 15 and a result integration and feedback module 16.

[0051] A method for non-verbal communication between a medical professional and a patient, comprising the following steps:

[0052] Step 1: Create an icon library. Use an icon classification method to design and import a variety of image cards, including daily needs, common discomforts, and emotional expressions. Use visual design tools to ensure that each icon is concise and easy to understand. Generate an icon library.

[0053] Step 2: User Interface Design: Build the user interface based on user experience design principles, design icons of different colors and sizes to enhance recognizability, and adopt responsive design to ensure that the interface can be displayed well on different devices; generate the user interface;

[0054] Step 3: Direct pointing interaction: Using a pointing selection algorithm, patients are allowed to select icons by simply pointing their fingers. A click feedback mechanism is designed to ensure that the patient's selection is correctly recognized. The pointing selection result is generated.

[0055] Step 4: Eye and head movement recognition: Using highly sensitive sensor technology, the system captures the patient's eye and head movements in real time, uses image processing algorithms to identify the patient's intentions, and designs an interface to convert the movements into cursor movements or selections, generating the movement recognition results.

[0056] Step 5: Interact with the touch screen and touchpad; Based on touch technology, design an information platform to provide audio feedback for key operations, use gesture recognition algorithms, set simple gestures to help patients express their needs, and generate touch interaction results;

[0057] Step 6: Result integration and feedback mechanism; combining the pointing selection result of step 3, the action recognition result of step 4, and the touch interaction result of step 5, the data fusion algorithm is used to integrate the information, and visual or auditory feedback is provided to confirm the correctness of the system's interpretation; generating an integrated feedback result.

[0058] Furthermore, step 1: icon library establishment includes the following specific steps:

[0059] Sub-step 1.1: Needs analysis: Through research and interviews, collect information on the non-verbal communication needs of doctors, nurses, and patients, determine the necessary icon types and content, and generate a needs list.

[0060] Sub-step 1.2: Icon design: Use visual design tools to design various icons based on the requirements list, ensuring they are concise and easy to understand; generate a draft of the icons;

[0061] Sub-step 1.3: Classify icons; classify the designed icons according to daily needs, common discomforts, and emotional expressions for ease of use; and generate a classified icon library;

[0062] Sub-step 1.4: User feedback: Collect feedback from medical staff and patients on the icons through user testing, make necessary modifications, and generate the final icon library.

[0063] Next, step 2: user interface design; the specific steps are as follows:

[0064] Sub-step 2.1: Interface prototype design: Based on user experience principles, build a preliminary prototype of the interface, set icons of different colors and sizes, and generate the interface prototype;

[0065] Sub-step 2.2: Responsive design: Use responsive design methods to ensure the interface displays well on different devices, test its compatibility, and generate an adaptive interface.

[0066] Sub-step 2.3: User testing: Conduct user experience testing, observe user interactions on the interface, collect feedback for optimization, and generate a user feedback report.

[0067] Sub-step 2.4: Interface optimization; adjust and optimize the interface based on user feedback reports to ensure smooth user operations; generate the final user interface.

[0068] Further, step 3: directly point to the interaction method; the specific steps are as follows:

[0069] Sub-step 3.1: Pointing selection algorithm design; Design a simple pointing selection algorithm that allows patients to select icons by pointing their fingers; Generate a selection algorithm;

[0070] Sub-step 3.2: Click feedback mechanism; add a click feedback mechanism to the selection algorithm to ensure that the patient's choice is correctly identified; generate a feedback mechanism;

[0071] Sub-step 3.3: Interaction testing: verify the accuracy of the pointing selection through user testing, collect data for analysis, and generate test results;

[0072] Sub-step 3.4: Algorithm adjustment: Based on the test results, make necessary adjustments to the pointing selection algorithm to improve accuracy; and generate optimized selection results.

[0073] Next, step 4: eye and head gesture recognition; the specific steps are as follows:

[0074] Sub-step 4.1: Sensor technology selection: Select a high-sensitivity sensor that can capture the patient's eye and head movements in real time; generate a sensor configuration;

[0075] Sub-step 4.2: Image processing algorithm design; using image processing algorithms, analyze the data captured by the sensor to identify the patient's intention; generate an intention recognition algorithm;

[0076] Sub-step 4.3: Action conversion mechanism: Design a mechanism to convert the recognized actions into cursor movement or selection to facilitate patient interaction; generate action conversion results;

[0077] Sub-step 4.4: System testing; perform system testing to verify the accuracy of eye gaze and head movement recognition, make necessary adjustments, and generate the final action recognition results.

[0078] Further, step 5: touch the screen and interact with the touchpad; the specific steps are as follows:

[0079] Sub-step 5.1: Touch technology selection: Select appropriate touch technology to support touch and gesture interaction and ensure user experience; generate touch technology configuration;

[0080] Sub-step 5.2: Audio feedback design; Design audio feedback for touch interactions to enhance the user's sense of confirmation; Create an audio feedback mechanism;

[0081] Sub-step 5.3: Gesture recognition algorithm development; Develop a simple gesture recognition algorithm to enable patients to express their needs through gestures; Generate a gesture recognition algorithm;

[0082] Sub-step 5.4: Interaction test; perform interaction test to verify the accuracy and smoothness of touch screen and touchpad operation; generate touch interaction results.

[0083] Furthermore, step 6: result integration and feedback mechanism; the specific steps are as follows:

[0084] Sub-step 6.1: Data fusion algorithm design; Design a data fusion algorithm to integrate the results of steps 3, 4, and 5 to form a comprehensive output; Generate a fusion algorithm;

[0085] Sub-step 6.2: Feedback mechanism establishment; Establish visual and auditory feedback mechanisms to confirm the correctness of the system's interpretation; Generate feedback mechanisms;

[0086] Sub-step 6.3: System testing: Conduct overall system testing to ensure that all modules function properly after integration and that feedback is effective; generate a system test report;

[0087] Sub-step 6.4: Optimize the integration results; according to the test report, optimize the integration results and feedback mechanism as necessary; generate the final integration feedback results.

[0088] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A non-verbal communication platform between medical staff and patients, comprising a communication platform device (1), It is characterized in that The communication platform device (1) comprises an icon library module (11), a user interface module (12), a direct pointing interaction module (13), an action recognition module (14), a touch screen and touch pad interaction module (15) and a result integration and feedback module (16).

2. A method of non-verbal communication between medical staff and patients. It is characterized in that The following steps are involved: Step 1: Establish an icon library; adopt the icon classification method, design and import a variety of picture cards, including daily needs, common discomforts and emotional expressions, and use visual design tools to ensure that each icon is concise and easy to understand; Generate icon library; Step 2: User interface design: Based on the user experience design principles, build the user interface, design icons of different colors and sizes to enhance recognizability, and adopt responsive design to ensure that the interface can be well displayed on different devices; Generate user interface; Step 3: Directly point to the interaction method; A pointing selection algorithm is used to allow patients to select icons through simple finger pointing movements, and a click feedback mechanism is designed to ensure that the patient's selection is correctly recognized; a pointing selection result is generated; Step 4: Eye and head movement recognition: Use high-sensitivity sensor technology to capture the patient's eye and head movements in real time, use image processing algorithms to identify the patient's intentions, and design an interface to convert the movements into cursor movements or selections. Generate action recognition results; Step 5: Touch the screen to interact with the trackpad; Based on touch technology, the information platform is designed to provide audio feedback for key operations, and gesture recognition algorithms are used to set simple gestures to help patients express their needs; Generate touch interaction results; Step 6: Result integration and feedback mechanism: Combine the pointing selection result of step 3, the action recognition result of step 4, and the touch interaction result of step 5, integrate the information through the data fusion algorithm, and provide visual or auditory feedback to confirm the correctness of the system's interpretation; Generate integrated feedback results.

3. A method for non-verbal communication between medical staff and patients according to claim 2, It is characterized in that Step 1: Icon library establishment; the specific steps are as follows: Sub-step 1.1: Demand analysis: Through research and interviews, collect the non-verbal communication needs of medical staff and patients, and determine the necessary icon types and contents; Generate a list of requirements; Sub-step 1.2: Icon design; Use visual design tools to design various icons based on the requirements list, ensuring they are concise and easy to understand; Generate a first draft of the icon; Sub-step 1.3: Icon classification; Classify the designed icons according to daily needs, common discomforts and emotional expressions for easy use; generate a classified icon library; Sub-step 1.4: User feedback; Through user testing, collect feedback from medical staff and patients on the icons and make necessary modifications; Generate the final icon library.

4. A method for non-verbal communication between medical staff and patients according to claim 2, It is characterized in that Step 2: User interface design; the specific steps are as follows: Sub-step 2.1: Interface prototype design; Based on the user experience principles, build a preliminary prototype of the interface, set icons of different colors and sizes; generate an interface prototype; Sub-step 2.2: Responsive design: Use responsive design methods to ensure that the interface displays well on different devices and test its compatibility; Generate an adaptation interface; Sub-step 2.3: User testing; conduct user experience testing, observe user operations on the interface, collect feedback for optimization; generate user feedback reports; Sub-step 2.4: Interface optimization; Adjust and optimize the interface based on user feedback reports to ensure smooth user operations; generate the final user interface.

5. A method for non-verbal communication between medical staff and patients according to claim 2, It is characterized in that Step 3: Directly point to the interaction method; the specific steps are as follows: Sub-step 3.1: Pointing selection algorithm design; Design a simple pointing selection algorithm that allows patients to select icons by pointing their fingers; Generate a selection algorithm; Sub-step 3.2: Click feedback mechanism; add a click feedback mechanism to the selection algorithm to ensure that the patient's choice is correctly identified; generate a feedback mechanism; Sub-step 3.3: Interaction testing; verify the accuracy of the pointing selection through user testing, collect data for analysis; generate test results; Sub-step 3.4: Algorithm adjustment; According to the test results, necessary adjustments are made to the pointing selection algorithm to improve accuracy and generate optimized selection results.

6. A method for non-verbal communication between medical staff and patients according to claim 2, It is characterized in that Step 4: Eye and head movement recognition; the specific steps are as follows: Sub-step 4.1: Sensor technology selection; select a high-sensitivity sensor that can capture the patient's eye and head movements in real time; generate a sensor configuration; Sub-step 4.2: Image processing algorithm design: Using image processing algorithms, analyze the data captured by the sensor to identify the patient's intention; Generate intent recognition algorithms; Sub-step 4.3: Action conversion mechanism: Design a mechanism to convert the recognized actions into cursor movement or selection to facilitate patient interaction; Generate action conversion results; Sub-step 4.4: System testing: Perform system testing to verify the accuracy of eye and head movement recognition and make necessary adjustments; Generate the final action recognition result.

7. A method for non-verbal communication between medical staff and patients according to claim 2, It is characterized in that Step 5: Touch the screen to interact with the touchpad; the specific steps are as follows: Sub-step 5.1: Touch technology selection; select appropriate touch technology to support touch and gesture interaction and ensure user experience; generate touch technology configuration; Sub-step 5.2: Audio feedback design: Design audio feedback for touch interaction to enhance the user's sense of operation confirmation; Generate audio feedback mechanism; Sub-step 5.3: Gesture recognition algorithm development; Develop a simple gesture recognition algorithm to enable patients to express their needs through gestures; Generate a gesture recognition algorithm; Sub-step 5.4: Interaction test; perform interaction test to verify the accuracy and smoothness of touch screen and touchpad operation; generate touch interaction results.

8. A method for non-verbal communication between medical staff and patients according to claim 2, It is characterized in that Step 6: Result integration and feedback mechanism; the specific steps are as follows: Sub-step 6.1: Data fusion algorithm design: Design a data fusion algorithm to integrate the results of steps 3, 4, and 5 to form a comprehensive output; Generate fusion algorithm; Sub-step 6.2: Feedback mechanism establishment; Establish visual and auditory feedback mechanisms to confirm the correctness of the system's interpretation; Generate feedback mechanisms; Sub-step 6.3: System testing; conduct overall system testing to ensure that all modules function properly after integration and that feedback is effective; generate a system test report; Sub-step 6.4: Optimize the integration results; According to the test report, make necessary optimizations to the integration results and feedback mechanisms; Generate final integrated feedback results.

Images