Intelligent nursing interaction system based on voice instruction recognition

Abstract

The invention discloses an intelligent nursing interaction system based on voice instruction recognition, and relates to the technical field of nursing, and the technical scheme is characterized in that a user performs real-time voice stream recording and then generates a voice file; using the voice large model to identify the voice file and then outputting text information; analyzing the text information by using a large language model to obtain nursing object information and a document operation instruction; obtaining a corresponding nursing object ID from a database according to the analyzed nursing object information; obtaining an address of a target document from a knowledge base according to the analyzed document operation instruction; generating feedback execution information based on the nursing object ID and the document address; a voiceprint feature file and an instruction intention mapping model of the nursing object are established, the voiceprint feature file comprises voiceprint baseline parameters and disease related voice features of the nursing object, and the instruction intention mapping model is used for associating voice instructions and nursing demand types; the method has the effect that the instruction input efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of nursing technology, and more specifically, to a smart nursing interaction system based on voice command recognition. Background Technology

[0002] In traditional nursing settings, the transmission and processing of nursing information often rely on manual operations. For example, nurses need to personally record patients' nursing needs, vital signs, and other information, and then manually search for patient records and relevant nursing documents. This process is not only time-consuming but also prone to errors in information recording or delays in retrieval. Furthermore, there is a lack of efficient and accurate methods for recognizing and analyzing patients' nursing needs expressed verbally, making it difficult to quickly translate voice commands into specific nursing procedures. Moreover, there is no mechanism for collecting and analyzing the voice characteristics related to the patient's condition during the nursing process, hindering the timely detection of subtle changes in the patient's condition and thus affecting the timeliness and effectiveness of nursing care. Summary of the Invention

[0003] In view of the shortcomings of existing technologies, the purpose of this invention is to provide a smart nursing interaction system based on voice command recognition.

[0004] To achieve the above objectives, the present invention provides the following technical solution: A smart nursing interaction system based on voice command recognition includes: Input module: After the user inputs real-time voice stream, an audio file is generated; the audio file is recognized using a large voice model and then the text information is output. Parsing module: Uses a large language model to parse text information to obtain nursing object information and document operation instructions; retrieves the corresponding nursing object ID from the database based on the parsed nursing object information; retrieves the address of the target document from the knowledge base based on the parsed document operation instructions; generates feedback execution information based on the nursing object ID and document address; Construction module: Establish a voiceprint feature profile and instruction intent mapping model for nursing subjects. The voiceprint feature profile includes the voiceprint baseline parameters of the nursing subject and the speech features associated with the symptoms. The instruction intent mapping model is used to associate speech instructions with nursing need types. Recognition module: Recognizes the real-time speech stream to obtain the real-time voiceprint baseline parameters of the nursing object, and the voiceprint feature file parses the real-time voiceprint baseline parameters to obtain real-time disease-related speech features; Processing module: Performs voiceprint verification and instruction segmentation on the real-time voice stream to obtain the voice instructions to be processed; calls the instruction intent mapping model to parse the intent of the voice instructions to be processed and generate real-time nursing need tags. The generation module processes real-time nursing needs tags, real-time symptom-related voice features, feedback execution information, nursing action instruction sets, and nursing data to generate nursing interaction reports.

[0005] Preferably, the real-time nursing need tags, real-time symptom-related voice features, feedback execution information, nursing action instruction sets, and nursing data are processed to generate a nursing interaction report, specifically including the following steps: Based on the real-time symptom-related speech features of the nursing patients, weights are assigned to the real-time nursing need labels to obtain a priority nursing need sequence; a nursing action instruction set is generated based on the priority nursing need sequence. Nursing interaction reports are generated based on the feedback from the patients, nursing action instructions, and nursing data.

[0006] Preferably, establishing a voiceprint feature profile of the nursing subject and a mapping model of instruction intent specifically includes the following steps: Collect speech samples from nursing subjects in resting, active, and pain states; Voiceprint feature parameters are extracted from language samples; wherein, the voiceprint feature parameters include fundamental frequency, spectral entropy, formant frequency and speech rhythm; Label the symptom status of each voice sample with the medical record data of the nursing subjects; Voiceprint profiles are created based on the correlation between voiceprint characteristics and disease status. A corpus of voice commands collected from nursing process instructions; A deep learning algorithm is used to train an instruction intent mapping model. The text features and acoustic features of the speech instruction corpus are input into the instruction intent mapping model to obtain the corresponding nursing need types.

[0007] Preferably, the real-time voice stream is subjected to voiceprint verification and instruction segmentation to obtain the voice instruction to be processed, specifically including the following steps: The acquired real-time speech stream is processed by frame segmentation to extract the voiceprint features of each frame. The voiceprint feature is compared with the baseline parameters of the voiceprint feature file to calculate the voiceprint matching degree. When the voiceprint matching degree is higher than the preset voiceprint matching threshold, it is determined to be valid speech; Real-time speech streams are segmented into speech commands to be processed based on speech pause features and semantic punctuation features. When the voiceprint matching degree is lower than the preset voiceprint matching threshold, authentication is triggered.

[0008] Preferably, the instruction intent mapping model is invoked to parse the intent of the voice instruction to be processed and generate real-time nursing need labels, specifically including the following steps: The text information and acoustic features converted from the voice command to be processed are input into the command intent mapping model to obtain the corresponding nursing need type, expected effect description and initial urgency. Preliminary nursing need tags are generated based on the nursing need type and expected outcome description; wherein, the preliminary nursing need tags include the need content and the execution target; Collect real-time vital sign data of the nursing subject; wherein, the vital sign data includes heart rate variability, respiratory rate, skin resistance and body temperature; Real-time vital signs data are processed with the urgency of nursing needs to generate a real-time nursing needs sequence.

[0009] Preferably, real-time vital sign data is processed with the urgency of nursing needs to generate a real-time nursing need sequence, specifically including the following steps: Establish a correlation between real-time vital signs data and the urgency of nursing needs; The initial urgency level of the initial care needs label is adjusted based on real-time vital signs data; The system then generates a real-time nursing needs sequence based on the revised urgency values.

[0010] Preferably, generating a nursing action instruction set based on a priority nursing needs sequence specifically includes the following steps: The demand execution terminal is obtained by matching the demand tags according to the priority nursing demand sequence. The demand execution terminal includes a main execution terminal and an auxiliary execution terminal. Construct a nursing execution terminal mapping table that associates nursing need types with corresponding execution terminals; wherein, the execution terminals include intelligent nursing beds, infusion control devices, drug delivery devices, and manual nursing terminals; A set of nursing action instructions is generated based on the functional parameters of the execution terminal and the specific content of nursing needs; wherein, the instruction set includes action type, execution force, duration and safety boundary parameters.

[0011] Preferably, the method further includes the following steps: The performance evaluation value is obtained by weighting the feedback emotion index and the rate of change of physiological response data. If the performance evaluation value is within the preset performance range, the current execution parameters of the execution terminal will be maintained. If the performance evaluation value is not within the preset performance range, the current execution parameters of the execution terminal will be adjusted.

[0012] Preferably, the method further includes the following steps: Real-time capture of the patient's voice feedback during the care process; After extracting words expressing satisfaction and words expressing discomfort from the feedback voice, a feedback sentiment index is generated. Collect physiological response data of the nursing subjects; wherein, the physiological response data includes the rate of change and fluctuation range of vital sign parameters.

[0013] Preferably, a nursing interaction report is generated based on the patient's feedback, nursing action instruction set, and nursing data, specifically including the following steps: Extract feedback execution information, nursing action instruction sets, and nursing data from the nursing interaction process; The nursing data includes voiceprint verification results, instruction segmentation records, intent parsing logs, execution parameter adjustment data, and final physiological response data. A nursing interaction report is constructed by using time-series analysis to analyze feedback execution information, nursing action instruction sets, and nursing data; wherein, the nursing interaction report includes instruction response time, execution accuracy, and number of feedback adjustments.

[0014] Compared with the prior art, the present invention has the following beneficial effects: This invention's input module supports real-time voice stream input and generates voice files. It then uses a large-scale voice model to accurately recognize and output text information, improving the efficiency of instruction input and enabling nurses to convey nursing intentions more quickly. The parsing module uses a large-scale language model to parse text information, obtaining nursing subject information and document operation instructions. It retrieves the nursing subject ID from the database and the target document address from the knowledge base, thereby generating feedback execution information. This achieves rapid matching of nursing subject information and document resources, reducing the time cost of manual searching and ensuring that nursing work can be carried out efficiently based on accurate information and resources, avoiding delays in the nursing process due to information errors or untimely resource retrieval. The construction module establishes a voiceprint feature profile of the nursing subject and an instruction intent mapping model. The voiceprint feature profile includes voiceprint baseline parameters and symptom-related voice features, while the instruction intent mapping model associates voice instructions with nursing need types. This lays the foundation for subsequent precision nursing based on voiceprints and instructions, enabling targeted nursing services based on the unique voiceprint of the nursing subject and the intent reflected in the instructions. The recognition module identifies real-time voiceprint baseline parameters from the real-time voice stream and combines them with the voiceprint feature profile to parse real-time symptom-related voice features. This system helps to promptly capture changes in the patient's voiceprint related to their symptoms, enabling nursing staff to take early intervention measures and safeguard the patient's health. The processing module verifies the real-time voice stream and segments the commands to be processed, then uses a command intent mapping model to generate real-time nursing need labels. This process ensures the accuracy and standardization of voice command processing, making nursing work more targeted and avoiding misunderstandings or omissions. The generation module combines real-time nursing need labels, real-time symptom-related voice features, feedback execution information, nursing action command sets, and nursing data processing to generate a nursing interaction report. This facilitates closed-loop management and continuous improvement of nursing work. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the modules of the intelligent nursing interaction system based on voice command recognition proposed in this invention; Figure 2 This is a schematic diagram illustrating the method for obtaining a real-time nursing demand sequence in a smart nursing interaction system based on voice command recognition, as proposed in this invention. Figure 3 This is a schematic diagram of the intelligent nursing interaction system based on voice command recognition proposed in this invention. Detailed Implementation

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0017] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0018] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0019] Reference Figures 1-3 As shown.

[0020] The embodiments further illustrate the intelligent nursing interaction system based on voice command recognition proposed in this invention.

[0021] A smart nursing interaction system based on voice command recognition includes: Input module: After the user inputs real-time voice stream, an audio file is generated; the audio file is recognized using a large voice model and then the text information is output. Parsing module: Uses a large language model to parse text information to obtain nursing object information and document operation instructions; retrieves the corresponding nursing object ID from the database based on the parsed nursing object information; retrieves the address of the target document from the knowledge base based on the parsed document operation instructions; generates feedback execution information based on the nursing object ID and document address; For example, a nurse might say, "I'm going to measure patient Li Si's temperature. The target temperature should be recorded as 36.5℃, and the target heart rate as 75 beats per minute." This real-time audio stream is recorded and generated as an audio file. A large-scale speech model is then used to process this audio file. The speech model can recognize the speech content, convert the speech signal into text, and finally output the corresponding text information. In this case, the output text information is: "I'm going to measure patient Li Si's temperature. The target temperature should be recorded as 36.5℃, and the target heart rate as 75 beats per minute."

[0022] The parsing module uses a large language model to analyze the text information output by the input module, specifically the text about measuring patient Li Si's body temperature (target temperature: 36.5℃, target heart rate: 75 beats / minute). The large language model's semantic understanding capabilities extract the nursing subject information and document operation instructions. The nursing subject information is clearly defined as patient Li Si, and the document operation instructions are for measuring body temperature and heart rate. The parsing module uses the extracted nursing subject information (patient Li Si) and the database (which stores all patient information, including a unique identifier, the nursing subject ID) to find Li Si's nursing subject ID (let's say 002). Simultaneously, based on the parsed document operation instructions for measuring body temperature and heart rate, the parsing module searches the knowledge base. The knowledge base stores the target document addresses for various document operations; after searching, the address of the document related to measuring body temperature and heart rate is found, generating feedback execution information.

[0023] Construction module: Establish the voiceprint feature profile and instruction intent mapping model of the nursing object. The voiceprint feature profile contains the voiceprint baseline parameters of the nursing object and the speech features associated with the disease. The instruction intent mapping model is used to associate speech instructions with nursing needs types. Recognition module: Recognizes the real-time speech stream to obtain the real-time voiceprint baseline parameters of the nursing object, and the voiceprint feature file parses the real-time voiceprint baseline parameters to obtain real-time disease-related speech features; Processing module: Performs voiceprint verification and instruction segmentation on the real-time voice stream to obtain the voice instructions to be processed; calls the instruction intent mapping model to parse the intent of the voice instructions to be processed and generate real-time nursing need tags. The generation module processes real-time nursing needs tags, real-time symptom-related voice features, feedback execution information, nursing action instruction sets, and nursing data to generate nursing interaction reports.

[0024] When a user records a real-time voice stream, a corresponding voice file is generated. A large-scale voice model is then used to recognize the voice file, converting the voice content into text information to provide text material for subsequent information processing.

[0025] A large language model is used to parse the text information output by the input module, extracting nursing object information and document operation instructions. Based on the parsed nursing object information, the corresponding nursing object ID is retrieved from the database; based on the parsed document operation instructions, the address of the target document is retrieved from the knowledge base. Feedback execution information is generated based on the nursing object ID and document address, clarifying the basis for subsequent execution operations in nursing work.

[0026] A voiceprint feature profile and instruction intent mapping model for nursing patients are established. The voiceprint feature profile includes the patient's voiceprint baseline parameters and speech features associated with their symptoms, reflecting the patient's speech characteristics under different health conditions. The instruction intent mapping model is used to associate speech instructions with types of nursing needs.

[0027] When a real-time audio stream is input, it is recognized to obtain the real-time voiceprint baseline parameters of the patient. The voiceprint feature profile is then analyzed to obtain real-time symptom-related speech features, which are used to determine the patient's current speech performance related to their symptoms.

[0028] First, voiceprint verification is performed on the real-time voice stream to ensure the accuracy of the voice source. At the same time, the voice stream is segmented into instructions, dividing the continuous voice stream into voice instructions to be processed. The instruction intent mapping model established by the construction module is called to perform intent parsing on the voice instructions to be processed to generate real-time nursing need labels, thereby clarifying the specific direction of nursing needs.

[0029] The system processes real-time nursing needs tags, real-time symptom-related voice features, feedback execution information, nursing action instruction sets, and nursing data to generate a nursing interaction report, which comprehensively and in detail presents the entire nursing interaction process and related results.

[0030] The process of generating a nursing interaction report by processing real-time nursing need tags, real-time symptom-related voice features, feedback execution information, nursing action instruction sets, and nursing data includes the following steps: Based on the real-time symptom-related speech features of the nursing patients, weights are assigned to the real-time nursing need labels to obtain a priority nursing need sequence; a nursing action instruction set is generated based on the priority nursing need sequence. Nursing interaction reports are generated based on the feedback from the patients, nursing action instructions, and nursing data.

[0031] This application prioritizes nursing needs and generates a set of nursing action instructions. The system assigns weights to real-time nursing need labels based on the patient's real-time symptom-related speech features, thus obtaining a priority nursing need sequence. For example, if the patient's speech features related to acute chest pain are prominent, such as rapid speech accompanied by painful groans, the weight of the real-time nursing need label for relieving acute chest pain will be significantly increased, and its priority in the priority nursing need sequence will be higher. After obtaining this priority nursing need sequence, a set of nursing action instructions is generated based on this sequence. Assuming the highest priority need is relieving acute chest pain, the nursing action instruction set includes specific and targeted instructions such as keeping the patient in a supine position, immediately notifying the doctor, and preparing electrocardiogram monitoring equipment and emergency medications.

[0032] The system comprehensively collects feedback from patients regarding the execution of nursing actions, such as whether chest pain has been relieved or whether other discomfort has occurred after the actions are performed. It also integrates the generated nursing action instruction set and various nursing data produced during the nursing process, such as vital sign monitoring data (heart rate, blood pressure), and specific time records of nursing operations, such as the time of instructing the patient to lie flat and the time of notifying the doctor. The nursing interaction report generated by comprehensively processing and analyzing this feedback, nursing action instruction set, and nursing data provides a comprehensive and detailed presentation of the entire nursing process. For example, the report clearly shows that due to the obvious symptom-related vocal characteristics of acute chest pain, this need is high in the priority sequence, thus executing the nursing actions of placing the patient in a supine position and notifying the doctor. The report also displays the changes in the patient's chest pain symptoms and fluctuations in vital signs after these actions, providing strong evidence for subsequent nursing work evaluation and optimization.

[0033] Establishing a voiceprint profile of the nursing patient and a mapping model of instruction intent includes the following steps: Collect speech samples from nursing subjects in resting, active, and pain states; Voiceprint feature parameters are extracted from language samples; these parameters include fundamental frequency, spectral entropy, formant frequency, and speech rhythm. Label the symptom status of each voice sample with the medical record data of the nursing subjects; Voiceprint profiles are created based on the correlation between voiceprint characteristics and disease status. A corpus of voice commands collected from nursing process instructions; A deep learning algorithm is used to train an instruction intent mapping model. The text features and acoustic features of the speech instruction corpus are input into the instruction intent mapping model to obtain the corresponding nursing need types.

[0034] Comprehensive voice data was collected from nursing subjects in resting, active, and pain states. Signal processing algorithms were used to analyze the fundamental frequency, spectral entropy, formant frequencies, and speech rhythm from the voice samples. The fundamental frequency parameter reflects the pitch of the voice; for example, during pain, the fundamental frequency increases from 120Hz at rest to over 160Hz due to physiological stress. Spectral entropy measures the spectral complexity of the voice; during active states, the increased respiratory rate causes the spectral entropy value to increase from 0.6 at rest to approximately 0.8. Formant frequencies are related to the physiological state of the vocal tract; when patients have respiratory inflammation, the F1 frequency shifts by about 100Hz. Speech rhythm is determined by the syllable intervals; during symptom states, the pause frequency increases from the normal 10 times per minute to over 15 times per minute.

[0035] If a resting voice sample is accompanied by a medical record showing a blood glucose level of 8.5 mmol / L and the fundamental frequency of the voice is stable at 130 Hz, it can be labeled as "Type II Diabetes - Resting - High Blood Glucose"; if the active voice sample shows the patient feeling weak while walking and has a heart rate of 95 beats / minute, it can be labeled as "Diabetes Complications - Active - Mild Weakness".

[0036] The extracted parameters are associated with and stored with tags to form a dynamic profile. For example, in a patient's voiceprint profile, the resting fundamental frequency range is recorded as 110-140Hz. When the real-time voice fundamental frequency is consistently higher than 150Hz and the spectral entropy exceeds 0.9, the system automatically associates it with symptoms indicating pain or stress response, providing a benchmark for subsequent real-time identification.

[0037] A corpus of voice commands collected from nursing procedures was developed, employing a CNN-LSTM hybrid neural network architecture. The input layer simultaneously receives textual and acoustic features. Taking "emergency help - difficulty breathing" as an example, the model learns the combination patterns of textual keywords and acoustic features for this type of command through training. When voice input is received in real-time recognition, the model quickly outputs the corresponding nursing need type, along with a description of the expected effect ("immediate oxygen supply required") and a 4-level urgency score. The entire training process is iteratively optimized using real-world medical scenario data.

[0038] The process of performing voiceprint verification and instruction segmentation on real-time audio streams to obtain the audio instructions to be processed includes the following steps: The acquired real-time speech stream is processed by frame segmentation to extract the voiceprint features of each frame. The voiceprint feature is compared with the baseline parameters of the voiceprint feature file to calculate the voiceprint matching degree. When the voiceprint matching degree is higher than the preset voiceprint matching threshold, it is determined to be valid speech; Real-time speech streams are segmented into speech commands to be processed based on speech pause features and semantic punctuation features. When the voiceprint matching degree is lower than the preset voiceprint matching threshold, authentication is triggered.

[0039] This application first performs frame segmentation processing on the acquired real-time speech stream. This involves dividing the continuous speech signal into numerous short frames according to time intervals. This frame segmentation operation allows for the extraction of voiceprint features from each frame, such as fundamental frequency, spectrum, and formants. These features are crucial for determining the validity of the speech and performing instruction segmentation. When a patient expresses "I have chest pain and want to have an electrocardiogram," the frame segmentation process breaks this sentence down into multiple short frames, each from which unique voiceprint features can be extracted.

[0040] The extracted voiceprint features of each frame are compared with baseline parameters in a pre-established voiceprint feature archive to calculate the voiceprint matching degree. The voiceprint feature archive stores the baseline parameters of the patient's voiceprint in different states; for example, a patient's fundamental frequency at rest is usually between 120-150Hz. The degree of matching is obtained by comparing the fundamental frequency and other parameters in the real-time extracted voiceprint features with these baseline parameters.

[0041] First, it's necessary to clarify that the voiceprint feature file stores various baseline parameters of the patient's voiceprint under different states, such as the normal range or typical values ​​of fundamental frequency, spectrum, and formant parameters. After extracting the voiceprint features of each frame from the real-time speech stream, these extracted voiceprint features are compared one by one with the corresponding baseline parameters in the voiceprint feature file. Taking fundamental frequency as an example, assuming the baseline parameter range of a patient's fundamental frequency in the voiceprint feature file is 120-150Hz, while the fundamental frequency in a real-time extracted frame of speech voiceprint features is 135Hz, then a match is determined in the fundamental frequency feature. A similar approach is used for the spectrum and formant voiceprint features, comparing the real-time extracted feature values ​​with the baseline parameter range or typical values.

[0042] The real-time extracted voiceprint features are combined into a feature vector, and the baseline parameters in the voiceprint feature file are also combined into a corresponding vector. The cosine similarity between these two vectors is then calculated, and this similarity value serves as an important indicator of voiceprint matching accuracy. Furthermore, different voiceprint features are assigned different weights because different voiceprint features have varying degrees of importance in identifying individuals. For example, the fundamental frequency is more important than some less important spectral features, and these are given higher weights. A comprehensive calculation is then performed to obtain a voiceprint matching score that reflects the degree of matching between the real-time voiceprint and the baseline parameters. This comparison and calculation process determines the matching between the real-time voiceprint and the baseline parameters in the voiceprint feature file, thus providing a basis for subsequent operations such as determining the validity of the speech.

[0043] If the voiceprint matching degree is higher than the preset voiceprint matching threshold, it indicates that the voice comes from a registered and legitimate care subject and is judged as valid voice. At this time, the real-time voice stream is segmented based on voice pause features and semantic punctuation features to obtain the voice commands to be processed. Taking a patient saying "I have chest pain and want to have an electrocardiogram" as an example, there is a natural pause between "chest pain" and "want to have an electrocardiogram" in the voice. At the same time, semantically, these two parts can also be considered as independent statements. Based on this voice pause and semantic punctuation-like separation, the sentence is segmented into two voice commands to be processed: "I have chest pain" and "want to have an electrocardiogram," which facilitates subsequent operations such as intent parsing for different commands.

[0044] If the voiceprint matching accuracy is lower than the preset voiceprint matching threshold, it means that the voice is not from a registered patient. To ensure system security and the validity of the command, an identity verification process is triggered. Identity verification may involve the voice sender entering their bed number and answering preset questions related to themselves. Only after successful identity verification will the voice be considered valid and proceed to subsequent processing steps, thereby preventing invalid voice commands from unrelated personnel from interfering with the normal operation of the system.

[0045] The instruction intent mapping model is invoked to parse the intent of the voice instruction to be processed and generate real-time nursing need labels. This process includes the following steps: The text information and acoustic features converted from the voice command to be processed are input into the command intent mapping model to obtain the corresponding nursing need type, expected effect description and initial urgency. Preliminary nursing need tags are generated based on the description of nursing need type and expected outcome; the preliminary nursing need tags include the need content and the execution target; Collect real-time vital sign data of the nursing subjects; among which, vital sign data include heart rate variability, respiratory rate, skin resistance and body temperature; Real-time vital signs data are processed with the urgency of nursing needs to generate a real-time nursing needs sequence.

[0046] This application first converts the voice command to be processed into text information and extracts the corresponding acoustic features. For example, if a patient says, "My chest hurts a lot, please get me an electrocardiogram," speech recognition technology converts this sentence into text information, while acoustic feature extraction captures the rise in fundamental frequency and changes in spectral entropy during speech. This text information and acoustic features are then input into a command intent mapping model. This model, trained on a large amount of nursing scenario data, can output the corresponding nursing need type, expected outcome description, and initial urgency level based on the input content. The command intent mapping model determines the nursing need type as a cardiac-related urgent examination, the expected outcome description as quickly identifying the cause of chest pain and ruling out heart disease, and sets the initial urgency level to "urgent."

[0047] Preliminary nursing need labels are generated based on the type of nursing need and the description of the expected outcome. The preliminary nursing need label includes the content of the need and the execution target. The content of the need is the specific nursing request, such as performing an electrocardiogram (ECG) examination. The execution target specifies the objects needed to fulfill the need, such as the ECG equipment and the on-duty nurse.

[0048] Real-time vital sign data of the patient are collected, including heart rate variability, respiratory rate, skin resistance, and body temperature. Heart rate variability reflects the autonomic nervous system's regulatory function; abnormal values ​​indicate stress or other conditions. A normal adult respiratory rate is 12-20 breaths per minute; abnormal increases or decreases are clinically significant. Skin resistance reflects the level of sympathetic nervous system activity and tends to decrease during pain and anxiety. Normal body temperature ranges from 36-37.2°C; abnormal temperatures suggest infection. If the patient not only complains of chest pain but also has a respiratory rate exceeding 30 breaths per minute (far above the normal range) and significant fluctuations in heart rate variability, this provides crucial physiological evidence for assessing the urgency of the patient's care needs.

[0049] Real-time vital sign data is processed in conjunction with the urgency of nursing needs to generate a real-time nursing need sequence. The urgency of the nursing need is adjusted based on the degree of abnormality in the vital sign data. For example, if the initial urgency of a nursing need is marked as urgent, but the real-time respiratory rate is 30 breaths / minute and the heart rate variability is significantly abnormal, the urgency of the nursing need is increased by combining these vital sign data. All nursing needs are then sorted according to their urgency to generate a real-time nursing need sequence. The most urgent nursing needs are addressed first.

[0050] Real-time vital signs data are processed in conjunction with the urgency of nursing needs to generate a real-time nursing needs sequence, which includes the following steps: Establish a correlation between real-time vital signs data and the urgency of nursing needs; The initial urgency level of the initial care needs label is adjusted based on real-time vital signs data; The system then generates a real-time nursing needs sequence based on the revised urgency values.

[0051] This application first establishes the correlation between real-time vital signs data and the urgency of nursing needs. This requires combining medical knowledge and a large amount of clinical data to clarify the impact of changes in different vital signs on the urgency of nursing needs. For example, the respiratory rate of a normal adult is in the range of 12-20 breaths / minute. When the respiratory rate exceeds 30 breaths / minute, it indicates that the patient has a more serious respiratory problem, and the corresponding urgency of nursing needs will increase significantly. Similarly, the normal range of heart rate is 60-100 beats / minute. If the heart rate suddenly rises to above 120 beats / minute, accompanied by other abnormal signs, the urgency of the related nursing needs will increase.

[0052] The initial urgency level of the initial care need label is adjusted based on the real-time collected vital sign data. For example, if the initial care need label is for oxygen therapy and the initial urgency level is set to moderate, but real-time monitoring shows the patient's respiratory rate has reached 35 breaths / minute and blood oxygen saturation has dropped to 88%, while normal blood oxygen saturation should be ≥95%, according to the established correlation, this situation constitutes a severe respiratory abnormality, and the urgency level of this care need needs to be adjusted from moderate to urgent.

[0053] All nursing needs are sorted according to the revised urgency values ​​to generate a real-time nursing need sequence. For example, after revision, the urgency of providing oxygen to a patient becomes urgent, the urgency of assisting the patient to turn over is low, and the urgency of measuring the patient's body temperature is moderate. The generated real-time nursing need sequence will first provide oxygen, then measure the body temperature, and finally assist in turning over, thus ensuring that the most urgent nursing needs are addressed first, thereby improving the timeliness and effectiveness of nursing care.

[0054] The process of generating a nursing action instruction set based on a priority sequence of nursing needs includes the following steps: The demand execution terminals are obtained by matching the demand tags according to the priority nursing demand sequence. The demand execution terminals include the main execution terminal and the auxiliary execution terminal. Construct a nursing execution terminal mapping table that associates nursing need types with corresponding execution terminals; wherein, the execution terminals include intelligent nursing beds, infusion control devices, drug delivery devices, and manual nursing terminals; Nursing action instruction sets are generated based on the functional parameters of the execution terminal and the specific content of nursing needs; the instruction set includes action type, execution force, duration and safety boundary parameters.

[0055] This application first matches the corresponding demand execution terminal based on the demand tags in the priority nursing demand sequence. The demand execution terminal is divided into a main execution terminal and an auxiliary execution terminal. For example, when the demand tag is to assist the patient in adjusting their position to relieve back pressure, the main execution terminal is matched with a smart nursing bed, and the auxiliary execution terminal is a manual nursing terminal used to assist in observing the patient's condition during the adjustment of the smart nursing bed.

[0056] A mapping table of nursing execution terminals needs to be constructed, linking nursing need types to corresponding execution terminals. For example, if the nursing need type is position adjustment, the corresponding execution terminal would be a smart nursing bed; infusion management corresponds to infusion control equipment; and medication administration corresponds to a drug delivery device.

[0057] Nursing action instruction sets are generated based on the functional parameters of the execution terminal and the specific content of the nursing needs. Taking the nursing need of assisting a patient to adjust their position to a semi-recumbent position using a smart nursing bed as an example, the functional parameters of the smart nursing bed include the adjustable angle range of the headboard and the speed of raising and lowering. In the nursing action instruction set generated based on the specific nursing needs, the action type is adjusting the angle of the smart nursing bed's headboard. The execution force must ensure smooth bed raising and lowering to avoid patient discomfort. The duration is set to adjust the headboard from a horizontal position to a 30° semi-recumbent position, taking approximately 10 seconds. The safety boundary parameter is that the headboard adjustment angle should not exceed 60° to prevent the patient from slipping. Another example is the nursing need of administering intravenous antibiotics using a drug delivery device. The functional parameters of the drug delivery device include the injection speed range and dosage accuracy. The generated instruction set shows the action type as intravenous injection, the execution force as an injection rate of 20 drops per minute, the duration as an injection of 10ml of medication, and the safety boundary parameter as an injection rate not exceeding 30 drops per minute to avoid excessive cardiac burden. In this way, the nursing execution terminal can accurately and safely complete nursing actions.

[0058] It also includes the following steps: The performance evaluation value is obtained by weighting the feedback emotion index and the rate of change of physiological response data. If the performance evaluation value is within the preset performance range, the current execution parameters of the execution terminal will be maintained. If the performance evaluation value is not within the preset performance range, the current execution parameters of the execution terminal will be adjusted.

[0059] This application first calculates the performance evaluation value by weighting the feedback emotion index and the rate of change of physiological response data. The feedback emotion index is obtained through the patient's facial expressions and verbal feedback. For example, if the patient shows comfort and relaxation while receiving care, the emotion index is higher. The rate of change of physiological response data measures the speed of change of physiological indicators such as vital signs, heart rate, and blood pressure. For example, when performing a nursing operation to adjust the patient's position on an intelligent nursing bed, if the patient's feedback is relatively comfortable, the emotion index is higher. At the same time, if the patient's heart rate is monitored and steadily changes from 90 beats / minute to 85 beats / minute, and the rate of change of physiological response data is relatively stable, the performance evaluation value of this nursing operation is obtained by weighting these two data.

[0060] The operation of the execution terminal's current execution parameters is determined based on the relationship between the performance evaluation value and the preset performance range. If the performance evaluation value is within the preset performance range, it indicates that the current execution parameters of the execution terminal are appropriate and can achieve a good nursing effect, so the current execution parameters of the execution terminal are maintained. For example, taking the intelligent nursing bed's position adjustment as an example, if the calculated performance evaluation value is within the preset good range, then the intelligent nursing bed will continue to operate according to the current angle and speed execution parameters.

[0061] If the performance evaluation value is not within the preset performance range, the current execution parameters of the execution terminal need to be adjusted. For example, if a patient reports significant discomfort and a low mood index when using an infusion control device, and the physiological response data shows a rapid increase in heart rate, and the weighted performance evaluation value is not within the preset range, then the execution parameters of the infusion control device need to be adjusted, such as slowing down the infusion rate, and the performance evaluation reassessed to ensure that the nursing procedure achieves good results.

[0062] It also includes the following steps: Real-time capture of the patient's voice feedback during the care process; After extracting words expressing satisfaction and words expressing discomfort from the feedback voice, a feedback sentiment index is generated. Collect physiological response data of the nursing subjects; among which, physiological response data includes the rate of change and fluctuation range of vital signs parameters.

[0063] This application first captures the patient's voice feedback during the nursing process in real time. For example, when a caregiver uses a smart nursing bed to adjust a patient's position, the patient may say things like, "This position is very comfortable," or "This angle is a little painful for me." These voices will be captured by the system in a timely manner.

[0064] The captured feedback speech is analyzed to extract words expressing satisfaction and expressions of discomfort, thereby generating a feedback sentiment index. For example, words like "comfortable" and "satisfied" will increase the feedback sentiment index, while expressions of discomfort such as "painful" and "uncomfortable" will decrease it. For instance, if a patient says the posture is okay but a little cramped, the satisfaction word "okay" and the discomfort word "a little cramped" are extracted and combined to generate the corresponding feedback sentiment index.

[0065] Physiological response data of the nursing subjects are collected, including the rate of change and fluctuation range of vital signs parameters. Vital signs parameters include heart rate, blood pressure, and respiratory rate. For example, during position adjustments, the rate of change in the patient's heart rate is monitored. If the patient's original heart rate was 70 beats / minute and it changes to 75 beats / minute after adjustment, the rate of change is (75-70) / 70≈7.14%. Simultaneously, the fluctuation range of blood pressure is monitored to see if it remains stable or experiences significant fluctuations. By combining feedback emotional indices and physiological response data, the effectiveness of nursing care can be more comprehensively and accurately assessed, providing a basis for subsequent adjustments to nursing procedures.

[0066] Based on the patient's feedback, nursing action instructions, and nursing data, a nursing interaction report is generated, which includes the following steps: Extract feedback execution information, nursing action instruction sets, and nursing data from the nursing interaction process; The nursing data includes voiceprint verification results, instruction segmentation records, intent parsing logs, execution parameter adjustment data, and final physiological response data. A nursing interaction report was constructed using time-series analysis to analyze feedback execution information, nursing action instruction sets, and nursing data. The nursing interaction report includes instruction response time, execution accuracy, and number of feedback adjustments.

[0067] This application first extracts feedback execution information, nursing action instruction sets, and nursing data from the nursing interaction process. The nursing data includes voiceprint verification results, instruction segmentation records, intent parsing logs, execution parameter adjustment data, and final physiological response data. In the nursing interaction process of adjusting the patient's position on the intelligent nursing bed, the feedback execution information is the patient's voice feedback indicating satisfaction with the adjusted position; the nursing action instruction set is adjusting the head of the intelligent nursing bed to a 30° semi-recumbent position at a speed of 10° per minute; the voiceprint verification result in the nursing data shows a 95% match between the patient's voiceprint and the baseline voiceprint in the file; the instruction segmentation record segments the patient's voice instruction "Adjust the bed to a semi-recumbent position" into "Adjust the intelligent nursing bed position to a semi-recumbent position"; the intent parsing log records the process of parsing the intent of the instruction; the execution parameter adjustment data shows that the speed was adjusted to 8° per minute due to the patient's slightly faster feedback during the actual adjustment; and the final physiological response data shows that the patient's heart rate was 75 beats per minute and blood pressure was stable after the adjustment.

[0068] A time-series analysis method was used to process the extracted feedback execution information, nursing action instruction sets, and nursing data to construct a nursing interaction report. The time-series analysis method chronologically organizes the sequence of events and related data in the nursing process. The constructed nursing interaction report includes instruction response time, execution accuracy, and the number of feedback adjustments. This information allows for a clear and accurate review and evaluation of the entire nursing interaction process.

[0069] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A smart nursing interaction system based on voice command recognition, characterized in that, include: Input module: Generates audio files after users input real-time audio streams; After recognizing the speech file using a large speech model, the text information is output. Parsing module: Uses a large language model to parse text information to obtain nursing object information and document operation instructions; retrieves the corresponding nursing object ID from the database based on the parsed nursing object information; retrieves the address of the target document from the knowledge base based on the parsed document operation instructions; generates feedback execution information based on the nursing object ID and document address; Construction module: Establish a voiceprint feature profile and instruction intent mapping model for nursing subjects. The voiceprint feature profile includes the voiceprint baseline parameters of the nursing subject and the speech features associated with the symptoms. The instruction intent mapping model is used to associate speech instructions with nursing need types. Recognition module: Recognizes the real-time speech stream to obtain the real-time voiceprint baseline parameters of the nursing object, and the voiceprint feature file parses the real-time voiceprint baseline parameters to obtain real-time disease-related speech features; Processing module: Performs voiceprint verification and instruction segmentation on the real-time voice stream to obtain the voice instructions to be processed; calls the instruction intent mapping model to parse the intent of the voice instructions to be processed and generate real-time nursing need tags. Generation module: Processes real-time nursing needs tags, real-time symptom-related voice features, feedback execution information, nursing action instruction sets, and nursing data to generate nursing interaction reports.

2. The intelligent nursing interaction system based on voice command recognition according to claim 1, characterized in that, The process of generating a nursing interaction report by processing real-time nursing need tags, real-time symptom-related voice features, feedback execution information, nursing action instruction sets, and nursing data includes the following steps: Based on the real-time symptom-related speech features of the nursing patients, weights are assigned to the real-time nursing need labels to obtain a priority nursing need sequence; a nursing action instruction set is then generated based on the priority nursing need sequence. Nursing interaction reports are generated based on the feedback from the patients, nursing action instructions, and nursing data.

3. The intelligent nursing interaction system based on voice command recognition according to claim 2, characterized in that, Establishing a voiceprint profile of the nursing patient and a mapping model of instruction intent includes the following steps: Collect voice samples from nursing subjects in resting, active, and pain states; Voiceprint feature parameters are extracted from language samples; wherein, the voiceprint feature parameters include fundamental frequency, spectral entropy, formant frequency and speech rhythm; Label the symptom status of each voice sample with the medical record data of the nursing subjects; Voiceprint profiles are created based on the correlation between voiceprint characteristics and disease status. A corpus of voice commands collected from nursing process instructions; A deep learning algorithm is used to train an instruction intent mapping model. The text features and acoustic features of the speech instruction corpus are input into the instruction intent mapping model to obtain the corresponding nursing need types.

4. The intelligent nursing interaction system based on voice command recognition according to claim 3, characterized in that, The process of performing voiceprint verification and instruction segmentation on real-time audio streams to obtain the audio instructions to be processed includes the following steps: The acquired real-time speech stream is processed by frame segmentation to extract the voiceprint features of each frame. The voiceprint feature is compared with the baseline parameters of the voiceprint feature file to calculate the voiceprint matching degree. When the voiceprint matching degree is higher than the preset voiceprint matching threshold, it is determined to be valid speech; Real-time speech streams are segmented into speech commands to be processed based on speech pause features and semantic punctuation features. When the voiceprint matching degree is lower than the preset voiceprint matching threshold, authentication is triggered.

5. The intelligent nursing interaction system based on voice command recognition according to claim 4, characterized in that, The instruction intent mapping model is invoked to parse the intent of the voice instruction to be processed and generate real-time nursing need labels. This process includes the following steps: The text information and acoustic features converted from the voice command to be processed are input into the command intent mapping model to obtain the corresponding nursing need type, expected effect description and initial urgency. Preliminary nursing need tags are generated based on the nursing need type and expected outcome description; wherein, the preliminary nursing need tags include the need content and the execution target; Collect real-time vital sign data of the nursing subject; wherein, the vital sign data includes heart rate variability, respiratory rate, skin resistance and body temperature; Real-time vital signs data are processed with the urgency of nursing needs to generate a real-time nursing needs sequence.

6. The intelligent nursing interaction system based on voice command recognition according to claim 5, characterized in that, Real-time vital signs data are processed in conjunction with the urgency of nursing needs to generate a real-time nursing needs sequence, which includes the following steps: Establish a correlation between real-time vital signs data and the urgency of nursing needs; The initial urgency level of the initial care needs label is adjusted based on real-time vital signs data; The system then generates a real-time nursing needs sequence based on the revised urgency values.

7. The intelligent nursing interaction system based on voice command recognition according to claim 6, characterized in that, The process of generating a nursing action instruction set based on a priority sequence of nursing needs includes the following steps: The demand execution terminal is obtained by matching the demand tags according to the priority nursing demand sequence. The demand execution terminal includes a main execution terminal and an auxiliary execution terminal. Construct a nursing execution terminal mapping table that associates nursing need types with corresponding execution terminals; wherein, the execution terminals include intelligent nursing beds, infusion control devices, drug delivery devices, and manual nursing terminals; A set of nursing action instructions is generated based on the functional parameters of the execution terminal and the specific content of nursing needs; wherein, the instruction set includes action type, execution force, duration and safety boundary parameters.

8. The intelligent nursing interaction system based on voice command recognition according to claim 7, characterized in that, It also includes the following steps: The performance evaluation value is obtained by weighting the feedback emotion index and the rate of change of physiological response data. If the performance evaluation value is within the preset performance range, the current execution parameters of the execution terminal will be maintained. If the performance evaluation value is not within the preset performance range, the current execution parameters of the execution terminal will be adjusted.

9. The intelligent nursing interaction system based on voice command recognition according to claim 8, characterized in that, It also includes the following steps: Real-time capture of the patient's voice feedback during the care process; After extracting words expressing satisfaction and words expressing discomfort from the feedback voice, a feedback sentiment index is generated. Collect physiological response data of the nursing subjects; wherein, the physiological response data includes the rate of change and fluctuation range of vital sign parameters.

10. The intelligent nursing interaction system based on voice command recognition according to claim 9, characterized in that, Based on the patient's feedback, nursing action instructions, and nursing data, a nursing interaction report is generated, which includes the following steps: Extract feedback execution information, nursing action instruction sets, and nursing data from the nursing interaction process; The nursing data includes voiceprint verification results, instruction segmentation records, intent parsing logs, execution parameter adjustment data, and final physiological response data. A nursing interaction report is constructed by using time-series analysis to analyze feedback execution information, nursing action instruction sets, and nursing data; wherein, the nursing interaction report includes instruction response time, execution accuracy, and number of feedback adjustments.

Images