Medical voice interaction intelligent name card

Abstract

This invention provides a medical voice-interactive smart badge, including a badge body for wearing on medical personnel; a battery module, located within the badge body, for powering various modules; the invention utilizes a bone conduction vibration sensor and an inertial measurement unit attached to the chest for low-power initial trigger detection, waking up the microphone array and positioning module only when the medical personnel are detected speaking or moving, avoiding continuous high-power operation and extending the badge's battery life; simultaneously, it identifies the current dialogue scenario by collecting environmental audio, location information, and posture data after wake-up, and uses corresponding activation conditions to determine the scenario, switching the badge from standby to working state only during actual medical-related dialogue, thus solving the problem of accidental triggering in non-medical scenarios and eliminating the need for manual button operation by medical personnel, achieving hands-free and scenario-adaptive intelligent activation.

Description

Technical Field

[0001] This invention relates to a medical voice-interactive smart badge, belonging to the technical field of medical information and voice interaction. Background Technology

[0002] In clinical medical settings, healthcare professionals frequently need to record patient information, issue medical orders, and query test results. Traditional manual data entry methods are inefficient and increase the risk of cross-infection. To address this, medical voice-interactive badges have emerged in recent years, using voice recognition to assist healthcare professionals in completing medical record entry and information retrieval. However, existing badge activation methods primarily include physical button triggering, wireless signal proximity detection, or generic voice wake-up words. Physical button triggering requires healthcare professionals to free their hands, which is extremely inconvenient in scenarios where hands are occupied, such as during ward rounds or surgeries. Wireless signal proximity detection relies solely on distance, making it prone to accidental activation when healthcare professionals pass by a patient's bedside, rather than during medical conversations. For example, conversations with nurses during shift changes or reassuring conversations with family members can also incorrectly activate the system, resulting in invalid data collection and wasted battery power. Generic voice wake-up words cannot distinguish the speaker's identity or intention, and other people in the environment may also activate the badge. Therefore, this paper proposes a medical voice-interactive smart badge. Summary of the Invention

[0003] In view of this, the present invention provides a medical voice-interactive smart badge to solve or alleviate the technical problems existing in the prior art, and at least provides a beneficial alternative.

[0004] The technical solution of the present invention is as follows: a medical voice-interactive smart badge, including a badge body, for wearing on medical staff; A battery module, located inside the work badge body, is used to power each module; A multimodal sensing module is disposed in the body of the name tag, including an inertial measurement unit and a bone conduction vibration sensor. The inertial measurement unit is configured to collect the wearer's posture data, and the bone conduction vibration sensor is attached to the back of the name tag and fits against the wearer's chest, and is configured to collect vocal cord vibration signals transmitted through the bones. A voice acquisition module, located inside the work badge, includes a microphone array and is configured to acquire ambient audio signals. The positioning module is located inside the badge and is configured to collect the wearer's real-time location information. An edge computing module, located within the employee badge body, is electrically connected to a multimodal perception module, a voice acquisition module, and a positioning module. The edge computing module is configured to receive vocal cord vibration signals acquired by a bone conduction vibration sensor and attitude data acquired by an inertial measurement unit, and determine whether preset preliminary triggering conditions are met. These preliminary triggering conditions include the bone conduction vibration sensor detecting vocal cord vibration signals or the inertial measurement unit detecting movement. When the preliminary triggering conditions are met, the microphone array and positioning module are activated. The current dialogue scenario is identified based on the audio signals, location information, and attitude data acquired after activation. Based on the identified dialogue scenario, an activation condition corresponding to that scenario is applied for judgment. When the judgment result meets the activation condition, the employee badge body is switched from standby mode to working mode. In operation, the collected environmental audio signals are separated by voiceprint to distinguish between medical staff voiceprint channels and non-medical staff voiceprint channels. The separated audio signals are then subjected to speech recognition to obtain text. The patient's personal identity information in the text is then dynamically desensitized using a named entity recognition model to generate desensitized text data. A communication module, located within the employee ID card, is configured to send the de-identified text data to the hospital server.

[0005] More preferably, the standby state is a low-power mode, in which the microphone array, the communication module and the positioning module are in a sleep state, and the inertial measurement unit and the bone conduction vibration sensor collect data at a first sampling rate; The working state is a full-function operation mode. In this working state, the microphone array collects ambient audio signals at a second sampling rate higher than the first sampling rate, and the positioning module continuously collects location information. When the initial triggering condition is met, the edge computing module wakes up the microphone array and the positioning module, and further determines whether the precise activation condition is met based on the audio signal and location information collected after wake-up. When the precise activation conditions are met, the work badge body is switched to the working state.

[0006] More preferably, the edge computing module identifies dialogue scenarios by classifying them into doctor-patient dialogue, medical staff dialogue, doctor-patient family dialogue, or consultation dialogue based on the wearer's orientation, voiceprint recognition results, voice keywords, and location information. The wearer's orientation is calculated from the posture data collected by the inertial measurement unit. Specifically, when the bone conduction vibration sensor detects that the wearer has started to speak, the heading angle of the inertial measurement unit at this time is recorded as a reference. Subsequently, the change angle of the wearer's orientation relative to the reference is calculated by integrating the gyroscope.

[0007] More preferably, when the identified dialogue scenario is a doctor-patient dialogue, the precise activation conditions include the wearer being speaking, the wearer being in the treatment area, the wearer facing the patient, the presence of non-medical voiceprints in the environment, and the speech semantic intent being related to treatment. When the identified dialogue scenario is a medical conversation, the precise activation conditions include the wearer being speaking, the presence of other medical voiceprints in the environment, and the speech semantic intent being related to medical work.

[0008] Further preferred, when the identified dialogue scenario is a dialogue between a doctor, a patient, or a family member, the precise activation conditions include the wearer being speaking, the presence of a voiceprint in the environment that is neither a medical professional nor a registered medical professional, and the semantic intent of the voice being related to informing the patient of their condition or obtaining informed consent. When the identified dialogue scenario is a consultation dialogue, the precise activation conditions include the wearer being speaking, the presence of other medical staff voiceprints in the environment, and the speech semantic intent being related to diagnosis discussion or treatment plan formulation.

[0009] More preferably, the inertial measurement unit includes an accelerometer and a gyroscope; The edge computing module calculates the tilt angle of the wearer's torso based on accelerometer data to identify standing, sitting, bending or walking postures, and calculates the change in the wearer's orientation angle based on gyroscope data. The edge computing module determines whether the wearer is speaking and determines the start and end times of speaking based on the short-time energy of the vibration signal collected by the bone conduction vibration sensor. When the short-time energy exceeds a preset threshold and the duration is greater than milliseconds, it is determined that the wearer has started speaking.

[0010] More preferably, the edge computing module performs voiceprint separation on the audio signal by first performing beamforming processing to enhance the sound in the direction in front of the wearer based on the wearer orientation information provided by the inertial measurement unit; then, it uses a temporal convolutional network architecture to distinguish the audio signal into different speaker channels, wherein the medical voiceprint channel is identified based on preset registered voiceprint features, and the non-medical voiceprint channel is identified through a dynamic clustering algorithm.

[0011] More preferably, the edge computing module dynamically desensitizes the text obtained by speech recognition of the separated audio signal. Specifically, it identifies personal identity information through a named entity recognition model. The personal identity information includes at least one of the following: patient name, ID number, phone number, home address, and social security number. The identified personal identity information is then replaced with the corresponding masking character to generate the desensitized text data.

[0012] More preferably, the edge computing module is also used to adopt different processing strategies based on the identified dialogue scenario. When a doctor-patient dialogue is identified, the system distinguishes between the voiceprint channels of medical staff and non-medical staff, generates a structured medical record document, and completely desensitizes sensitive patient information. When a medical staff conversation is identified, different medical staff voiceprint channels are distinguished, and a medical staff handover record or nursing notes are generated, and the record is restricted from being entered into the main medical record. When a conversation between a doctor, patient, and family member is identified, the voiceprint channels of medical staff and family members are distinguished, and a record of the patient's condition is generated. When a consultation dialogue is identified, different medical staff voiceprint channels are distinguished, and consultation discussion minutes are generated.

[0013] A further preferred embodiment includes bone conduction headphones, which are independent of and wirelessly connected to the employee badge body, and configured to receive decision support information returned by the hospital server and broadcast it to the wearer.

[0014] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions: This invention utilizes a bone conduction vibration sensor and an inertial measurement unit attached to the chest for low-power initial trigger detection. The microphone array and positioning module are only activated when medical staff are detected speaking or moving, avoiding continuous high-power operation and extending the badge's battery life. Simultaneously, the current dialogue scenario is identified by collecting environmental audio, location information, and posture data after activation. Based on different scenarios, corresponding activation conditions are applied to determine the scenario. The badge is switched from standby to working state only during actual medical-related dialogue, thus solving the problem of accidental triggering in non-medical scenarios. It also eliminates the need for medical staff to manually press buttons, achieving intelligent activation that is hands-free and scene-adaptive.

[0015] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the structure of the work badge body in this invention.

[0017] Figure 2 This is a schematic diagram of the structure of the multimodal perception module and the voice acquisition module in this invention.

[0018] in: 10-Employee badge body; 20-Battery module; 30-Multimodal sensing module; 31-Inertial measurement unit; 32-Bone conduction vibration sensor; 40-Voice acquisition module; 41-Microphone array; 50-Positioning module; 60-Edge computing module; 70-Communication module; 80-Bone conduction headphones. Detailed Implementation To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0019] In the description of this invention, it should be noted that when an element is referred to as being "fixed to" or "set on" another element, it can be directly on or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to or indirectly connected to the other element.

[0020] In the description of this invention, it should be noted that the terms "center," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. "Several" means one or more, unless otherwise explicitly specified.

[0021] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0022] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0023] like Figure 1-2 As shown, this embodiment of the invention provides a medical voice-interactive smart badge, including a badge body 10, for wearing on medical personnel; In one embodiment, the battery module 20 is disposed within the work badge body 10 and is used to supply power to each module; The multimodal sensing module 30 is located inside the badge body 10 and includes an inertial measurement unit 31 and a bone conduction vibration sensor 32. The inertial measurement unit 31 is configured to collect the wearer's posture data, and the bone conduction vibration sensor 32 is attached to the back of the badge body 10 and fits against the wearer's chest, and is configured to collect vocal cord vibration signals transmitted through the bones. The voice acquisition module 40 is installed inside the work badge body 10 and includes a microphone array 41, configured to acquire ambient audio signals. The positioning module 50 is installed inside the work badge body 10 and is configured to collect the wearer's real-time location information. An edge computing module 60, located within the employee badge body 10, is electrically connected to the multimodal perception module 30, the voice acquisition module 40, and the positioning module 50. The edge computing module 60 is configured to receive vocal cord vibration signals acquired by the bone conduction vibration sensor 32 and posture data acquired by the inertial measurement unit 31, and determine whether preset preliminary triggering conditions are met. These preliminary triggering conditions include the bone conduction vibration sensor 32 detecting vocal cord vibration signals or the inertial measurement unit 31 detecting movement. When the preliminary triggering conditions are met, the microphone array 41 and the positioning module 50 are activated. The current dialogue scenario is identified based on the audio signals, location information, and posture data acquired after activation. Based on the identified dialogue scenario, activation conditions corresponding to that scenario are used for judgment. When the judgment result meets the activation conditions, the employee badge body 10 is switched from standby mode to working mode. In operation, the collected environmental audio signals are separated by voiceprint to distinguish between medical staff voiceprint channels and non-medical staff voiceprint channels. The separated audio signals are then subjected to speech recognition to obtain text. The patient's personal identity information in the text is then dynamically desensitized using a named entity recognition model to generate desensitized text data. The communication module 70 is located inside the work badge body 10 and is configured to send de-identified text data to the hospital server.

[0024] In this embodiment, the inertial measurement unit 31 adopts a six-axis sensor, integrating a three-axis accelerometer and a three-axis gyroscope, and is fixed on the circuit board inside the name tag body 10. When the name tag is worn on the chest, the coordinate system of the inertial measurement unit 31 forms a fixed mapping relationship with the coordinate system of the wearer's torso. The bone conduction vibration sensor 32 is set on the back of the name tag body 10, and its vibration sensing surface protrudes from the back of the shell. When worn, it fits closely to the chest of the medical staff and can collect the vocal cord vibration signal transmitted through the bones, while effectively shielding the airborne noise in the environment.

[0025] The microphone array 41 consists of four microphones arranged linearly on the side of the name tag body 10, with the openings facing the wearer. This array supports beamforming technology, which can amplify specific sound sources according to their direction and suppress interfering sounds from other directions.

[0026] The positioning module 50 integrates a global navigation satellite system receiver unit, a Bluetooth beacon receiver unit, and an inertial navigation unit. In the hospital indoor environment, it can distinguish ward areas, emergency room areas, intensive care unit areas, nurse station areas, office areas, and corridor areas by deploying Bluetooth beacons in locations such as corridors, ward entrances, and nurse stations.

[0027] The edge computing module 60 uses a main control chip with a built-in neural network processing unit and is electrically connected to the multimodal perception module 30, the voice acquisition module 40, and the positioning module 50. The edge computing module 60 can efficiently run algorithms such as voiceprint separation, speech recognition, and desensitization.

[0028] The communication module 70 supports mobile communication and wireless LAN, and is used to establish a secure connection with the hospital's private server to transmit de-identified text data.

[0029] In one embodiment, the standby state is a low-power mode. In the standby state, the microphone array 41, communication module 70, and positioning module 50 are in a sleep state, and the inertial measurement unit 31 and bone conduction vibration sensor 32 collect data at a first sampling rate. The working state is a full-function operation mode. In the working state, the microphone array 41 collects ambient audio signals at a second sampling rate higher than the first sampling rate, and the positioning module 50 continuously collects location information. When the preliminary triggering conditions are met, the edge computing module 60 wakes up the microphone array 41 and the positioning module 50, and further determines whether the precise activation conditions are met based on the audio signals and location information collected after wake-up. When the precise activation conditions are met, the badge body 10 is switched to the working state.

[0030] In this embodiment, the badge body 10 has two operating modes: standby and working. Standby is the default state, and its goal is to minimize power consumption. In standby mode, the microphone array 41, communication module 70, and positioning module 50 are all in sleep mode, without data acquisition or transmission. The inertial measurement unit 31 and bone conduction vibration sensor 32 acquire data at a first sampling rate. The edge computing module 60 then runs only a very simple judgment logic: receiving the vocal cord vibration signal acquired by the bone conduction vibration sensor 32 and the attitude data acquired by the inertial measurement unit 31, and determining whether the preset preliminary triggering conditions are met. The preliminary triggering conditions include: the bone conduction vibration sensor 32 detects a vocal cord vibration signal, i.e., the wearer begins to speak, or the inertial measurement unit 31 detects movement, such as an acceleration change exceeding a preset threshold. These two conditions constitute an "OR" logic; if either one is met, it is considered that voice interaction may be required.

[0031] When the initial triggering conditions are met, such as when a doctor begins to speak, the edge computing module 60 immediately wakes up the microphone array 41 and the positioning module 50. After waking up, the microphone array 41 begins to collect ambient audio signals, and the positioning module 50 begins to acquire real-time location information. At this time, the edge computing module 60 performs dialogue scene recognition based on the audio signals, location information, and existing posture data collected after waking up. After recognizing the current dialogue scene, the edge computing module 60 uses the activation conditions corresponding to that scene for judgment. This stage is called precise activation condition judgment.

[0032] If the precise activation condition judgment result is true, the edge computing module 60 switches the work badge body 10 from the standby state to the working state. In the working state, the microphone array 41 collects the ambient audio signal at a second sampling rate higher than the first sampling rate, the positioning module 50 continuously collects the location information, the edge computing module 60 performs voiceprint separation, speech recognition and dynamic desensitization processing on the audio signal to generate desensitized text data, and the communication module 70 establishes a connection with the hospital server and sends the desensitized text data.

[0033] If the precise activation condition judgment result is false, such as the doctor just coughing, or the doctor chatting with colleagues in the corridor, or the doctor not talking to the patient when passing by the bed, then the name tag body 10 will not enter the working state, and the microphone array 41 and the positioning module 50 will re-enter sleep mode to save power.

[0034] Through this two-level judgment mechanism, the badge body 10 can maximize battery life while ensuring functional accuracy. The initial triggering condition relies only on the low-power bone conduction vibration sensor 32 and inertial measurement unit 31, without waking up the high-power microphone array 41 and positioning module 50; these modules are only woken up for precise judgment after the initial condition is met, thus avoiding continuous high-power operation.

[0035] In one embodiment, the edge computing module 60 identifies dialogue scenarios by classifying them into doctor-patient dialogue, medical staff dialogue, doctor-patient family dialogue, or consultation dialogue based on the wearer's orientation, voiceprint recognition results, voice keywords, and location information. The wearer's orientation is calculated from the attitude data collected by the inertial measurement unit 31. Specifically, when the bone conduction vibration sensor 32 detects that the wearer has started speaking, it records the heading angle of the inertial measurement unit 31 at this time as a reference, and then calculates the change angle of the wearer's orientation relative to the reference by integrating the gyroscope.

[0036] In this embodiment, calculating the wearer's orientation is fundamental to scene recognition. When the bone conduction vibration sensor 32 detects that the wearer has started speaking, the edge computing module 60 records the heading angle of the inertial measurement unit 31 at this time as a reference. Subsequently, the edge computing module 60 integrates the angular velocity data output by the gyroscope to calculate the change angle of the wearer's orientation relative to the reference in real time. For example, if the reference is 0 degrees and the current integration result is 15 degrees, then the wearer's orientation has deviated 15 degrees to the right relative to the initial reference. This angle is compared with a preset threshold; if it is less than the threshold, it is determined that the wearer is still facing the original direction; otherwise, it is determined that the orientation has changed.

[0037] When recognizing dialogue scenarios, the edge computing module 60 integrates the following four types of information: wearer orientation, voiceprint recognition results, voice keywords, and location information provided by the positioning module 50. Specifically: If the wearer is facing the patient's bed, the orientation angle range of each bed is pre-calibrated via Bluetooth beacons, and the voiceprint separation results show the presence of non-medical voiceprints (i.e., speakers not registered as medical personnel), and the voice keywords contain treatment-related content such as "where do I feel unwell," "blood sugar level," and "medication," and the positioning module 50 shows that the current location is in the ward area or emergency room area, then it is identified as a doctor-patient dialogue.

[0038] If the voiceprint separation results show that there are two or more registered medical staff voiceprints, and the voice keywords include content such as shift handover, medical orders, and nursing care, it will be identified as a medical staff conversation regardless of the wearer's orientation.

[0039] If the voiceprint separation results show the presence of a voiceprint that is neither a medical professional nor a registered medical professional, i.e., a family member's voiceprint, and the voice keywords contain words such as "family member," "consent form," "information about the condition," and "signature," then it is identified as a dialogue between a doctor, patient, and family member.

[0040] If the voiceprint separation results show that there are multiple registered medical voiceprints, and the voice keywords contain words such as "diagnosis", "discussion", "plan", and "consultation", then it is identified as a consultation dialogue.

[0041] Through the above multi-dimensional fusion judgment, the work badge body 10 can accurately distinguish different dialogue scenarios, providing a basis for adopting different activation conditions.

[0042] In one embodiment, when the identified dialogue scenario is a doctor-patient dialogue, the precise activation conditions include the wearer being speaking, the wearer being in the treatment area, the wearer facing the patient, the presence of a non-medical voiceprint in the environment, and the speech semantic intent being related to diagnosis and treatment. When the identified dialogue scenario is a medical staff dialogue, the precise activation conditions include the wearer being speaking, the presence of other medical staff voiceprints in the environment, and the speech semantic intent being related to medical work. When the identified dialogue scenario is a doctor-patient-family dialogue, the precise activation conditions include the wearer being speaking, the presence of a non-medical staff voiceprint that is not registered, and the speech semantic intent being related to informing the patient of their condition or obtaining informed consent. When the identified dialogue scenario is a consultation dialogue, the precise activation conditions include the wearer being speaking, the presence of other medical staff voiceprints in the environment, and the speech semantic intent being related to discussing a diagnosis or formulating a treatment plan.

[0043] In this embodiment, when the identified dialogue scenario is a doctor-patient conversation, the precise activation condition requires the simultaneous fulfillment of five sub-conditions: the wearer is speaking, as confirmed by the bone conduction vibration sensor 32; the wearer is in a treatment area, as confirmed by the positioning module 50, including ward areas, emergency room areas, intensive care unit areas, etc.; the wearer is facing the patient, as confirmed by the inertial measurement unit 31; non-medical voiceprints are present in the environment, as confirmed by voiceprint separation; and the semantic intent of the speech is related to diagnosis and treatment, as confirmed by the lightweight semantic model. Only when all five conditions are met will the system switch to the working state. This design can avoid accidental activation when a doctor passes by the bedside, and also avoid accidental activation when a doctor and nurse discuss non-diagnosis and treatment matters in the ward.

[0044] When the identified dialogue scenario is a medical-staff conversation, the precise activation conditions are relatively lenient: the wearer only needs to be speaking, other medical-staff voiceprints must be present in the environment, and the semantic intent of the speech must be related to medical work, such as shift handover, medical orders, or nursing care. These activation conditions do not require the wearer to be facing the patient, nor do they require the presence of non-medical-staff voiceprints. This design facilitates the automatic recording of shift handover content and nursing notes when doctors and nurses communicate in areas such as nurses' stations and offices.

[0045] When the identified dialogue scenario is a conversation between a medical staff member, a patient, or a family member, the precise activation conditions include: the wearer is speaking, there is a voiceprint in the environment that is neither a medical professional nor a registered medical professional, and the semantic intent of the speech is related to informing the patient of their condition or obtaining informed consent. This condition does not require the wearer to be facing the patient, as family members may not be at the bedside, but it requires that the semantic content involve informing or obtaining consent to avoid recording irrelevant casual conversations.

[0046] When the identified dialogue scenario is a consultation, the precise activation conditions include: the wearer is speaking, other medical professionals' voiceprints are present in the environment, and the semantic intent of the speech is related to diagnostic discussion or treatment plan formulation. This condition does not require the wearer to be facing the patient, as consultations may take place in an office or meeting room, but it does require the semantic content to involve diagnostic and treatment decisions.

[0047] By configuring differentiated activation conditions for different scenarios, the system not only ensures the complete recording of critical medical conversations but also avoids unnecessary conversations from interfering with the system, while saving power and network resources.

[0048] In one embodiment, the inertial measurement unit 31 includes an accelerometer and a gyroscope; the edge computing module 60 calculates the tilt angle of the wearer's torso based on accelerometer data to identify standing, sitting, bending, or walking postures, and calculates the change in the wearer's orientation angle based on gyroscope data; the edge computing module 60 determines whether the wearer is speaking and determines the start and end times of speech based on the short-time energy of the vibration signal collected by the bone conduction vibration sensor 32. When the short-time energy exceeds a preset threshold and the duration is greater than 100 milliseconds, it is determined that the wearer has started speaking. The specific method of the edge computing module 60 for voiceprint separation of the audio signal is as follows: first, beamforming processing is performed to enhance the sound facing the wearer's front direction based on the wearer's orientation information provided by the inertial measurement unit 31; then, a temporal convolutional network architecture is used to distinguish the audio signal into different speaker channels, where the medical voiceprint channel is identified based on preset registered voiceprint features, and the non-medical voiceprint channel is identified through a dynamic clustering algorithm. The edge computing module 60 dynamically desensitizes the text obtained from the speech recognition of the separated audio signal. Specifically, it identifies personal identity information through a named entity recognition model. The personal identity information includes at least one of the following: patient name, ID number, phone number, home address, and social security number. The identified personal identity information is then replaced with the corresponding masking character to generate desensitized text data.

[0049] In this embodiment, the accelerometer in the inertial measurement unit 31 outputs triaxial acceleration values. The edge computing module 60 calculates the components of gravitational acceleration along each axis to obtain the wearer's torso pitch angle. A pitch angle between 80 and 100 degrees is determined to be a standing posture, between 70 and 80 degrees to a sitting posture, and between 30 and 70 degrees to a bent-over posture. When the accelerometer data exhibits periodic fluctuations, it is determined to be a walking state; when the fluctuation amplitude suddenly increases and exceeds a threshold, it is determined to be a rapid movement state. The gyroscope outputs triaxial angular velocity values, and the edge computing module 60 integrates the angular velocity to obtain the angle change. Combined with a reference reference, the absolute angle of the wearer's orientation can be obtained in real time.

[0050] The vibration signal collected by the bone conduction vibration sensor 32 is calculated by the edge computing module 60 using its short-term energy. When the short-term energy exceeds a preset threshold and the duration is greater than 100 milliseconds, it is determined that the wearer has started speaking; when the short-term energy is lower than the threshold and the duration is greater than 200 milliseconds, it is determined that speaking has ended. By detecting the rising and falling edges of the signal, the start and end times of speaking can be accurately determined.

[0051] The edge computing module 60 performs voiceprint separation on the audio signal as follows: First, beamforming is performed. Based on the wearer's orientation information provided by the inertial measurement unit 31, the beam is directed in front of the wearer, i.e., the direction the medical staff are facing, thereby enhancing the sound in that direction and suppressing interference from the sides and rear. After beamforming, the edge computing module 60 uses a temporal convolutional network architecture to distinguish the audio signal into different speaker channels. The medical staff voiceprint channel is identified based on pre-registered voiceprint features. The hospital establishes a voiceprint template for each medical staff member and stores it in the name tag body 10. The non-medical staff voiceprint channel is identified through a dynamic clustering algorithm.

[0052] After voiceprint separation, the edge computing module 60 performs speech recognition on the audio from each channel, converting the audio into text. Subsequently, the text undergoes dynamic desensitization. Desensitization uses a named entity recognition model, which can identify personally identifiable entities in the text, including patient names, ID numbers, phone numbers, home addresses, and social security numbers. After identifying these entities, the edge computing module 60 replaces them with corresponding masking characters. The desensitized text retains complete medical semantic information but removes all personally identifiable content.

[0053] In one embodiment, the edge computing module 60 is further configured to employ different processing strategies based on the identified dialogue scenario. When the dialogue is identified as a doctor-patient conversation, it distinguishes between the voiceprint channels of medical staff and non-medical staff, generates a structured medical record document, and completely desensitizes sensitive patient information. When the dialogue is identified as a medical staff conversation, it distinguishes between different medical staff voiceprint channels, generates a medical staff handover record or nursing notes, and restricts the record from being included in the main medical record. When the dialogue is identified as a conversation between a doctor, patient, and family member, it distinguishes between the voiceprint channels of medical staff and family members, and generates a record of patient information. When the dialogue is identified as a consultation conversation, it distinguishes between different medical staff voiceprint channels, and generates consultation discussion minutes. The system also includes a bone conduction headset 80, which is independent of and wirelessly connected to the employee badge body 10, and configured to receive decision support information returned by the hospital server and broadcast it to the wearer.

[0054] In this embodiment, the bone conduction headphones 80 adopt an ear-hook structure, fitting snugly above the cheekbone area when worn. Therefore, the external auditory canal of medical personnel is fully open when wearing them, allowing them to simultaneously hear the decision information broadcast by the bone conduction headphones and ambient sounds. The bone conduction headphones 80 are wirelessly connected to the work badge body 10 via Bluetooth. When the hospital server returns decision support information, the communication module 70 receives the information and forwards it to the bone conduction headphones 80 for immediate broadcast.

[0055] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A medical voice-interactive smart badge, characterized in that: Includes the work badge (10), which is worn by medical staff; A battery module (20) is disposed inside the work badge body (10) and is used to supply power to each module; A multimodal sensing module (30) is disposed in the nameplate body (10) and includes an inertial measurement unit (31) and a bone conduction vibration sensor (32). The inertial measurement unit (31) is configured to collect the wearer's posture data. The bone conduction vibration sensor (32) is attached to the back of the nameplate body (10) and fits against the wearer's chest. It is configured to collect vocal cord vibration signals transmitted through the bones. The voice acquisition module (40) is located inside the work badge body (10) and includes a microphone array (41) configured to acquire ambient audio signals; A positioning module (50) is installed inside the work badge body (10) and configured to collect the wearer's real-time location information; An edge computing module (60) is located inside the work badge body (10) and is electrically connected to a multimodal perception module (30), a voice acquisition module (40), and a positioning module (50). The edge computing module (60) is configured to receive vocal cord vibration signals collected by a bone conduction vibration sensor (32) and posture data collected by an inertial measurement unit (31), and determine whether a preset preliminary triggering condition is met. The preliminary triggering condition includes the detection of vocal cord vibration signals by the bone conduction vibration sensor (32) or the detection of movement by the inertial measurement unit (31). When the preliminary triggering condition is met, the microphone array (41) and the positioning module (50) are woken up. The current dialogue scenario is identified based on the audio signals, location information, and posture data collected after wake-up. The activation condition corresponding to the identified dialogue scenario is used for judgment. When the judgment result meets the activation condition, the work badge body (10) is switched from standby state to working state. In operation, the collected environmental audio signals are separated by voiceprint to distinguish between medical staff voiceprint channels and non-medical staff voiceprint channels. The separated audio signals are then subjected to speech recognition to obtain text. The patient's personal identity information in the text is then dynamically desensitized using a named entity recognition model to generate desensitized text data. The communication module (70) is located inside the work badge body (10) and is configured to send the desensitized text data to the hospital server.

2. The medical voice-interactive smart badge according to claim 1, characterized in that: The standby state is a low-power mode. In the standby state, the microphone array (41), the communication module (70) and the positioning module (50) are in a sleep state, and the inertial measurement unit (31) and the bone conduction vibration sensor (32) collect data at a first sampling rate. The working state is a full-function operation mode. In the working state, the microphone array (41) collects ambient audio signals at a second sampling rate higher than the first sampling rate, and the positioning module (50) continuously collects location information. When the initial triggering conditions are met, the edge computing module (60) wakes up the microphone array (41) and the positioning module (50), and further determines whether the precise activation conditions are met based on the audio signals and location information collected after wake-up. When the precise activation conditions are met, the work badge body (10) is switched to the working state.

3. The medical voice-interactive smart badge according to claim 1, characterized in that: The edge computing module (60) identifies dialogue scenarios by classifying them into doctor-patient dialogue, medical staff dialogue, doctor-patient family dialogue, or consultation dialogue based on the wearer's orientation, voiceprint recognition results, voice keywords, and location information. The wearer's orientation is calculated from the posture data collected by the inertial measurement unit (31). Specifically, when the bone conduction vibration sensor (32) detects that the wearer has started to speak, the heading angle of the inertial measurement unit (31) at this time is recorded as a reference. Subsequently, the change angle of the wearer's orientation relative to the reference is calculated by integrating the gyroscope.

4. A medical voice-interactive smart badge according to claim 3, characterized in that: When the identified dialogue scenario is a doctor-patient dialogue, the precise activation conditions include the wearer being speaking, the wearer being in the treatment area, the wearer facing the patient, the presence of non-medical voiceprints in the environment, and the speech semantic intent being related to treatment. When the identified dialogue scenario is a medical conversation, the precise activation conditions include the wearer being speaking, the presence of other medical voiceprints in the environment, and the speech semantic intent being related to medical work.

5. A medical voice-interactive smart badge according to claim 1, characterized in that: When the identified dialogue scenario is a dialogue between a doctor, a patient, or a family member, the precise activation conditions include the wearer being speaking, the presence of a voiceprint from a non-medical staff member who is not a registered medical staff member in the environment, and the semantic intent of the voice being related to informing the wearer of their condition or obtaining informed consent. When the identified dialogue scenario is a consultation dialogue, the precise activation conditions include the wearer being speaking, the presence of other medical staff voiceprints in the environment, and the speech semantic intent being related to diagnosis discussion or treatment plan formulation.

6. A medical voice-interactive smart badge according to claim 1, characterized in that: The inertial measurement unit (31) includes an accelerometer and a gyroscope; The edge computing module (60) calculates the tilt angle of the wearer's torso based on accelerometer data to identify standing, sitting, bending or walking postures, and calculates the change in the wearer's orientation angle based on gyroscope data. The edge computing module (60) determines whether the wearer is speaking and determines the start and end time of speaking based on the short-time energy of the vibration signal collected by the bone conduction vibration sensor (32). When the short-time energy exceeds a preset threshold and the duration is greater than 100 milliseconds, it is determined that the wearer has started speaking.

7. A medical voice-interactive smart badge according to claim 6, characterized in that: The edge computing module (60) performs voiceprint separation on the audio signal by first performing beamforming processing and enhancing the sound in the direction in front of the wearer based on the wearer orientation information provided by the inertial measurement unit (31); then, it uses a temporal convolutional network architecture to distinguish the audio signal into different speaker channels, where the medical voiceprint channel is identified based on the preset registered voiceprint features, and the non-medical voiceprint channel is identified through a dynamic clustering algorithm.

8. A medical voice-interactive smart badge according to claim 3, characterized in that: The edge computing module (60) dynamically desensitizes the text obtained by speech recognition of the separated audio signal. Specifically, it identifies personal identity information through a named entity recognition model. The personal identity information includes at least one of the following: patient name, ID number, telephone number, home address, and social security number. The identified personal identity information is then replaced with the corresponding masking character to generate the desensitized text data.

9. A medical voice-interactive smart badge according to claim 1, characterized in that: The edge computing module (60) is also used to adopt different processing strategies according to the identified dialogue scenario. When a doctor-patient dialogue is identified, the system distinguishes between the voiceprint channels of medical staff and non-medical staff, generates a structured medical record document, and completely desensitizes sensitive patient information. When a medical staff conversation is identified, different medical staff voiceprint channels are distinguished, and a medical staff handover record or nursing notes are generated, and the record is restricted from being entered into the main medical record. When a conversation between a doctor, patient, and family member is identified, the voiceprint channels of medical staff and family members are distinguished, and a record of the patient's condition is generated. When a consultation dialogue is identified, different medical staff voiceprint channels are distinguished, and consultation discussion minutes are generated.

10. A medical voice-interactive smart badge according to claim 1, characterized in that: It also includes a bone conduction headset (80), which is independent of the nameplate body (10) and wirelessly connected to the nameplate body (10), and is configured to receive decision support information returned by the hospital server and broadcast it to the wearer.

Images