System for providing speech to text based hospital paging service

The STT-based hospital call service system addresses low patient usage of bedhead units by enabling efficient two-way communication and EMR integration through voice-to-text conversion, enhancing patient-staff interaction and reducing nurse visits.

KR103046372B1Active Publication Date: 2026-11-11한정민
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
KR1020230028872
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-03-06
Publication Date
2026-11-11
Estimated Expiration
2043-03-06

AI Technical Summary

Technical Problem

Existing hospital bedhead units have low patient usage rates due to being designed primarily for medical staff, leading to inefficient communication and unnecessary nurse visits, and lack of systems for two-way voice communication and integration with EMR.

Method used

A STT-based hospital call service system that converts voice input to text and vice versa, enabling two-way communication, recording patient requests linked to EMR, and supporting multiple languages.

Benefits of technology

Facilitates efficient patient-staff communication, reduces unnecessary nurse visits, and ensures seamless integration with EMR by converting voice to text and vice versa, supporting multilingual interactions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112023025035486-PAT00004_ABST
    Figure 112023025035486-PAT00004_ABST
Patent Text Reader

Abstract

A STT-based hospital call service provision system is provided, comprising a call device that receives voice utterance when a call button is pushed, converts it into STT (Speech to Text), and transmits the text; a user terminal that outputs the text transmitted from the call device; a receiving unit that receives the text from the call device; a transmitting unit that delivers the transmitted text to the user terminal; and a call service provision server that includes an output unit that outputs the text until a confirmation button is pressed on the user terminal.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to a STT-based hospital call service provision system, and provides a system that converts voice speech input into a call device into STT and transmits it to medical staff. Background Technology

[0002] As the aging of the population progresses rapidly while the government's fiscal capacity shrinks, the efficiency and informatization of the medical device sector are emerging as critical challenges of the times. Currently, bedhead units, installed on the wall above the beds of inpatients, are medical devices equipped with power, medical oxygen, pagers, microphones, and lighting for the purpose of patient treatment and care. They are used comprehensively in wards and emergency rooms of various hospitals, including tertiary and large hospitals, regardless of medical specialty. Unique among medical devices, they are capable of communicating with nurses within the hospital, and in line with the trend of integrating IT and communication technologies with medical devices as we enter the information age, they are expected to potentially replace nursing staff in the future. To enhance patient satisfaction, hospital facilities and medical equipment must be provided for comfortable hospital stays and seamless use; however, despite being the medical device located closest to the patient and having the highest exposure frequency, bedhead units have been found to have a very low usage rate by patients. Although they are medical devices used jointly by patients and medical specialists, they are designed primarily for medical staff use, causing difficulties for patients to utilize them.

[0003] At this time, methods utilizing a pager that supports voice communication or transmitting a signal indicating the end of the IV drip to an LED lamp in the nursing station have been researched and developed. In this regard, prior art Korean Published Patent No. 2021-0127890 (published on October 25, 2021) and Korean Registered Patent No. 10-0634820 (published on October 20, 2006) respectively disclose a configuration that supports voice and data communication and performs payment via a mobile integrated service when making a hospital reservation, and a configuration that detects the amount of IV drip and outputs it as voice and melody while simultaneously providing guidance via an LED in the nursing station.

[0004] However, in the former case, hospital reservations are handled via mobile-based O2O (Online to Offline) services and do not utilize pagers installed in the hospital; similarly, in the latter case, the system only notifies patients when their IV fluids have run out and is not configured to transmit patient requests to the nursing station or, conversely, convey messages from the nursing station to the patient. Since mobile phone use is prohibited in hospitals due to various equipment, pagers must be used. However, the pagers currently installed on bed-head units are equipped only with a call button; consequently, pressing the button results in unnecessary duplication of movement, as a nurse must visit the patient, ask what is needed, and then return with equipment to fulfill the request. Therefore, research and development of a system capable of transmitting requests to medical staff via voice commands is required. The problem to be solved

[0005] One embodiment of the present invention provides a STT-based hospital call service provision system that enables two-way communication by converting voice utterance into STT (Speech to Text) and transmitting it to be displayed on the screen of a user terminal when voice utterance is input to a call device, and transmitting it back to the call device as TTS (Text to Speech) when text is input from the user terminal; records logs of patient requests and treatment details linked to EMR (Electronic Medical Records) or stores the history of requests corresponding to the patient ID through multiple speaker identification; and ensures that there are no communication problems between foreigners and medical staff through multilingual support. However, the technical problem to be solved by this embodiment is not limited to the technical problem described above, and other technical problems may exist. means of solving the problem

[0006] As a technical means for achieving the aforementioned technical task, one embodiment of the present invention includes a calling service providing server comprising: a calling device that receives a voice utterance when a call button is pushed, converts it into STT (Speech to Text), and then transmits the text; a receiving unit that receives the text transmitted from the calling device and outputs the text transmitted from the calling device; a transmitting unit that transmits the transmitted text to the user terminal; and an output unit that causes the text to be output at the user terminal until a confirmation button is pressed. Effects of the invention

[0007] According to any one of the means for solving the problem of the present invention described above, when a voice utterance is input to a calling device, the voice utterance is converted into STT (Speech to Text) and transmitted to be displayed on the screen of a user terminal, and when text is input from the user terminal, it is transmitted back to the calling device as TTS (Text to Speech) to enable two-way communication; through multiple speaker identification, logs of patient requests and treatment details are recorded to be linked to EMR (Electronic Medical Records) or the history of requests is stored corresponding to the patient ID, and through multilingual support, there are no problems in communication between foreigners and medical staff. Brief explanation of the drawing

[0008] FIG. 1 is a diagram illustrating an STT-based hospital call service provision system according to an embodiment of the present invention. Figure 2 is a block diagram illustrating a call service provider server included in the system of Figure 1. FIGS. 3 and 4 are drawings for illustrating an embodiment in which an STT-based hospital call service according to an embodiment of the present invention is implemented. FIG. 5 is an operation flowchart illustrating a method for providing an STT-based hospital call service according to an embodiment of the present invention. Specific details for implementing the invention

[0009] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0010] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected" but also cases where they are "electrically connected" with other elements interposed between them. Furthermore, when a part is described as "including" a component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components, and it should be understood that this does not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0011] Terms such as “about,” “substantially,” etc., used throughout the specification, are used to mean at or near the stated value when inherent manufacturing and material tolerances are presented in the stated meaning, and are used to prevent unscrupulous infringers from unfairly exploiting the disclosure in which precise or absolute values ​​are mentioned to aid in understanding the invention. Terms such as “step” or “step of” used throughout the specification of the invention do not mean “step for”.

[0012] In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized by one piece of hardware. Meanwhile, "part" is not limited to software or hardware, and "part" may be configured to reside in an addressable storage medium or configured to run on one or more processors. Accordingly, as an example, "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." In addition, the components and '~parts' may be implemented to play one or more CPUs within the device or secure multimedia card.

[0013] Some of the operations or functions described herein as being performed by a terminal, device, or device may instead be performed by a server connected to said terminal, device, or device. Likewise, some of the operations or functions described as being performed by a server may also be performed by a terminal, device, or device connected to said server.

[0014] In this specification, some of the operations or functions described as mapping or matching with a terminal may be interpreted as meaning mapping or matching the terminal's unique number or personal identification information, which is the terminal's identifying data.

[0015] The present invention will be described in detail below with reference to the attached drawings.

[0016] FIG. 1 is a diagram illustrating an STT-based hospital call service providing system according to an embodiment of the present invention. Referring to FIG. 1, the STT-based hospital call service providing system (1) may include at least one user terminal (100), a call service providing server (300), and at least one calling device (400). However, since the STT-based hospital call service providing system (1) of FIG. 1 is merely an embodiment of the present invention, the present invention is not to be interpreted as being limited by FIG. 1.

[0017] At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1, at least one user terminal (100) can be connected to a call service provider server (300) through the network (200). And, the call service provider server (300) can be connected to at least one user terminal (100) and at least one call device (400) through the network (200). Also, at least one call device (400) can be connected to the call service provider server (300) through the network (200).

[0018] Here, a network refers to a connection structure capable of exchanging information among individual nodes, such as multiple terminals and servers. Examples of such networks include Local Area Networks (LANs), Wide Area Networks (WANs), the World Wide Web (WWW), wired and wireless data networks, telephone networks, and wired and wireless television networks. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc.

[0019] In the following, the term "at least one" is defined as a term including both singular and plural forms, and it will be obvious that even if the term "at least one" does not exist, each component may exist in a singular or plural form and may mean singular or plural. Furthermore, whether each component is provided in a singular or plural form may be changed according to the embodiment.

[0020] At least one user terminal (100) may be a terminal of a medical professional connected to a call service provider server (300) using a web page, app page, program, or application related to an STT-based hospital call service. Additionally, the user terminal (100) may be a terminal that outputs text of a call device (400) via the call service provider server (300), and may be a terminal that inputs text and transmits a message to a call device (400) via the call service provider server (300).

[0021] Here, at least one user terminal (100) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a laptop, desktop, or laptop equipped with a navigation system or a web browser. At this time, at least one user terminal (100) may be implemented as a terminal capable of connecting to a remote server or terminal via a network. At least one user terminal (100) may include all kinds of handheld-based wireless communication devices, such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone, smartpad, tablet PC, etc.

[0022] The call service providing server (300) may be a server that provides an STT-based hospital call service web page, app page, program, or application. Additionally, the call service providing server (300) may be a server that stores the ID, installed room number, bed number, etc. of at least one call device (400). Furthermore, the call service providing server (300) may be a server that creates a communication channel to connect the user terminal (100) and the call device (400), transmits text received from the call device (400) to the user terminal (100), and transmits text entered from the user terminal (100) to the call device (400).

[0023] Here, the call service providing server (300) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a laptop, desktop, laptop, etc. equipped with a navigation system and a web browser.

[0024] At least one calling device (400) may be a device including STT (Speech to Text) that converts speech utterance into text using a web page, app page, program, or application related to an STT-based hospital calling service. When a call button of the calling device (400) is selected, it may be a device that transmits a message to a user terminal (100) via a calling service provider server (300), and may be a device that outputs a message from the user terminal (100) as speech utterance.

[0025] Here, at least one calling device (400) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a laptop, desktop, or laptop equipped with a navigation system or a web browser. At this time, at least one calling device (400) may be implemented as a terminal capable of connecting to a remote server or terminal via a network. At least one paging device (400) may include all kinds of handheld-based wireless communication devices, such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone, smartpad, tablet PC, etc.

[0026] FIG. 2 is a block diagram for explaining a call service provider server included in the system of FIG. 1, and FIG. 3 and FIG. 4 are drawings for explaining an embodiment in which an STT-based hospital call service according to an embodiment of the present invention is implemented.

[0027] Referring to FIG. 2, the call service providing server (300) may include a receiving unit (310), a transmitting unit (320), an output unit (330), a bidirectional unit (340), a multiplexer identification unit (350), a log storage unit (360), and a multilingual support unit (370).

[0028] When a call service providing server (300) or another server (not shown) operating in conjunction with one embodiment of the present invention transmits an STT-based hospital call service application, program, app page, web page, etc. to at least one user terminal (100) and at least one call device (400), the at least one user terminal (100) and at least one call device (400) may install or open the STT-based hospital call service application, program, app page, web page, etc. Additionally, a service program may be operated on at least one user terminal (100) and at least one call device (400) using a script executed in a web browser. Here, a web browser refers to a program that enables the use of web (WWW: World Wide Web) services and receives and displays hypertext described in HTML (Hyper Text Mark-up Language), and includes, for example, Netscape, Explorer, Chrome, etc. In addition, "application" refers to an application on a terminal, and includes, for example, an app running on a mobile terminal (smartphone).

[0029] Referring to FIG. 2, the receiver (310) can receive text from the calling device (400). When the calling button is pushed, the calling device (400) can receive voice utterance, convert it into STT (Speech to Text), and then transmit the text. At this time, depending on the unit cost, complexity of configuration, design, computing resources, and networking resources required to manufacture the calling device (400), the calling device (400) may perform STT, but the receiver (310) may only receive voice utterance and convert it into STT. This is not limited to this, as the end at which voice utterance is received and STT is performed may be changed according to the embodiment. At this time, it is clarified that in addition to the STT function described above in the server-client structure of the present invention, the function described below may also be implemented in the calling service providing server (300), which is the server, or in the calling device (400), depending on the embodiment in which the calling device (400) is implemented, so the arrangement of each function and configuration is not limited to the embodiment described below.

[0030] Speaker Recognition

[0031] Speaker recognition refers to identifying a speaker by extracting unique features from a speech signal produced by a person. For speaker recognition, methods can be used to recognize the speaker by modeling it using Vector Quantization (VQ), Hidden Markov Model (HMM), Gaussian Mixture Model (GMM), Support Vector Machine (SVM), Dynamic Time Warping (DTW), and Artificial Neural Networks (ANN), by extracting features from the speech signal using Mel-Frequency Cepstral Coefficients (MFCC) or Linear Predictive Cepstral Coefficients (LPCC).

[0032] Since most STT engines are English-based, in one embodiment of the present invention, a speech recognition engine developed by ETRI (Electronics and Telecommunications Research Institute) may be used. STT (Speech To Text) is a deep neural network-based speech recognition engine integrated with signal processing algorithms. It is optimized for Korean language recognition and consists of a search engine for utterance verification using real-time word duration modeling, a preprocessing module for dynamic feature extraction for automatic speech recognition, an advanced acoustic model based on deep learning, and a recognition postprocessing tool that supports restoring word segments from sequences of morphemes output by the speech recognition engine and outputting them in sentence units. STT requires training an acoustic model and a language model to produce answers optimized for the domain language. While the language model only requires securing a corpus used in the domain, training the acoustic model requires a text set of utterance content. Accordingly, accuracy can be improved by training the acoustic model using a dataset of speech utterances and dialects from each age group.

[0033] The transmission unit (320) can transmit the transmitted text to the user terminal (100). The user terminal (100) can output the text transmitted from the calling device (400). The calling device (400) may be an IoT (Internet of Things) based device. It may also be a device that is easy to attach and detach and can be installed anywhere, while being able to connect with each user terminal (100) and the calling service provider server (300) periodically or in real time. Alternatively, the calling device (400) may be a device that is connected to an AP (Access Point) to which the user terminal (100) is connected, or connected to the calling service provider server (300) via the user terminal (100) or connected to the AP.

[0034] The output unit (330) outputs text from the user terminal (100) until a confirmation button is entered. If the text is to be displayed for only a short time and then disappear, it may be displayed and then disappear while medical staff, such as nurses, are away from their posts; therefore, it can be configured to be displayed until a confirmation button is entered. In this case, if nurses or other medical staff possess a pager, the text may be transmitted to the pager, and in this case, when a confirmation button is entered on the pager, the text disappears from the screen but remains in the log described later.

[0035] The bidirectional unit (340) can convert text input from the user terminal (100) into Text to Speech (TTS) and transmit it to the calling device (400) for output. At this time, the calling device (400) may include a microphone capable of receiving voice utterance. It may also include a speaker that outputs voice utterance output via the TTS transmitted from the user terminal (100). Additionally, the calling device (400) may further include a display that outputs the text input from the user terminal (100).

[0036] The multiple speaker identification unit (350) inputs the voice utterance input from the calling device (400) into a speaker identification algorithm that is pre-configured to identify the speaker, and can transmit the speaker information of the identified speaker to the user terminal (100).

[0037] <Multiple Speaker Identification>

[0038] If a multi-bed room is used instead of a single room and multiple beds are placed, it is possible that not just one patient is speaking; thus, cases of multiple speakers speaking simultaneously or overlapping speech may occur. To address this, a speech frequency correlation-based speech recognition method can be utilized to recognize speech utterances and speakers using Speech to Text (STT). By extracting only the speech from mixed audio and analyzing it in the frequency domain, residual noise remaining in the speech can be removed through correlation operations on frequency components. By distinguishing between speech signals and noise signals through correlation operations in the frequency domain between speech signals, the STT recognition rate can be improved, and speech signals can be extracted more efficiently.

[0039] <stft>

[0040] Since audio signals are represented as changes in one-dimensional signal strength over time, it is difficult to distinguish the characteristics of each signal based solely on changes in signal strength in the time domain when various signals are mixed. Due to the nature of audio signals, which are very high-dimensional and composed of the sum of various frequencies, there are limitations in identifying characteristics in the time domain.

[0041]

[0042]

[0043] Equation 1 represents the Fourier Transform, an operation that transforms a function in the time domain into the frequency domain by expressing it as the sum of periodic functions with various frequencies. f(x) represents the signal in the time domain, F(u) represents the signal in the frequency domain, and Equation 2 represents Euler's formula. When Equation 1 is applied to an audio signal, the frequency domain components of the corresponding signal can be obtained. For example, while it is difficult to distinguish individual signals when signals with multiple frequencies are mixed and superimposed, applying the Fourier Transform makes it possible to distinguish the magnitudes corresponding to each frequency.

[0044]

[0045] The Short-Time Fourier Transform (STFT) is a method that applies a Fourier transform to data by dividing it into small intervals over time. It is used to efficiently analyze information regarding signals where frequency components change over time. Equation 3 represents the STFT transformation equation, where w(t) indicates the window function. A spectrogram is a three-dimensional graph in which the result of applying the STFT to an audio signal is set with time and frequency as the horizontal and vertical axes, respectively, and the intensity of each frequency is represented by a color change in dB units. For example, when an audio signal to which the STFT has been applied is represented as a spectrogram, the colors are distinguished according to the intensity of the frequencies; colors closer to yellow indicate stronger frequency intensity, while colors closer to purple indicate weaker frequency intensity. The intensity of frequencies in a spectrogram can be examined using the graph.

[0046] Deep Learning

[0047] A spectrogram can be represented as a 2D image in which time information is expressed on the horizontal axis and frequency magnitude in dB units on the vertical axis by dividing a 1D audio signal into intervals of a certain length and applying a Fourier transform to each interval, and it is possible to use it as training data for a deep learning program based on a Convolutional Neural Network (CNN) used in image processing.

[0048] <svs>

[0049] SVS (Singing Voice Separation) is a deep learning network trained to separate voice signals and instrument signals from an audio signal containing both voice and instrument sounds by using two types of audio signals: one in which the singer's voice is recorded along with the audio, and another in which only the instrument sounds are recorded before the voice. The process of separating the audio signal is completed by converting it into a spectrogram, detecting changes in the spectrum to segment the sound source based on the point where the spectral changes abruptly, performing voice separation using the learned information, and restoring the separated spectrogram data back into an audio signal.

[0050] Voice Frequency-Based Noise Filtering

[0051] Since there are limitations in the process of separating audio signals into music and speech using SVS, such as residual noise remaining that is not completely eliminated or new noise being generated due to distortion during the reconstruction of the separated audio sources, a process to remove this noise is necessary to improve STT recognition rates. Therefore, to address the issue of reduced STT recognition rates caused by non-speech elements, a residual noise removal filter utilizing frequency domain correlation can be employed. The speaker's voice signal is separated from an audio signal containing a mixture of multiple voice signals using SVS. By applying STFT to the separated voice signal to convert it to the frequency domain, the frequency components of each segment of the voice signal can be identified, and voices other than the speaker's are filtered out using a filter implemented through frequency domain correlation operations. Of course, it goes without saying that various methods to distinguish multiple speakers and identify each participant can be utilized in addition to the methods described above.

[0052] The log storage unit (360) can store the history of voice utterances collected from the calling device (400) as a history log to be output from the user terminal (100) and synchronized to be recorded in the Electronic Medical Records (EMR). For example, there are cases where a nurse is called after finishing an IV drip, but a call may also be made to complain of various symptoms to medical staff, such as suddenly having a stomachache, feeling unwell, or having leg discomfort. In each case, medical staff record everything in the EMR, but they do not record the text of the voice utterance exactly as it is. If they forget to record it immediately, the EMR will be saved with the symptoms missing from the record, even though the symptoms existed. Accordingly, each history can be stored as a log and written as an attached document in the EMR. Furthermore, if this history is built into big data, meaningful information can be obtained. For example, if a patient hospitalized for disease A possesses physical data B and has condition C while complaining of symptom E, immediate information can be output even in the absence of medical staff, such as "E should be tested or suspected." To this end, the following configurations may be added. Of course, the following configurations are not mandatory.

[0053] Medical data is heterogeneous data that can be classified into structured, unstructured, and image data. The Knowledge Acquisition and Inference Layer (KAIL) provides a foundation for extracting knowledge from data through the support of dialogue-based interactions. KAIL consists of six sub-components: Actionable Knowledge Evolution and Inferencing (AKEI), Automatic Case Generator (ACG), Structured Knowledge Acquisition (SKA), Descriptive Knowledge Acquisition (DKA), Image-based Knowledge Acquisition (IKA), and Dialogue-based Knowledge Acquisition (DBKA). Most hospitals store patient information in their medical record systems in the form of structured data (medical records), unstructured data, and images. This data contains various pattern information ranging from the diagnosis of a patient's disease to treatment and follow-up measures.

[0054] In this context, SKA is responsible for knowledge extraction from structured data. Since the importance of attributes within the data varies, SKA initially selects and extracts only the features critical for knowledge generation, and various machine learning algorithms are applied to the selected feature set to perform knowledge extraction. DKA is responsible for knowledge extraction from unstructured data, utilizing natural language processing techniques. Medical imaging also plays a crucial role in diagnosis and treatment, and IKA assists in the generation of medical knowledge by providing insights into these images. IKA can perform image preprocessing, data segmentation, Region of Interest (RI) definition, defect detection, and deep learning-based model comparison. To facilitate platform access and usage and to streamline knowledge acquisition, IMP provides a conversation-based interface, and knowledge extraction is performed on the conversation content by DBKA. DBKA converts the conversation content into text, after which DKA extracts the key concepts and relationships within the dialogue. DBKA incorporates semantic information into the conversation to grasp the intent and extract knowledge within the appropriate context. The four components described can express the extracted knowledge in the form of IF-THEN rules.

[0055] IMP can integrate generated IF-THEN rules with the hospital's existing system knowledge, and utilizes Ripple Down Rules (RDR), a hierarchical knowledge representation model, to integrate them into a single model. In RDR, each rule is grouped with a central case, which is referred to as a cornerstone case. Therefore, ACG is utilized to generate cases for rules where no cases exist. ACG automatically generates cases by analyzing the attribute types and values ​​of the rules, and these cases become the cornerstone cases for the corresponding rules. All rules, along with their cases, are integrated into the RDR model by AKEI, and the knowledge within the RDR is managed by domain experts. RDR was selected for knowledge modeling and representation because it facilitates the addition, evolution, and verification of new knowledge, allowing physicians to consider only the minimum requirements.

[0056] In addition, one embodiment of the present invention may construct big data regarding the diagnosis, judgment, and consulting results of medical data and perform knowledge base modeling capable of improving the reliability of comprehensive judgment. To this end, an algorithm through deep learning analysis may be designed to calculate and verify test result indices, and further modeling may be performed to provide comprehensive checkup information through judgment knowledge. In a general health checkup process, reliability and speed can be improved by adding knowledge base modeling that applies artificial intelligence analysis technology between the examination stage and the comprehensive judgment stage. A cloud environment must be integrated to construct a big data-based AI analysis-type health checkup knowledge base model. That is, a cloud environment must be established that is advantageous for scalability based on usage rather than on a server-by-server basis by dividing the resources of a large-capacity server and allocating them as independent resources, and which offers superior performance and safety compared to a dedicated server. Based on a cloud environment, an algorithm can be designed and applied to construct a database capable of assigning meaning to current test results by analyzing data received from the checkup center, and based on this, AI analysis can be introduced to utilize the analyzed and processed data for comprehensive judgment.

[0057] The multilingual support unit (370) can detect the language input from the calling device (400), convert the result of translating the language into STT, and transmit it to the user terminal (100). A Neural Machine Translation (NMT) model can be used to enable multilingual support, and the Neural Machine Translation model may be a model specialized in medical-related domain knowledge. Accordingly, a machine translation program specialized in the domain can be used. To this end, Neural Machine Translation (NMT) can be used, and a Neural Machine Translation model specialized in domain knowledge can be used for sentence translation including terms frequently used in the medical field.

[0058] Transfer learning

[0059] Transfer learning aims to solve new problems faster or with better solutions by applying previously learned knowledge. In this context, transfer learning is performed by fine-tuning a model pre-trained on large amounts of data using the Pretrain-Finetuning Approach (PFA) to the data required for specific tasks. When using this learning method, the knowledge acquired by the pre-trained model can be effectively transferred. Therefore, when transfer learning is applied, superior performance can be expected while using a smaller amount of training data when creating a new model using an existing model. In the field of machine translation, where parallel translation data is scarce, this cross-domain knowledge transfer can be demonstrated as effective. Accordingly, in one embodiment of the present invention, a neural network machine translation model can be trained by adding training data containing domain knowledge used by each medical review platform to a model pre-trained on a large amount of data.

[0060] Text Data Augmentation

[0061] Data Augmentation (DA) is a method to overcome data scarcity by generating new data through slight modifications to existing data. This method can be actively used for image data augmentation in the field of computer vision. In the case of natural language processing, it has not been easy to utilize DA because changing even a single word can completely alter the meaning of a sentence; however, when Text Data Augmentation is applied to Easy Data Augmentation (EDA) techniques, DA can have a significant impact on actual performance improvement in natural language processing. Accordingly, in one embodiment of the present invention, four EDA techniques—Synonym Replacement (SR), Random Insertion (RI), Random Swap (RS), and Random Deletion (RD)—can be used for translation text data augmentation. RI involves inserting random words into a sentence, which can effectively increase translation data pairs by changing the expression while preserving the intent (Label) of the existing text data. By applying this technique, data augmentation can be effectively attempted even when translation text data containing specific domain knowledge is scarce.

[0062] <Technical Term Extraction>

[0063] Technical Term Extraction (TTE) is a method used for effective technical term learning. Deep learning-based models learn autonomously through large corpora, identifying patterns and rules in source sentences to predict words. In other words, the accuracy of term prediction increases only when sentences containing the term to be predicted exist frequently in the corpus; however, this type of learning makes it difficult to effectively address the problem of a lack of translation text data containing specific domain knowledge. This is because even if translation data is augmented by applying Text DA, generating and training translation text pairs that use specific technical terms redundantly still requires significant time and resources.

[0064] Accordingly, in one embodiment of the present invention, to solve this problem, a pair of technical terms used in text to which DA has been applied can be extracted and additionally trained. By applying this method, effective training of a neural network model is possible even with a small domain corpus. Text data is augmented by inserting technical terms into a frame corpus (a set of sentences that are repeatedly used to insert technical terms), and the technical term-based data generated in this way is added to a domain corpus. Then, technical terms used in the data obtained by performing text data augmentation are extracted and added to the final domain corpus, and this corpus can be used as training data for a neural network model.

[0065] Hereinafter, the operation process according to the configuration of the call service providing server of FIG. 2 described above will be explained in detail with reference to FIG. 3 and FIG. 4. However, it is obvious that the embodiment is merely one of the various embodiments of the present invention and is not limited thereto.

[0066] Referring to FIG. 3, (a) a call service providing server (300) registers at least one call device (400), and (b) when a voice utterance is input and converted into text after a call button is pushed, the converted text is transmitted to be displayed on the screen of a user terminal (100). Then, the call service providing server (300) can create a bidirectional channel to convert the text input from the user terminal (100) back into voice and then output it from the call device (400) as in (c). In addition, the call service providing server (300) can use multiple speaker identification as in (d), taking into account cases where multiple people use one room and one call device (400) is provided for each room, or cases where one call device (400) exists for each bed but the voices of others may be mixed.

[0067] As shown in (a) of FIG. 4, the call service providing server (300) maps the input from the call device (400) to a patient ID (speaker ID), classifies and stores it as a history log, and can store it as an attached document in the EMR by date. In addition, the call service providing server (300) can enable domain-specific translation using a corpus specialized in the medical field (b), and can be implemented to guide the call device (400) after it is confirmed after being displayed on a monitor as shown in (c).

[0068] As for the details regarding the STT-based hospital call service provision method of FIGS. 2 to 4 that are not described, they are identical to or can be easily inferred from the details described above regarding the STT-based hospital call service provision method through FIG. 1, so further explanation will be omitted.

[0069] FIG. 5 is a diagram illustrating the process of transmitting and receiving data between each component included in the STT-based hospital call service providing system of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of the process of transmitting and receiving data between each component will be described through FIG. 5, but the present invention is not limited to such an embodiment, and it is obvious to those skilled in the art that the process of transmitting and receiving data illustrated in FIG. 5 may be modified according to various embodiments described above.

[0070] Referring to FIG. 5, the call service provider server receives text from the call device (S5100) and transmits the transmitted text to the user terminal (S5200).

[0071] Also, the call service provider server outputs text from the user terminal until a confirmation button is entered (S5300).

[0072] The order of the steps described above (S5100~S5300) is merely an example and is not limited thereto. That is, the order of the steps described above (S5100~S5300) may vary, and some of these steps may be executed simultaneously or deleted.

[0073] As for the details regarding the STT-based hospital call service provision method of Fig. 5 that are not described, they are identical to or can be easily inferred from the details described above regarding the STT-based hospital call service provision method through Figs. 1 to 4, so further explanation will be omitted.

[0074] A method for providing an STT-based hospital call service according to one embodiment described through FIG. 5 may also be implemented in the form of a recording medium containing computer-executable instructions, such as an application or program module executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, as well as removable and non-removable media. Additionally, a computer-readable medium may include all computer storage media. Computer storage media include both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data.

[0075] The method for providing an STT-based hospital call service according to one embodiment of the present invention described above may be executed by an application basically installed on a terminal (which may include a program included in a platform or operating system, etc., basically installed on the terminal), or by an application (i.e., a program) directly installed by a user on a master terminal through an application providing server, such as an application store server, an application, or a web server related to the service. In this sense, the method for providing an STT-based hospital call service according to one embodiment of the present invention described above may be implemented as an application (i.e., a program) that is basically installed on the terminal or directly installed by a user, and may be recorded on a computer-readable recording medium such as a terminal.

[0076] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0077] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention.< / svs> < / stft>

Claims

Claim 1 A STT-based hospital call service providing system comprising: a calling device that receives voice utterance when a call button is pushed, converts it into STT (Speech to Text), and then transmits the text; a user terminal that outputs the text transmitted from the calling device; and a calling service providing server having a receiving unit that receives the text transmitted from the calling device, a transmitting unit that transmits the transmitted text to the user terminal, an output unit that causes the text to be displayed at the user terminal until a confirmation button is input, and a bidirectional unit that converts the text into TTS (Text to Speech) when text is input at the user terminal and transmits it to the calling device for output. Claim 2 delete Claim 3 The STT-based hospital call service providing system according to claim 1, further comprising: a call service providing server that inputs a voice utterance input from the call device into a pre-configured speaker identification algorithm to identify the speaker, and transmits speaker information of the identified speaker to the user terminal. Claim 4 In claim 3, the STT-based hospital call service providing system is further characterized by including a log storage unit in the call service providing server that stores a history of voice utterances collected from the call device as a history log to output from the user terminal and synchronizes it to be recorded as an EMR (Electronic Medical Records). Claim 5 The STT-based hospital call service providing system according to claim 1, further comprising: a multilingual support unit in which the call service providing server detects a language input from the call device, converts the result of translating the language into STT, and transmits it to the user terminal. Claim 6 A STT-based hospital call service providing system according to claim 1, wherein the calling device is an IoT (Internet of Things) based device.

Citation Information

Patent Citations

  • Nursing service support system

    JP2020030468A

  • Patient monitoring system and nurse call system

    JP2022156958A

  • System and method of patient management using wearable device

    KR1020170063275A

  • Multifunction control system for medical environment improvement

    KR1020180095964A

  • Apparatus for managing sickroom

    KR1020220121084A