Methods and Systems for Operating Room Medical Order Management Based on Speech Recognition

Abstract

This invention provides a method and system for operating room medical order management based on speech recognition, relating to the field of speech recognition technology. The invention pre-collects doctors' voice commands, extracts key voice commands, divides them into different time segments, and calculates multi-order Mel-frequency cepstral coefficients to form a reference sequence. It then collects the surgeon's voice commands in real time, extracts key voice commands to determine drug types, divides time segments according to the number of time segments for the same drug type in the database, and evaluates the matching degree with the database through cosine similarity and energy entropy analysis, dynamically marking key and non-key segments. Finally, it combines cosine similarity and normalized speech similarity to generate a comprehensive speech similarity score, thereby determining the surgeon's identity and access level. Drug retrieval is performed only if the access is sufficient; otherwise, confirmation is obtained from the doctor with the highest access level. This process ensures the standardization and safety of drug retrieval during surgery.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, specifically to a method and system for operating room medical order management based on speech recognition. Background Technology

[0002] In the operating room environment, the efficiency and accuracy of medical order management are directly related to patient safety and the smooth progress of surgery. An efficient medical order management system helps doctors quickly obtain the necessary medications, ensuring the successful execution of the procedure. Traditional operating room medical order management relies heavily on written records or manual input into electronic systems, which suffers from slow response times, susceptibility to errors in information transmission, and potential for inappropriate medication use.

[0003] In recent years, speech recognition technology has been introduced into the medical field, but existing solutions still face many challenges in the application of high-noise, high-pressure environments such as operating rooms. For example, background noise interferes with speech recognition accuracy, differences in pronunciation habits and speaking speeds among different doctors lead to recognition errors, and the management of access to high-risk drugs lacks a dynamic verification mechanism. These problems may lead to delays or errors in the execution of medical orders, increasing medical risks.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for operating room medical order management based on speech recognition, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for operating room medical order management based on speech recognition, comprising the following steps:

[0008] Step 1: Collect voice commands from each doctor in advance. The voice commands include drug name, drug dosage and administration method. Extract key voice commands from the voice commands. The key voice commands are the voice of drug name. Divide the key voice commands into several time segments at equal intervals. Calculate the multi-order Mel frequency cepstral coefficients of each time segment to form a reference sequence for each time segment. Summarize the reference sequences of the same key voice command as voice features. Set doctor access level and drug risk level. Based on voice features, doctor access level and drug risk level, establish a doctor-drug access database.

[0009] Step 2: During the operation, the surgeon's key voice commands are collected and calculated in real time. The commands are segmented according to the number of time segments that are the same as the key voice commands in the database. The real-time sequence of each time segment of the surgeon's key voice commands is calculated. The cosine similarity between the real-time sequence and the reference sequence under each time segment is calculated. The average cosine similarity under all time segments is taken to reflect the matching degree between the surgeon's key voice commands and the key voice commands in the database.

[0010] Step 3: Calculate the energy entropy of each time segment based on the surgeon's key voice commands and the key voice commands in the database. Use the 25th percentile of the energy entropy of all time segments as the key segment boundary value. If the energy entropy of a time segment is less than the key segment boundary value, mark the time segment as a non-key segment; otherwise, mark it as a key segment. Assign weights to key segments and non-key segments respectively. Perform dynamic time warping on the surgeon's key voice commands and the key voice commands in the database to generate a global alignment distance. Generate a normalized speech similarity based on the global alignment distance.

[0011] Step 4: Generate a comprehensive speech similarity based on cosine similarity and normalized speech similarity, and determine the speech feature with the highest comprehensive speech similarity in the doctor-drug permission database to determine the identity and permission level of the current attending physician and to determine the drugs that need to be called.

[0012] Step 5: When the surgeon's authority level meets the level required to retrieve medication, the medication will be retrieved according to the surgeon's key voice command. If the surgeon's authority level does not meet the level required to retrieve medication, the doctor with the highest authority level in the operating room will confirm the medication via a second voice command.

[0013] Furthermore, the principle underlying the calculation of the Mel-frequency cepstral coefficients of key speech commands is as follows:

[0014] The formula used to pre-emphasize key speech commands within a time segment is as follows:

[0015]

[0016] Among them, y q (t) represents the value of the signal at time t after pre-emphasis in the q-th time segment, where q represents the index of the time segment and q∈[1,Q], Q represents the total number of time segments, and t represents the time variable. Let represent the signal values ​​of the q-th time segment at time t and time t-1, respectively, and let a represent the preemphasis coefficient, where 0.90 ≤ a ≤ 0.97;

[0017] The pre-emphasized signal is segmented into short-time frame signals, and the segmented signals are then windowed using the following formula:

[0018]

[0019] Among them, y q w(n) represents the signal after windowing at the nth time frame within the qth time segment, where n is the time frame index, N is the frame length, and w(n) represents the Hamming window function for the nth time frame. This represents the short-time frame signal of the nth time frame after segmentation in the qth time segment;

[0020] The formula used to perform a Fast Fourier Transform on the windowed signal is:

[0021]

[0022] Among them, X q (k) represents the frequency domain signal at the k-th frequency in the q-th time segment, j0 represents the imaginary unit, k represents the frequency index with a value range of [0, N-1], and e represents the base of the natural logarithm;

[0023] The generated Mel frequency cepstral coefficients are:

[0024]

[0025] Among them, E q (m) represents the energy of the m-th Mel filter in the q-th time segment, where m represents the index of the Mel filter in the filter bank, M represents the number of Mel filters, and H m (k) The value of the m-th Mel filter at the k-th frequency, MFCC q (i) represents the i-th order Mel frequency cepstral coefficient in the q-th time segment, where i represents the index of the order of the Mel frequency cepstral coefficient, and i∈[1,13], MFCC q (1) MFCC q (2), ..., MFCC q (13) Combine them into a reference sequence for the q-th time segment. This represents the Discrete Cosine Transform (DCT) kernel.

[0026] Furthermore, the principle underlying the establishment of the doctor-drug access database is as follows:

[0027] Based on the pharmacological risks of drugs, usage scenarios, and specific hospital management policies, drugs are classified into four access levels: low risk, low-to-medium risk, medium-to-high risk, and high risk. Based on doctors' professional titles and clinical experience, doctors are classified into four access levels. Doctors at each access level can access drugs at the same level and lower levels. The doctor-drug access database contains doctors' access levels, drug risk levels, and voice characteristics of different doctors issuing requests for different drugs.

[0028] Furthermore, the Mel-frequency cepstral coefficients (MFCCs) of the surgeon's key voice commands were calculated, and the same number of time segments were extracted. q (i), and MFCC′ q (1) MFCC′ q (2), ..., MFCC′ q (13) Combine to form the reference sequence for the q-th time segment;

[0029] The formula used to calculate cosine similarity is:

[0030]

[0031] Among them, S q MFCC' represents the cosine similarity between the real-time sequence and the reference sequence at the q-th time segment. q (i) represents the i-th order Mel-frequency cepstral coefficient (MFCC) of the surgeon's key voice command in the q-th time segment. q (i) represents the i-th order Mel frequency cepstral coefficient of the key speech command in the database at the q-th time frame, and S represents the matching degree of the key speech command.

[0032] Furthermore, the principle underlying the generation of normalized speech similarity is as follows:

[0033] The formula used to calculate the energy entropy of the surgeon's key voice commands in the q-th time segment is:

[0034]

[0035] Among them, H′ q The energy entropy of the surgeon's key voice command at time segment q is ||MFCC′. q (i)||1 represents MFCC′ q L1 norm of (i);

[0036] The formula used to calculate the energy entropy of key speech commands in the database for the q-th time segment is:

[0037]

[0038] Among them, H q ||MFCC represents the energy entropy of key voice commands in the database at the q-th time segment. q (i)||1 represents MFCC q L1 norm of (i);

[0039] Assign weights to key segments and non-key segments respectively:

[0040]

[0041] Among them, w′ q w represents the weight of the q-th time segment in the surgeon's key voice commands. q This represents the weight of the q-th time segment in the key voice commands in the database;

[0042] The formula used to generate the global alignment distance by dynamically time-normalizing the surgeon's key voice commands and key voice commands in the database is as follows:

[0043]

[0044] Where D(r,s) represents the minimum cumulative distance between the time segment of the r-th surgeon's key voice command and the time segment of the s-th key voice command in the database, r represents the time segment index of the surgeon's key voice command, and r∈[1,Q], s represents the time segment index of the key voice command in the database, and s∈[1,Q], w′ r w represents the weight of the r-th time segment in the surgeon's key voice commands. s MFCC′ represents the weight of the s-th time segment in the key voice commands of the database. r (i) represents the i-th order Mel-frequency cepstral coefficient (MFCC) of the time segment of the r-th key speech command from the chief surgeon. s (i) represents the i-th order Mel frequency cepstral coefficient of the time segment of the key speech command in the s-th database;

[0045] Create a matrix D of size Q×Q, initialize the boundary conditions as D(0,0)=0, D(r,0)=+∞, D(0,s)=+∞, fill D(r,s) into matrix D, where r and s correspond to the row and column positions in the matrix, respectively. After filling matrix D, the value D(Q,Q) in the lower right corner of the matrix is ​​the global alignment distance.

[0046] The formula used to generate normalized speech similarity is:

[0047]

[0048] Among them, S DTW This represents the normalized speech similarity.

[0049] Furthermore, the formula used to generate the comprehensive speech similarity score is as follows:

[0050]

[0051] Where K represents the overall speech similarity, k1 and k2 represent the weight coefficients of cosine similarity and normalized speech similarity, respectively, and k1 > k2.

[0052] This invention also provides a system for operating room medical order management based on speech recognition. The system is used to implement the aforementioned method for operating room medical order management based on speech recognition, specifically including:

[0053] The data preprocessing module is used to pre-collect the voice commands of each doctor, which include drug name, drug dosage and administration method. Key voice commands are extracted from the voice commands, which are the voice commands of drug names. The key voice commands are divided into several time segments at equal intervals. The multi-order Mel frequency cepstral coefficients of each time segment are calculated to form a reference sequence for each time segment. The reference sequences of the same key voice command are summarized as voice features. Doctor permission level and drug risk level are set. Based on voice features, doctor permission level and drug risk level, a doctor-drug permission database is established.

[0054] The real-time data acquisition module is used to collect and calculate the surgeon's key voice commands in real time during the operation. It segments the commands according to the number of time segments that are the same as the key voice commands in the database, calculates the real-time sequence of each time segment of the surgeon's key voice commands, calculates the cosine similarity between the real-time sequence and the reference sequence under each time segment, and takes the average of the cosine similarity under all time segments to reflect the matching degree between the surgeon's key voice commands and the key voice commands in the database.

[0055] The dynamic adjustment module is used to calculate the energy entropy of each time segment based on the surgeon's key voice commands and key voice commands in the database. The 25th percentile of the energy entropy of all time segments is used as the key segment boundary value. When the energy entropy of a time segment is less than the key segment boundary value, the time segment is marked as a non-key segment; otherwise, it is marked as a key segment. Weights are assigned to key segments and non-key segments respectively. Dynamic time warping is performed on the surgeon's key voice commands and key voice commands in the database to generate a global alignment distance. Normalized speech similarity is generated based on the global alignment distance.

[0056] The voice matching module is used to generate a comprehensive voice similarity based on cosine similarity and normalized voice similarity, and to determine the voice feature with the highest comprehensive voice similarity in the doctor-drug permission database, so as to determine the identity and permission level of the current surgeon and determine the drug to be called.

[0057] The instruction confirmation module is used to retrieve medications based on the surgeon's key voice commands when the surgeon's permission level meets the permission level for retrieving medications. If the surgeon's permission level does not meet the permission level for retrieving medications, the doctor with the highest permission level in the operating room will confirm the second voice command.

[0058] Compared with the prior art, the beneficial effects of the present invention are:

[0059] This invention improves speech recognition accuracy and reduces the probability of misrecognition by pre-collecting each doctor's voice commands and calculating Mel-frequency cepstral coefficients as speech features. It establishes a doctor-drug access database by setting doctor access levels and drug risk levels, ensuring that only doctors with the appropriate permissions can access specific drugs. This multi-factor authentication mechanism significantly enhances the security of drug management and reduces medical risks caused by improper access.

[0060] This invention also collects key voice commands from the surgeon in real time during surgery, segments the surgeon's voice commands and key voice commands in the database according to the same number of time segments, calculates the cosine similarity between the two for each time segment, and determines the degree of voice matching from static features; it identifies key segments through energy entropy analysis, and performs dynamic time warping based on different weights for key segments, performing voice matching from dynamic features. Combining static and dynamic features, it can identify both the timbre and speech rate characteristics of the voice, ensuring the accuracy of voice matching. At the same time, energy entropy is introduced during dynamic time warping to determine the weight of key segments and non-key segments, expanding the influence of key segments and further improving the accuracy of voice matching; based on the voice recognition results, it determines whether to call medication, and performs secondary confirmation when permissions are exceeded to avoid incorrect medication retrieval and ensure the safety of the surgery. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the method flow of an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram illustrating the improvement in recognition accuracy of key segments before and after weighting in an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of the system modules in an embodiment of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0065] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0066] Example:

[0067] Please see Figures 1 to 2 The present invention provides a technical solution:

[0068] A method for operating room medical order management based on speech recognition, comprising the following steps:

[0069] Step 1: Collect voice commands from each doctor in advance. The voice commands include drug name, drug dosage and administration method. Extract key voice commands from the voice commands. The key voice commands are the voice of drug name. Divide the key voice commands into several time segments at equal intervals. Calculate the multi-order Mel frequency cepstral coefficients of each time segment to form a reference sequence for each time segment. Summarize the reference sequences of the same key voice command as voice features. Set doctor access level and drug risk level. Based on voice features, doctor access level and drug risk level, establish a doctor-drug access database.

[0070] In this embodiment, the principle underlying the calculation of the Mel frequency cepstral coefficients of key voice commands is as follows:

[0071] The formula used to pre-emphasize key speech commands within a time segment is as follows:

[0072]

[0073] Among them, y q (t) represents the value of the signal at time t after pre-emphasis in the q-th time segment, where q represents the index of the time segment and q∈[1,Q], Q represents the total number of time segments, and t represents the time variable. Let represent the signal values ​​of the q-th time segment at time t and time t-1, respectively, and let a represent the preemphasis coefficient, where 0.90 ≤ a ≤ 0.97;

[0074] During the propagation of a speech signal, the energy of high-frequency components typically decays faster than that of low-frequency components. Pre-emphasizing speech commands aims to compensate for the energy attenuation of the high-frequency portion of the speech signal, enhance high-frequency features, make the spectrum flatter, and avoid the loss of high-frequency information during subsequent feature extraction. Typically, the pre-emphasis coefficient ranges from 0.90 to 0.97. The larger the pre-emphasis coefficient, the more obvious the high-frequency enhancement effect; the smaller the pre-emphasis coefficient, the weaker the high-frequency enhancement effect. The signal strength after pre-emphasis is directly proportional to the pre-emphasis coefficient and the original signal strength.

[0075] The pre-emphasized signal is segmented into short-time frame signals, and the segmented signals are then windowed using the following formula:

[0076]

[0077] Among them, y q w(n) represents the signal after windowing at the nth time frame within the qth time segment, where n is the time frame index, N is the frame length, and w(n) represents the Hamming window function for the nth time frame. This represents the short-time frame signal of the nth time frame after segmentation in the qth time segment;

[0078] Continuous speech signals are segmented into short frames, typically 20 to 40 milliseconds per frame, to capture the local features of the signal. The signal after segmentation may be discontinuous at the frame edges, which may introduce false high-frequency components in the frequency domain. Windowing can smooth the ends of the frame and make the periodicity of the signal more obvious. Hamming windowing achieves a good balance between time domain and frequency domain performance while effectively reducing spectral leakage.

[0079] The formula used to perform a Fast Fourier Transform on the windowed signal is:

[0080]

[0081] Among them, X q (k) represents the frequency domain signal at the k-th frequency in the q-th time segment, j0 represents the imaginary unit, k represents the frequency index with a value range of [0, N-1], and e represents the base of the natural logarithm;

[0082] The time-domain signal is transformed into a frequency-domain signal through Fast Fourier Transform (FFT). The original speech signal is a waveform in the time domain, and its amplitude changes with time, which cannot directly reflect the frequency components of the sound. Through FFT, each frame of speech signal is transformed from the time domain to the frequency domain, and the change of the signal's energy distribution with frequency is obtained. The logarithm of the energy output of each Mel filter is taken to generate the logarithmic energy, and the discrete cosine transform is performed on the logarithmic energy to generate the Mel frequency cepstral coefficients.

[0083] The generated Mel frequency cepstral coefficients are:

[0084]

[0085] Among them, E q (m) represents the energy of the m-th Mel filter in the q-th time segment, where m represents the index of the Mel filter in the filter bank, M represents the number of Mel filters, and H m (k) The value of the m-th Mel filter at the k-th frequency, MFCC q (i) represents the i-th order Mel frequency cepstral coefficient in the q-th time segment, where i represents the index of the order of the Mel frequency cepstral coefficient, and i∈[1,13], MFCC q (1) MFCC q (2), ..., MFCC q (13) Combine them into a reference sequence for the q-th time segment. This represents the Discrete Cosine Transform (DCT) kernel.

[0086] In each time segment, the Mel frequency cepstral coefficients are calculated from orders 1 to 13, including low-frequency components of orders 1 to 5 and mid-to-high-frequency components of orders 6 to 13. Low-frequency classification mainly captures the static features of the vocal tract, such as timbre and vocal habits, to distinguish the identities of different speakers. Mid-to-high-frequency components are used to reflect the dynamic details of speech and to perform semantic content recognition. If the order is higher, more high-frequency noise will be introduced. Therefore, the Mel frequency cepstral coefficients of order 13 are selected to form a 13-dimensional vector. The length of the 13-dimensional vector is more suitable for the DTW algorithm. When the dimension is higher, the alignment complexity increases significantly, which affects the real-time performance.

[0087] Mel frequency cepstral coefficients (MFCCs) compress the short-time power spectrum of a speech signal into a few low-dimensional coefficients through a Mel filter bank and DCT transform. Low-frequency MFCCs distinguish the vocal characteristics of different speakers, while high-frequency MFCCs identify specific semantic content.

[0088] In this embodiment, the principle upon which the doctor-drug permission database is established is as follows:

[0089] Based on the pharmacological risks of drugs, usage scenarios, and specific hospital management policies, drugs are classified into four access levels: low risk, low-to-medium risk, medium-to-high risk, and high risk. Based on doctors' professional titles and clinical experience, doctors are classified into four access levels. Doctors at each access level can access drugs at the same level and lower levels. The doctor-drug access database contains doctors' access levels, drug risk levels, and voice characteristics of different doctors issuing requests for different drugs.

[0090] Low-risk drugs refer to routine drugs used in surgery, such as saline and 5% glucose. These drugs have low risk and wide applicability, and it is necessary to ensure that most doctors can directly use them. As the risk level of a drug increases, the corresponding risk is greater and the scope of application is more limited. Using such drugs requires a certain level of experience and medical expertise from the doctor. The doctor's access level reflects their medical expertise; the higher the access level, the more types of drugs the doctor can directly use.

[0091] Step 2: During the operation, the surgeon's key voice commands are collected and calculated in real time. The commands are segmented according to the number of time segments that are the same as the key voice commands in the database. The real-time sequence of each time segment of the surgeon's key voice commands is calculated. The cosine similarity between the real-time sequence and the reference sequence under each time segment is calculated. The average cosine similarity under all time segments is taken to reflect the matching degree between the surgeon's key voice commands and the key voice commands in the database.

[0092] In this embodiment, based on the key voice commands of the attending physician collected in real time, key voice commands of the same drug type in the database are identified, and the key voice commands of the attending physician are segmented according to the number of the same time segments of the same drug type. The method for identifying the drug type is as follows: based on the end-to-end speech recognition model, the key voice commands in the database and the key voice commands of the attending physician are processed in the same way, the key voice commands are converted into text, and the text information of the database and the key voice commands of the attending physician are directly matched.

[0093] The Mel-frequency cepstral coefficients (MFCCs) of the surgeon's key voice commands were calculated using the same method as in step 1, and the same number of time segments were extracted. q (i), and MFCC′ q (1) MFCC′ q (2), ..., MFCC′ q (13) Combine to form the reference sequence for the q-th time segment;

[0094] The formula used to calculate cosine similarity is:

[0095]

[0096] Among them, S q MFCC' represents the cosine similarity between the real-time sequence and the reference sequence at the q-th time segment. q (i) represents the i-th order Mel-frequency cepstral coefficient (MFCC) of the surgeon's key voice command in the q-th time segment. q (i) represents the i-th order Mel frequency cepstral coefficient of the key speech command in the database at the q-th time frame, and S represents the matching degree of the key speech command.

[0097] Within the same time segment, the average cosine similarity of the real-time sequence and the reference sequence is calculated to measure the similarity between the real-time sequence and the reference sequence within this time segment. The matching degree of key speech commands is then calculated based on the average cosine similarity across all time segments to reflect the similarity between the surgeon's key speech commands and those in the database. The range of cosine similarity is [-1, 1], therefore the similarity range of key speech commands is also [-1, 1]. Cosine similarity reflects the directional consistency between two sequences; the closer the cosine similarity is to 1, the more consistent the directions of the two sequences, and the more similar the sequences. The closer the cosine similarity is to 0, the greater the directional deviation between the two sequences, and the less consistent the sequences. The closer the matching degree of key speech commands is to 1, the closer the cosine similarity of the sequences in each time segment is to 1, indicating a higher matching degree for each time segment. Therefore, the higher the matching degree of key speech commands obtained by summing and averaging the cosine similarities of these time segments, the higher the matching degree of key speech commands. The matching degree of key speech commands mainly reflects the static feature matching situation, without considering their dynamic changes over time.

[0098] Step 3: Calculate the energy entropy of each time segment based on the surgeon's key voice commands and the key voice commands in the database. Use the 25th percentile of the energy entropy of all time segments as the key segment boundary value. If the energy entropy of a time segment is less than the key segment boundary value, mark the time segment as a non-key segment; otherwise, mark it as a key segment. Assign weights to key segments and non-key segments respectively. Perform dynamic time warping on the surgeon's key voice commands and the key voice commands in the database to generate a global alignment distance. Generate a normalized speech similarity based on the global alignment distance.

[0099] In this embodiment, the principle underlying the generation of normalized speech similarity is as follows:

[0100] The formula used to calculate the energy entropy of the surgeon's key voice commands in the q-th time segment is:

[0101]

[0102] Among them, H′ q The energy entropy of the surgeon's key voice command at time segment q is ||MFCC′. q (i)||1 represents MFCC′ q L1 norm of (i);

[0103] The formula used to calculate the energy entropy of key speech commands in the database for the q-th time segment is:

[0104]

[0105] Among them, Hq ||MFCC represents the energy entropy of key voice commands in the database at the q-th time segment. q (i)||1 represents MFCC q L1 norm of (i);

[0106] Energy entropy is the application of information entropy to the distribution of signal energy. By calculating the energy entropy of each time segment, key segments and non-key segments are distinguished. Key segments represent effective information in speech, such as clearly pronounced drug names, dosages, and usage methods. Non-key segments contain features such as breathing sounds, background noise, and unimportant speech information. In the DTW alignment process, the matching error of key segments has a greater impact on the global similarity, thus more accurately reflecting the substantial differences in doctors' pronunciation. The higher the speech energy entropy of a certain time segment, the more uniform the energy distribution and the richer the information; the lower the energy entropy, the lower the information content. The 25th percentile of the speech energy entropy of different time segments is used as the key segment boundary. Speech segments with energy entropy higher than the key segment boundary are selected as key segments, and speech segments with energy entropy lower than the key segment boundary are non-key segments. By quantifying the information density of speech segments through energy entropy, noise suppression and key feature enhancement are achieved.

[0107] The formula for calculating the L1 norm is:

[0108] Assign weights to key segments and non-key segments respectively:

[0109]

[0110] Among them, w′ q w represents the weight of the q-th time segment in the surgeon's key voice commands. q This represents the weight of the q-th time segment in the key voice commands in the database;

[0111] Key segments are less affected by noise and contain more semantically important information, thus having a more significant impact on cumulative distance and therefore have higher weights. Non-key segments have higher similarity among different speakers and are more susceptible to environmental noise, thus having lower weights.

[0112] The formula used to generate the global alignment distance by dynamically time-normalizing the surgeon's key voice commands and key voice commands in the database is as follows:

[0113]

[0114] Where D(r,s) represents the minimum cumulative distance between the time segment of the r-th surgeon's key voice command and the time segment of the s-th key voice command in the database, r represents the time segment index of the surgeon's key voice command, and r∈[1,Q], s represents the time segment index of the key voice command in the database, and s∈[1,Q], w′r represents the weight of the r-th time segment in the surgeon's key voice command, w s Let MFCC'r(i) represent the weight of the s-th time segment in the key speech commands in the database, and let MFCC'r(i) represent the i-th order Mel-frequency cepstral coefficient of the time segment of the r-th key speech command from the attending physician. s (i) represents the i-th order Mel frequency cepstral coefficient of the time segment of the key speech command in the s-th database;

[0115] Create a matrix D of size Q×Q, initialize the boundary conditions as D(0,0)=0, D(r,0)=+∞, D(0,s)=+∞, fill D(r,s) into matrix D, where r and s correspond to the row and column positions in the matrix, respectively. After filling matrix D, the value D(Q,Q) in the lower right corner of the matrix is ​​the global alignment distance.

[0116] Dynamic Time Warping (DTW) measures the temporal similarity between key speech commands from the surgeon and key speech commands in the database. It can handle non-linear deformations in time series. In speech recognition, different people may have different speech rates when saying the same word; weighted DTW aligns these sequences. Minimum Cumulative Distance (MCD) represents the minimum cumulative cost of aligning the Mel-frequency cepstral coefficients (MFCCs) of the r-th time segment of the surgeon's key speech command with the s-th time segment of the key speech command in the database. The smaller the MCD, the smaller the feature difference, and the higher the similarity between the two speech commands. The values ​​of r and s are the same as q, determining the elements in different rows and columns of matrix D. The minimum cumulative distance is related to MFCC′. r (i) and MFCC s (i) is proportional to the Euclidean distance between them, and proportional to the minimum cumulative distance of the paths from D(r,s) to the horizontal, vertical, and diagonal positions. This reflects the direct difference in features between two time frames, w′ q ·w n This represents the weights of key segments and non-key segments, and w′ q >w q This is used to highlight the contribution of key segments; The algorithm selects the path with the smallest cumulative distance from three possible paths to ensure the lowest possible total cost of alignment. These three possible paths represent the paths from D(r,s) to the horizontal, vertical, and diagonal positions in matrix D. The global alignment distance is the final output of the DTW algorithm, representing the minimum cost required to align two speech commands. The accuracy of the minimum cumulative distance judgment is improved after weighting by key segment weights, as shown in Table 1.

[0117] Table 1

[0118]

[0119]

[0120] The formula used to generate normalized speech similarity is:

[0121]

[0122] Among them, S DTW This represents the normalized speech similarity.

[0123] Normalized speech similarity reflects the degree of temporal alignment matching of speech sequences, ranging from (0,1], S DTW The closer the value is to 1, the lower the alignment cost between the two sequences on the time axis, indicating a high degree of temporal and feature matching between the surgeon's key voice commands and the key voice commands in the database. It divides the global alignment distance by the sum of the frame lengths of the two speech commands, normalizes the distance to the average alignment cost per time frame, and normalized speech similarity is inversely proportional to the global alignment distance.

[0124] Step 4: Generate a comprehensive speech similarity based on cosine similarity and normalized speech similarity, and determine the speech feature with the highest comprehensive speech similarity in the doctor-drug permission database to determine the identity and permission level of the current attending physician and to determine the drugs that need to be called.

[0125] In this embodiment, the formula used to generate the comprehensive speech similarity is:

[0126]

[0127] Where K represents the overall speech similarity, k1 and k2 represent the weight coefficients of cosine similarity and normalized speech similarity, respectively, and k1 > k2.

[0128] Cosine similarity and normalized speech similarity affect the overall speech similarity from static and dynamic perspectives, respectively. It is a nonlinear transformation of cosine similarity; when S approaches 1, The value of will increase sharply, indicating that the static features of the frequency have reached a high degree of matching. These static features include timbre, pronunciation habits, etc.; S DTW It reflects the degree of temporal matching after dynamic time warping, and captures the dynamic changes of speech commands on the time axis, such as speaking rhythm and speech rate. It makes up for the shortcomings of cosine similarity in temporal aspects. In speech recognition, static features have a higher degree of distinguishing the speaker's identity, while temporal features are mainly affected by environmental noise and speech rate changes. To ensure the accuracy of recognition, static features are given higher weights. Therefore, k1 > k2, and k1 = 0.7 and k2 = 0.3 are taken.

[0129] Step 5: When the surgeon's authority level meets the level required to retrieve medication, the medication will be retrieved according to the surgeon's key voice command. If the surgeon's authority level does not meet the level required to retrieve medication, the doctor with the highest authority level in the operating room will confirm the medication via a second voice command.

[0130] In this embodiment, if the surgeon's permission level meets the permission level for retrieving medication, it means that the surgeon's permission level is the same as or higher than the risk level of the medication, and the surgeon has the corresponding authority to prepare and transport the medication directly according to the surgeon's voice instructions. If the surgeon's permission level does not meet the requirements, the surgeon with the highest permission level in the operating room is notified to confirm the surgeon's voice instructions a second time. After confirmation, the medication is prepared and transported, and the record is saved for subsequent review.

[0131] Please see Figure 3 The present invention also provides a system for operating room medical order management based on speech recognition. The system is used to implement the above-mentioned method for operating room medical order management based on speech recognition, specifically including:

[0132] The data preprocessing module is used to pre-collect the voice commands of each doctor, which include drug name, drug dosage and administration method. Key voice commands are extracted from the voice commands, which are the voice commands of drug names. The key voice commands are divided into several time segments at equal intervals. The multi-order Mel frequency cepstral coefficients of each time segment are calculated to form a reference sequence for each time segment. The reference sequences of the same key voice command are summarized as voice features. Doctor permission level and drug risk level are set. Based on voice features, doctor permission level and drug risk level, a doctor-drug permission database is established.

[0133] The real-time data acquisition module is used to collect and calculate the surgeon's key voice commands in real time during the operation. It segments the commands according to the number of time segments that are the same as the key voice commands in the database, calculates the real-time sequence of each time segment of the surgeon's key voice commands, calculates the cosine similarity between the real-time sequence and the reference sequence under each time segment, and takes the average of the cosine similarity under all time segments to reflect the matching degree between the surgeon's key voice commands and the key voice commands in the database.

[0134] The dynamic adjustment module is used to calculate the energy entropy of each time segment based on the surgeon's key voice commands and key voice commands in the database. The 25th percentile of the energy entropy of all time segments is used as the key segment boundary value. When the energy entropy of a time segment is less than the key segment boundary value, the time segment is marked as a non-key segment; otherwise, it is marked as a key segment. Weights are assigned to key segments and non-key segments respectively. Dynamic time warping is performed on the surgeon's key voice commands and key voice commands in the database to generate a global alignment distance. Normalized speech similarity is generated based on the global alignment distance.

[0135] The voice matching module is used to generate a comprehensive voice similarity based on cosine similarity and normalized voice similarity, and to determine the voice feature with the highest comprehensive voice similarity in the doctor-drug permission database, so as to determine the identity and permission level of the current surgeon and determine the drug to be called.

[0136] The instruction confirmation module is used to retrieve medications based on the surgeon's key voice commands when the surgeon's permission level meets the permission level for retrieving medications. If the surgeon's permission level does not meet the permission level for retrieving medications, the doctor with the highest permission level in the operating room will confirm the second voice command.

[0137] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0138] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0139] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0140] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for operating room medical order management based on speech recognition, characterized in that, The specific steps include: Step 1: Collect voice commands from each doctor in advance. The voice commands include drug name, drug dosage and administration method. Extract key voice commands from the voice commands. The key voice commands are the voice of drug name. Divide the key voice commands into several time segments at equal intervals. Calculate the multi-order Mel frequency cepstral coefficients of each time segment to form a reference sequence for each time segment. Summarize the reference sequences of the same key voice command as voice features. Set doctor access level and drug risk level. Based on voice features, doctor access level and drug risk level, establish a doctor-drug access database. Step 2: During the operation, the surgeon's key voice commands are collected and calculated in real time. The commands are segmented according to the number of time segments that are the same as the key voice commands in the database. The real-time sequence of each time segment of the surgeon's key voice commands is calculated. The cosine similarity between the real-time sequence and the reference sequence under each time segment is calculated. The average cosine similarity under all time segments is taken to reflect the matching degree between the surgeon's key voice commands and the key voice commands in the database. Step 3: Calculate the energy entropy of each time segment based on the surgeon's key voice commands and the key voice commands in the database. Use the 25th percentile of the energy entropy of all time segments as the key segment boundary value. If the energy entropy of a time segment is less than the key segment boundary value, mark the time segment as a non-key segment; otherwise, mark it as a key segment. Assign weights to key segments and non-key segments respectively. Perform dynamic time warping on the surgeon's key voice commands and the key voice commands in the database to generate a global alignment distance. Generate a normalized speech similarity based on the global alignment distance. Step 4: Generate a comprehensive speech similarity based on cosine similarity and normalized speech similarity, and determine the speech feature with the highest comprehensive speech similarity in the doctor-drug permission database to determine the identity and permission level of the current attending physician and to determine the drugs that need to be called. Step 5: When the surgeon's authority level meets the level required to retrieve medication, the medication will be retrieved according to the surgeon's key voice command. If the surgeon's authority level does not meet the level required to retrieve medication, the doctor with the highest authority level in the operating room will confirm the medication via a second voice command.

2. The method for operating room medical order management based on speech recognition according to claim 1, characterized in that: The principle underlying the calculation of the Mel frequency cepstral coefficients of the key speech commands in step 1 is as follows: The formula used to pre-emphasize key speech commands within a time segment is as follows: Among them, y q (t) represents the value of the signal at time t after pre-emphasis in the q-th time segment, where q represents the index of the time segment and q∈[1,Q], Q represents the total number of time segments, and t represents the time variable. Let represent the signal values ​​of the q-th time segment at time t and time t-1, respectively, and let a represent the preemphasis coefficient, where 0.90 ≤ a ≤ 0.97; The pre-emphasized signal is segmented into short-time frame signals, and the segmented signals are then windowed using the following formula: Among them, y q w(n) represents the signal after windowing at the nth time frame within the qth time segment, where n is the time frame index, N is the frame length, and w(n) represents the Hamming window function for the nth time frame. This represents the short-time frame signal of the nth time frame after segmentation in the qth time segment; The formula used to perform a Fast Fourier Transform on the windowed signal is: Among them, X q (k) represents the frequency domain signal at the k-th frequency in the q-th time segment, j0 represents the imaginary unit, k represents the frequency index with a value range of [0, N-1], and e represents the base of the natural logarithm; The generated Mel frequency cepstral coefficients are: Among them, E q (m) represents the energy of the m-th Mel filter in the q-th time segment, where m represents the index of the Mel filter in the filter bank, M represents the number of Mel filters, and H m (k) The value of the m-th Mel filter at the k-th frequency, MFCC q (i) represents the i-th order Mel frequency cepstral coefficient in the q-th time segment, where i represents the index of the order of the Mel frequency cepstral coefficient, and i∈[1,13], MFCC q (1) MFCC q (2), ..., MFCC q (13) Combine them into a reference sequence for the q-th time segment. This represents the Discrete Cosine Transform (DCT) kernel.

3. The method for operating room medical order management based on speech recognition according to claim 1, characterized in that: The principle underlying the establishment of the doctor-drug access database in step 1 is as follows: Based on the pharmacological risks of drugs, usage scenarios, and specific hospital management policies, drugs are classified into four access levels: low risk, low-to-medium risk, medium-to-high risk, and high risk. Based on doctors' professional titles and clinical experience, doctors are classified into four access levels. Doctors at each access level can access drugs at the same level and lower levels. The doctor-drug access database contains doctors' access levels, drug risk levels, and voice characteristics of different doctors issuing requests for different drugs.

4. The method for operating room medical order management based on speech recognition according to claim 2, characterized in that: In step 2, the Mel-frequency cepstral coefficients (MFCCs) of the surgeon's key voice commands are calculated using the same steps as in claim 2, and the same number of time segments are extracted. q (i), and MFCC′ q (1) MFCC′ q (2), ..., MFCC′ q (13) Combine to form the reference sequence for the q-th time segment; The formula used to calculate cosine similarity is: Among them, S q MFCC' represents the cosine similarity between the real-time sequence and the reference sequence at the q-th time segment. q (i) represents the i-th order Mel-frequency cepstral coefficient (MFCC) of the surgeon's key voice command in the q-th time segment. q (i) represents the i-th order Mel frequency cepstral coefficient of the key speech command in the database at the q-th time frame, and S represents the matching degree of the key speech command.

5. A method for operating room medical order management based on speech recognition according to claim 4, characterized in that: The principle underlying the generation of normalized speech similarity in step 3 is as follows: The formula used to calculate the energy entropy of the surgeon's key voice commands in the q-th time segment is: Among them, H′ q The energy entropy of the surgeon's key voice command at time segment q is ||MFCC′. q (i)||1 represents MFCC′ q L1 norm of (i); The formula used to calculate the energy entropy of key speech commands in the database for the q-th time segment is: Among them, H q ||MFCC represents the energy entropy of key voice commands in the database at the q-th time segment. q (i)||1 represents MFCC q L1 norm of (i); Assign weights to key segments and non-key segments respectively: Among them, w′ q w represents the weight of the q-th time segment in the surgeon's key voice commands. q This represents the weight of the q-th time segment in the key voice commands in the database; The formula used to generate the global alignment distance by dynamically time-normalizing the surgeon's key voice commands and key voice commands in the database is as follows: Where D(r,s) represents the minimum cumulative distance between the time segment of the r-th surgeon's key voice command and the time segment of the s-th key voice command in the database, r represents the time segment index of the surgeon's key voice command, and r∈[1,Q], s represents the time segment index of the key voice command in the database, and s∈[1,Q], w′ r w represents the weight of the r-th time segment in the surgeon's key voice commands. s MFCC′ represents the weight of the s-th time segment in the key voice commands of the database. r (i) represents the i-th order Mel-frequency cepstral coefficient (MFCC) of the time segment of the r-th key speech command from the chief surgeon. s (i) represents the i-th order Mel frequency cepstral coefficient of the time segment of the key speech command in the s-th database; Create a matrix D of size Q×Q, initialize the boundary conditions as D(0,0)=0, D(r,0)=+∞, D(0,s)=+∞, fill D(r,s) into matrix D, where r and s correspond to the row and column positions in the matrix, respectively. After filling matrix D, the value D(Q,Q) in the lower right corner of the matrix is ​​the global alignment distance. The formula used to generate normalized speech similarity is: Among them, S DTW This represents the normalized speech similarity.

6. A method for operating room medical order management based on speech recognition according to claim 5, characterized in that: The formula used to generate the comprehensive speech similarity in step 4 is as follows: Where K represents the overall speech similarity, k1 and k2 represent the weight coefficients of cosine similarity and normalized speech similarity, respectively, and k1 > k2.

7. A system for operating room medical order management based on speech recognition, characterized in that: The system is used for the real-time operation room medical order management application based on speech recognition as described in any one of claims 1-6, specifically including: The data preprocessing module is used to pre-collect the voice commands of each doctor, which include drug name, drug dosage and administration method. Key voice commands are extracted from the voice commands, which are the voice commands of drug names. The key voice commands are divided into several time segments at equal intervals. The multi-order Mel frequency cepstral coefficients of each time segment are calculated to form a reference sequence for each time segment. The reference sequences of the same key voice command are summarized as voice features. Doctor permission level and drug risk level are set. Based on voice features, doctor permission level and drug risk level, a doctor-drug permission database is established. The real-time data acquisition module is used to collect and calculate the surgeon's key voice commands in real time during the operation. It segments the commands according to the number of time segments that are the same as the key voice commands in the database, calculates the real-time sequence of each time segment of the surgeon's key voice commands, calculates the cosine similarity between the real-time sequence and the reference sequence under each time segment, and takes the average of the cosine similarity under all time segments to reflect the matching degree between the surgeon's key voice commands and the key voice commands in the database. The dynamic adjustment module is used to calculate the energy entropy of each time segment based on the surgeon's key voice commands and key voice commands in the database. The 25th percentile of the energy entropy of all time segments is used as the key segment boundary value. When the energy entropy of a time segment is less than the key segment boundary value, the time segment is marked as a non-key segment; otherwise, it is marked as a key segment. Weights are assigned to key segments and non-key segments respectively. Dynamic time warping is performed on the surgeon's key voice commands and key voice commands in the database to generate a global alignment distance. Normalized speech similarity is generated based on the global alignment distance. The voice matching module is used to generate a comprehensive voice similarity based on cosine similarity and normalized voice similarity, and to determine the voice feature with the highest comprehensive voice similarity in the doctor-drug permission database, so as to determine the identity and permission level of the current surgeon and determine the drug to be called. The instruction confirmation module is used to retrieve medications based on the surgeon's key voice commands when the surgeon's permission level meets the permission level for retrieving medications. If the surgeon's permission level does not meet the permission level for retrieving medications, the doctor with the highest permission level in the operating room will confirm the second voice command.

Images