Preoperative voice recognition and interaction evaluation method

Abstract

The invention discloses a preoperative voice recognition and interaction evaluation method, which comprises the following steps that: voice data of a patient and personal information data of the patient are combined, and the voice data can reflect voice characteristics and voice expression ability of the patient; personal information data can provide more information about the health condition and life style of the patient, and more comprehensive information is provided for medical evaluation. Secondly, the pronunciation dictionary is introduced into the interaction evaluation model, the voice recognition model is helped to more accurately understand the voice information of the patient, a more accurate evaluation result close to reality is provided, and the accuracy and reliability of the voice recognition model are further improved; and finally, by designing patient evaluation indexes, the satisfaction degree and trust degree of the patients on medical services are increased, and communication and cooperation between the patients and medical staff are promoted.

Description

Technical Field

[0001] The present invention relates to the field of speech interaction and index evaluation technology, and in particular to a preoperative speech recognition and interaction evaluation method. Background Art

[0002] Speech recognition technology is a technology that enables smart devices to understand human speech. It mainly involves multidisciplinary sciences such as digital signal processing, artificial intelligence, linguistics, mathematical statistics, acoustics, emotion and psychology.

[0003] In recent years, with the rise of artificial intelligence, speech recognition technology has made great breakthroughs in both theory and application, and has begun to move from the laboratory to the market and has gradually entered our daily lives. Speech recognition has now been used in many fields, mainly including speech recognition dictation devices, voice paging and Q&A platforms, autonomous advertising platforms, intelligent customer service, etc. Therefore, the ability to understand speech recognition and human-computer interaction is an important indicator for evaluating the intelligence of intelligent voice products.

[0004] However, we have adopted many model training or robot learning technologies to make the machine reasonably evaluate the patient and understand the patient's intention before the operation, but the effect is not ideal. It is often due to insufficient consideration, insufficient recognition of sound waveforms, and not close to real life. How to improve the accuracy of the model for sound recognition and evaluate and analyze patients is still the focus and hot spot of research and development personnel. Summary of the invention

[0005] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0006] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a pre-operative speech recognition and interactive evaluation method to solve the problems mentioned in the background technology.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a preoperative speech recognition and interactive evaluation method, comprising:

[0008] Acquire the patient's voice data and personal information data before the operation, convert the voice data into a voice signal, and perform feature extraction on the voice signal;

[0009] Establishing a speech recognition model and an interactive evaluation model, calculating a feature score of each extracted signal feature vector in the speech recognition model through a linear sequence of Mel frequency cepstral coefficients, and storing the feature score in a feature table;

[0010] Convert the patient's personal information data into feature values, establish a connectivity graph between the feature table and the feature values, introduce a pronunciation dictionary, train the pronunciation dictionary according to the feature table, and use the pronunciation dictionary as input to the interactive evaluation model;

[0011] According to the output results of the interactive evaluation model, a patient evaluation index is designed to obtain the patient's interactive evaluation score before surgery.

[0012] As a preferred solution of the preoperative speech recognition and interactive evaluation method of the present invention, the speech data and personal information data of the patient before the operation are obtained, the speech data is converted into a speech signal, and the feature extraction of the speech signal is performed, including:

[0013] Before extracting features from the speech signal, the speech signal is filtered into a waveform signal, and the waveform signal is divided into an obstacle-free waveform signal and an obstacle-containing waveform signal.

[0014] As a preferred solution of the preoperative speech recognition and interactive evaluation method of the present invention, wherein:

[0015] Also includes:

[0016] The emotional features in the barrier-free waveform signal and the barrier-affected waveform signal are analyzed, and the emotional features in the barrier-affected waveform signal are marked and rearranged.

[0017] As a preferred solution of the preoperative speech recognition and interactive evaluation method of the present invention, wherein:

[0018] A linear sequence of Mel-frequency cepstral coefficients, consisting of:

[0019] Define the speech signal process and obtain a linear sequence of Mel-frequency cepstral coefficients:

[0020]

[0021] Among them, M represents the unit vector in all extracted signals; α represents the unobstructed waveform signal; β represents the obstructed waveform signal; θ represents the label of the emotional feature; x and y represent the feature vectors of each extracted signal i.

[0022] As a preferred solution of the preoperative speech recognition and interactive evaluation method of the present invention, wherein: calculating the feature score of each extracted signal feature vector in the speech recognition model, and storing the feature score in the feature table, comprises:

[0023] The feature table is defined as a data structure table, wherein the rows of the data structure table correspond to feature vectors, and the columns correspond to the scores of the feature vectors.

[0024] As a preferred solution of the preoperative speech recognition and interactive evaluation method of the present invention, the patient's personal information data is converted into feature values, and a connectivity graph between the feature table and the feature values ​​is established, including:

[0025] The characteristic values ​​include multiple types of values ​​of the patient's personal information. If the multiple types of values ​​are all contained in the corresponding cells in the characteristic table, the connectivity graph is established; otherwise, the characteristic values ​​of the patient's personal information data are reconverted.

[0026] As a preferred solution of the preoperative speech recognition and interactive evaluation method of the present invention, wherein: training a pronunciation dictionary according to the feature table and using the pronunciation dictionary as the input of the interactive evaluation model includes:

[0027] The process of training the pronunciation dictionary using the feature table adopts cross training, and each cross training result is transmitted to the next training result to update the current training process of the model.

[0028] As a preferred solution of the preoperative speech recognition and interactive evaluation method of the present invention, wherein: according to the output result of the interactive evaluation model, a patient evaluation index is designed to obtain the patient's interactive evaluation score before surgery, including:

[0029] The patient evaluation indicators include the patient's physical condition, psychological condition, pain condition and satisfaction evaluation;

[0030] The interactive evaluation score is replaced according to the current training process of the interactive evaluation model.

[0031] Compared with the prior art, the invention has the following beneficial effects: the invention obtains the speech data and personal information data of the patient before the operation, converts the speech data into a speech signal, and extracts features from the speech signal; establishes a speech recognition model and an interactive evaluation model, calculates the feature score of each extracted signal feature vector in the speech recognition model through a linear sequence of Mel-frequency cepstral coefficients, and stores the feature score in a feature table; converts the patient's personal information data into a feature value, establishes a connectivity graph between the feature table and the feature value, introduces a pronunciation dictionary, trains the pronunciation dictionary according to the feature table, and uses the pronunciation dictionary as the input of the interactive evaluation model; designs a patient evaluation index according to the output result of the interactive evaluation model, and obtains the patient's Pre-operative interactive evaluation score; first, the present invention combines the patient's voice data and the patient's personal information data, wherein the voice data can reflect the patient's voice characteristics and voice expression ability, while the personal information data can provide more information about the patient's health status and lifestyle, providing more comprehensive information for medical evaluation; secondly, the pronunciation dictionary is introduced into the interactive evaluation model to help the speech recognition model to more accurately understand the patient's voice information, provide more accurate and practical evaluation results, and further improve the accuracy and reliability of the speech recognition model; finally, by designing patient evaluation indicators, the patient's satisfaction and trust in medical services are increased, and communication and cooperation between patients and medical staff are promoted. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0033] Figure 1 The overall flow chart of the preoperative speech recognition and interactive evaluation method according to one embodiment of the present invention;

[0034] Figure 2 The model loss degree of the preoperative speech recognition and interactive evaluation method according to an embodiment of the present invention;

[0035] Figure 3 This is a speech recognition waveform diagram of the preoperative speech recognition and interactive evaluation method described in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0038] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0039] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0040] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0041] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0042] Example 1

[0043] Reference Figure 1, which is the first embodiment of the present invention, provides a preoperative speech recognition and interactive evaluation method, including:

[0044] S1. Acquire the patient's voice data and personal information data before the operation, convert the voice data into a voice signal, and perform feature extraction on the voice signal;

[0045] Specifically, the patient's voice data before the operation is obtained through a condition description voice collection device;

[0046] Further, the speech data is converted into a speech signal, and features of the speech signal are extracted;

[0047] Specifically, feature extraction adopts hierarchical processing rules;

[0048] It should be noted that the hierarchical processing rules are adopted because in the actual process, collecting the patient's description of the disease characteristics involves personal privacy, so the patient's consent is required, and the processing load of subsequent feature extraction is reduced;

[0049] Specifically, the hierarchical processing rules are as follows:

[0050] The description of the patient's condition is set as the primary processing, and is stored in the form of a word sequence; each word is weighted by adding or subtracting weights; sensitive words are placed in the position with the highest weight, that is, the sensitive words are weighted, such as "cancer +10", "lesion +7" and "chemotherapy +5"; the rest of the description is set as the secondary processing, and the rest of the description is weighted, such as "a little -3", "slightly -8" and "mild -4";

[0051] It should be noted that the values ​​of these weightings can be adjusted according to actual needs;

[0052] Furthermore, before extracting features from the speech signal, the speech signal is filtered into a waveform signal, and the waveform signal is divided into an unobstructed waveform signal and an obstructed waveform signal;

[0053] Specifically, the obstacle-free waveform is a waveform with a slow peak, and the obstacle waveform signal is a waveform with a steep and fast peak;

[0054] It should be noted that in order to enable the subsequent speech recognition model to understand the patient's speech information more accurately, we considered the situation where the patient was deaf-mute or normal; and found that the waveform of the speech of the deaf-mute was steep and fast, while the waveform of the speech of the normal person was a relatively slow peak waveform;

[0055] Further, analyzing the emotional features in the barrier-free waveform signal and the barrier-free waveform signal, marking and rearranging the emotional features in the barrier-free waveform signal;

[0056] Specifically, the emotional features in the obstructed waveform signal are marked as "1", and the emotional features in the unobstructed waveform signal are marked as "0"; the rearrangement method is: all "1"s are arranged together until no "1" appears;

[0057] It should be noted that arranging speech signals with the same waveform helps optimize the feature extraction process and can be used for data visualization, which improves the accuracy of subsequent model evaluation;

[0058] S2, establishing a speech recognition model and an interactive evaluation model, calculating a feature score of each extracted signal feature vector in the speech recognition model through a linear sequence of Mel frequency cepstral coefficients, and storing the feature score in a feature table;

[0059] Specifically, the speech recognition model and the interactive evaluation model are established as follows:

[0060]

[0061] Where W represents a multi-type value of the patient's personal information, It represents the emotional feature label, t=1 represents the current model state is "loading"; Y represents the speech recognition result; G represents the feature table T and feature value The connected graph of Q v |W v-1 It is represented by the probability Q of filling in the previous cell v under multiple types of values, Represented as the output of the interactive evaluation model, It represents the pronunciation dictionary; R represents the model training process; S represents the evaluation score;

[0062] Furthermore, the speech signal process is defined to obtain a linear sequence of Mel-frequency cepstral coefficients:

[0063]

[0064] Where M represents the unit vector in all extracted signals; α represents the unobstructed waveform signal; β represents the obstructed waveform signal; θ represents the label of the emotional feature; x and y represent the feature vectors of each extracted signal i;

[0065] Further, the feature scores are stored in a feature table, and the feature table is defined as a data structure table, wherein the rows of the structure table correspond to the feature vectors, and the columns correspond to the scores of the feature vectors;

[0066] Specifically, the feature table is a multidimensional data table, which uses a pointer chain connection method, that is, calling a feature vector can get the score of the feature vector without looking up the specific content contained in the column, which saves the model running time;

[0067] S3, converting the patient's personal information data into feature values, establishing a connectivity graph between the feature table and the feature values, introducing a pronunciation dictionary, training the pronunciation dictionary according to the feature table, and using the pronunciation dictionary as the input of the interactive evaluation model;

[0068] Further, the converted feature values ​​include multiple types of values ​​of the patient's personal information. If the multiple types of values ​​are all contained in the corresponding cells in the feature table, then the connectivity graph is established; otherwise, the feature values ​​of the patient's personal information data are reconverted;

[0069] Specifically, the multi-type values ​​include: basic information: the patient's name, gender, age, date of birth, etc.; address information: including the patient's address, zip code, etc.; contact information: including the patient's mobile phone number, landline number, email address, etc.; medical information: including the patient's medical record number, treatment department, medical insurance type, etc.; social information: including the patient's occupation, education level, marital status, etc.;

[0070] It should be noted that classifying all the patient information into multi-type values ​​can fully consider the actual patient situation and provide an accurate data set for the training of the pronunciation dictionary;

[0071] Furthermore, the process of training the pronunciation dictionary using the feature table adopts cross training, and each cross training result is transferred to the next training result to update the current training process of the model;

[0072] Specifically, the steps before the crossover processing are: randomly divide the data set in the feature table into k parts; extract one of the data sets as the validation set each time without repeated extraction, and use the remaining k-1 parts as the validation set for crossover model training, and train a total of k times;

[0073] It should be noted that traditional crossover can only be used on small sample data sets, otherwise it is difficult to identify trends in the data set, resulting in errors. By establishing a feature table, we pass each crossover training result to the next training result and update the current training process of the model, which can effectively solve the problem that the traditional crossover method can only train small-scale data.

[0074] S4. Design patient evaluation indicators based on the output results of the interactive evaluation model to obtain the patient's interactive evaluation score before surgery;

[0075] Furthermore, the patient evaluation indicators include the patient's physical condition, psychological condition, pain condition, and satisfaction evaluation;

[0076] Furthermore, the interactive evaluation score is replaced by the score according to the current training process of the interactive evaluation model;

[0077] Specifically, before designing the patient evaluation indicators, a patient questionnaire needs to be designed, as shown in Table 1;

[0078] Table 1 Questionnaire

[0079]

[0080] Specifically, the patient evaluation indicators refer to Table 2;

[0081] Table 2 Evaluation indicators

[0082] index result Score gender Male, accurate 100 age 45, accurate 100 Symptoms (viral cold) Common influenza 70 Initial diagnosis score precise 86

[0083] It should be noted that by designing patient evaluation indicators and survey questionnaires, patients' satisfaction and trust in medical services are increased, and communication and cooperation between patients and medical staff are promoted.

[0084] Example 2

[0085] Reference Figure 2 and Figure 3 , which is the second embodiment of the present invention, and which provides a preoperative speech recognition and interactive evaluation method, comprising: verifying and explaining the beneficial effects of the solution of the present invention;

[0086] We collected patient information datasets from hospitals in a certain area and added a lot of noise. We put all the datasets in the same area through cross-training. We can visually compare the differences between the data sets and observe the waveform signals under complex noise. Figure 3 ;

[0087] It can be seen that under complex noise, the sound can still be propagated stably, indicating that the model conditions established in the solution of the present invention are fully considered and can remain stable all the time; we also tested the loss value of the model and found that according to the slight interval iteration, the loss value did not exceed 0.1, indicating that the model training of the present invention meets the actual expectations and can maintain a low loss value under the premise of high accuracy; thus improving the accuracy and reliability of the speech recognition model.

[0088] Those skilled in the art will appreciate that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0089] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0093] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A preoperative speech recognition and interactive evaluation method, characterized in that: include: Acquire the patient's voice data and personal information data before the operation, convert the voice data into a voice signal, and perform feature extraction on the voice signal; Establishing a speech recognition model and an interactive evaluation model, calculating a feature score of each extracted signal feature vector in the speech recognition model through a linear sequence of Mel frequency cepstral coefficients, and storing the feature score in a feature table; Convert the patient's personal information data into feature values, establish a connectivity graph between the feature table and the feature values, introduce a pronunciation dictionary, train the pronunciation dictionary according to the feature table, and use the pronunciation dictionary as input to the interactive evaluation model; According to the output results of the interactive evaluation model, a patient evaluation index is designed to obtain the patient's interactive evaluation score before surgery.

2. The preoperative speech recognition and interactive evaluation method according to claim 1, characterized in that: Acquiring the patient's voice data and personal information data before the operation, converting the voice data into a voice signal, and performing feature extraction on the voice signal, including: Before extracting features from the speech signal, the speech signal is filtered into a waveform signal, and the waveform signal is divided into an obstacle-free waveform signal and an obstacle-containing waveform signal.

3. The preoperative speech recognition and interactive evaluation method according to claim 2, characterized in that: Also includes: The emotional features in the barrier-free waveform signal and the barrier-affected waveform signal are analyzed, and the emotional features in the barrier-affected waveform signal are marked and rearranged.

4. The preoperative speech recognition and interactive evaluation method according to claim 2 or 3, characterized in that: A linear sequence of Mel-frequency cepstral coefficients, consisting of: Define the speech signal process and obtain a linear sequence of Mel-frequency cepstral coefficients: Among them, M represents the unit vector in all extracted signals; α represents the unobstructed waveform signal; β represents the obstructed waveform signal; θ represents the label of the emotional feature; x and y represent the feature vectors of each extracted signal i.

5. The preoperative speech recognition and interactive evaluation method according to claim 4, characterized in that: Calculating a feature score for each extracted signal feature vector in the speech recognition model and storing the feature score in a feature table includes: The feature table is defined as a data structure table, wherein the rows of the data structure table correspond to feature vectors, and the columns correspond to the scores of the feature vectors.

6. The preoperative speech recognition and interactive evaluation method according to claim 5, characterized in that: The patient's personal information data is converted into feature values, and a connectivity graph between the feature table and the feature values ​​is established, including: The characteristic values ​​include multiple types of values ​​of the patient's personal information. If the multiple types of values ​​are all contained in the corresponding cells in the characteristic table, the connectivity graph is established; otherwise, the characteristic values ​​of the patient's personal information data are reconverted.

7. The preoperative speech recognition and interactive evaluation method according to claim 6, characterized in that: Introducing a pronunciation dictionary, training the pronunciation dictionary according to the feature table, and using the pronunciation dictionary as an input of an interactive evaluation model, including: The process of training the pronunciation dictionary using the feature table adopts cross-training, and each cross-training result is transmitted to the next training result to update the current training process of the model.

8. The preoperative speech recognition and interactive evaluation method according to claim 7, characterized in that: According to the output results of the interactive evaluation model, patient evaluation indicators are designed to obtain the patient's interactive evaluation score before surgery, including: The patient evaluation indicators include the patient's physical condition, psychological condition, pain condition and satisfaction evaluation; The interactive evaluation score is replaced according to the current training process of the interactive evaluation model.

Images