A medical dialogue question and answer method and device, computer equipment and storage medium

Abstract

The application relates to the technical field of medical question and answer, and discloses a medical dialogue question and answer method, device, computer equipment and storage medium, wherein the medical dialogue question and answer method first performs discrete coding on a medical question voice signal through an audio coding model to obtain audio coding detail features containing semantic information, emotional color and context relationship; secondly, the audio coding detail features are represented in a vector form through an encoding vectorization model to obtain question vector representation data; finally, an answer is generated for the question vector representation data through a multi-head attention large model to obtain a medical answer voice signal corresponding to the medical question voice signal. The beneficial effect is that the multi-head attention large model is input with voice detail information required for accurately understanding a patient's question, the obtained answer is not only accurate but also can simulate empathy, and the humanistic care for the patient is fully reflected, so that the user can be better guided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical question and answer, and in particular relates to a medical dialogue question and answer method and device, a computer device and a storage medium. BACKGROUND

[0002] Ophthalmic patients often face the challenge of visual impairment and need voice guidance to assist cooperation during eye examination. This highlights the urgent need for voice services. Voice services not only convey tone and emotion, but also help to better understand the patient's intention, thereby significantly improving patient compliance.

[0003] In related technologies, a medical intelligent voice service system usually converts an audio signal into text through a voice-to-text model, and then generates an answer through a text input question and answer model.

[0004] However, the voice detail information processing level of the medical intelligent voice service in related technologies needs to be improved, and therefore a new medical dialogue question and answer method is urgently needed. SUMMARY

[0005] The present application provides a medical dialogue question and answer method, device, computer device and storage medium, which solves the technical problem of the need to improve the voice detail information processing level of the medical intelligent voice service in related technologies, and achieves the technical effect of more accurate understanding of patient problems by voice services, more accurate answers and the ability to simulate empathy.

[0006] In order to achieve the above purpose, the main technical solution adopted by the present application comprises:

[0007] In a first aspect, the present application provides a medical dialogue question and answer method, which comprises:

[0008] obtaining a medical question voice signal and inputting the medical question voice signal into a medical question and answer model; wherein the medical question and answer model comprises an audio coding model, a coding vectorization model and a multi-head attention large model connected in turn;

[0009] directly discretely coding the medical question voice signal through the audio coding model to obtain audio coding detail features, and inputting the audio coding detail features into the coding vectorization model; wherein the audio coding detail features are used to represent semantic information, emotional color and context relationship in the medical question voice signal at the same time;

[0010] vectorizing the audio coding detail features through the coding vectorization model to obtain question vector representation data, and inputting the question vector representation data into the multi-head attention large model; wherein the question vector representation data is used to represent semantic association information of the medical question voice signal;

[0011] generate an answer to the question vector representation data by the multi-head attention large model to output a medical answer voice signal corresponding to the medical question voice signal.

[0012] Optionally, the multi-head attention large model is sequentially connected with a vector conversion model and an audio decoding model; the generating an answer to the question vector representation data by the multi-head attention large model to output a medical answer voice signal corresponding to the medical question voice signal comprises:

[0013] extracting features from the question vector representation data by the multi-head attention large model to obtain intermediate vector representation data;

[0014] inputting the intermediate vector representation data into the vector conversion model to obtain intermediate coding detail features;

[0015] inputting the intermediate coding detail features into the audio decoding model for decoding to obtain the medical answer voice signal.

[0016] Optionally, the audio encoding model and the audio decoding model are obtained in the following manner:

[0017] constructing a dialog audio dataset in the medical field;

[0018] training an initial encoding model and an initial decoding model synchronously using the dialog audio dataset to obtain the audio encoding model and the audio decoding model.

[0019] Optionally, the encoding vectorization model and the vector conversion model are obtained in the following manner:

[0020] discretely encoding the dialog audio dataset using the audio encoding model to obtain a feature coding dataset corresponding to the dialog audio dataset;

[0021] training an initial encoding vectorization model and an initial vector conversion model synchronously using the feature coding dataset to obtain the encoding vectorization model and the vector conversion model.

[0022] Optionally, the multi-head attention large model is obtained in the following manner:

[0023] vectorizing representation of the feature coding dataset using the encoding vectorization model to obtain an audio vector dataset corresponding to the feature coding dataset;

[0024] training an initial large model using the audio vector dataset to obtain the multi-head attention large model.

[0025] Optionally, the training of the initial large model by using the audio vector dataset comprises:

[0026] training of the initial large model by using the audio vector dataset to obtain a first intermediate large model;

[0027] obtaining a specified scene medical conversation audio, fine-tuning the model parameters of the intermediate large model by using a specified scene vector dataset corresponding to the specified scene medical conversation audio to obtain a second intermediate large model;

[0028] obtaining evaluation feedback data for the prediction result of the second intermediate large model; wherein the evaluation feedback data is used to represent an improvement direction of the second intermediate large model which does not meet the preset prediction requirement in the prediction result;

[0029] continuing training of the second intermediate large model by using specific conversation audio for the improvement direction to obtain the multi-head attention large model.

[0030] Optionally, the conversation audio dataset in the medical field is constructed by the following method:

[0031] obtaining real medical voice conversation audio, real medical text conversation and open source text conversation in the medical field;

[0032] taking the real medical voice conversation audio as a main dataset;

[0033] taking deductive medical conversation audio and open source medical conversation audio as a supplementary dataset; wherein the conversation audio dataset comprises the main dataset and the supplementary dataset; the deductive medical conversation audio is obtained by deducting the real medical text conversation, and the open source medical conversation audio is obtained by voice synthesis of the open source text conversation.

[0034] In a second aspect, an embodiment of the present application provides a medical conversation question and answer device, the device comprising:

[0035] a voice signal acquisition module configured to acquire a medical question voice signal and input the medical question voice signal into a medical question and answer model; wherein the medical question and answer model comprises an audio coding model, a coding vectorization model and a multi-head attention large model connected in sequence;

[0036] a discrete coding module configured to directly perform discrete coding on the medical question voice signal by using the audio coding model to obtain audio coding detail features, and input the audio coding detail features into the coding vectorization model; wherein the audio coding detail features are used to represent semantic information, emotional color and context relationship in the medical question voice signal at the same time;

[0037] a vectorization module configured to vectorize the audio coding detail features by the coding vectorization model to obtain question vector representation data, and input the question vector representation data into the multi-head attention large model, wherein the question vector representation data is configured to represent semantic association information of the medical question voice signal;

[0038] an answer generation module configured to generate an answer by the multi-head attention large model based on the question vector representation data, to output a medical answer voice signal corresponding to the medical question voice signal.

[0039] In a third aspect, an embodiment of the present application provides a computer device, comprising:

[0040] a memory and a processor, which are communicatively connected with each other, and the memory stores computer instructions, and the processor executes the computer instructions to perform the method described in any one of the preceding aspects.

[0041] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are configured to cause a computer to perform the method described in any one of the preceding aspects.

[0042] In the present application, first, the medical question voice signal is discretely coded by an audio coding model to obtain audio coding detail features containing semantic information, emotional color and context relationship; second, the audio coding detail features are vectorized by a coding vectorization model to obtain question vector representation data; finally, the question vector representation data is answered by a multi-head attention large model to obtain a medical answer voice signal corresponding to the medical question voice signal. By inputting the voice detail information required for accurately understanding the medical question into the multi-head attention large model, the obtained answer is not only accurate, but also can simulate empathy, fully embodies the humanistic care for patients, and thus can better assist and guide the user. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the specific embodiments or prior art in the present application, the drawings needed in the specific embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0044] Figure 1 a flowchart of the medical dialogue question and answer method provided by the embodiments of the present application;

[0045] Figure 2A schematic diagram of a medical question and answer model provided for an embodiment of the present specification;

[0046] Figure 3 A schematic diagram of a medical question and answer model provided for an embodiment of the present specification;

[0047] Figure 4 A flowchart of a medical dialogue question and answer method provided for an embodiment of the present specification;

[0048] Figure 5 A model training flowchart of a medical dialogue question and answer method provided for an embodiment of the present specification;

[0049] Figure 6 A model training flowchart of a medical dialogue question and answer method provided for an embodiment of the present specification;

[0050] Figure 7 A model training flowchart of a medical dialogue question and answer method provided for an embodiment of the present specification;

[0051] Figure 8 A model training flowchart of a medical dialogue question and answer method provided for an embodiment of the present specification;

[0052] Figure 9 A schematic diagram of a medical dialogue question and answer device provided for an embodiment of the present specification;

[0053] Figure 10 A structural schematic diagram of a computer device provided for an embodiment of the present specification. DETAILED DESCRIPTION

[0054] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0055] Ophthalmic patients, especially the elderly population, urgently need voice guidance during eye examination. However, the prior art usually adopts a voice-to-text processing method, which converts the patient's voice into text and then inputs the medical question and answer model to generate an answer. This processing procedure inevitably leads to the loss of detailed information contained in the voice, such as rich semantics and subtle emotions, making it difficult for these systems to perceive the semantic and emotional state of the patient as acutely as a human doctor, which may result in incorrect answers or lack of necessary empathy, failing to provide the required psychological comfort and support for the patient. In addition, China is a complex language environment with multiple languages and dialects, and factors such as language differences, individual language habits, and regional accents significantly affect the accuracy of voice-to-text technology. These challenges not only highlight the inadequacy of related technology, but also indicate the urgency of developing more accurate voice recognition and understanding technology in the medical field, so as to provide ophthalmic patients with a more personalized and high-quality medical experience.

[0056] Based on this, the present application provides a medical dialogue question and answer method, which first encodes the medical question voice signal through an audio encoding model to obtain audio encoding detail features containing semantic information, emotional color and context relationship; secondly, the audio encoding detail features are represented by a vector through an encoding vectorization model to obtain question vector representation data; finally, the question vector representation data is generated by a multi-head attention large model to obtain the medical answer voice signal corresponding to the medical question voice signal. By inputting the voice detail information required for accurate understanding of the medical question to the multi-head attention large model, the answer obtained is not only accurate, but also can simulate empathy, fully reflecting the humanistic care for patients, so as to better assist in guiding the user.

[0057] According to an embodiment of the present application, a medical dialogue question and answer method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0058] Please refer to Figure 1 The medical dialogue question and answer method provided by the embodiment of the present application includes the following steps:

[0059] S110, obtaining a medical question voice signal and inputting the medical question voice signal to a medical question and answer model.

[0060] The medical question voice signal can be various voice information related to health, symptoms or treatment expressed by a patient in a medical scene. These voice signals can include the patient describing a disease, asking a question, expressing an emotion or feedback on a treatment process, such as "I have dry eye", "Is glaucoma easy to treat?", "I have used chloramphenicol eye drops for a week, but there is no improvement", and the like. Please refer to Figure 2 The medical question and answer model includes an audio coding model, an encoding vectorization model and a multi-head attention large model connected in sequence.

[0061] S120, directly discretely encode the medical question voice signal through the audio coding model to obtain audio coding detail features, and input the audio coding detail features into the encoding vectorization model.

[0062] The audio coding model can extract features from the medical question voice signal, thereby realizing discretely encoding the medical question voice signal to obtain its audio coding detail features. The audio coding detail features are used to represent the semantic information, emotional color and context relationship in the medical question voice signal at the same time.

[0063] S130, the audio coding detail features are represented by the encoding vectorization model to obtain question vector representation data, and the question vector representation data are input into the multi-head attention large model.

[0064] The audio coding detail features include a plurality of features extracted from the medical question voice signal, and the encoding vectorization model converts the audio coding detail features of the medical question into a vector form, so that each feature becomes a dimension in the vector. This process maps the features of the audio signal into a high-dimensional space, so that complex data relationships can be processed in a multi-dimensional space. The question vector representation data are used to represent the semantic association information of the medical question voice signal, including the semantic information, emotional color and context relationship of the medical question.

[0065] S140, the multi-head attention large model generates answers to the question vector representation data to output the medical answer voice signal corresponding to the medical question voice signal.

[0066] Specifically, the multi-head attention large model uses multiple attention heads to focus on different parts of the input question vector representation data in parallel, extracting rich contextual information. Further, through the self-attention mechanism, the information extracted by different heads is integrated to better understand the semantic information, emotional color and contextual relationship of the question. Further, based on the result of the above information fusion, the corresponding answer is generated. Because the multi-head attention large model has the ability to understand the semantic information, emotional color and contextual relationship contained in the question, the generated answer is not only accurate, but also has a doctor-like empathy, reflecting humanistic care. This feature enables the medical question and answer model to better meet the emotional needs and psychological support of patients when responding to them.

[0067] It can be understood that the answer output by the multi-head attention large model is in vector form, and further conversion into discrete encoding and decoding is required to obtain the medical answer voice signal corresponding to the medical question voice signal.

[0068] In the above embodiment, first, the medical question voice signal is discretely encoded by the audio encoding model to obtain audio encoding detail features containing semantic information, emotional color and contextual relationship; second, the audio encoding detail features are vectorized by the encoding vectorization model to obtain question vector representation data; finally, the multi-head attention large model is used to generate answers to the question vector representation data to obtain the medical answer voice signal corresponding to the medical question voice signal. By inputting the voice detail information required to accurately understand the patient's question to the multi-head attention large model, the answer obtained is not only accurate, but also can simulate empathy, fully reflecting the humanistic care for patients, thereby better assisting and guiding users.

[0069] In some embodiments, referring to Figure 3 , the multi-head attention large model is sequentially connected with a vector conversion model and an audio decoding model. That is, the medical question and answer model includes an audio encoding model, an encoding vectorization model, a multi-head attention large model, a vector conversion model and an audio decoding model connected in sequence. The multi-head attention large model is used to generate answers to the question vector representation data to output the medical answer voice signal corresponding to the medical question voice signal, referring to Figure 4 , the steps include:

[0070] S210, feature extraction is performed on the question vector representation data by the multi-head attention large model to obtain intermediate vector representation data.

[0071] S220, the intermediate vector representation data is input into the vector conversion model to obtain intermediate encoding detail features.

[0072] S230, the intermediate encoding detail features are input into the audio decoding model for decoding to obtain the medical answer voice signal.

[0073] The function of the vector conversion model is to convert the intermediate vector representation data into discrete form encoding, i.e., intermediate encoded detail features. The function of the audio decoding model is to decode the intermediate encoded detail features in discrete form to obtain a continuous medical answer voice signal.

[0074] Specifically, after the question vector representation data is input into the multi-head attention large model, the multi-head attention large model extracts features from the question vector representation data, and uses multiple attention heads to focus on different features extracted in parallel to extract rich context information. Further, through the self-attention mechanism, the information extracted by different heads is integrated to better understand the semantic information, emotional color and context relationship of the question. Based on the result of the above information fusion, the corresponding intermediate vector representation data is generated. Then, the intermediate vector representation data is converted into intermediate encoded detail features through the vector conversion model. Finally, the intermediate encoded detail features are converted into a medical answer voice signal through the audio decoding model.

[0075] In the above embodiment, first, the multi-head attention large model is used to extract features from the question vector representation data, thereby obtaining intermediate vector representation data. Second, the intermediate vector representation data is input into the vector conversion model to obtain intermediate encoded detail features. Finally, the intermediate encoded detail features are decoded by the audio decoding model to generate a voice signal of the medical answer. Such voice output enables ophthalmic patients to conveniently obtain the required medical question answers, and these answers are accurate and empathetic, making patients feel the humanistic care like a doctor, thereby improving the medical experience of ophthalmic patients.

[0076] In some embodiments, referring to Figure 5 , the audio encoding model and the audio decoding model are obtained by the following method:

[0077] S310, constructing a dialogue audio dataset in the medical field.

[0078] S320, synchronously training the initial encoding model and the initial decoding model using the dialogue audio dataset to obtain the audio encoding model and the audio decoding model.

[0079] Specifically, the dialogue audio dataset in the medical field can be real ophthalmic doctor-patient dialogue recordings about the patient's condition, medication or treatment, which have different emotional colors, different scenes and features reflecting the doctor's humanistic care. In addition, the dialogue audio dataset can also be audio generated by real people acting on real medical text dialogues or audio generated by text-to-speech technology. It should be noted that the audio data generated by the text-to-speech technology needs to have natural voice features, including changes in intonation, emotion and speech rate.

[0080] Exemplarily, based on the reconstruction paradigm of the audio signal, the initial encoding model and the initial decoding model are synchronously trained by using the dialogue audio data set in the medical field, and the key steps are as follows:

[0081] 1) Select 80% of the data in the dialogue audio data set as the training set, and the remaining 20% of the data as the validation set; select a suitable loss function (such as mean square error loss function or cross entropy loss function) to measure the gap between the reconstructed audio signal output by the initial decoding model and the initial audio signal input to the initial encoding model; select a suitable optimization algorithm (such as Adam optimization algorithm) to update the model parameters to minimize the loss.

[0082] 2) Train the model by various methods, such as self-supervised learning method, semi-supervised learning method, and residual vector quantizer technology. When the initial encoding model and the initial decoding model trained by each method respectively have desired encoding and decoding capabilities, including correctly encoding and decoding the semantic information, emotional color and context relationship in the training data, the training is completed. By comparing the performance of these methods in audio feature extraction and information preservation, an optimal model is selected, mainly from the aspects of encoding efficiency, information retention degree and computational complexity. High encoding efficiency means that in the least amount of encoding, that is, in the least amount of audio encoding detail features, high-quality audio signals can be effectively represented and restored. The information retention degree can be the degree of key information retained by the reconstructed audio signal from the original audio signal. The computational complexity can be the length of time required for the encoding and decoding process or the computer resources required. The performance of each model in encoding efficiency, information retention degree and computational complexity can be considered comprehensively to select an optimal model.

[0083] 3) Adjust the optimizable model parameters to fine-tune the optimal model with the data of the validation set, and the main training goal is to achieve a desired information retention degree, such as 90% frame preservation rate, with the least amount of audio encoding detail features. After fine-tuning is completed, the audio encoding model and the audio decoding model are obtained.

[0084] In the above embodiment, by constructing a dialogue audio data set in the medical field, and synchronously training the initial encoding model and the initial decoding model by using the dialogue audio data set, the audio encoding model and the audio decoding model are obtained. The audio encoding model can provide data support for the training of the initial encoding vectorization model and the initial vector conversion model by discretely encoding the medical dialogue audio data set. At the same time, the model can extract the detail information in the medical dialogue audio, laying a good encoding foundation for the transmission of the detail information of the medical problem voice signal to the multi-head attention large model. The audio decoding model provides a good decoding foundation for accurately and empathetically delivering the medical problem answers to the patient in the form of voice.

[0085] In some embodiments, referring to Figure 6 The encoding vectorization model and the vector conversion model are obtained by the following manner:

[0086] S410, discretely encoding the dialogue audio data set by using the audio encoding model to obtain a feature encoding data set corresponding to the dialogue audio data set.

[0087] S420, synchronously training the initial encoding vectorization model and the initial vector conversion model by using the feature encoding data set to obtain the encoding vectorization model and the vector conversion model.

[0088] Specifically, the dialogue audio data set is discretely encoded by using the audio encoding model trained and optimized to obtain a feature encoding data set corresponding to the dialogue audio data set.

[0089] Exemplarily, based on the reconstruction paradigm of the feature encoding data, the initial encoding vectorization model and the initial vector conversion model are synchronously trained by using the feature encoding data set in the ophthalmic medical field, and the key steps are as follows:

[0090] 1) Select 80% of the data in the feature encoding data set as the training set, and the remaining 20% of the data as the validation set; select a suitable loss function (such as mean square error loss function or cross-entropy loss function) to measure the gap between the reconstructed feature encoding data output by the initial vector conversion model and the feature encoding data input by the initial encoding vectorization model; select a suitable optimization algorithm (such as Adam optimization algorithm) to update the model parameters to minimize the loss.

[0091] 2) Train the model by various methods, such as self-supervised learning method, semi-supervised learning method, factor analysis-based method and adversarial generative network technology. When the initial encoding vectorization model and the initial vector conversion model trained by each method respectively have the expected encoding and vectorization capabilities, including the ability to encode and vectorize the semantic information, emotional color and context relationship in the training data, the training is completed. By comparing the performance of these methods in terms of semantic retention, vector space efficiency and computing performance, an optimal model is selected. Semantic retention refers to the degree to which features are retained during encoding and vectorization. The spatial efficiency of the vector refers to the effectiveness of the memory resources required for storing and representing the vector relative to the dimension and data volume of the vector. A good vector representation method needs to find a suitable balance between semantic retention and vector space efficiency, and also takes into account the computing performance. The performance of each model in terms of semantic retention, vector space efficiency and computing performance can be considered comprehensively to select one as the optimal model.

[0092] 3) By adjusting the model parameters that can be optimized, the optimal model is fine-tuned with the data of the validation set. The main training goal is to achieve the desired semantic retention, such as 90% semantic retention, with the least vector. After fine-tuning, the encoding vectorization model and the vector conversion model are obtained.

[0093] In the above embodiment, first, the dialogue audio data set is discretely encoded using the audio encoding model to obtain a feature encoding data set corresponding to the dialogue audio data set; second, the initial encoding vectorization model and the initial vector conversion model are trained simultaneously using the feature encoding data set to obtain the encoding vectorization model and the vector conversion model. The encoding vectorization model can vectorize the feature encoding data set of the medical dialogue, providing data support for the training of the initial large model. At the same time, this model can convert the detailed information of the medical dialogue audio from encoding to vector format, providing an encoding vectorization basis for the smooth transmission of detailed information of the medical question voice signal to the large model. The vector conversion model provides a good vectorization decoding basis for delivering accurate and empathetic medical question answers to patients.

[0094] In some embodiments, referring to Figure 7 , the multi-head attention large model is obtained by the following method:

[0095] S510, vectorizing the feature encoding data set using the encoding vectorization model to obtain an audio vector data set corresponding to the feature encoding data set.

[0096] S520, training the initial large model using the audio vector data set to obtain the multi-head attention large model.

[0097] Wherein, the initial large model is an untrained large model, which can be trained using the audio vector data set to learn the semantic information, emotional color and context relationship in the audio vector data set.

[0098] Specifically, the feature encoding data set is vectorized using the encoding vectorization model obtained by training and fine-tuning to obtain an audio vector data set corresponding to the feature encoding data set.

[0099] Exemplarily, when training the initial large model with the audio vector dataset, the input includes auxiliary information corresponding to each piece of data in addition to the audio vector dataset. The auxiliary information exists in the form of labels, including text labels, emotion labels (such as happy, angry, anxious), and context relationship labels (such as appointment inquiry, symptom inquiry, and drug inquiry) of each piece of data. The text labels help the initial large model capture the relationship between audio and language; the emotion labels help the initial large model identify and understand the emotional changes in the audio; and the context relationship labels help the initial large model better understand the scene of the audio data, thereby capturing deeper semantic relationships. When the initial large model learns the audio vector dataset, the training is completed, and the multi-head attention large model is obtained.

[0100] Exemplarily, the training of the initial large model needs to balance between the data volume and the token number of the audio vector dataset. The training parameters of the large language model can be used as a reference. For example, the medical question and answer knowledge base of the language model of the same level of knowledge volume has 2000 tokens, and 100,000 pieces of text data are used for training. According to this estimation, the token number of the medical question and answer knowledge base of the voice large model is 6000 to 8000, and about 300,000 pieces of data are needed for training. Generally, the data volume of the audio vector dataset is fixed. If the token number used for initial training is 7000 and it is found that the training effect of the initial large model is not good due to too many tokens, the token aggregation degree can be improved by adjusting the parameters of the audio encoding model and the encoding vectorization model to reduce the number of output tokens until the initial large model reaches the expected training effect. Conversely, if the training effect of the initial large model is better than expected, the token aggregation degree can be reduced by adjusting the parameters of the audio encoding model and the encoding vectorization model to increase the number of output tokens until the initial large model reaches the expected training effect.

[0101] In the above embodiment, the feature encoding dataset is vectorized by the encoding vectorization model to obtain an audio vector dataset corresponding to the feature encoding dataset, and the initial large model is trained with the audio vector dataset to obtain a multi-head attention large model. The audio vector dataset contains rich medical dialogue detail information, which provides support for subsequent applications. When the input medical question contains these details, the multi-head attention large model can quickly combine them with the existing knowledge base to generate accurate and empathetic answers.

[0102] In some embodiments, referring to Figure 8 , the initial large model is trained with the audio vector dataset to obtain a multi-head attention large model, and the steps include:

[0103] S610, training the initial large model by using the audio vector dataset to obtain a first intermediate large model.

[0104] The audio vector dataset is obtained by using an encoding vectorization model to vectorize the feature encoding dataset.

[0105] Specifically, when the initial large model is trained by using the audio vector dataset, the audio vector dataset and the auxiliary information corresponding to each piece of data are input. The auxiliary information exists in the form of labels, including text labels, emotion labels (such as happy, angry, and anxious) and context relationship labels (such as appointment inquiry, symptom inquiry, and drug inquiry) of each piece of data. The text labels help the initial large model to capture the relationship between the audio and the language; the emotion labels help the initial large model to identify and understand the emotional changes in the audio; and the context relationship labels help the initial large model to better understand the scene and background of the audio data, so as to capture deeper semantic relationships. After the initial large model learns the audio vector dataset, the training is completed, and the first intermediate large model is obtained.

[0106] S620, obtaining specified scene medical conversation audio, fine-tuning the model parameters of the intermediate large model by using the specified scene vector dataset corresponding to the specified scene medical conversation audio to obtain a second intermediate large model.

[0107] The specified scene can be a common scene in ophthalmic medical conversation, such as initial diagnosis consultation, vision examination result discussion, disease diagnosis communication, treatment plan discussion, preoperative preparation and postoperative care, follow-up and review, etc. The specified scene medical conversation audio can be audio data for supplementary training of some scenes with poor training effect on the first intermediate large model. The specified scene vector dataset can be a dataset that can be input into the intermediate large model for training, which is obtained by discretely encoding the specified scene medical conversation audio by an audio encoding model and then vectorizing by an encoding vectorization model.

[0108] Specifically, after obtaining the first intermediate large model, the common problems in the specified scene are asked to the first intermediate large model by manual, and the first intermediate large model is tested. After the test, for some scenes with poor training effect found in the test process, the specified scene vector dataset is used for supplementary training to fine-tune the model parameters of the intermediate large model, and the second intermediate large model is obtained. For example, if the first intermediate large model does not answer correctly when asked about what a certain drug is used for, the first intermediate large model can be supplemented and trained by using the vector dataset of the introduction of the drug. After the training is completed, the second intermediate large model is obtained.

[0109] S630, obtaining evaluation feedback data for the prediction result of the second intermediate large model.

[0110] The evaluation feedback data is used to represent the improvement direction of the second intermediate large model that does not meet the preset prediction requirements in the prediction result.

[0111] Specifically, the evaluation feedback data for the prediction result of the second intermediate large model can be obtained through the evaluation feedback of human experts. After the training of the second intermediate large model is completed, a strict comparative experiment can be performed on the test set to evaluate its performance. A multi-dimensional index system can be used, including but not limited to: speech recognition accuracy, semantic accuracy of dialogue content, empathy level of model performance, naturalness and clarity of generated speech, processing ability of professional medical terms, system response speed and other evaluation indexes. Through the organization of a professional artificial evaluation team, the prediction result of the second intermediate large model is scored, and the scoring result is statistically analyzed to obtain the evaluation feedback data for the prediction result of the large model.

[0112] S640, continue training the second intermediate large model using the specific dialogue audio for the improvement direction to obtain a multi-head attention large model.

[0113] The specific dialogue audio can be a data set for supplementary training of the improvement direction.

[0114] Specifically, the specific dialogue audio can be discretely encoded by an audio encoding model to obtain audio encoding detail features of the specific dialogue audio, and then the audio encoding detail features are represented by a vectorization model to obtain vector representation data of the specific dialogue audio. Finally, the vector representation data and the corresponding auxiliary data are input into the second intermediate large model for training, and the multi-head attention large model is obtained after the training is completed.

[0115] In the above embodiments, first, the initial large model is trained using the audio vector data set to obtain the first intermediate large model; second, the specified scene medical dialogue audio is obtained, and the model parameters of the intermediate large model are fine-tuned using the specified scene vector data set corresponding to the specified scene medical dialogue audio to obtain the second intermediate large model; and finally, based on the evaluation feedback data for the prediction result of the second intermediate large model, the second intermediate large model is continuously trained using the specific dialogue audio for the improvement direction to obtain a multi-head attention large model. Through multi-stage training and evaluation, a speech large model that performs excellently in the medical dialogue scene is obtained, which provides more accurate and empathetic answers for patients and provides a better speech interaction experience for patients.

[0116] In some embodiments, the dialogue audio dataset in the medical field is constructed by: obtaining real medical voice dialogue audio, real medical text dialogue, and open source text dialogue in the medical field; taking the real medical voice dialogue audio as the main dataset; taking the deduced medical dialogue audio and the open source medical dialogue audio as the supplementary dataset; wherein the dialogue audio dataset comprises the main dataset and the supplementary dataset; the deduced medical dialogue audio is obtained by scene deduction on the real medical text dialogue, and the open source medical dialogue audio is obtained by voice synthesis on the open source text dialogue.

[0117] The real medical voice dialogue audio is derived from real medical scenarios, captures natural voice characteristics, reflects real communication methods, and the dialogue occurs in actual medical environments, providing vivid backgrounds and real dialogue content, which helps to improve the understanding ability of the medical question and answer model in real scenarios. As the main dataset in the dialogue audio dataset, the real medical voice dialogue audio provides the most real and valuable learning materials for medical question and answer model training.

[0118] The deduced medical dialogue audio provides diverse medical scenarios by simulating or creating dialogue scenes, enhances the breadth of dialogue content, and ensures high audio quality during production, so that the model can receive clear and easy-to-understand human voices when learning, which helps to improve the human voice recognition accuracy of the medical question and answer model.

[0119] The open source medical dialogue audio contains a large amount of ophthalmic professional knowledge, which can be used as a supplement to ophthalmic medical knowledge background, enhance the professional knowledge reserve of the model, and help to improve the problem of high error rate in recognizing professional medical terminology due to insufficient professional knowledge in large models in related technologies.

[0120] Exemplarily, the construction of the dialogue audio dataset in the medical field includes:

[0121] 1) Real medical voice dialogue audio: obtain more than 100,000 minutes of real clinical dialogue recordings, and preferentially select dialogue audio with clear medical problem description, direct emotional expression, professional content, and humanistic care.

[0122] 2) Real medical text dialogue: obtain more than 260,000 real ophthalmic clinical text dialogue records, and preferentially select dialogue scenes with the most representative and universal, such as initial consultation, vision test result discussion, disease diagnosis communication, treatment plan discussion, preoperative preparation and postoperative care, follow-up and review, etc. Real people perform professional dialogue deduction according to these scenes to generate high-quality common scene dialogue audio data.

[0123] 3) Open-source text dialogue: Obtain more than 5,000,000 large-scale language model research field open-source medical related dialogue data, and screen out the text dialogue content closely related to ophthalmic medicine from it, and then convert it into audio form through advanced text-to-speech technology, and try to preserve the natural speech characteristics, including intonation, emotion and speech speed changes, during conversion.

[0124] It should be noted that the data of the dialogue audio dataset needs to be cleaned, desensitized and labeled in the data preparation stage. Data cleaning mainly includes noise removal and deletion of irrelevant statements (such as greetings or colloquial expressions) and inappropriate content for training (such as impolite language). This process can be completed by manual or rule setting for small models, and finally confirmed by artificial comprehensive review. Data desensitization mainly removes personal privacy information to protect the identity and sensitive data of users. Data labeling mainly adds emotional color labels and context relationship labels to dialogue audio data. Emotional color labels reflect the emotional state of the speaker, such as calm, angry and happy, while context relationship labels are information related to the dialogue scene, such as asking about the disease and seeking advice. Through accurate data labeling, it can help the medical question and answer model better understand and analyze the emotional changes and dialogue intentions of patients.

[0125] In the above embodiment, by using real medical voice dialogue audio as the main dataset, using real human voice and real scene can significantly improve the understanding ability of the large model to natural communication. At the same time, the supplement of the medical dialogue audio and the open-source medical dialogue audio enriches the dialogue scene and professional knowledge, and enhances the adaptability of the model to diversified communication and the understanding ability of professional terms. Integrating data from three different sources helps to cover a wide range of medical dialogue scenarios and improve the adaptability and generalization ability of the medical question and answer model.

[0126] Please refer to Figure 9 In the embodiment of the present application, a medical dialogue question and answer device 700 is also provided, which comprises a voice signal acquisition module 710, a discrete coding module 720, a vectorization module 730 and an answer generation module 740.

[0127] The voice signal acquisition module 710 is used to acquire a medical question voice signal and input the medical question voice signal to a medical question and answer model; wherein the medical question and answer model comprises an audio coding model, a coding vectorization model and a multi-head attention large model connected in sequence.

[0128] The discrete coding module 720 is configured to directly discretely code the medical question voice signal by using an audio coding model to obtain audio coding detail features, and input the audio coding detail features into an encoding vectorization model; wherein the audio coding detail features are used to represent semantic information, emotional color and context relationship in the medical question voice signal at the same time.

[0129] The vectorization module 730 is configured to perform vectorization representation on the audio coding detail features by using the encoding vectorization model to obtain question vector representation data, and input the question vector representation data into a multi-head attention large model; wherein the question vector representation data is used to represent semantic association information of the medical question voice signal.

[0130] The answer generation module 740 is configured to perform answer generation on the question vector representation data by using the multi-head attention large model to output a medical answer voice signal corresponding to the medical question voice signal.

[0131] In some embodiments, the medical dialogue question and answer device 700 further comprises a feature extraction module, a vector data input module and a feature input module:

[0132] The feature extraction module is configured to perform feature extraction on the question vector representation data by using the multi-head attention large model to obtain intermediate vector representation data.

[0133] The vector data input module is configured to input the intermediate vector representation data into a vector conversion model to obtain intermediate coding detail features.

[0134] The feature input module is configured to input the intermediate coding detail features into an audio decoding model for decoding to obtain the medical answer voice signal.

[0135] In some embodiments, the model training module in the medical dialogue question and answer device 700 further comprises an audio data set construction module and a synchronous training module:

[0136] The audio data set construction module is configured to construct a dialogue audio data set in the medical field.

[0137] The synchronous training module is configured to perform synchronous training on the initial coding model and the initial decoding model by using the dialogue audio data set to obtain the audio coding model and the audio decoding model.

[0138] In some embodiments, the model training module in the medical dialogue question and answer device 700 further comprises a dialogue discrete coding module and a synchronous training module:

[0139] The dialogue discrete coding module is configured to discretely code the dialogue audio data set by using the audio coding model to obtain a feature coding data set corresponding to the dialogue audio data set.

[0140] The synchronous training module is configured to perform synchronous training on the initial encoding vectorization model and the initial vector conversion model by using the feature coding dataset, to obtain the encoding vectorization model and the vector conversion model.

[0141] In some embodiments, the training module of the model in the medical dialogue question and answer device 700 further comprises a dataset vectorization module and a large model training module:

[0142] The dataset vectorization module is configured to perform vectorization representation on the feature coding dataset by using the encoding vectorization model, to obtain an audio vector dataset corresponding to the feature coding dataset.

[0143] The large model training module is configured to train the initial large model by using the audio vector dataset, to obtain the multi-head attention large model.

[0144] In some embodiments, the training module of the model in the medical dialogue question and answer device 700 further comprises a large model training module, a large model fine-tuning module, a feedback data acquisition module, and a supplementary training module:

[0145] The large model training module is configured to train the initial large model by using the audio vector dataset, to obtain a first intermediate large model.

[0146] The large model fine-tuning module is configured to acquire a specified scene medical dialogue audio, fine-tune the model parameters of the intermediate large model by using a specified scene vector dataset corresponding to the specified scene medical dialogue audio, to obtain a second intermediate large model.

[0147] The feedback data acquisition module is configured to acquire evaluation feedback data for the prediction result of the second intermediate large model; wherein the evaluation feedback data is used to represent an improvement direction that does not meet the preset prediction requirement in the prediction result of the second intermediate large model.

[0148] The supplementary training module is configured to continue training the second intermediate large model by using a specific dialogue audio for the improvement direction, to obtain the multi-head attention large model.

[0149] In some embodiments, the training module of the model in the medical dialogue question and answer device 700 further comprises a dialogue audio dataset construction module:

[0150] The dialogue audio dataset construction module is configured to acquire real medical voice dialogue audio, real medical text dialogue, and open source text dialogue in the medical field; the real medical voice dialogue audio is used as a main dataset; the deductive medical dialogue audio and the open source medical dialogue audio are used as supplementary datasets; wherein the dialogue audio dataset comprises the main dataset and the supplementary dataset; the deductive medical dialogue audio is obtained by deductively performing scene deduction on the real medical text dialogue; and the open source medical dialogue audio is obtained by performing voice synthesis on the open source text dialogue.

[0151] Further function description of each module and unit is the same as the corresponding embodiment described above, and will not be repeated here.

[0152] The medical conversation question and answer device in the embodiment is in the form of a functional unit. The unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory executing one or more software or fixed programs, and / or other devices that can provide the above functions.

[0153] Please refer to Figure 10 , Figure 10 is a structural schematic diagram of a computer device provided by the embodiment of the present application, as Figure 10 shown, the computer device includes one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Various components are communicatively connected to each other by different buses, and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display GUI graphics information on an external input / output device, such as a display device coupled to the interface. In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple storage devices, if necessary. Similarly, multiple computer devices can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 10 In the embodiment, the processor 10 is taken as an example.

[0154] The processor 10 can be a central processor, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a general array logic, or any combination thereof.

[0155] The memory 20 stores instructions executable by the at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0156] The memory 20 can include a program storage area and a data storage area, where the program storage area can store an operating system, application programs required for at least one function, and the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, which can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0157] The memory 20 can include a volatile memory, such as a random access memory, and can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid state disk, and can also include a combination of the above-mentioned types of memories.

[0158] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 can be connected by a bus or other means, Figure 10 The connection by the bus is taken as an example.

[0159] The input device 30 can receive inputted digital or character information, and generate key signal inputs related to the user settings and function controls of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), a tactile feedback device (e.g., a vibration motor), etc. The display device includes, but is not limited to, a liquid crystal display, a light-emitting diode, a display, and a plasma display. In some alternative embodiments, the display device can be a touch screen.

[0160] The embodiments of the present application further provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or be implemented as computer code originally stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0161] The embodiments of the present application provide a computer program product, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of any of the embodiments of the present application.

[0162] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

[0163] The system, device, module or unit illustrated in the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0164] For the convenience of description, the above device is described as various units divided by functions. Of course, the functions of each unit can be implemented in the same or more software and / or hardware in the implementation of the present application.

[0165] Those skilled in the art will appreciate that embodiments of the application can be readily used as a method, a system or a computer program product. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0166] The application is described in relation to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It is understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0167] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0169] It should also be noted that the terms "comprises" and / or "comprising", or "includes" and / or "including" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof. In the context of the specification, the term "and / or" means "and" or "or", and the term "or" means "and" or "or". In the context of the specification, the term "exemplary" means "example" or "an example of".

[0170] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0171] The above only describes the embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

[0172] Although the embodiments of the present application are described in conjunction with the drawings, those skilled in the art can make various modifications and changes without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A medical dialogue question answering method, characterized by, The method comprises: acquiring a medical question voice signal and inputting the medical question voice signal into a medical question and answer model; wherein the medical question and answer model comprises an audio coding model, an encoding vectorization model and a multi-head attention large model connected in sequence; directly discretely coding the medical question voice signal through the audio coding model to obtain audio coding detail features, and inputting the audio coding detail features into the encoding vectorization model; wherein the audio coding detail features are used to simultaneously represent semantic information, emotional color and context relationship in the medical question voice signal; vectorizing the audio coding detail features through the encoding vectorization model to obtain question vector representation data, and inputting the question vector representation data into the multi-head attention large model; wherein the question vector representation data is used to represent semantic association information of the medical question voice signal; the multi-head attention large model is connected in sequence with a vector conversion model and an audio decoding model; the multi-head attention large model is used to extract features from the question vector representation data to obtain intermediate vector representation data; inputting the intermediate vector representation data into the vector conversion model to obtain intermediate coding detail features; inputting the intermediate coding detail features into the audio decoding model for decoding to obtain a medical answer voice signal.

2. The method of claim 1, wherein, The audio coding model and the audio decoding model are obtained in the following manner: constructing a dialogue audio data set in the medical field; synchronously training an initial coding model and an initial decoding model using the dialogue audio data set to obtain the audio coding model and the audio decoding model.

3. The method of claim 2, wherein, The encoding vectorization model and the vector conversion model are obtained in the following manner: discretely coding the dialogue audio data set using the audio coding model to obtain a feature coding data set corresponding to the dialogue audio data set; synchronously training an initial encoding vectorization model and an initial vector conversion model using the feature coding data set to obtain the encoding vectorization model and the vector conversion model.

4. The method of claim 3, wherein, The multi-head attention large model is obtained in the following manner: vectorizing the feature coding data set using the encoding vectorization model to obtain an audio vector data set corresponding to the feature coding data set; training an initial large model using the audio vector data set to obtain the multi-head attention large model.

5. The method of claim 4, wherein, The training of the initial large model using the audio vector data set to obtain the multi-head attention large model comprises: training the initial large model using the audio vector data set to obtain a first intermediate large model; acquiring a specified scene medical dialogue audio, fine-tuning model parameters of the intermediate large model using a specified scene vector data set corresponding to the specified scene medical dialogue audio to obtain a second intermediate large model; acquiring evaluation feedback data for a prediction result of the second intermediate large model; wherein the evaluation feedback data is used to represent an improvement direction of the prediction result of the second intermediate large model that does not meet a preset prediction requirement; The second intermediate large model is continuously trained using specific dialogue audio for the direction to be improved, and the multi-head attention large model is obtained.

6. The method of any one of claims 2 to 4, wherein the dialogue audio dataset of the medical field is constructed by: obtaining real medical voice dialogue audio, real medical text dialogue, and open source text dialogue of the medical field; taking the real medical voice dialogue audio as a main dataset; Deductive medical dialog audio and open source medical dialog audio are used as supplemental data sets; wherein, the dialogue audio dataset comprises the main dataset and the supplementary dataset; the medical dialogue audio is obtained by scene deduction on the real medical text dialogue, and the open source medical dialogue audio is obtained by voice synthesis on the open source text dialogue.

7. A device for implementing the medical dialogue question answering method according to any one of claims 1-6, characterized in that, The device comprises: a voice signal acquisition module configured to acquire a medical question voice signal and input the medical question voice signal into a medical question and answer model; wherein the medical question and answer model comprises an audio coding model, a coding vectorization model, and a multi-head attention large model connected in sequence; a discrete coding module configured to directly perform discrete coding on the medical question voice signal through the audio coding model to obtain audio coding detail features, and input the audio coding detail features into the coding vectorization model; wherein the audio coding detail features are used to represent semantic information, emotional color, and context relationship in the medical question voice signal simultaneously; a vectorization module configured to perform vectorization representation on the audio coding detail features through the coding vectorization model to obtain question vector representation data, and input the question vector representation data into the multi-head attention large model; wherein the question vector representation data is used to represent semantic association information of the medical question voice signal; an answer generation module configured to perform answer generation on the question vector representation data through the multi-head attention large model to output a medical answer voice signal corresponding to the medical question voice signal.

8. A computer device, comprising: comprise: a memory and a processor, which are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing a computer to execute the method of any one of claims 1 to 6.

Images