Medical dialogue question and answer method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of medical question answering, and discloses a medical dialogue question answering method and device, computer equipment and a storage medium. Obtaining audio coding detail features including semantic information, emotion colors and contextual relationships; secondly, performing vectorization representation on the audio coding detail features through a coding vectorization model to obtain problem vector representation data; and finally, performing answer generation on the question vector representation data through a multi-head attention big model to obtain a medical answer voice signal corresponding to the medical question voice signal. The multi-head attention big model has the beneficial effects that the voice detail information required for accurately understanding the problem of the patient is input into the multi-head attention big model, so that the obtained answer is not only accurate, but also can emulate and embody the same principle, and fully embody humanistic care for the patient, so that the user can be better assisted and guided.

Description

Technical Field

[0001] The present application relates to the field of medical question-and-answer technology, and in particular to a medical dialogue question-and-answer method, apparatus, computer device, and storage medium. Background Art

[0002] Ophthalmology patients often face challenges with visual impairment and need voice guidance to assist in the eye examination process. This highlights the urgent need for voice services. Voice services can not only convey tone and emotion, but also help to more accurately understand the patient's intentions, thereby significantly improving patient compliance.

[0003] In related technologies, medical intelligent voice service systems usually use a speech-to-text model to convert audio signals into text, and then generate answers through a text input question-and-answer model.

[0004] However, the level of voice detail information processing of medical intelligent voice services in related technologies needs to be improved, so there is an urgent need to propose a new medical dialogue question and answer method. Summary of the invention

[0005] The present application provides a medical dialogue question-and-answer method, apparatus, computer equipment and storage medium, which solves the technical problem in the related art that the level of voice detail information processing of medical intelligent voice services needs to be improved, and achieves the technical effect that the voice service has a more precise understanding of the patient's questions, the answers are more accurate and can simulate empathy.

[0006] In order to achieve the above objectives, the main technical solutions adopted in this application include:

[0007] In a first aspect, an embodiment of the present application provides a medical dialogue question-answering method, the method comprising:

[0008] Acquire a medical question voice signal, and input the medical question voice signal into a medical question-answering model; wherein the medical question-answering model includes an audio coding model, a coding vectorization model, and a multi-head attention large model connected in sequence;

[0009] Directly discretize the medical problem speech signal through the audio coding model to obtain audio coding detail features, and input the audio coding detail features into the coding vectorization model; wherein the audio coding detail features are used to simultaneously characterize the semantic information, emotional color and contextual relationship in the medical problem speech signal;

[0010] The audio coding detail features are vectorized by the coding vectorization model to obtain question vector representation data, and the question vector representation data is input into the multi-head attention large model; wherein the question vector representation data is used to characterize the semantic association information of the medical question speech signal;

[0011] The multi-head attention large model is used to generate an answer to the question vector representation data to output a medical answer voice signal corresponding to the medical question voice signal.

[0012] Optionally, the multi-head attention large model is sequentially connected to a vector conversion model and an audio decoding model; the multi-head attention large model is used to generate an answer to the question vector representation data to output a medical answer voice signal corresponding to the medical question voice signal, including:

[0013] Perform feature extraction on the problem vector representation data through the multi-head attention large model to obtain intermediate vector representation data;

[0014] Inputting the intermediate vector representation data into the vector conversion model to obtain intermediate encoding detail features;

[0015] The intermediate encoded detail features are input into the audio decoding model for decoding to obtain the medical answer voice signal.

[0016] Optionally, the audio encoding model and the audio decoding model are obtained in the following manner:

[0017] Build a conversation audio dataset in the medical field;

[0018] The initial encoding model and the initial decoding model are synchronously trained using the conversation audio data set to obtain the audio encoding model and the audio decoding model.

[0019] Optionally, the encoding vectorization model and the vector conversion model are obtained in the following manner:

[0020] Using the audio coding model to discretely encode the conversation audio dataset to obtain a feature coding dataset corresponding to the conversation audio dataset;

[0021] The feature coding data set is used to synchronously train an initial coding vectorization model and an initial vector conversion model to obtain the coding vectorization model and the vector conversion model.

[0022] Optionally, the multi-head attention large model is obtained by:

[0023] Using the coding vectorization model to vectorize the feature coding data set to obtain an audio vector data set corresponding to the feature coding data set;

[0024] The audio vector data set is used to train the initial large model to obtain the multi-head attention large model.

[0025] Optionally, the using the audio vector data set to train the initial large model to obtain the multi-head attention large model includes:

[0026] Using the audio vector data set to train the initial large model to obtain a first intermediate large model;

[0027] Acquire a medical conversation audio of a specified scene, and fine-tune the model parameters of the intermediate large model using a specified scene vector data set corresponding to the medical conversation audio of the specified scene to obtain a second intermediate large model;

[0028] Obtaining evaluation feedback data for the prediction results of the second intermediate large model; wherein the evaluation feedback data is used to characterize the direction to be improved in the prediction results of the second intermediate large model that do not meet the preset prediction requirements;

[0029] The second intermediate large model is further trained using the specific dialogue audio for the direction to be improved to obtain the multi-head attention large model.

[0030] Optionally, the conversation audio dataset in the medical field is constructed in the following manner:

[0031] Obtain audio of real medical voice conversations, real medical text conversations, and open source text conversations in the medical field;

[0032] Using the real medical voice conversation audio as the main data set;

[0033] The deduced medical conversation audio and the open source medical conversation audio are used as supplementary data sets; wherein, the conversation audio data set includes the main data set and the supplementary data set; the deduced medical conversation audio is obtained by scene deduction of the real medical text conversation, and the open source medical conversation audio is obtained by speech synthesis of the open source text conversation.

[0034] In a second aspect, an embodiment of the present application provides a medical dialogue question-and-answer device, the device comprising:

[0035] A speech signal acquisition module, used to acquire a medical question speech signal, and input the medical question speech signal into a medical question answering model; wherein the medical question answering model includes an audio coding model, a coding vectorization model and a multi-head attention large model connected in sequence;

[0036] A discrete coding module, used for directly performing discrete coding on the medical problem speech signal through the audio coding model to obtain audio coding detail features, and inputting the audio coding detail features into the coding vectorization model; wherein the audio coding detail features are used to simultaneously characterize the semantic information, emotional color and contextual relationship in the medical problem speech signal;

[0037] A vectorization module, used for vectorizing the audio coding detail features through the coding vectorization model to obtain question vector representation data, and inputting the question vector representation data into the multi-head attention large model; wherein the question vector representation data is used to characterize the semantic association information of the medical question speech signal;

[0038] An answer generation module is used to generate an answer to the question vector representation data through the multi-head attention large model to output a medical answer voice signal corresponding to the medical question voice signal.

[0039] In a third aspect, an embodiment of the present application provides a computer device, including:

[0040] A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes any of the above-mentioned methods by executing the computer instructions.

[0041] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute any of the methods described above.

[0042] In this application, the medical question voice signal is first discretely encoded through an audio coding model to obtain audio coding detail features containing semantic information, emotional color and contextual relationship; secondly, the audio coding detail features are vectorized through a coding vectorization model to obtain question vector representation data; finally, the multi-head attention large model is used to generate answers for the question vector representation data to obtain the medical answer voice signal corresponding to the medical question voice signal. By inputting the voice detail information required for accurate understanding of medical problems into the multi-head attention large model, the answer obtained is not only accurate, but also can simulate empathy, fully reflecting the humanistic care for patients, so as to better assist and guide users. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A flowchart of the medical dialogue question-and-answer method provided in the embodiment of this specification;

[0045] Figure 2A schematic diagram of a medical question-answering model provided in an embodiment of this specification;

[0046] Figure 3 A schematic diagram of a medical question-answering model provided in an embodiment of this specification;

[0047] Figure 4 A flowchart of the medical dialogue question-and-answer method provided in the embodiment of this specification;

[0048] Figure 5 A model training flow chart of the medical dialogue question-answering method provided in the embodiments of this specification;

[0049] Figure 6 A model training flow chart of the medical dialogue question-answering method provided in the embodiments of this specification;

[0050] Figure 7 A model training flow chart of the medical dialogue question-answering method provided in the embodiments of this specification;

[0051] Figure 8 A model training flow chart of the medical dialogue question-answering method provided in the embodiments of this specification;

[0052] Fig. 9 A schematic diagram of a medical dialogue question-and-answer device provided in an embodiment of this specification;

[0053] Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0055] Ophthalmological patients, especially the elderly, are in urgent need of voice guidance during eye examinations. However, existing technologies usually use speech-to-text processing to convert the patient's voice into text and then input it into the medical question-and-answer model to generate answers. This processing flow inevitably leads to the loss of detailed information contained in the voice, such as rich semantics and subtle emotions, making it difficult for these systems to perceive the patient's semantic and emotional state as keenly as human doctors, which may lead to wrong answers or lack of necessary empathy, and fail to provide patients with the psychological comfort and support they need. In addition, my country is a complex language environment with multiple languages ​​and dialects. Factors such as language differences, individual language habits and regional accents significantly affect the accuracy of speech-to-text technology. These challenges not only highlight the shortcomings of related technologies, but also show the urgency of developing more accurate speech recognition and understanding technologies in the medical field, so as to provide ophthalmic patients with a more humane and high-quality medical experience.

[0056] Based on this, the present application provides a medical dialogue question-answering method, which first discretely encodes the medical question voice signal through an audio coding model to obtain audio coding detail features containing semantic information, emotional color and contextual relationship; secondly, the audio coding detail features are vectorized through a coding vectorization model to obtain question vector representation data; finally, the multi-head attention large model is used to generate answers for the question vector representation data to obtain the medical answer voice signal corresponding to the medical question voice signal. By inputting the voice detail information required for accurate understanding of medical problems into the multi-head attention large model, the answer obtained is not only accurate, but also can simulate empathy, fully reflecting the humanistic care for patients, so as to better assist and guide users.

[0057] According to an embodiment of the present application, an embodiment of a medical dialogue question and answer method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] See also Figure 1 The medical dialogue question-answering method provided in the embodiment of the present application includes the following steps:

[0059] S110. Acquire a medical question voice signal, and input the medical question voice signal into a medical question-answering model.

[0060] Among them, medical problem voice signals can be various sound information related to health, symptoms or treatment expressed by patients in medical scenarios. These voice signals can include patients describing their condition, asking questions, expressing emotions or feedback on the treatment process, such as "I have dry eyes", "Is glaucoma easy to treat", "I have used this chloramphenicol eye drops for a week, but there is no improvement", etc. Please refer to Figure 2 ,The medical question answering model includes a sequentially connected audio encoding model, an encoding vectorization model and a multi-head attention large model.

[0061] S120. Directly discretize the medical problem speech signal through the audio coding model to obtain audio coding detail features, and input the audio coding detail features into the coding vectorization model.

[0062] The audio coding model can extract features from the medical problem speech signal, thereby realizing discrete coding of the medical problem speech signal and obtaining its audio coding detail features. The audio coding detail features are used to simultaneously characterize the semantic information, emotional color and contextual relationship in the medical problem speech signal.

[0063] S130. Vectorize the audio coding detail features through the coding vectorization model to obtain question vector representation data, and input the question vector representation data into the multi-head attention large model.

[0064] The audio coding detail features include multiple features extracted from the medical problem speech signal. The coding vectorization model converts the audio coding detail features of the medical problem into a vector form, so that each feature becomes a dimension in the vector. This process maps the features of the audio signal to a high-dimensional space, so that complex data relationships can be processed in a multi-dimensional space. The question vector representation data is used to characterize the semantic association information of the medical problem speech signal, including the semantic information, emotional color and contextual relationship of the medical problem.

[0065] S140. Generate an answer to the question vector representation data through a multi-head attention large model to output a medical answer voice signal corresponding to the medical question voice signal.

[0066] Specifically, the multi-head attention model uses multiple attention heads to focus on different parts of the input question vector representation data in parallel, extracting rich contextual information. Furthermore, through the self-attention mechanism, the information extracted by different heads is integrated to better understand the semantic information, emotional color and contextual relationship of the question. Furthermore, based on the results of the above information fusion, the corresponding answer is generated. Because the multi-head attention model has the ability to understand the semantic information, emotional color and contextual relationship contained in the question, the generated answer is not only accurate, but also full of doctor-like empathy, reflecting humanistic care. This feature enables the medical question-answering model to better meet the emotional needs and psychological support of patients when responding to them.

[0067] It is understandable that the answer output by the multi-head attention model is in vector form, and it needs to go through the process of conversion into discrete encoding and decoding to obtain the medical answer voice signal corresponding to the medical question voice signal.

[0068] In the above embodiment, the medical question voice signal is first discretely encoded by the audio coding model to obtain audio coding detail features containing semantic information, emotional color and contextual relationship; secondly, the audio coding detail features are vectorized by the coding vectorization model to obtain the question vector representation data; finally, the multi-head attention large model is used to generate the answer to the question vector representation data to obtain the medical answer voice signal corresponding to the medical question voice signal. By inputting the voice detail information required to accurately understand the patient's problem into the multi-head attention large model, the answer obtained is not only accurate, but also can simulate empathy, fully reflecting the humanistic care for the patient, so as to better assist and guide the user.

[0069] In some embodiments, see Figure 3 , the multi-head attention model is followed by a vector conversion model and an audio decoding model. That is, the medical question-answering model includes an audio encoding model, an encoding vectorization model, a multi-head attention model, a vector conversion model, and an audio decoding model connected in sequence. The multi-head attention model is used to generate answers to the question vector representation data to output the medical answer voice signal corresponding to the medical question voice signal. Please refer to Figure 4 , the steps include:

[0070] S210. Perform feature extraction on the question vector representation data through a large multi-head attention model to obtain intermediate vector representation data.

[0071] S220, input the intermediate vector representation data into the vector conversion model to obtain intermediate encoding detail features.

[0072] S230. Input the intermediate encoded detail features into the audio decoding model for decoding to obtain a medical answer voice signal.

[0073] The function of the vector conversion model is to convert the intermediate vector representation data into discrete form encoding, that is, the intermediate encoding detail features. The function of the audio decoding model is to decode the discrete form of the intermediate encoding detail features to obtain a continuous medical answer voice signal.

[0074] Specifically, after the question vector representation data is input into the multi-head attention large model, the multi-head attention large model extracts features from the question vector representation data, and uses multiple attention heads to focus on the extracted different features in parallel to extract rich contextual information. Furthermore, through the self-attention mechanism, the information extracted by different heads is integrated to better understand the semantic information, emotional color and contextual relationship of the question. Based on the results of the above information fusion, the corresponding intermediate vector representation data is generated. Then, the intermediate vector representation data is converted into intermediate encoding detail features through the vector conversion model. Finally, the intermediate encoding detail features are converted into medical answer voice signals through the audio decoding model.

[0075] In the above embodiment, the problem vector representation data is firstly feature extracted through the multi-head attention large model to obtain the intermediate vector representation data. Secondly, these intermediate vector representation data are input into the vector conversion model to obtain the intermediate coding detail features. Finally, the intermediate coding detail features are decoded by the audio decoding model to generate the voice signal of the medical answer. Such voice output enables ophthalmic patients to easily obtain the answers to the medical questions they need, and these answers are accurate and empathetic, allowing patients to feel the humanistic care like a doctor, thereby improving the medical experience of ophthalmic patients.

[0076] In some embodiments, see Figure 5 , the audio encoding model and audio decoding model are obtained in the following way:

[0077] S310. Build a conversation audio dataset in the medical field.

[0078] S320: synchronously train the initial encoding model and the initial decoding model using the conversation audio data set to obtain an audio encoding model and an audio decoding model.

[0079] Specifically, the conversation audio dataset in the medical field can include recordings of conversations between real ophthalmologists and patients about their condition, medicine or treatment. These recordings have different emotional colors, different scenes and characteristics that reflect the doctor's humanistic care. In addition, the conversation audio dataset can also be audio generated by real medical text conversations performed by real people or audio generated by text-to-speech technology. It should be noted that the audio data generated by these text-to-speech technologies need to have natural speech characteristics, including changes in intonation, emotion and speech speed.

[0080] For example, based on the reconstruction paradigm of audio signals, the initial encoding model and the initial decoding model are trained synchronously using a medical conversation audio dataset. The key steps are as follows:

[0081] 1) Select 80% of the data in the conversation audio dataset as the training set and the remaining 20% ​​of the data as the validation set; select a suitable loss function (such as the mean square error loss function or the cross entropy loss function) to measure the gap between the reconstructed audio signal output by the initial decoding model and the initial audio signal input to the initial encoding model; select a suitable optimization algorithm (such as the Adam optimization algorithm) to update the model parameters to minimize the loss.

[0082] 2) The model is trained by a variety of methods, such as self-supervised learning methods, semi-supervised learning methods, and residual vector quantizer technology. When the initial encoding model and the initial decoding model trained by each method have the expected encoding and decoding capabilities, including the correct encoding and decoding of the semantic information, emotional color and contextual relationship in the training data, the training is completed. By comparing the performance of these methods in audio feature extraction and information retention, an optimal model is selected, mainly evaluated from the aspects of coding efficiency, information retention and computational complexity. High coding efficiency means that the high-quality audio signal can be effectively represented and restored with the least amount of coding, that is, the least amount of audio coding detail features. Information retention can be the degree to which the key information of the original audio signal is retained by the reconstructed audio signal. Computational complexity can be measured by the length of time required for the encoding and decoding process or the required computer resources. The performance of each model in terms of coding efficiency, information retention and computational complexity can be comprehensively considered, and one model can be selected as the optimal model.

[0083] 3) By adjusting the optimizable model parameters, the optimal model is tuned using the validation set data. The main training goal is to achieve the desired information retention, such as 90% frame rate, with the least audio encoding detail features. After the tuning is completed, the audio encoding model and audio decoding model are obtained.

[0084] In the above embodiment, by constructing a conversation audio dataset in the medical field, and using the conversation audio dataset to synchronously train the initial coding model and the initial decoding model, an audio coding model and an audio decoding model are obtained. The audio coding model can provide data support for the training of the initial coding vectorization model and the initial vector conversion model by discretely encoding the medical conversation audio dataset. At the same time, the model can extract detailed information from the medical conversation audio, and lay a good coding foundation for the detailed information of the medical problem voice signal to the multi-head attention large model. The audio decoding model provides a good decoding foundation for delivering accurate and empathetic answers to medical problems to patients in the form of voice.

[0085] In some embodiments, see Figure 6 , the encoding vectorization model and vector conversion model are obtained in the following way:

[0086] S410. Discretely encode the conversation audio data set using an audio coding model to obtain a feature coding data set corresponding to the conversation audio data set.

[0087] S420. Use the feature coding data set to synchronously train the initial coding vectorization model and the initial vector conversion model to obtain a coding vectorization model and a vector conversion model.

[0088] Specifically, the conversation audio dataset is discretely encoded using the trained and tuned audio coding model to obtain a feature coding dataset corresponding to the conversation audio dataset.

[0089] For example, based on the reconstruction paradigm of feature coding data, the initial coding vectorization model and the initial vector conversion model are trained synchronously using the feature coding dataset in the field of ophthalmology. The key steps are as follows:

[0090] 1) Select 80% of the data in the feature encoding dataset as the training set and the remaining 20% ​​of the data as the validation set; select a suitable loss function (such as the mean square error loss function or the cross entropy loss function) to measure the gap between the reconstructed feature encoding data output by the initial vector conversion model and the feature encoding data input by the initial encoding vectorization model; select a suitable optimization algorithm (such as the Adam optimization algorithm) to update the model parameters to minimize the loss.

[0091] 2) The model is trained by various methods, such as self-supervised learning methods, semi-supervised learning methods, factor analysis-based methods, and adversarial generative network technology. When the initial encoding vectorization model and the initial vector conversion model trained by each method have the ability to meet the expected encoding steering and vector transcoding, including the ability to encode the semantic information, emotional color, and contextual relationship in the training data, the training is completed. By comparing the performance of these methods in terms of semantic retention, vector space efficiency, and computing performance, an optimal model is selected. Semantic retention refers to the degree to which features are retained during the encoding steering and vector transcoding process. The spatial efficiency of the vector refers to the effectiveness of the memory resources required to store and represent the vector relative to the dimension and data volume of the vector. A good vector representation method needs to find a suitable balance between semantic retention and vector space efficiency, and take into account computing performance. The performance of each model in terms of semantic retention, vector space efficiency, and computing performance can be comprehensively considered to select one as the optimal model.

[0092] 3) By adjusting the optimizable model parameters, the optimal model is tuned using the validation set data. The main training goal is to achieve the desired semantic retention with the least vectors, such as 90% semantic retention. After the tuning is completed, the encoding vectorization model and the vector conversion model are obtained.

[0093] In the above embodiment, the conversation audio data set is first discretely encoded using the audio coding model to obtain a feature coding data set corresponding to the conversation audio data set; secondly, the initial coding vectorization model and the initial vector conversion model are synchronously trained using the feature coding data set to obtain a coding vectorization model and a vector conversion model. The coding vectorization model can vectorize the feature coding data set of the medical conversation and provide data support for the training of the initial large model. At the same time, the model can convert the detailed information of the medical conversation audio from the code to the vector format, providing a coding vectorization basis for the smooth transmission of the detailed information of the medical problem voice signal to the large model. The vector conversion model provides a good vectorized decoding basis for delivering accurate and empathetic answers to medical problems to patients.

[0094] In some embodiments, see Figure 7 , we get a large multi-head attention model by:

[0095] S510. Use a coding vectorization model to vectorize the feature coding data set to obtain an audio vector data set corresponding to the feature coding data set.

[0096] S520. Use the audio vector data set to train the initial large model to obtain a multi-head attention large model.

[0097] Among them, the initial large model is an untrained large model, which can be trained using the audio vector dataset to learn the semantic information, emotional color and contextual relationship in the audio vector dataset.

[0098] Specifically, the trained and tuned coding vectorization model is used to vectorize the feature coding data set to obtain an audio vector data set corresponding to the feature coding data set.

[0099] Exemplarily, when the initial large model is trained using an audio vector dataset, the input includes auxiliary information corresponding to each piece of data in addition to the audio vector dataset. These auxiliary information exist in the form of labels, including text labels, emotional labels (such as happy, angry, anxious), and contextual relationship labels (such as appointment inquiries, symptom inquiries, and medication inquiries) for each piece of data. Text labels help the initial large model capture the relationship between audio and language; emotional labels help the initial large model identify and understand emotional changes in audio; and contextual relationship labels can help the initial large model better understand the scene of the audio data, thereby capturing deeper semantic relationships. When the initial large model has learned the audio vector dataset, the training is completed, and a multi-head attention large model is obtained.

[0100] Exemplarily, the training of the initial large model requires a balance between the data volume and the number of tokens in the audio vector data set. The training parameters of the large language model can be used as a reference. For example, the medical question and answer knowledge base using the language large model of the same level of knowledge has 2,000 tokens, and 100,000 text data are used for training. Based on this, it is estimated that the number of tokens in the medical question and answer knowledge base using the voice large model is 6,000 to 8,000, and the training requires about 300,000 data. In general, the data volume of the audio vector data set is fixed. If the initial training uses 7,000 tokens, if it is found that the training effect of the initial large model is not good due to too many tokens, the parameters of the audio coding model and the coding vectorization model can be adjusted to increase the aggregation degree of the tokens to reduce the number of output tokens until the initial large model achieves the expected training effect. On the contrary, if it is found that the training effect of the initial large model exceeds expectations, the parameters of the audio coding model and the coding vectorization model can also be adjusted to reduce the aggregation degree of the tokens to increase the number of output tokens until the initial large model achieves the expected training effect.

[0101] In the above embodiment, the feature coding data set is vectorized by the coding vectorization model to obtain the audio vector data set corresponding to the feature coding data set, and then the initial large model is trained by the audio vector data set to obtain the multi-head attention large model. The rich medical conversation details contained in the audio vector data set provide support for subsequent applications. When the input medical question contains these details, the multi-head attention large model can quickly combine them with the existing knowledge base to generate accurate and empathetic answers.

[0102] In some embodiments, see Figure 8 , use the audio vector dataset to train the initial large model to obtain a multi-head attention large model. The steps include:

[0103] S610. Train the initial large model using the audio vector data set to obtain a first intermediate large model.

[0104] The audio vector data set is obtained by vectorizing the feature coding data set using a coding vectorization model.

[0105] Specifically, when the initial large model is trained using the audio vector dataset, the audio vector dataset and the auxiliary information corresponding to each piece of data are input. The auxiliary information exists in the form of labels, including text labels, emotional labels (such as happy, angry, anxious), and contextual relationship labels (such as appointment inquiries, symptom inquiries, and medication inquiries) for each piece of data. Text labels help the initial large model capture the relationship between audio and language; emotional labels help the initial large model identify and understand emotional changes in audio; and contextual relationship labels can help the initial large model better understand the scene and background of the audio data, thereby capturing deeper semantic relationships. When the initial large model has learned the audio vector dataset, the training is completed and the first intermediate large model is obtained.

[0106] S620. Obtain the medical conversation audio of the specified scenario, and use the specified scenario vector data set corresponding to the medical conversation audio of the specified scenario to fine-tune the model parameters of the intermediate large model to obtain the second intermediate large model.

[0107] Among them, the specified scene can be a common scene in ophthalmic medical conversations, such as initial consultation, discussion of vision test results, disease diagnosis communication, treatment plan discussion, preoperative preparation and postoperative care, follow-up and reexamination, etc. The specified scene medical conversation audio can be audio data for supplementary training of certain scenes with poor training effects of the first intermediate large model. The specified scene vector data set can be a data set that can be input into the intermediate large model for training by discretely encoding the specified scene medical conversation audio through an audio coding model and then vectorizing it through a coding vectorization model.

[0108] Specifically, after obtaining the first intermediate large model, it is necessary to manually ask the first intermediate large model common questions in the specified scenario to test the first intermediate large model. After the test is completed, for some scenarios with poor training results found during the test, the specified scenario vector data set is used for supplementary training to fine-tune the model parameters of the intermediate large model and obtain the second intermediate large model. For example, if the first intermediate large model is asked what a certain drug treats, and the first intermediate large model does not answer correctly, the first intermediate large model can be supplemented with training using a vector data set that introduces the drug. After the training is completed, the second intermediate large model is obtained.

[0109] S630: Obtain evaluation feedback data for the prediction results of the second intermediate large model.

[0110] The evaluation feedback data is used to characterize the directions for improvement in the prediction results of the second intermediate large model that do not meet the preset prediction requirements.

[0111] Specifically, the evaluation feedback data for the prediction results of the second intermediate large model can be obtained through the evaluation feedback of human experts. After the training of the second intermediate large model is completed, rigorous comparative experiments can be carried out on the test set to evaluate its performance. A multi-dimensional indicator system can be adopted, including but not limited to: speech recognition accuracy, semantic accuracy of conversation content, empathy level of model performance, naturalness and clarity of generated speech, processing ability of professional medical terms, system response speed and other evaluation indicators. By organizing a professional manual evaluation team, the prediction results of the second intermediate large model are scored, and the scoring results are statistically analyzed to obtain evaluation feedback data for the prediction results of the large model.

[0112] S640. Continue training the second intermediate large model using the specific conversation audio for the direction to be improved to obtain a multi-head attention large model.

[0113] Among them, specific conversation audio can be a data set for supplementary training in the direction to be improved.

[0114] Specifically, the audio coding model can be used to discretely encode the specific conversation audio to obtain the audio coding detail features of the specific conversation audio, and then the audio coding detail features can be vectorized through the coding vectorization model to obtain the vector representation data of the specific conversation audio, and finally the vector representation data and the corresponding auxiliary data can be input into the second intermediate large model for training. After the training is completed, the multi-head attention large model is obtained.

[0115] In the above embodiment, the initial large model is first trained using the audio vector data set to obtain the first intermediate large model; secondly, the medical conversation audio of the specified scene is obtained, and the model parameters of the intermediate large model are fine-tuned using the specified scene vector data set corresponding to the medical conversation audio of the specified scene to obtain the second intermediate large model; finally, based on the evaluation feedback data of the prediction results of the second intermediate large model, the second intermediate large model is further trained using the specific conversation audio for the direction to be improved to obtain the multi-head attention large model. Through multi-stage training and evaluation, a voice large model with excellent performance in medical conversation scenarios is obtained, providing patients with more accurate and empathetic answers, so that patients have a better voice interaction experience.

[0116] In some embodiments, a conversation audio dataset in the medical field is constructed in the following manner: obtaining real medical voice conversation audio, real medical text conversations, and open source text conversations in the medical field; using real medical voice conversation audio as the main dataset; using deduced medical conversation audio and open source medical conversation audio as supplementary datasets; wherein the conversation audio dataset includes the main dataset and the supplementary dataset; the deduced medical conversation audio is obtained by scene deduction of real medical text conversations, and the open source medical conversation audio is obtained by speech synthesis of open source text conversations.

[0117] Among them, the audio of real medical voice conversations originates from real medical scenarios, captures natural voice features, reflects the real way of communication, and the conversations take place in actual medical environments, providing vivid backgrounds and real conversation content, which helps improve the medical question-answering model's ability to understand real-life scenarios. As the main data set in the conversation audio data set, real medical voice conversation audio provides the most authentic and valuable learning materials for the training of medical question-answering models.

[0118] The interpreted medical conversation audio provides diverse medical scenarios and enhances the breadth of conversation content through simulated or created conversation scenarios. The high audio quality is ensured during the production process, so that the model can receive clear and easy-to-understand real voices when learning, which helps to improve the real voice recognition accuracy of the medical question-and-answer model.

[0119] Open source medical conversation audio contains a lot of ophthalmic expertise, which can serve as a supplement to the ophthalmic medical knowledge background, enhance the model's professional knowledge reserves, and help improve the problem of high error rate in identifying professional medical terms due to insufficient professional knowledge in large models in related technologies.

[0120] Exemplarily, the construction of a conversation audio dataset in the medical field includes:

[0121] 1) Real medical voice conversation audio: Obtain more than 100,000 minutes of real clinical conversation recordings, and give priority to conversation audio with clear descriptions of medical problems, direct emotional expression, professional content and humanistic care.

[0122] 2) Real medical text dialogue: We obtained more than 260,000 real ophthalmology clinical text dialogue records, and gave priority to the most representative and common dialogue scenarios, such as initial consultation, discussion of vision test results, disease diagnosis communication, treatment plan discussion, preoperative preparation and postoperative care, follow-up and reexamination, etc. Real people performed professional dialogue interpretation based on these scenarios to generate high-quality audio data of dialogues in common scenarios.

[0123] 3) Open source text dialogue: Obtain more than 5,000,000 open source medical-related dialogue data in the field of large-scale language model research, and screen out text dialogue content closely related to ophthalmic medicine, and then convert it into audio form through advanced text-to-speech technology, while retaining natural speech features as much as possible during the conversion, including changes in intonation, emotion, and speaking speed.

[0124] It should be noted that the data of the conversation audio dataset needs to be cleaned, desensitized and labeled during the data preparation stage. Data cleaning mainly includes de-noising and deleting statements that are not related to training (such as greetings or colloquial expressions) and content that is not suitable for training (such as impolite words). This process can be completed manually or by setting rules for small models, and ultimately needs to be confirmed by manual comprehensive review. Data desensitization mainly removes personal privacy information to protect the user's identity and sensitive data. Data labeling mainly adds emotional color labels and contextual relationship labels to conversation audio data. Emotional color labels reflect the speaker's emotional state, such as calm, angry and happy, while contextual relationship labels are information related to the conversation scene, such as asking about the condition and seeking advice. By accurately labeling data, medical question-answering models can better understand and analyze patients' emotional changes and conversation intentions.

[0125] In the above embodiment, by using real medical voice conversation audio as the main data set, using real voices and real scenes, the large model's ability to understand natural communication can be significantly improved. At the same time, the supplement of deduced medical conversation audio and open source medical conversation audio enriches the conversation scenes and professional knowledge, and enhances the model's adaptability to diversified communication and understanding of professional terms. Integrating data from these three different sources helps cover a wide range of medical conversation scenarios and improves the adaptability and generalization of medical question-and-answer models.

[0126] See also Fig. 9 In an embodiment of the present application, a medical dialogue question-and-answer device 700 is also provided, and the medical dialogue question-and-answer device 700 includes a speech signal acquisition module 710, a discrete encoding module 720, a vectorization module 730 and an answer generation module 740.

[0127] The speech signal acquisition module 710 is used to acquire the medical question speech signal and input the medical question speech signal into the medical question answering model; wherein the medical question answering model includes an audio coding model, a coding vectorization model and a multi-head attention large model connected in sequence;

[0128] The discrete coding module 720 is used to directly perform discrete coding on the medical problem speech signal through the audio coding model to obtain audio coding detail features, and input the audio coding detail features into the coding vectorization model; wherein the audio coding detail features are used to simultaneously characterize the semantic information, emotional color and contextual relationship in the medical problem speech signal;

[0129] The vectorization module 730 is used to vectorize the audio coding detail features through the coding vectorization model to obtain question vector representation data, and input the question vector representation data into the multi-head attention large model; wherein the question vector representation data is used to represent the semantic association information of the medical question speech signal;

[0130] The answer generation module 740 is used to generate answers to the question vector representation data through a multi-head attention model to output a medical answer voice signal corresponding to the medical question voice signal.

[0131] In some embodiments, the medical dialogue question-answering device 700 further includes a feature extraction module, a vector data input module, and a feature input module:

[0132] The feature extraction module is used to extract features from the question vector representation data through a multi-head attention model to obtain intermediate vector representation data;

[0133] A vector data input module is used to input the intermediate vector representation data into the vector conversion model to obtain the intermediate encoding detail features;

[0134] The feature input module is used to input the intermediate encoded detail features into the audio decoding model for decoding to obtain the medical answer voice signal.

[0135] In some embodiments, the training module of the model in the medical dialogue question-answering device 700 further includes an audio data set construction module and a synchronization training module:

[0136] Audio dataset construction module, used to build conversation audio datasets in the medical field;

[0137] The synchronous training module is used to synchronously train the initial encoding model and the initial decoding model using the conversation audio data set to obtain the audio encoding model and the audio decoding model.

[0138] In some embodiments, the training module of the model in the medical dialogue question-answering device 700 further includes a dialogue discrete encoding module and a synchronization training module:

[0139] A dialogue discrete coding module is used to discretely encode the dialogue audio data set using an audio coding model to obtain a feature coding data set corresponding to the dialogue audio data set;

[0140] The synchronous training module is used to synchronously train the initial coding vectorization model and the initial vector conversion model using the feature coding data set to obtain the coding vectorization model and the vector conversion model.

[0141] In some embodiments, the model training module in the medical dialogue question-answering device 700 further includes a data set vectorization module and a large model training module:

[0142] A data set vectorization module is used to vectorize the feature coding data set using the coding vectorization model to obtain an audio vector data set corresponding to the feature coding data set;

[0143] The large model training module is used to train the initial large model using the audio vector dataset to obtain a multi-head attention large model.

[0144] In some embodiments, the model training module in the medical dialogue question-answering device 700 further includes a large model training module, a large model fine-tuning module, a feedback data acquisition module, and a supplementary training module:

[0145] A large model training module, used to train the initial large model using the audio vector data set to obtain a first intermediate large model;

[0146] A large model fine-tuning module is used to obtain the medical conversation audio of the specified scene, and use the specified scene vector data set corresponding to the medical conversation audio of the specified scene to fine-tune the model parameters of the intermediate large model to obtain a second intermediate large model;

[0147] A feedback data acquisition module, used to acquire evaluation feedback data for the prediction results of the second intermediate large model; wherein the evaluation feedback data is used to characterize the direction to be improved in the prediction results of the second intermediate large model that do not meet the preset prediction requirements;

[0148] The supplementary training module is used to continue training the second intermediate large model using specific dialogue audio for the direction to be improved, so as to obtain a multi-head attention large model.

[0149] In some embodiments, the training module of the model in the medical dialogue question-answering device 700 further includes a dialogue audio data set construction module:

[0150] The conversation audio dataset construction module is used to obtain real medical voice conversation audio, real medical text conversation and open source text conversation in the medical field; the real medical voice conversation audio is used as the main dataset; the deduced medical conversation audio and open source medical conversation audio are used as supplementary datasets; among which, the conversation audio dataset includes the main dataset and the supplementary dataset; the deduced medical conversation audio is obtained by scene deduction of real medical text conversation, and the open source medical conversation audio is obtained by speech synthesis of open source text conversation.

[0151] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0152] The medical dialogue question-and-answer device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0153] See also Fig.10 , Fig.10 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application, such as Fig.10 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.10 A processor 10 is taken as an example.

[0154] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0155] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0156] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0157] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0158] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0159] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0160] The embodiment of the present application also provides a computer-readable storage medium. The above method according to the embodiment of the present application can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0161] The embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of any embodiment of the present application.

[0162] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

[0163] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0164] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0170] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0171] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

[0172] Although the embodiments of the present application have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A medical dialogue question-answering method, characterized in that: The method comprises: Acquire a medical question voice signal, and input the medical question voice signal into a medical question-answering model; wherein the medical question-answering model includes an audio coding model, a coding vectorization model, and a multi-head attention large model connected in sequence; Directly discretize the medical problem speech signal through the audio coding model to obtain audio coding detail features, and input the audio coding detail features into the coding vectorization model; wherein the audio coding detail features are used to simultaneously characterize the semantic information, emotional color and contextual relationship in the medical problem speech signal; The audio coding detail features are vectorized by the coding vectorization model to obtain question vector representation data, and the question vector representation data is input into the multi-head attention large model; wherein the question vector representation data is used to characterize the semantic association information of the medical question speech signal; The multi-head attention large model is used to generate an answer to the question vector representation data to output a medical answer voice signal corresponding to the medical question voice signal.

2. The method according to claim 1, characterized in that The multi-head attention large model is sequentially connected to a vector conversion model and an audio decoding model; the multi-head attention large model is used to generate an answer to the question vector representation data to output a medical answer voice signal corresponding to the medical question voice signal, including: Perform feature extraction on the problem vector representation data through the multi-head attention large model to obtain intermediate vector representation data; Inputting the intermediate vector representation data into the vector conversion model to obtain intermediate encoding detail features; The intermediate encoded detail features are input into the audio decoding model for decoding to obtain the medical answer voice signal.

3. The method according to claim 2, characterized in that The audio encoding model and the audio decoding model are obtained by: Build a conversation audio dataset in the medical field; The initial encoding model and the initial decoding model are synchronously trained using the conversation audio data set to obtain the audio encoding model and the audio decoding model.

4. The method according to claim 3, characterized in that The encoding vectorization model and the vector conversion model are obtained by: Using the audio coding model to discretely encode the conversation audio dataset to obtain a feature coding dataset corresponding to the conversation audio dataset; The feature coding data set is used to synchronously train an initial coding vectorization model and an initial vector conversion model to obtain the coding vectorization model and the vector conversion model.

5. The method according to claim 4, characterized in that The multi-head attention model is obtained by the following method: Using the coding vectorization model to vectorize the feature coding data set to obtain an audio vector data set corresponding to the feature coding data set; The audio vector data set is used to train the initial large model to obtain the multi-head attention large model.

6. The method according to claim 5, characterized in that The method of training the initial large model using the audio vector data set to obtain the multi-head attention large model includes: Using the audio vector data set to train the initial large model to obtain a first intermediate large model; Acquire a medical conversation audio of a specified scene, and fine-tune the model parameters of the intermediate large model using a specified scene vector data set corresponding to the medical conversation audio of the specified scene to obtain a second intermediate large model; Obtaining evaluation feedback data for the prediction results of the second intermediate large model; wherein the evaluation feedback data is used to characterize the direction to be improved in the prediction results of the second intermediate large model that do not meet the preset prediction requirements; The second intermediate large model is further trained using the specific dialogue audio for the direction to be improved to obtain the multi-head attention large model.

7. According to the method according to any one of claims 3 to 5, the medical field dialogue audio dataset is constructed by: Obtain audio of real medical voice conversations, real medical text conversations, and open source text conversations in the medical field; Using the real medical voice conversation audio as the main data set; The audio of deduced medical conversations and open-source medical conversation audio are used as supplementary datasets; among them, The conversation audio data set includes the main data set and the supplementary data set; the interpreted medical conversation audio is obtained by performing scene interpretation of the real medical text conversation, and the open source medical conversation audio is obtained by performing speech synthesis of the open source text conversation.

8. A medical dialogue question-and-answer device, characterized in that: The device comprises: A speech signal acquisition module, used to acquire a medical question speech signal, and input the medical question speech signal into a medical question answering model; wherein the medical question answering model includes an audio coding model, a coding vectorization model and a multi-head attention large model connected in sequence; A discrete coding module, used for directly performing discrete coding on the medical problem speech signal through the audio coding model to obtain audio coding detail features, and inputting the audio coding detail features into the coding vectorization model; wherein the audio coding detail features are used to simultaneously characterize the semantic information, emotional color and contextual relationship in the medical problem speech signal; A vectorization module, used for vectorizing the audio coding detail features through the coding vectorization model to obtain question vector representation data, and inputting the question vector representation data into the multi-head attention large model; wherein the question vector representation data is used to characterize the semantic association information of the medical question speech signal; An answer generation module is used to generate an answer to the question vector representation data through the multi-head attention large model to output a medical answer voice signal corresponding to the medical question voice signal.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images