Intelligent reply method and device based on medical treatment, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an intelligent reply method and device based on medical treatment, electronic equipment and a storage medium, and belongs to the technical field of medical science and technology and artificial intelligence. The method comprises the following steps: constructing target medical problem indication information for problem voiceprint features, problem text features and target medical knowledge features acquired from target voice data containing medical data; and indicating the medical health model to reply according to the target medical question indication information. According to the embodiment of the invention, the target medical question indication information is constructed according to the question voiceprint feature, the question text feature and the target medical knowledge feature, different individual features and corresponding specific symptoms of the consulting user can be considered, comprehensive target medical question indication information is constructed, the accuracy of reply is ensured, and the user experience is improved. And the medical health model is indicated to answer according to the target medical question indication information, so that the accuracy of intelligent answering is improved.

Description

Technical Field

[0001] The present application relates to the fields of medical technology and artificial intelligence technology, and in particular to a medical-based intelligent response method and device, electronic device and storage medium. Background Art

[0002] At present, natural language processing technology is usually used to convert user questions into structured queries, and relevant information of the questions is retrieved from the database for answering. For example, in the medical technology scenario, the user asks "I often feel tired and have mild chest pain recently. What might this be?" The symptoms in the user's question are analyzed through natural language processing technology, and relevant medical information of "fatigue" and "chest pain" is retrieved from the search database, and the answer content "This may be caused by anemia, heart disease or lack of sleep" is obtained. However, this method often relies on a general database to answer user questions, and cannot give targeted responses based on the specific individual differences of users, resulting in a low accuracy rate of intelligent responses. Therefore, how to improve the accuracy of intelligent responses has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a medical-based intelligent response method and device, electronic device and storage medium, aiming to improve the accuracy of user risk diagnosis.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a medical-based intelligent reply method, the method comprising:

[0005] Acquire target speech data containing medical questions;

[0006] Performing voiceprint recognition on the target voice data to obtain problematic voiceprint features;

[0007] Performing speech recognition on the target speech data to obtain target medical question text data, and performing word embedding processing on the target medical question text data to obtain question text features;

[0008] Matching the question text features with a pre-built medical health vector knowledge base to obtain target medical knowledge features;

[0009] Constructing target medical problem indication information based on the problem voiceprint feature, the problem text feature and the target medical knowledge feature;

[0010] The pre-trained medical health model is instructed to generate a response according to the target medical question indication information to obtain target response data that matches the target voice data.

[0011] In some embodiments, constructing target medical problem indication information based on the problem voiceprint feature, the problem text feature, and the target medical knowledge feature includes:

[0012] The question voiceprint feature and the question text feature are combined to obtain the question voice and text joint feature;

[0013] The problem voiceprint feature is integrated with the target medical knowledge feature to obtain the problem voiceprint knowledge joint feature;

[0014] The problem voice-text joint feature and the problem voiceprint knowledge joint feature are input into a preset indication template to construct indication information, so as to obtain the target medical problem indication information.

[0015] In some embodiments, the step of fusing the question voiceprint feature and the question text feature to obtain the question voice and text joint feature includes:

[0016] Based on a preset acoustic unit, the question voiceprint feature and the question text feature are aligned to obtain an aligned question text feature and an aligned question voiceprint feature;

[0017] The alignment question text feature and the alignment question voiceprint feature are fused using a preset shared encoder to obtain the question voice-text joint feature.

[0018] In some embodiments, matching the question text features with a pre-built medical health vector knowledge base to obtain target medical knowledge features includes:

[0019] The medical health vector knowledge base is subjected to low-dimensional projection through a preset low-dimensional projection matrix to obtain a low-dimensional vector knowledge base; the low-dimensional vector knowledge base is stored in a preset candidate knowledge hash bucket;

[0020] The question text features are low-dimensionally projected by the low-dimensional projection matrix to obtain a low-dimensional question text vector; the low-dimensional question text vector is stored in a preset text hash bucket;

[0021] Performing target adjacent distance screening on the candidate knowledge hash buckets according to the text hash bucket to obtain a target knowledge hash bucket;

[0022] Acquire candidate medical and health knowledge features from the target knowledge hash bucket, and calculate similarity between the candidate medical and health knowledge features and the question text features to obtain a target similarity;

[0023] The candidate medical and health knowledge features are screened according to the target similarity to obtain the target medical knowledge features.

[0024] In some embodiments, instructing the pre-trained medical health model to generate a response according to the target medical question indication information to obtain target response data that matches the target voice data includes:

[0025] Using the medical health model to perform intent recognition on the target medical problem indication information to obtain the target indication information intent;

[0026] Performing an indication search on the target medical problem indication information according to the target indication information intention to obtain indication search information;

[0027] Logical reasoning and association are performed on the indicated retrieval information to obtain target response data that matches the target voice data.

[0028] In some embodiments, performing voiceprint recognition on the target voice data to obtain the problematic voiceprint features includes:

[0029] Performing frame processing on the target voice data to obtain a target voice frame signal;

[0030] Perform voiceprint recognition on the target speech frame signal to obtain the problem voiceprint feature.

[0031] In some embodiments, before matching the question text features with a pre-built medical health vector knowledge base to obtain target medical knowledge features, the method further includes:

[0032] Acquire medical health data, and perform semantic segmentation on the medical health data to obtain segmented medical health data;

[0033] Performing word embedding processing on the segmented medical and health data to obtain segmented medical and health vectors;

[0034] Constructing a medical health vector index for the segmented medical health vector; constructing a question text vector index for the question text feature;

[0035] The segmented medical health vector and the question text features are vector-stored according to the medical health vector index and the question text vector index to obtain the medical health vector knowledge base.

[0036] To achieve the above-mentioned purpose, a second aspect of the embodiment of the present application proposes a medical-based intelligent answering device, the device comprising:

[0037] A question voice acquisition module, used to acquire target voice data containing medical questions;

[0038] A voiceprint recognition module, used to perform voiceprint recognition on the target voice data to obtain problematic voiceprint features;

[0039] A speech recognition and word embedding module, used to perform speech recognition on the target speech data to obtain target medical question text data, and perform word embedding processing on the target medical question text data to obtain question text features;

[0040] A knowledge matching module, used to match the question text features with a pre-built medical health vector knowledge base to obtain target medical knowledge features;

[0041] An indication information construction module, used for constructing target medical problem indication information based on the problem voiceprint feature, the problem text feature and the target medical knowledge feature;

[0042] The reply module is used to generate a reply according to the target medical question indication information instructing the pre-trained medical health model to obtain target reply data that matches the target voice data.

[0043] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0044] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0045] The medical-based intelligent reply method and device, electronic device and storage medium proposed in the present application obtain target voice data containing medical questions; perform voiceprint recognition on the target voice data to obtain question voiceprint features; perform voice recognition on the target voice data to obtain target medical question text data, and perform word embedding processing on the target medical question text data to obtain question text features; match the question text features with a pre-constructed medical health vector knowledge base to obtain target medical knowledge features; construct target medical question indication information based on the question voiceprint features, question text features and target medical knowledge features; instruct a pre-trained medical health model to generate a reply based on the target medical question indication information to obtain target reply data that matches the target voice data. This application first matches the question text features with a pre-constructed medical health vector knowledge base, which can match the user's questions with the medical symptoms in the knowledge base, ensure that the subsequently generated answers are closely related to the user's question information, and facilitate improving the accuracy of subsequent intelligent replies; secondly, the target medical question indication information is constructed based on the question voiceprint features, question text features and target medical knowledge features, which can take into account the different individual characteristics and corresponding specific symptoms of the consulting users, and construct comprehensive target medical question indication information, which helps to ensure the accuracy of the replies; finally, the pre-trained medical health model is instructed to generate replies based on the target medical question indication information, which helps to improve the accuracy of the intelligent replies. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of a medical-based intelligent reply method provided in an embodiment of the present application;

[0047] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0048] Figure 3 is another flow chart of the medical-based intelligent reply method provided in an embodiment of the present application;

[0049] Figure 4 yes Figure 1 Flow chart of step S104 in FIG.

[0050] Figure 5 yes Figure 1 Flow chart of step S105 in FIG.

[0051] Figure 6 yes Figure 5 Flow chart of step S501 in FIG.

[0052] Figure 7 yes Figure 1 Flow chart of step S106 in FIG.

[0053] Figure 8is a schematic diagram of the structure of a medical-based intelligent answering device provided in an embodiment of the present application;

[0054] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] First, some nouns involved in this application are analyzed:

[0059] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0060] Based on this, the embodiments of the present application provide a medical-based intelligent response method and device, an electronic device and a storage medium, aiming to improve the accuracy of user risk diagnosis.

[0061] The medical-based intelligent reply method and device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the medical-based intelligent reply method in the embodiments of the present application is described.

[0062] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0063] The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, processing / interaction systems, mechatronics, etc. Artificial intelligence software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0064] The medical-based intelligent reply method provided in the embodiment of the present application relates to the fields of medical technology and artificial intelligence technology. The medical-based intelligent reply method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements a medical-based intelligent reply method, etc., but is not limited to the above forms.

[0065] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0066] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user image data, user video data, and business text data, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or jump to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0067] Figure 1 is an optional flow chart of the medical-based intelligent reply method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0068] Step S101, obtaining target voice data containing medical questions.

[0069] Step S102: Perform voiceprint recognition on the target voice data to obtain problematic voiceprint features.

[0070] Step S103, performing speech recognition on the target speech data to obtain target medical question text data, and performing word embedding processing on the target medical question text data to obtain question text features.

[0071] Step S104, matching the question text features with the pre-built medical health vector knowledge base to obtain target medical knowledge features.

[0072] Step S105, constructing target medical problem indication information based on the question voiceprint features, question text features and target medical knowledge features.

[0073] Step S106, instructing the pre-trained medical health model to generate a response according to the target medical question indication information, and obtaining target response data that matches the target voice data.

[0074] Steps S101 to S106 illustrated in the embodiment of the present application are as follows: acquiring target speech data containing medical questions; performing voiceprint recognition on the target speech data to obtain question voiceprint features; performing speech recognition on the target speech data to obtain target medical question text data, and performing word embedding processing on the target medical question text data to obtain question text features; matching the question text features with a pre-constructed medical and health vector knowledge base to obtain target medical knowledge features; constructing target medical question indication information based on the question voiceprint features, question text features, and target medical knowledge features; instructing a pre-trained medical and health model to generate a reply based on the target medical question indication information to obtain target reply data that matches the target speech data. This application first matches the question text features with a pre-constructed medical health vector knowledge base, which can match the user's questions with the medical symptoms in the knowledge base, ensure that the subsequently generated answers are closely related to the user's question information, and facilitate improving the accuracy of subsequent intelligent replies; secondly, the target medical question indication information is constructed based on the question voiceprint features, question text features and target medical knowledge features, which can take into account the different individual characteristics and corresponding specific symptoms of the consulting users, and construct comprehensive target medical question indication information, which helps to ensure the accuracy of the replies; finally, the pre-trained medical health model is instructed to generate replies based on the target medical question indication information, which helps to improve the accuracy of the intelligent replies.

[0075] In step S101 of some embodiments, specifically, the target voice data includes a medical question, which is raised by a medical consultation user.

[0076] For example, in a medical technology scenario, the target voice data can be voice questions asked by the user, such as "What are the early symptoms of myocarditis?" or "I often feel tired and have mild chest pain recently. What might this be?"

[0077] Specifically, target voice data containing medical questions can be obtained from the enterprise backend database.

[0078] See also Figure 2 In some embodiments, step S102 includes but is not limited to steps S201 to S202:

[0079] Step S201, performing frame processing on the target speech data to obtain a target speech frame signal.

[0080] Step S202: Perform voiceprint recognition on the target speech frame signal to obtain the problem voiceprint feature.

[0081] In step S201 of some embodiments, the target speech frame signal refers to a speech signal obtained by segmenting the target speech data according to a fixed time length.

[0082] Specifically, before the target speech data is framed, a filter is used to remove noise (such as environmental noise, echo, etc.) in the target speech data, and the amplitude of the target speech data is standardized to have a uniform amplitude range.

[0083] Specifically, the target speech data after denoising and standardization processing is segmented according to the audio signal and a certain number of seconds to obtain the target speech frame signal.

[0084] In step S202 of some embodiments, the problematic voiceprint features include but are not limited to the fundamental frequency, timbre, speaking speed, rhythm and formant of the target speech frame signal.

[0085] Specifically, the fundamental frequency is the lowest frequency of periodic vibration in the sound signal, which reflects the pitch of the sound and can be used to preliminarily determine the gender of the sound. The fundamental frequency can be determined by obtaining the autocorrelation function of each frame of the target speech frame signal and finding the frame signal position of the first zero crossing point based on the autocorrelation function, thereby determining the relationship between the frame signal position and the sampling rate.

[0086] For example, the fundamental frequency of a male voice might be between 80-200 Hz, while the fundamental frequency of a female voice might be between 160-500 Hz.

[0087] Specifically, the gender of the voice can be further determined in combination with the timbre. First, a spectrum is obtained by performing a fast Fourier transform on each frame of the speech signal, and the spectrum is filtered through a set of Mel-scale filters to simulate the nonlinear frequency perception of the human ear. The logarithmic energy of the output of each filter is further taken, and finally the logarithmic energy is discrete cosine transformed to obtain Mel-frequency cepstral coefficients. The timbre is represented by the Mel-frequency cepstral coefficients.

[0088] Specifically, the speech rate is usually quantified by the number of syllables or phonemes uttered per second, and the unit may be syllables / second or phoneme / second. The speech rate can be calculated by using speech recognition technology to segment the speech frame signal into independent syllables or phonemes, and counting the total duration of the entire speech frame signal and the number of syllables or phonemes.

[0089] Specifically, rhythm is an indicator that describes the strength and duration of speech, and is usually related to the pause and stress characteristics of speech. Rhythm extraction can divide speech frame signals into smaller units (such as syllables) and identify rhythmic patterns in speech (such as stress position, pause time, etc.) to determine the rhythm.

[0090] Specifically, the formant refers to the peak value in the speech spectrogram, which reflects the vocal tract shape and pronunciation method of the speech. The extraction of the formant can be determined by detecting the peak position in the spectrogram, and since the formant position changes over time, the formant position can also be smoothed to reduce the impact of noise.

[0091] Specifically, the characteristics of the consulting user can be determined through voiceprint features. Since there are significant differences in the fundamental frequency ranges of men and women, the fundamental frequency of men is usually lower, while the fundamental frequency of women is higher. The gender of the consulting user can be preliminarily identified by analyzing the fundamental frequency, and the gender identification can be assisted by combining the timbre characteristics of the consulting user to determine the gender of the consulting user. Furthermore, with age, the structure of a person's vocal cords and pronunciation methods will change, resulting in changes in fundamental frequency and timbre characteristics. By identifying changes in fundamental frequency and timbre characteristics, the age stage of the consulting user can also be predicted. Since people of different ages also have differences in speech speed and rhythm characteristics, speech speed and rhythm characteristics can also be used to assist gender identification, thereby determining the age of the consulting user.

[0092] Specifically, due to differences in pronunciation habits in different regions, timbre and resonance peak features can be used to assist in identifying the geographical location of the consulting user. People in different regions may have unique dialect characteristics, and dialect feature recognition can also be performed on speech frame signals. The geographical location of the consulting user can be ultimately determined by combining dialect features.

[0093] In this embodiment, by performing voiceprint recognition on the target speech frame signal, it is helpful to determine the specific individual differences of different users, and different prompt information can be generated according to the differences of different users, so as to provide personalized and accurate responses to users later.

[0094] In step S103 of some embodiments, specifically, the speech frame signal of the target speech data is converted into a series of digital features through automatic speech recognition technology, and these digital features can reflect the acoustic characteristics of the target speech data (such as pitch, sound intensity, sound length, etc.); finally, the possible pronunciation unit sequence is predicted based on the acoustic characteristics, and the pronunciation unit sequence is combined into meaningful words and sentences in combination with grammatical rules and context information to obtain the target medical problem text data.

[0095] For example, in medical and technological scenarios, the target voice data may be voice questions asked by users, such as "What are the early symptoms of myocarditis?" or "I often feel tired and have mild chest pain recently. What might this be?" These voice questions can be recognized as text data through automatic speech recognition technology.

[0096] Specifically, word embedding is a technology that converts text data into numerical vectors. These vectors can capture the semantic information and contextual relationships of words. By performing word embedding processing on the target medical problem text data, the target medical problem text data can be effectively converted into a vector representation in a high-dimensional space.

[0097] Furthermore, before word embedding, it is usually necessary to preprocess the target medical problem text data (such as word segmentation, stop word removal, stem extraction or word form restoration, etc.), which helps to eliminate noise in the target medical problem text data and improve the quality of word embedding; then, the preprocessed target medical problem text data is subjected to Embedding processing to generate a vector representation for each word in the target medical problem text data. These vector representations are the question text features, and these vector representations not only contain the semantic information of the word itself, but also reflect the semantic similarity between words through the relative position relationship in the vector space.

[0098] For example, in medical and technological scenarios, if a user asks, "I have been having headaches recently, what should I do?" this question is first converted into text by speech recognition technology, and then processed through word embedding. Each word (such as "headache", "recently", "often") will be converted into a corresponding vector representation to determine the characteristics of the question text.

[0099] See also Figure 3 In some embodiments, before step S104, the medical-based intelligent answer method may include but is not limited to steps S301 to S304:

[0100] Step S301, obtaining medical health data, and performing semantic segmentation on the medical health data to obtain segmented medical health data.

[0101] Step S302: perform word embedding processing on the segmented medical and health data to obtain segmented medical and health vectors.

[0102] Step S303, constructing a medical health vector index for the segmented medical health vectors; and constructing a question text vector index for the question text features.

[0103] Step S304, the medical health vector and question text features are segmented and stored in vectors according to the medical health vector index and the question text vector index to obtain a medical health vector knowledge base.

[0104] In step S301 of some embodiments, specifically, in medical and technological scenarios, medical health data includes but is not limited to disease information, drug information, diagnostic guidelines, medical papers, clinical data and other data.

[0105] Specifically, the goal of semantic segmentation is to divide a long text into multiple semantically coherent paragraphs or sentences while ensuring that each part contains a complete information unit.

[0106] Specifically, when segmenting medical and health data, we first need to determine the segmentation strategy (such as sentence-based segmentation, paragraph-based segmentation, or topic-based segmentation, etc.), and then use the Splitter tool to automatically identify the semantic boundaries and semantic structures of medical and health data (such as topic structure, transition relationship structure, etc.), and locate the target semantic boundaries according to the segmentation strategy. By combining the semantic structure, the medical and health data can be automatically segmented.

[0107] For example, if the medical health data is "Heart disease is one of the leading causes of death worldwide. It includes coronary artery disease, myocardial infarction, heart failure, etc. Coronary artery disease is caused by insufficient blood supply to the coronary arteries. Myocardial infarction is usually caused by complete blockage of the coronary arteries." If it is based on sentence segmentation, this piece of medical health data is segmented into four sentences; if it is based on paragraph segmentation, the topic "heart disease" is identified. Since the medical health data is described around a "heart disease" subject, there is no need for paragraph segmentation; if it is based on topic segmentation, the medical health data is divided into three parts by identifying "heart disease", "coronary artery disease" and "myocardial infarction" as different topics.

[0108] In this embodiment, by performing semantic segmentation on medical health data, segmented medical health data is obtained, which can meet the context window size restrictions of subsequent medical health models, and also help ensure that each text fragment contains complete semantic information, so that subsequent models can more accurately understand the meaning of each fragment in the medical health data, thereby generating more accurate response content.

[0109] In step S302 of some embodiments, specifically, segmenting the medical health vector means representing the meaning of segmenting the medical health data through a high-dimensional vector space.

[0110] Specifically, the method of performing word embedding processing on the segmented medical and health data to obtain the segmented medical and health vector is consistent with the method of performing word embedding processing on the target medical question text data to obtain the question text features, which will not be repeated here.

[0111] In step S303 of some embodiments, specifically, the segmented medical health vectors and question text features can be clustered using the DBSCAN clustering algorithm to determine cluster categories, and based on the cluster categories, an index item is constructed for each cluster category. The index item generally includes the center vector of the cluster, a vector list of the cluster, and statistical information of the cluster (such as size, density, etc.).

[0112] Specifically, after constructing the vector index, we can use the vector index to retrieve and analyze data.

[0113] For example, in medical and technological scenarios, when doctors or researchers need to query specific disease information, vector indexes can quickly locate relevant medical records or research literature to provide timely and accurate information support.

[0114] In step S304 of some embodiments, specifically, the medical health vector knowledge base refers to a knowledge base that stores vector data.

[0115] Specifically, the Vectorstore system can be used to store segmented medical and health vectors and question text features in high-dimensional space. These vectors can be floating-point arrays, and the values ​​of the vectors in a certain dimension are represented by elements to obtain a medical and health vector knowledge base. Vectorstore also supports fast retrieval and similarity retrieval of vectors.

[0116] For example, in a medical technology scenario, if medical record retrieval is required, the patient's symptom description is obtained, the symptoms are converted into vectors, and then medical records similar to the symptoms are retrieved in the medical health vector knowledge base, so as to quickly obtain treatment experience and plans for similar cases.

[0117] In this embodiment, by vectorizing and storing the segmented medical health vectors and question text features according to the medical health vector index and the question text vector index, it is possible to convert unstructured data into structured vector form for storage through vectorization, making data that was originally difficult to index become indexable, which helps to optimize storage space and access speed and achieve efficient data retrieval.

[0118] See also Figure 4 In some embodiments, step S104 includes but is not limited to steps S401 to S405:

[0119] Step S401, low-dimensionally projecting the medical health vector knowledge base through a preset low-dimensional projection matrix to obtain a low-dimensional vector knowledge base; the low-dimensional vector knowledge base is stored in a preset candidate knowledge hash bucket.

[0120] Step S402, low-dimensionally projecting the question text features through a low-dimensional projection matrix to obtain a low-dimensional question text vector; the low-dimensional question text vector is stored in a preset text hash bucket.

[0121] Step S403, screening the candidate knowledge hash buckets by target adjacent distance according to the text hash bucket to obtain the target knowledge hash bucket.

[0122] Step S404, obtaining candidate medical and health knowledge features from the target knowledge hash bucket, and calculating similarity between the candidate medical and health knowledge features and the question text features to obtain target similarity.

[0123] Step S405, screening the candidate medical and health knowledge features according to the target similarity to obtain the target medical knowledge features.

[0124] In step S401 of some embodiments, specifically, the high-dimensional vector can be projected into the low-dimensional space through a preset low-dimensional projection matrix; wherein the preset low-dimensional projection matrix can be a matrix composed of random numbers, the size of the matrix is ​​the target low-dimensional value required for the high-dimensional vector, and the low-dimensional vector knowledge base is obtained by calculating the dot product between the medical health vector knowledge base and the low-dimensional projection matrix. Although the low-dimensional vector knowledge base has fewer dimensions than the medical health vector knowledge base, the low-dimensional vector knowledge base still retains similar features to the medical health vector knowledge base.

[0125] Furthermore, the vectors in the low-dimensional vector knowledge base are mapped to different candidate knowledge hash buckets using a hash function. The candidate knowledge hash buckets are used to store low-dimensional vector knowledge bases with similar features. Since the low-dimensional vector knowledge base uses locality-sensitive hashing, vectors with similarity are more likely to be mapped to the same or adjacent hash buckets.

[0126] In this embodiment, the medical health vector knowledge base is low-dimensionally projected through a preset low-dimensional projection matrix to obtain a low-dimensional vector knowledge base, which can realize the projection of high-dimensional data to a lower dimension, reduce computational complexity, improve matching efficiency while retaining the core features of the data.

[0127] In step S402 of some embodiments, specifically, a low-dimensional question text vector is obtained by calculating the dot product between the question text features and the low-dimensional projection matrix. Although the low-dimensional question text vector has fewer dimensions than the low-dimensional question text vector, the low-dimensional question text vector still retains similar features to the question text features.

[0128] Furthermore, the vectors in the low-dimensional question text vector are mapped to different text hash buckets using a hash function. The text hash bucket is used to store low-dimensional question text vectors with similar features, and because the low-dimensional question text vector also uses local sensitive hashing, similar low-dimensional question text vectors are more likely to be mapped to the same or adjacent hash buckets.

[0129] Specifically, based on the characteristics of local sensitive hashing, the text hash bucket and the candidate knowledge hash bucket can also be the same hash bucket.

[0130] In step S403 of some embodiments, specifically, target proximity distance screening can be performed by Retriever. In the field of natural language processing, Retriever generally refers to a query system or query component responsible for retrieving the most relevant information or documents for a given query from a large data set.

[0131] Specifically, Retriever can first determine the text hash bucket where the low-dimensional problem text vector to be found is located, and then determine the target knowledge hash bucket by retrieving the distance between the text hash bucket and the adjacent candidate knowledge hash bucket to find candidate medical and health knowledge features that may be similar to the low-dimensional problem text vector.

[0132] In this embodiment, the candidate knowledge hash buckets are screened based on the target adjacent distance according to the text hash bucket. Compared with a comprehensive search, this reduces the range of data that needs to be considered and improves the screening efficiency. In addition, by storing vector data in hash buckets, the content to be searched can be quickly located based on the vector index, which also improves the retrieval efficiency.

[0133] In step S404 of some embodiments, specifically, for the candidate medical and health knowledge features in the filtered target knowledge hash bucket, the similarity between the candidate medical and health knowledge features and the question text features can be calculated to obtain the candidate similarity between each candidate medical and health knowledge feature and the question text feature, and the largest similarity among the candidate similarities is selected as the target similarity.

[0134] Specifically, the similarity can be determined by calculating the cosine similarity or the Euclidean distance between the features.

[0135] In step S405 of some embodiments, specifically, the candidate medical and health knowledge features in the target candidate hash bucket are screened according to the target similarity to obtain the target medical knowledge features.

[0136] For example, in medical and technological scenarios, relevant medical features of "myocarditis" such as chest pain, palpitations, fatigue, shortness of breath or difficulty breathing can be screened out from candidate medical health knowledge features as target medical knowledge features based on the consulting user's input "What are the early symptoms of myocarditis?"

[0137] See also Figure 5 In some embodiments, step S105 includes but is not limited to steps S501 to S503:

[0138] Step S501, the question voiceprint feature and the question text feature are merged to obtain the question voice and text joint feature.

[0139] Step S502, the problem voiceprint feature is integrated with the target medical knowledge feature to obtain the problem voiceprint knowledge joint feature.

[0140] Step S503, inputting the problem voice-text joint feature and the problem voiceprint knowledge joint feature into a preset indication template to construct indication information, and obtaining the target medical problem indication information.

[0141] See also Figure 6 In some embodiments, step S501 includes but is not limited to steps S601 to S602:

[0142] Step S601, aligning the question voiceprint feature and the question text feature based on a preset acoustic unit to obtain an aligned question text feature and an aligned question voiceprint feature.

[0143] Step S602: using a preset shared encoder, the alignment question text feature and the alignment question voiceprint feature are merged to obtain the question voice and text joint feature.

[0144] In step S601 of some embodiments, the preset acoustic unit may be a hidden layer in a Transformer network structure, and the acoustic unit may be used as an intermediate representation to bridge question voiceprint features and question text features.

[0145] Specifically, the question voiceprint features can be converted into hidden phoneme feature sequences through the acoustic unit, and the words or characters in the question text features can also be mapped to the corresponding hidden phoneme feature space through the acoustic unit, so as to respectively align the question voiceprint features and the question text features in sequence through the hidden phoneme features, thereby mapping the question voiceprint features and the question text features to a unified hidden phoneme feature space to obtain aligned question text features and aligned question voiceprint features.

[0146] In step S602 of some embodiments, the preset shared encoder may also be a Transformer network. By inputting the alignment question text features and the alignment question voiceprint features into the Transformer network, the Transformer network can perform feature splicing on the alignment question text features and the alignment question voiceprint features, and learn the interaction and common features between the two modal features, and finally output the fused question voice-text joint features.

[0147] For example, in the medical and technology scenario, the question asked by the consulting user is "What are the early symptoms of heart disease?" First, the voiceprint features of the consulting user's question are analyzed to determine that the consulting user is a middle-aged woman from the Northeast region. By fusing the user's middle-aged woman, from the Northeast region, and "What are the early symptoms of heart disease?", the joint voiceprint feature of the question is obtained as a middle-aged woman from the Northeast region asking "What are the early symptoms of heart disease?"

[0148] Specifically, since heart disease may have different symptoms and pathogenesis between men and women, the early symptoms of heart disease are usually chest pain in men, while in women they may manifest as fatigue or shortness of breath. Also, since people in different age groups are susceptible to different diseases and have different needs for disease prevention and treatment, it is necessary to distinguish between age groups.

[0149] For example, osteoporosis is more common in older adults, but not usually in teenagers.

[0150] In this embodiment, by using a preset shared encoder to fuse the aligned question text features and the aligned question voiceprint features, it is possible to take into account the age, gender and regional characteristics of different individual consulting users, and align the question text to generate consulting questions containing individual user characteristics, which helps subsequent models to more accurately identify the specific symptoms and treatment plans of the consulting users, thereby improving the accuracy of intelligent replies.

[0151] In step S502 of some embodiments, specifically, the problem voiceprint feature can be converted into a vector of fixed length through a Transformer network, and the vector of the problem voiceprint feature and the vector of the target medical knowledge feature are concatenated, and the interaction and common meaning between the two features are learned, and finally the fused problem voiceprint knowledge joint feature is output.

[0152] For example, in the medical and technological scenario, the question raised by the consulting user is "How to prevent heart disease?" First, the voiceprint features of the consulting user's question are analyzed to determine that the consulting user is a middle-aged woman from the Northeast region. The target medical knowledge features can be the preventive measures for heart disease, such as keeping warm, high-fat and high-salt foods, cardiovascular, blood pressure, and blood lipid characteristics. By fusing the voiceprint features of the question with the target medical knowledge features, the joint features of the voiceprint knowledge of the question are obtained as follows: a question asked by a middle-aged woman from the Northeast region is "What are the early symptoms of heart disease?" It is necessary to keep warm, reduce the intake of high-fat and high-salt foods, and regularly measure cardiovascular, blood pressure, and blood lipids.

[0153] In step S503 of some embodiments, the instruction template is used to integrate information of the problem voice-text joint feature and the problem voiceprint knowledge joint feature.

[0154] For example, in the medical technology scenario, the instruction template can be: [gender], [age], user from [geographic location], asked a question about [question]. Based on the medical health vector knowledge base, the following are some suggestions: [question suggestion].

[0155] Furthermore, by inputting the joint features of the problem voice and text and the joint features of the problem voiceprint knowledge into a preset indication template to construct indication information, the target medical problem indication information can be obtained.

[0156] For example, a middle-aged woman from the northern region asked about how to prevent heart disease. According to the Medical Health Vector Knowledge Base, the following are some suggestions: pay attention to keeping warm, maintain a balanced diet, and reduce the intake of high-fat and high-salt foods. At the same time, have regular health check-ups and pay attention to blood pressure, blood lipids, and blood sugar levels.

[0157] In this embodiment, by using a preset shared encoder to fuse the alignment question text features and the alignment question voiceprint features, it is possible to take into account the different individual characteristics and corresponding specific symptoms of the consulting users, and generate detailed and personalized target medical problem indication information, thereby providing users with targeted health advice and information, helping to improve user satisfaction and the accuracy of intelligent replies, and enhancing the personalization and effectiveness of medical and health services.

[0158] See also Figure 7 In some embodiments, step S106 includes but is not limited to steps S701 to S703:

[0159] Step S701, using a medical health model to identify the intent of the target medical problem indication information to obtain the intent of the target indication information.

[0160] Step S702, performing indication retrieval on the target medical problem indication information according to the target indication information intention to obtain indication retrieval information.

[0161] Step S703, performing logical reasoning and association on the indicated retrieval information to obtain target response data that matches the target voice data.

[0162] In step S701 of some embodiments, specifically, the medical health model may be LLMs (Large Language Models), which is mainly composed of a Transformer network.

[0163] Specifically, Transformer can be used to extract key words of the target medical problem indication information, and the intent of the key words can be analyzed to obtain the target indication information intent.

[0164] For example, in a medical technology scenario, if the instruction information includes "I have been suffering from headaches, dizziness, and shortness of breath recently. What could be the cause?" Transformer can identify "headache", "dizziness", and "shortness of breath" as key symptoms, and perform intent analysis on the word "cause", indicating that the instruction information is intended to find the cause of the disease.

[0165] In this embodiment, by using the medical health model to identify the intent of the target medical problem indication information, it means that LLMs can deeply understand the indication information to identify the core intent of the knowledge information and provide relevant and accurate answers, which helps to improve the accuracy of intelligent replies.

[0166] In step S702 of some embodiments, the Transformer first obtains the indication vector index of the target medical problem indication information, and based on the indication vector index, performs indication retrieval from the medical health vector knowledge base to obtain indication retrieval information.

[0167] For example, in a medical technology scenario, if the target medical problem indication information is a question asked by a middle-aged woman from the northern region, "I have been suffering from frequent headaches, dizziness, and shortness of breath recently. What could be the cause?", then indication retrieval information such as heart disease or myocarditis can be found through the indication vector index.

[0168] In step S703 of some embodiments, specifically, LLMs will also combine Bayes' theorem to calculate the probability of a certain disease under given symptoms: P(disease|symptoms)=P(symptoms|disease)×P(disease) / P(symptoms). Wherein, P(disease|symptoms) is the probability that the patient has a certain disease based on the symptoms, P(symptoms|disease) is the probability of these symptoms if the patient has the disease, P(disease) is the prior probability that the patient has the disease without considering any symptoms, and P(symptoms) is the total probability of these symptoms.

[0169] For example, in the medical technology scenario, in the case of symptoms such as headache, dizziness and shortness of breath, P(symptoms|heart disease) = 0.7 means that if the patient has heart disease, there is a 70% probability of these symptoms, P(heart disease) = 0.05 means that there is a 5% prior probability of having heart disease in the population, P(symptoms) = 0.1 means that 10% of the population has these symptoms, then P(heart disease|symptoms) = 0.35, indicating that in the case of symptoms such as headache, dizziness and shortness of breath, the probability of the patient having heart disease is 35%.

[0170] Furthermore, if the target medical question indicates that a middle-aged woman from the northern region asked the question "I have been suffering from frequent headaches, dizziness, and shortness of breath recently. What could be the cause?", the indicated retrieval information is heart disease, and the target response content generated by LLMs in combination with logical reasoning and the medical health vector knowledge base is: Based on the symptoms you described, including headaches, dizziness, and shortness of breath, although these symptoms may be related to a variety of health conditions, symptom analysis shows that the probability that you have heart disease is about 35%. It should be noted that this probability is not a confirmed diagnosis, but a probability estimate based on the symptoms you provided and known medical data. In addition, regarding the symptoms you described, including headaches, dizziness, and shortness of breath, they may be cardiovascular problems caused by the cold winter. It is recommended that you record your symptoms and make an appointment with a doctor for face-to-face consultations and necessary physical examinations (such as electrocardiograms, blood pressure measurements, and blood tests). If symptoms suddenly worsen or new symptoms appear, such as severe chest pain, fainting, or slurred speech, please seek emergency medical help immediately.

[0171] Specifically, the target reply content can be text data or voice data, which is not limited here.

[0172] In this embodiment, by performing logical reasoning and association on the indicated retrieval information to obtain target response data that matches the target voice data, the model can comprehensively consider the problems described by the user to provide personalized and targeted suggestions. The model will also emphasize the limitations of probabilistic analysis and encourage users to conduct professional medical assessments, further improving the accuracy of intelligent responses.

[0173] The embodiment of the present application obtains target voice data containing medical questions; performs voiceprint recognition on the target voice data to obtain question voiceprint features; performs voice recognition on the target voice data to obtain target medical question text data, and performs word embedding processing on the target medical question text data to obtain question text features; matches the question text features with a pre-constructed medical health vector knowledge base to obtain target medical knowledge features; constructs target medical question indication information based on the question voiceprint features, question text features, and target medical knowledge features; generates a reply using a pre-trained medical health model according to the target medical question indication information to obtain target reply data that matches the target voice data. This application first matches the question text features with a pre-constructed medical health vector knowledge base, which can match the user's questions with the medical symptoms in the knowledge base, ensure that the subsequently generated answers are closely related to the user's question information, and facilitate improving the accuracy of subsequent intelligent replies; secondly, the target medical question indication information is constructed based on the question voiceprint features, question text features and target medical knowledge features, which can take into account the different individual characteristics and corresponding specific symptoms of the consulting users, and construct comprehensive target medical question indication information, which helps to ensure the accuracy of the replies; finally, the pre-trained medical health model is instructed to generate replies based on the target medical question indication information, which helps to improve the accuracy of the intelligent replies.

[0174] See also Figure 8 The embodiment of the present application also provides a medical-based intelligent reply device, which can implement the above-mentioned medical-based intelligent reply method, and the device includes:

[0175] A question voice acquisition module, used to acquire target voice data containing medical questions;

[0176] The voiceprint recognition module is used to perform voiceprint recognition on the target voice data to obtain the problem voiceprint features;

[0177] A speech recognition and word embedding module is used to perform speech recognition on the target speech data to obtain the target medical question text data, and perform word embedding processing on the target medical question text data to obtain question text features;

[0178] The knowledge matching module is used to match the question text features with the pre-built medical and health vector knowledge base to obtain the target medical knowledge features;

[0179] An indication information construction module is used to construct target medical problem indication information based on the problem voiceprint features, the problem text features and the target medical knowledge features;

[0180] The reply module is used to generate a reply based on the target medical question indication information by instructing the pre-trained medical health model to obtain target reply data that matches the target voice data.

[0181] The specific implementation of the medical-based intelligent reply device is basically the same as the specific implementation of the medical-based intelligent reply method mentioned above, and will not be repeated here.

[0182] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned medical-based intelligent reply method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0183] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0184] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0185] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store processing systems and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 902, and the processor 901 calls and executes the medical-based intelligent reply method of the embodiments of this application;

[0186] Input / output interface 903, used to implement information input and output;

[0187] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0188] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0189] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0190] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned medical-based intelligent response method is implemented.

[0191] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0192] The medical-based intelligent reply method, medical-based intelligent reply device, electronic device and storage medium provided in the embodiments of the present application obtain target voice data containing medical questions; perform voiceprint recognition on the target voice data to obtain question voiceprint features; perform voice recognition on the target voice data to obtain target medical question text data, and perform word embedding processing on the target medical question text data to obtain question text features; match the question text features with a pre-constructed medical health vector knowledge base to obtain target medical knowledge features; construct target medical question indication information based on the question voiceprint features, question text features and target medical knowledge features; instruct a pre-trained medical health model to generate a reply according to the target medical question indication information to obtain target reply data that matches the target voice data. This application first matches the question text features with a pre-constructed medical health vector knowledge base, which can match the user's questions with the medical symptoms in the knowledge base, ensure that the subsequently generated answers are closely related to the user's question information, and facilitate improving the accuracy of subsequent intelligent replies; secondly, the target medical question indication information is constructed based on the question voiceprint features, question text features and target medical knowledge features, which can take into account the different individual characteristics and corresponding specific symptoms of the consulting users, and construct comprehensive target medical question indication information, which helps to ensure the accuracy of the replies; finally, the pre-trained medical health model is instructed to generate replies based on the target medical question indication information, which helps to improve the accuracy of the intelligent replies.

[0193] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0194] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0195] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0196] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0197] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0198] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0199] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0200] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0202] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0203] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A medical-based intelligent reply method, characterized in that: The method comprises: Acquire target speech data containing medical questions; Performing voiceprint recognition on the target voice data to obtain problematic voiceprint features; Performing speech recognition on the target speech data to obtain target medical question text data, and performing word embedding processing on the target medical question text data to obtain question text features; Matching the question text features with a pre-built medical health vector knowledge base to obtain target medical knowledge features; Constructing target medical problem indication information based on the problem voiceprint feature, the problem text feature and the target medical knowledge feature; The pre-trained medical health model is instructed to generate a response according to the target medical question indication information to obtain target response data that matches the target voice data.

2. The method according to claim 1, characterized in that The step of constructing target medical problem indication information based on the problem voiceprint feature, the problem text feature and the target medical knowledge feature includes: The question voiceprint feature and the question text feature are combined to obtain the question voice and text joint feature; The problem voiceprint feature is integrated with the target medical knowledge feature to obtain the problem voiceprint knowledge joint feature; The problem voice-text joint feature and the problem voiceprint knowledge joint feature are input into a preset indication template to construct indication information, so as to obtain the target medical problem indication information.

3. The method according to claim 2, characterized in that The step of fusing the question voiceprint feature and the question text feature to obtain the question voice and text joint feature includes: Based on a preset acoustic unit, the question voiceprint feature and the question text feature are aligned to obtain an aligned question text feature and an aligned question voiceprint feature; The alignment question text feature and the alignment question voiceprint feature are fused using a preset shared encoder to obtain the question voice-text joint feature.

4. The method according to claim 1, characterized in that: The problem text features are matched with a pre-built medical health vector knowledge base to obtain target medical knowledge features, including: The medical health vector knowledge base is subjected to low-dimensional projection through a preset low-dimensional projection matrix to obtain a low-dimensional vector knowledge base; the low-dimensional vector knowledge base is stored in a preset candidate knowledge hash bucket; The question text features are low-dimensionally projected by the low-dimensional projection matrix to obtain a low-dimensional question text vector; the low-dimensional question text vector is stored in a preset text hash bucket; Performing target adjacent distance screening on the candidate knowledge hash buckets according to the text hash bucket to obtain a target knowledge hash bucket; Acquire candidate medical and health knowledge features from the target knowledge hash bucket, and calculate similarity between the candidate medical and health knowledge features and the question text features to obtain a target similarity; The candidate medical and health knowledge features are screened according to the target similarity to obtain the target medical knowledge features.

5. The method according to claim 1, characterized in that The step of instructing the pre-trained medical health model to generate a response according to the target medical question indication information to obtain target response data matching the target voice data includes: Using the medical health model to perform intent recognition on the target medical problem indication information to obtain the target indication information intent; Performing an indication search on the target medical problem indication information according to the target indication information intention to obtain indication search information; Logical reasoning and association are performed on the indicated retrieval information to obtain target response data that matches the target voice data.

6. The method according to any one of claims 1 to 5, characterized in that: The performing voiceprint recognition on the target voice data to obtain the problem voiceprint features includes: Performing frame processing on the target voice data to obtain a target voice frame signal; Perform voiceprint recognition on the target speech frame signal to obtain the problem voiceprint feature.

7. The method according to any one of claims 1 to 5, characterized in that: Before matching the question text features with the pre-built medical health vector knowledge base to obtain the target medical knowledge features, the method further includes: Acquire medical health data, and perform semantic segmentation on the medical health data to obtain segmented medical health data; Performing word embedding processing on the segmented medical and health data to obtain segmented medical and health vectors; Constructing a medical health vector index for the segmented medical health vector; constructing a question text vector index for the question text feature; The segmented medical health vector and the question text features are vector-stored according to the medical health vector index and the question text vector index to obtain the medical health vector knowledge base.

8. A medical-based intelligent answering device, characterized in that: The device comprises: A question voice acquisition module, used to acquire target voice data containing medical questions; A voiceprint recognition module, used to perform voiceprint recognition on the target voice data to obtain problematic voiceprint features; A speech recognition and word embedding module, used to perform speech recognition on the target speech data to obtain target medical question text data, and perform word embedding processing on the target medical question text data to obtain question text features; A knowledge matching module, used to match the question text features with a pre-built medical health vector knowledge base to obtain target medical knowledge features; An indication information construction module, used for constructing target medical problem indication information based on the problem voiceprint feature, the problem text feature and the target medical knowledge feature; The reply module is used to generate a reply according to the target medical question indication information instructing the pre-trained medical health model to obtain target reply data that matches the target voice data.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the medical-based intelligent response method described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the medical-based intelligent reply method according to any one of claims 1 to 7 is implemented.

Images