Interrogation abnormity detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an inquiry abnormity detection method and device, electronic equipment and a storage medium, and belongs to the technical field of medical science and technology and artificial intelligence. The method comprises the following steps: performing inquiry on target inquiry text data by utilizing an inquiry anomaly detection model to obtain inquiry reference data; and acquiring doctor inquiry data, and performing inquiry anomaly detection on the doctor inquiry data according to the inquiry reference data to obtain inquiry anomaly data. According to the embodiment of the invention, the target inquiry text data is inquired by using the inquiry anomaly detection model, so that the target inquiry text can be quickly and accurately analyzed and diagnosed, errors caused by human factors are reduced, the objectivity of the whole diagnosis process is ensured, and the inquiry accuracy is improved; and finally, performing inquiry abnormity detection on the doctor inquiry data according to the inquiry reference data, so that the difference between the doctor inquiry data and the doctor inquiry data can be found in time, the doctor can be helped to correct possible misdiagnosis or missed diagnosis in time, and the accuracy of inquiry is further improved.

Description

Technical Field

[0001] The present application relates to the fields of medical technology and artificial intelligence technology, and in particular to a method and device for detecting abnormalities in medical consultations, an electronic device, and a storage medium. Background Art

[0002] At present, the diagnosis of diseases is usually achieved by doctors combining the patient's self-reported symptoms, test results, and auxiliary examinations (such as blood tests, imaging tests, etc.). For example, in the medical technology scenario, for the diagnosis of cardiovascular disease, the doctor will use the patient's age, gender, family medical history, smoking history, blood pressure level, blood lipid status and other multi-dimensional information, combined with electrocardiogram, cardiac color ultrasound and other test results, to determine whether the patient has cardiovascular disease, so as to provide the patient with a corresponding treatment plan. However, this method often relies on the doctor's clinical experience and professional knowledge. Different doctors may have different diagnostic opinions on the symptoms and risks of the same patient due to differences in their clinical experience, knowledge background and personal judgment standards, resulting in a low accuracy rate of the consultation. Therefore, how to improve the accuracy of the consultation has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to provide a method and device for detecting abnormalities in medical consultation, an electronic device and a storage medium, aiming to improve the accuracy of medical consultation.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for detecting abnormalities in a medical consultation, the method comprising:

[0005] Acquire target medical consultation data, wherein the target medical consultation data includes target medical consultation image data and target medical consultation voice data;

[0006] Performing voice recognition on the target medical consultation voice data to obtain voice medical consultation text data;

[0007] Performing character recognition on the target medical consultation image data to obtain image medical consultation text data;

[0008] Using the image consultation text data, the voice consultation text data is subjected to text enhancement processing to obtain target consultation text data;

[0009] Conducting a medical consultation on the target medical consultation text data using a pre-trained medical consultation anomaly detection model to obtain medical consultation reference data;

[0010] The doctor's consultation data is acquired, and consultation anomaly detection is performed on the doctor's consultation data according to the consultation reference data to obtain consultation anomaly data.

[0011] In some embodiments, performing consultation anomaly detection on the doctor's consultation data according to the consultation reference data to obtain consultation anomaly data includes:

[0012] Calculating the similarity between the medical consultation reference data and the doctor's medical consultation data to obtain medical consultation similarity;

[0013] The doctor's consultation data is subjected to consultation anomaly detection according to the consultation similarity to obtain the consultation anomaly data.

[0014] In some embodiments, after performing consultation anomaly detection on the doctor's consultation data according to the consultation similarity and obtaining the consultation anomaly data, the method further includes:

[0015] Performing entity extraction on the abnormal medical consultation data to obtain abnormal medical consultation entities;

[0016] The doctor's consultation entity is updated according to the abnormal consultation entity to obtain an updated doctor's consultation entity;

[0017] The updated doctor consultation entity is consulted to obtain the target doctor consultation data.

[0018] In some embodiments, the step of using a pre-trained medical consultation anomaly detection model to conduct medical consultation on the target medical consultation text data to obtain medical consultation reference data includes:

[0019] Using the pre-trained medical inquiry anomaly detection model to perform feature extraction on the target medical inquiry text data to obtain target medical inquiry text features;

[0020] Performing inquiry entity boundary detection on the target inquiry text features to obtain the inquiry entity boundary;

[0021] According to the inquiry entity boundary, the target inquiry text feature is subjected to inquiry entity category identification to obtain the inquiry entity category;

[0022] The target medical inquiry text feature is queried according to the medical inquiry entity category to obtain the medical inquiry reference data; the medical inquiry reference data is represented by the entity form of the target medical inquiry text feature.

[0023] In some embodiments, the step of performing text enhancement processing on the voice consultation text data using the image consultation text data to obtain target consultation text data includes:

[0024] Obtain the preset medical consultation knowledge graph;

[0025] Performing feature extraction on the voice medical consultation text data to obtain voice medical consultation text features, and performing feature extraction on the image medical consultation text data to obtain image medical consultation text features;

[0026] Mapping the voice medical consultation text features and the image medical consultation text features onto the medical consultation knowledge graph to obtain a target medical consultation knowledge graph;

[0027] The target medical inquiry text data is obtained from the target medical inquiry knowledge graph.

[0028] In some embodiments, before the target medical consultation text data is diagnosing using the pre-trained medical consultation anomaly detection model to obtain medical consultation reference data, the method further includes:

[0029] Using the pre-trained medical consultation anomaly detection model to perform word embedding operation on the target medical consultation text data to obtain a medical consultation text matrix;

[0030] Obtaining an initial weight matrix of the medical inquiry text matrix, and obtaining an initial training parameter matrix of the initial weight matrix;

[0031] Performing symmetric decomposition on the initial training parameter matrix to obtain a decomposed symmetric matrix;

[0032] Score the initial weight matrix according to the decomposed symmetric matrix to obtain a weight importance score;

[0033] The pre-trained medical inquiry anomaly detection model is trained according to the weight importance score to obtain a trained medical inquiry anomaly detection model.

[0034] In some embodiments, the pre-trained medical consultation anomaly detection model is trained according to the weight importance score to obtain a trained medical consultation anomaly detection model, including:

[0035] Performing gradient update on the decomposed symmetric matrix according to a preset loss function and a preset learning rate to obtain an updated decomposed symmetric matrix;

[0036] The pre-trained medical consultation anomaly detection model is updated according to the updated decomposition symmetric matrix and the weight importance score to obtain the trained medical consultation anomaly detection model.

[0037] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a device for detecting abnormality in a medical consultation, the device comprising:

[0038] A medical consultation data acquisition module, used to acquire target medical consultation data, wherein the target medical consultation data includes target medical consultation image data and target medical consultation voice data;

[0039] A speech recognition module, used for performing speech recognition on the target medical consultation speech data to obtain speech medical consultation text data;

[0040] An image recognition module, used for performing character recognition on the target medical consultation image data to obtain image medical consultation text data;

[0041] A text enhancement module, used to perform text enhancement processing on the voice consultation text data using the image consultation text data to obtain target consultation text data;

[0042] A consultation module, used to use a pre-trained consultation anomaly detection model to conduct consultation on the target consultation text data to obtain consultation reference data;

[0043] The consultation anomaly detection module is used to obtain doctor consultation data, and perform consultation anomaly detection on the doctor consultation data according to the consultation reference data to obtain consultation anomaly data.

[0044] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0045] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0046] The present application proposes a method and device for detecting abnormal medical consultation, an electronic device, and a storage medium, which obtain target medical consultation data, including target medical consultation image data and target medical consultation voice data; perform voice recognition on the target medical consultation voice data to obtain voice medical consultation text data; perform character recognition on the target medical consultation image data to obtain image medical consultation text data; perform text enhancement processing on the voice medical consultation text data using the image medical consultation text data to obtain target medical consultation text data; perform medical consultation on the target medical consultation text data using a pre-trained medical consultation abnormality detection model to obtain medical consultation reference data; obtain doctor medical consultation data, and perform medical consultation abnormality detection on the doctor medical consultation data based on the medical consultation reference data to obtain medical consultation abnormality data. This application first uses the image consultation text data to perform text enhancement processing on the voice consultation text data to obtain the target consultation text data, which can effectively integrate the information from different data sources to obtain richer consultation data, so as to improve the accuracy of subsequent consultations; secondly, the target consultation text data is consulted using a pre-trained consultation anomaly detection model, which can quickly and accurately analyze and diagnose the target consultation text, reduce errors caused by human factors, ensure the objectivity of the entire diagnosis process, and improve the accuracy of the consultation; finally, the doctor's consultation data is subjected to consultation anomaly detection based on the consultation reference data, which can promptly discover the difference between the two, help doctors to promptly correct possible misdiagnosis or missed diagnosis, and further improve the accuracy of the consultation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a method for detecting abnormalities in medical consultation provided by an embodiment of the present application;

[0048] Figure 2 yes Figure 1 Flow chart of step S104 in FIG.

[0049] Figure 3 is another flow chart of the method for detecting abnormality in medical consultation provided by an embodiment of the present application;

[0050] Figure 4 yes Figure 3 Flow chart of step S305 in FIG.

[0051] Figure 5 yes Figure 1 Flow chart of step S105 in FIG.

[0052] Figure 6 yes Figure 1 Flow chart of step S106 in FIG.

[0053] Figure 7 is a structural schematic diagram of a device for detecting abnormalities in medical consultation provided in an embodiment of the present application;

[0054] Figure 8It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] First, some nouns involved in this application are analyzed:

[0059] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0060] Based on this, the embodiments of the present application provide a method and device for detecting abnormalities in medical consultation, an electronic device, and a storage medium, aiming to improve the accuracy of medical consultation.

[0061] The medical consultation anomaly detection method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the medical consultation anomaly detection method in the embodiments of the present application is described.

[0062] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0063] The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, processing / interaction systems, mechatronics, etc. Artificial intelligence software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0064] The method for detecting abnormal medical consultation provided in the embodiment of the present application relates to the fields of medical technology and artificial intelligence technology. The method for detecting abnormal medical consultation provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the method for detecting abnormal medical consultation, etc., but is not limited to the above forms.

[0065] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0066] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user image data, user video data, and business text data, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or jump to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0067] Figure 1 is an optional flowchart of the method for detecting abnormality in medical consultation provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0068] Step S101, obtaining target medical consultation data, where the target medical consultation data includes target medical consultation image data and target medical consultation voice data.

[0069] Step S102: performing voice recognition on the target medical consultation voice data to obtain voice medical consultation text data.

[0070] Step S103, performing character recognition on the target medical consultation image data to obtain image medical consultation text data.

[0071] Step S104, performing text enhancement processing on the voice consultation text data using the image consultation text data to obtain target consultation text data.

[0072] Step S105 , using the pre-trained medical consultation anomaly detection model to conduct medical consultation on the target medical consultation text data to obtain medical consultation reference data.

[0073] Step S106, obtaining doctor's consultation data, and performing consultation anomaly detection on the doctor's consultation data according to the consultation reference data to obtain consultation anomaly data.

[0074] Steps S101 to S106 shown in the embodiment of the present application are as follows: acquiring target medical consultation data, the target medical consultation data including target medical consultation image data and target medical consultation voice data; performing voice recognition on the target medical consultation voice data to obtain voice medical consultation text data; performing character recognition on the target medical consultation image data to obtain image medical consultation text data; performing text enhancement processing on the voice medical consultation text data using the image medical consultation text data to obtain target medical consultation text data; performing medical consultation on the target medical consultation text data using a pre-trained medical consultation anomaly detection model to obtain medical consultation reference data; acquiring doctor medical consultation data, and performing medical consultation anomaly detection on the doctor medical consultation data based on the medical consultation reference data to obtain medical consultation anomaly data. This application first uses the image consultation text data to perform text enhancement processing on the voice consultation text data to obtain the target consultation text data, which can effectively integrate the information from different data sources to obtain richer consultation data, so as to improve the accuracy of subsequent consultations; secondly, the target consultation text data is consulted using a pre-trained consultation anomaly detection model, which can quickly and accurately analyze and diagnose the target consultation text, reduce errors caused by human factors, ensure the objectivity of the entire diagnosis process, and improve the accuracy of the consultation; finally, the doctor's consultation data is subjected to consultation anomaly detection based on the consultation reference data, which can promptly discover the difference between the two, help doctors to promptly correct possible misdiagnosis or missed diagnosis, and further improve the accuracy of the consultation.

[0075] In step S101 of some embodiments, specifically, the target medical consultation data includes target medical consultation image data and target medical consultation voice data.

[0076] Specifically, the target medical consultation image data comes from the direct output of medical equipment (such as X-rays, CT scans, MRI images, etc.), photos provided by patients (such as wound photos, skin lesion photos, etc.), or medical records recorded by doctors.

[0077] Specifically, the target consultation voice data can be recorded using a recording device (such as a microphone, a smart phone, etc.) to record the doctor-patient conversation.

[0078] For example, in a medical technology scenario, if a patient visits a doctor due to cough and fever, the doctor will ask about the patient's name, age, duration of cough, fever temperature, medical history, and chest X-ray results during the consultation, and make a preliminary diagnosis of suspected acute bronchitis based on the information collected.

[0079] In step S102 of some embodiments, specifically, the speech frame signal of the target medical consultation voice data is converted into digital features through automatic speech recognition technology, and these digital features can reflect the characteristics of the target medical consultation voice data (such as pitch, sound intensity, etc.); finally, the possible pronunciation unit sequence is predicted based on the characteristics, and the pronunciation unit sequence is combined into words and sentences in combination with context information to finally obtain the voice consultation text data.

[0080] Furthermore, the voice consultation text data needs to be proofread to correct possible recognition errors or ambiguities.

[0081] For example, in a medical technology scenario, the target consultation voice data may be a conversation between a patient and a doctor. During the consultation, the doctor asks the patient's name, age, cough duration, and fever temperature. These voice conversations can be recognized as voice consultation text data through automatic speech recognition technology.

[0082] In step S103 of some embodiments, specifically, first, the target medical consultation image data is preprocessed (such as image enhancement, denoising, binarization, etc.); secondly, the character area in the target medical consultation image data is located by edge detection image processing technology, and the character area is further segmented into single characters; finally, the segmented character image is converted into text characters using optical character recognition technology, and the recognized text characters are integrated according to the original image layout to obtain image consultation text data.

[0083] For example, in a medical technology scenario, if a doctor is processing an image file containing a patient's medical history information, which is an image of a blood test report, the text information in the image of the blood test report can be automatically converted into an editable text format through optical character recognition technology to obtain image consultation text data containing the blood test report.

[0084] See also Figure 2 In some embodiments, step S104 includes but is not limited to steps S201 to S204:

[0085] Step S201, obtaining a preset medical consultation knowledge graph.

[0086] Step S202, extracting features from the voice medical consultation text data to obtain voice medical consultation text features, and extracting features from the image medical consultation text data to obtain image medical consultation text features.

[0087] Step S203, mapping the voice medical consultation text features and the image medical consultation text features to the medical consultation knowledge graph to obtain the target medical consultation knowledge graph.

[0088] Step S204, obtaining target medical inquiry text data from the target medical inquiry knowledge graph.

[0089] In step S201 of some embodiments, specifically, first, existing medical knowledge data is acquired, including disease symptoms, examination results, diagnostic methods, treatment plans, etc., and medical entities are extracted from the medical knowledge data, such as disease names, symptoms, drugs, examination items, etc.; secondly, medical entity relationships between medical entities are defined, such as symptoms-disease, disease-treatment plan, etc., and the medical entity relationships are constructed as edges in the medical consultation knowledge graph; finally, the extracted medical entities and medical entity relationships are stored in a graph database to obtain a medical consultation knowledge graph.

[0090] In step S202 of some embodiments, specifically, key features in the voice consultation text data can be extracted through word embedding operations, including lexical features (such as keywords, symptom descriptions, etc.), grammatical features (such as sentence structure, word order, etc.) and semantic features (such as text representation based on word vectors).

[0091] Specifically, the image consultation text features may be obtained by performing feature extraction on the image consultation text data through word embedding operation.

[0092] For example, in the medical technology scenario, the text features of a voice consultation may be that the patient reports that he has been coughing for a week and has a fever of 36.9°C; the text features of an image consultation may be that there is a nodule in the right lower lobe of the lung.

[0093] In step S203 of some embodiments, specifically, the voice consultation text features are first matched with entities and relationships in the consultation knowledge graph, that is, the disease names in the voice consultation text features are matched with the disease entities in the consultation knowledge graph, and the symptom descriptions are matched with the symptom attributes of the disease entities.

[0094] For example, in the medical field, the voice consultation text feature "the patient reported coughing for a week" can be mapped to the "cough" entity on the consultation graph, and the "upper respiratory tract infection" disease entity associated with "cough" can be found through knowledge reasoning, forming an entity relationship between cough and upper respiratory tract infection.

[0095] For example, in the medical field, the image consultation text feature "nodule shadow in the right lower lobe of the lung" can be mapped to the corresponding entity "nodule in the right lower lobe of the lung" in the consultation knowledge graph.

[0096] Specifically, the mapping results of the voice-text medical consultation text features and the image-text medical consultation text features are merged to obtain the target medical consultation knowledge graph.

[0097] For example, in the medical technology scenario, by combining the mapping of image consultation text features with voice consultation text features, we can obtain target consultation knowledge graphs including pneumonia-symptoms-nodules in the right lower lobe of the lung; cough-may cause-upper respiratory tract infection; pneumonia-symptoms-cough.

[0098] In this embodiment, by mapping the text features of voice medical consultation and the text features of image medical consultation to the medical consultation knowledge graph, the fusion of the text features of image medical consultation and the voice medical consultation features can be achieved. The accuracy of the text feature description is also enhanced through the reasoning ability of the medical consultation knowledge graph, which facilitates the subsequent improvement of the accuracy of the medical consultation.

[0099] In step S204 of some embodiments, specifically, the medical inquiry target is first determined, and the relevant target entities and target entity relationships are retrieved from the target medical inquiry knowledge graph according to the medical inquiry target, and all the queried information is sorted to form a complete medical inquiry report text.

[0100] For example, in the medical technology scenario, the target of the medical consultation query is the patient's symptoms (such as cough and fever). Diseases related to cough and fever may be upper respiratory tract infection or pneumonia, etc., retrieved from the target medical consultation knowledge graph. All queried entities and entity relationships (such as symptoms, signs, medical history, disease diagnosis suggestions, treatment suggestions, further examination suggestions, etc.) are compiled in the order of the consultation to form a complete medical consultation report text.

[0101] In this embodiment, by obtaining the target medical consultation text data from the target medical consultation knowledge graph, the diseases that may be caused by the symptoms in the medical consultation can be accurately located, and a large amount of user-personalized medical information can be stored through the target medical consultation knowledge graph, thereby providing personalized medical advice based on the specific symptoms and historical data of different patients, thereby facilitating improving the accuracy of subsequent medical consultations.

[0102] See also Figure 3 In some embodiments, before step S105, the method for detecting abnormality in medical consultation may include but is not limited to steps S301 to S305:

[0103] Step S301, using the pre-trained medical consultation anomaly detection model to perform word embedding operation on the target medical consultation text data to obtain a medical consultation text matrix.

[0104] Step S302, obtaining an initial weight matrix of the medical inquiry text matrix, and obtaining an initial training parameter matrix of the initial weight matrix.

[0105] Step S303, performing symmetric decomposition on the initial training parameter matrix to obtain a decomposed symmetric matrix.

[0106] Step S304, scoring the initial weight matrix according to the decomposed symmetric matrix to obtain a weight importance score.

[0107] Step S305 , training the pre-trained medical consultation anomaly detection model according to the weight importance scores to obtain a trained medical consultation anomaly detection model.

[0108] In step S301 of some embodiments, specifically, the pre-trained medical question anomaly detection model may be a large language model.

[0109] Specifically, the target medical inquiry text data may be embedded and mapped to a high-dimensional vector space to obtain a medical inquiry text matrix.

[0110] In step S302 of some embodiments, specifically, the initial weight matrix is ​​used to represent the weight of the medical consultation text matrix, and the initial weight matrix can be represented by W0.

[0111] Specifically, the initial training parameter matrix is ​​used to adjust the model performance during the fine-tuning process, that is, it is used to incrementally adjust the initial weight matrix during the fine-tuning process.

[0112] In step S303 of some embodiments, specifically, the decomposed symmetric matrix includes a singular value symmetric matrix, a left singular vector matrix and a right singular vector matrix, wherein the singular value symmetric matrix includes singular values, and the size of the singular value reflects the energy distribution of the initial weight matrix in each direction, that is, the larger the singular value, the higher the information content in the corresponding singular vector direction.

[0113] Specifically, the decomposed symmetric matrix is ​​used to represent the information content or importance of the initial weight matrix in different vector directions and guide the parameter budget allocation of the initial weight matrix.

[0114] In this embodiment, by performing symmetrical decomposition on the initial training parameter matrix to obtain a decomposed symmetric matrix, the large matrix can be decomposed into three smaller matrices to achieve dimensionality reduction of the initial training parameter matrix. The rank of the initial training parameter matrix can also be dynamically adjusted. The number of new parameters can be controlled by controlling the size of the rank, thereby reducing the training cost while maintaining the performance of the medical consultation anomaly detection model.

[0115] In step S304 of some embodiments, the weight importance score is an importance score for a triplet in the decomposed symmetric matrix; wherein the triplet is composed of all elements of the singular value symmetric matrix, the left singular vector matrix and the right singular vector matrix.

[0116] Specifically, the singular value symmetric matrix, the left singular vector matrix and the right singular vector matrix are used as triplets to score the importance, which can more intuitively represent the role and importance of each feature in the decomposition weight matrix.

[0117] Specifically, the weight importance score can be obtained by the following formula:

[0118]

[0119] Among them, Sk,i represents the weight importance score of the i-th triplet of the k-th decomposed symmetric matrix, s(γ k,i ) represents the weight importance score of the singular value symmetric matrix of the i-th triple of the k-th decomposition symmetric matrix, s(P k,ji ) represents the weight importance score of the jth left singular vector matrix of the i-th triple of the k-th decomposed symmetric matrix, s(Q l,ij ) represents the weight importance score of the j-th right singular vector matrix of the i-th triplet of the k-th decomposition symmetric matrix, d1 represents the dimension of the j-th left singular vector matrix of the i-th triplet of the k-th decomposition symmetric matrix, and d2 represents the dimension of the j-th right singular vector matrix of the i-th triplet of the k-th decomposition symmetric matrix.

[0120] For example, in a medical technology scenario, a patient describes his symptoms as cough, fever, and chest pain, and his past medical history is chronic bronchitis. The doctor initially diagnoses him with pneumonia and extracts features such as cough, fever, and chest pain through word embedding processing. An importance score is assigned to each feature based on the decomposition symmetric matrix of the singular value decomposition, where the weighted importance score is the overall importance score of the symptoms and past medical history for a diagnosis of pneumonia determined after combining the symptoms and past medical history features. The importance of the left singular vector represents the average contribution of all symptom features to the diagnosis of pneumonia, and the importance of the right singular vector represents the average contribution of past medical history to the diagnosis of pneumonia. The importance of the initial training parameter matrix represents the overall importance score for a diagnosis of pneumonia.

[0121] In this embodiment, by scoring the importance of the initial weight matrix according to the decomposed symmetric matrix, it is possible to facilitate subsequent fine-tuning of the medical interview anomaly detection model according to the weight importance scores, increase the training intensity of important parameters in the medical interview anomaly detection model, and reduce the attention to unimportant parameters in the medical interview anomaly detection model, thereby improving the efficiency of medical interview anomaly detection model training.

[0122] See also Figure 4 In some embodiments, step S305 includes but is not limited to steps S401 to S402:

[0123] Step S401, performing gradient update on the decomposed symmetric matrix according to a preset loss function and a preset learning rate to obtain an updated decomposed symmetric matrix.

[0124] Step S402, updating the pre-trained medical consultation anomaly detection model according to the updated decomposition symmetric matrix and the weight importance score to obtain a trained medical consultation anomaly detection model.

[0125] In step S401 of some embodiments, specifically, the learning rate is a hyperparameter used to control the weight update step size of the medical anomaly detection model during the training process.

[0126] Specifically, the preset loss function may be a cross entropy loss function.

[0127] Specifically, first obtain the training target of the triple constructed based on the singular value, the left singular vector element and the right singular vector element, obtain the gradient of the loss function with respect to the initial training parameter matrix, and update the gradients of the singular value symmetric matrix, the left singular vector matrix and the right singular vector matrix in the decomposed symmetric matrix according to the gradient and the learning rate using the stochastic gradient descent method to obtain the updated decomposed symmetric matrix.

[0128] In this embodiment, by performing gradient updating on the decomposed symmetric matrix according to a preset loss function and a preset learning rate, the convergence of the model can be accelerated, thereby facilitating subsequent improvement of the efficiency of model training.

[0129] In step S402 of some embodiments, specifically, the updating of the medical consultation anomaly detection model is achieved by discarding singular values ​​in the updated decomposition symmetry based on weight importance scores. Furthermore, the singular values ​​to be discarded can be removed by a pruning function to obtain a trained medical consultation anomaly detection model.

[0130] In this embodiment, the pre-trained medical consultation anomaly detection model is updated according to the updated decomposition symmetric matrix and the weight importance score, which can reduce the amount of parameter calculation in the model training process while ensuring the model accuracy, thereby improving the model training efficiency.

[0131] See also Figure 5 In some embodiments, step S105 includes but is not limited to steps S501 to S504:

[0132] Step S501 , using a pre-trained medical inquiry anomaly detection model to perform feature extraction on target medical inquiry text data to obtain target medical inquiry text features.

[0133] Step S502: performing inquiry entity boundary detection on the target inquiry text features to obtain the inquiry entity boundary.

[0134] Step S503, performing inquiry entity category recognition on the target inquiry text features according to the inquiry entity boundary to obtain the inquiry entity category.

[0135] Step S504, the target medical inquiry text feature is queried according to the medical inquiry entity category to obtain medical inquiry reference data; the medical inquiry reference data is represented by the entity form of the target medical inquiry text feature.

[0136] In step S501 of some embodiments, specifically, the medical question anomaly detection model is a large language model.

[0137] Specifically, the target medical inquiry text features are obtained by performing a word embedding operation on the target medical inquiry text data.

[0138] Specifically, by calculating the inquiry weight, key weight and value weight of the target inquiry text data, the target inquiry text data is mapped into a high-dimensional vector space to obtain the target inquiry text features represented by a matrix.

[0139] In step S502 of some embodiments, specifically, the boundary of the medical inquiry entity refers to the start position and the beginning position of the medical inquiry entity.

[0140] Specifically, the medical inquiry entity prediction is performed on the target medical inquiry text data through the medical inquiry anomaly detection model, and the start and end positions of the predicted entities are marked according to the BIOS (Begin, Inside, Outside) marking method.

[0141] For example, in a medical technology scenario, B-cough indicates the entity start position of the cough entity, while I-cough indicates the entity inner part (ie, the end position) of the cough entity.

[0142] In this embodiment, by performing inquiry entity boundary detection on the target inquiry text features, the start and end positions of the key entities in the target inquiry text features can be clearly defined, which helps assist doctors to quickly focus on the patient's specific health problems and improve the efficiency of the inquiry.

[0143] In step S503 of some embodiments, specifically, annotated entity category labels are obtained, and the entity labels include but are not limited to symptoms, diagnosed diseases, drugs, test results, etc.

[0144] For example, in a medical technology scenario, the target consultation text data is that the patient complains of coughing for a week, fever, and possible upper respiratory tract infection. The predicted consultation entities are cough, fever, and upper respiratory tract infection. Through the entity category label, the consultation entity category of cough and fever is determined to be symptoms, and the consultation entity category of upper respiratory tract infection is determined to be a diagnosed disease.

[0145] In this embodiment, by identifying the entity categories of the target medical inquiry text features according to the boundaries of the medical inquiry entities, different entity categories (such as symptoms, diagnosed diseases, drugs, etc.) can be quickly determined, which helps doctors quickly understand the patient's physical condition and helps improve the accuracy of the medical inquiry.

[0146] In step S504 of some embodiments, specifically, according to the inquiry entity boundary and inquiry entity category, the target inquiry text data can be transferred to the inquiry entity in a structured form to obtain inquiry reference data, wherein the inquiry reference data includes the inquiry entity name, entity category, entity start position and entity end position.

[0147] For example, in the medical technology scenario, the target consultation text feature is that the patient complains of repeated headache and fever symptoms in the past week and had acute pharyngitis last time. The doctor diagnosed it as possible influenza and recommended the use of antipyretic and analgesic drugs for treatment. The symptom entities are identified as headache and fever, the past medical history entity is acute pharyngitis, the time entity is lasting for one week, the diagnosed disease entity is influenza, and the drug entity is antipyretic and analgesic drugs. The final reference data is that the patient has symptoms of headache and fever lasting for one week and a past medical history of acute pharyngitis. He may have influenza and is recommended to use antipyretic and analgesic drugs.

[0148] See also Figure 6 In some embodiments, step S106 includes but is not limited to steps S601 to S602:

[0149] Step S601, calculating the similarity between the medical consultation reference data and the doctor's medical consultation data to obtain the medical consultation similarity.

[0150] Step S602: performing consultation anomaly detection on the doctor's consultation data according to the consultation similarity to obtain consultation anomaly data.

[0151] In step S601 of some embodiments, specifically, the medical question similarity can be calculated by cosine similarity, Jaccard similarity, Manhattan distance, or the like.

[0152] Specifically, the cosine similarity may be used to calculate the similarity between the consultation reference data and the doctor's consultation data to obtain a similarity score, which is usually between 0 and 1, where 0 indicates complete dissimilarity and 1 indicates complete similarity.

[0153] In step S602 of some embodiments, specifically, the consultation similarity is compared with a preset similarity threshold. If the consultation similarity is greater than or equal to the similarity threshold, it means that the similarity between the doctor's consultation data and the consultation reference data is high, and no abnormal marking is performed. If the consultation similarity is less than the similarity threshold, it means that the similarity between the doctor's consultation data and the consultation reference data is low, and the doctor's consultation data is marked as abnormal to obtain abnormal consultation data.

[0154] For example, in a medical technology scenario, the doctor's consultation data is that the patient has a cough for one week and is accompanied by a fever, and the consultation reference data is cough, lasting for one week, and fever. The cosine similarity algorithm is used to calculate the similarity between the two data, and the consultation similarity is 0.9. If the similarity threshold is 0.8, it is determined that the doctor's consultation data is highly similar to the consultation reference data, and no abnormal marking is performed.

[0155] For example, in a medical technology scenario, the doctor's consultation data is a patient's chief complaint of chest pain, located in the right lower abdomen, and the consultation reference data is a patient's chief complaint of abdominal pain, located in the right lower abdomen. The cosine similarity algorithm is used to calculate the similarity between the two data, and the consultation similarity is 0.4. Since chest pain and abdominal pain have different meanings and possible causes in medicine, it is determined that the similarity between the doctor's consultation data and the consultation reference data is low, and 0.4 is less than the similarity threshold of 0.8, then the chest pain is marked as abnormal, and the chest pain is used as abnormal consultation data.

[0156] In this embodiment, by performing consultation anomaly detection on the doctor's consultation data according to the consultation similarity, consultation anomaly data is obtained, which can accurately detect key information omitted during the consultation process, incorrectly described symptoms or inconsistent disease progression, thereby facilitating subsequent improvement of the accuracy of the consultation.

[0157] In some embodiments, after step S602, the abnormal consultation data is subjected to entity extraction to obtain the abnormal consultation entity; the doctor consultation entity is updated according to the abnormal consultation entity to obtain the updated doctor consultation entity; the updated doctor consultation entity is consulted to obtain the target doctor consultation data.

[0158] For example, in a medical technology scenario, the doctor's consultation data is that the patient complains of chest pain, located in the right lower abdomen, and the consultation reference data is that the patient complains of abdominal pain, located in the right lower abdomen. In this case, entity extraction is performed on chest pain and abdominal pain, and the entity type is symptom. The abdominal pain is updated for chest pain to obtain the target doctor's consultation data that the patient complains of abdominal pain, located in the right lower abdomen.

[0159] In this embodiment, by updating the doctor's consultation entity according to the abnormal consultation entity, the doctor can be promptly prompted with errors in the consultation information, reducing the risk of misdiagnosis or missed diagnosis by the doctor, helping the doctor to correct possible misdiagnosis or missed diagnosis in a timely manner, and improving the accuracy of the consultation.

[0160] The embodiment of the present application obtains target medical consultation data, which includes target medical consultation image data and target medical consultation voice data; performs voice recognition on the target medical consultation voice data to obtain voice medical consultation text data; performs character recognition on the target medical consultation image data to obtain image medical consultation text data; performs text enhancement processing on the voice medical consultation text data using the image medical consultation text data to obtain target medical consultation text data; performs medical consultation on the target medical consultation text data using a pre-trained medical consultation anomaly detection model to obtain medical consultation reference data; obtains doctor medical consultation data, and performs medical consultation anomaly detection on the doctor medical consultation data based on the medical consultation reference data to obtain medical consultation anomaly data. This application first uses the image consultation text data to perform text enhancement processing on the voice consultation text data to obtain the target consultation text data, which can effectively integrate the information from different data sources to obtain richer consultation data, so as to improve the accuracy of subsequent consultations; secondly, the target consultation text data is consulted using a pre-trained consultation anomaly detection model, which can quickly and accurately analyze and diagnose the target consultation text, reduce errors caused by human factors, ensure the objectivity of the entire diagnosis process, and improve the accuracy of the consultation; finally, the doctor's consultation data is subjected to consultation anomaly detection based on the consultation reference data, which can promptly discover the difference between the two, help doctors to promptly correct possible misdiagnosis or missed diagnosis, and further improve the accuracy of the consultation.

[0161] See also Figure 7 The embodiment of the present application further provides a device for detecting abnormality in medical consultation, which can implement the above-mentioned method for detecting abnormality in medical consultation, and the device includes:

[0162] A medical consultation data acquisition module is used to acquire target medical consultation data, the target medical consultation data includes target medical consultation image data and target medical consultation voice data;

[0163] A speech recognition module is used to perform speech recognition on the target medical consultation speech data to obtain speech consultation text data;

[0164] An image recognition module is used to perform character recognition on target medical consultation image data to obtain image medical consultation text data;

[0165] A text enhancement module is used to perform text enhancement processing on the voice consultation text data using the image consultation text data to obtain target consultation text data;

[0166] The consultation module is used to use the pre-trained consultation anomaly detection model to conduct consultation on the target consultation text data to obtain consultation reference data;

[0167] The consultation anomaly detection module is used to obtain the doctor's consultation data, and perform consultation anomaly detection on the doctor's consultation data based on the consultation reference data to obtain the consultation anomaly data.

[0168] The specific implementation of the medical question abnormality detection device is substantially the same as the specific implementation of the medical question abnormality detection method described above, and will not be described in detail herein.

[0169] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned abnormality detection method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0170] See also Figure 8 , Figure 8 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0171] The processor 801 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0172] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the processing system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the abnormality detection method for the medical consultation of the embodiment of this application;

[0173] Input / output interface 803, used to implement information input and output;

[0174] The communication interface 804 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0175] A bus 805 that transmits information between the various components of the device (e.g., the processor 801, the memory 802, the input / output interface 803, and the communication interface 804);

[0176] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0177] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned medical consultation abnormality detection method is implemented.

[0178] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0179] The embodiments of the present application provide a method for detecting anomalies in medical consultation, an apparatus for detecting anomalies in medical consultation, an electronic device, and a storage medium. The method comprises the following steps: obtaining target medical consultation data, wherein the target medical consultation data includes target medical consultation image data and target medical consultation voice data; performing voice recognition on the target medical consultation voice data to obtain voice medical consultation text data; performing character recognition on the target medical consultation image data to obtain image medical consultation text data; performing text enhancement processing on the voice medical consultation text data using the image medical consultation text data to obtain target medical consultation text data; performing medical consultation on the target medical consultation text data using a pre-trained medical consultation anomaly detection model to obtain medical consultation reference data; obtaining doctor medical consultation data, and performing medical consultation anomaly detection on the doctor medical consultation data according to the medical consultation reference data to obtain medical consultation anomaly data. This application first uses the image consultation text data to perform text enhancement processing on the voice consultation text data to obtain the target consultation text data, which can effectively integrate the information from different data sources to obtain richer consultation data, so as to improve the accuracy of subsequent consultations; secondly, the target consultation text data is consulted using a pre-trained consultation anomaly detection model, which can quickly and accurately analyze and diagnose the target consultation text, reduce errors caused by human factors, ensure the objectivity of the entire diagnosis process, and improve the accuracy of the consultation; finally, the doctor's consultation data is subjected to consultation anomaly detection based on the consultation reference data, which can promptly discover the difference between the two, help doctors to promptly correct possible misdiagnosis or missed diagnosis, and further improve the accuracy of the consultation.

[0180] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0181] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0182] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0184] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0185] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0186] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0187] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0190] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for detecting abnormalities in medical consultation, characterized in that: The method comprises: Acquire target medical consultation data, wherein the target medical consultation data includes target medical consultation image data and target medical consultation voice data; Performing voice recognition on the target medical consultation voice data to obtain voice medical consultation text data; Performing character recognition on the target medical consultation image data to obtain image medical consultation text data; Using the image consultation text data, the voice consultation text data is subjected to text enhancement processing to obtain target consultation text data; Conducting a medical consultation on the target medical consultation text data using a pre-trained medical consultation anomaly detection model to obtain medical consultation reference data; The doctor's consultation data is acquired, and consultation anomaly detection is performed on the doctor's consultation data according to the consultation reference data to obtain consultation anomaly data.

2. The method according to claim 1, characterized in that The step of performing abnormality detection on the doctor's consultation data according to the consultation reference data to obtain abnormal consultation data includes: Calculating the similarity between the medical consultation reference data and the doctor's medical consultation data to obtain medical consultation similarity; The doctor's consultation data is subjected to consultation anomaly detection according to the consultation similarity to obtain the consultation anomaly data.

3. The method according to claim 2, characterized in that After performing abnormal consultation detection on the doctor's consultation data according to the consultation similarity and obtaining the abnormal consultation data, the method further includes: Performing entity extraction on the abnormal medical consultation data to obtain abnormal medical consultation entities; The doctor's consultation entity is updated according to the abnormal consultation entity to obtain an updated doctor's consultation entity; The updated doctor consultation entity is consulted to obtain the target doctor consultation data.

4. The method according to claim 1, characterized in that: The method of using the pre-trained medical consultation anomaly detection model to conduct medical consultation on the target medical consultation text data to obtain medical consultation reference data includes: Using the pre-trained medical inquiry anomaly detection model to perform feature extraction on the target medical inquiry text data to obtain target medical inquiry text features; Performing inquiry entity boundary detection on the target inquiry text features to obtain the inquiry entity boundary; According to the inquiry entity boundary, the target inquiry text feature is subjected to inquiry entity category identification to obtain the inquiry entity category; The target medical inquiry text feature is queried according to the medical inquiry entity category to obtain the medical inquiry reference data; the medical inquiry reference data is represented by the entity form of the target medical inquiry text feature.

5. The method according to claim 1, characterized in that The step of performing text enhancement processing on the voice consultation text data using the image consultation text data to obtain target consultation text data includes: Obtain the preset medical consultation knowledge graph; Performing feature extraction on the voice medical consultation text data to obtain voice medical consultation text features, and performing feature extraction on the image medical consultation text data to obtain image medical consultation text features; Mapping the voice medical consultation text features and the image medical consultation text features onto the medical consultation knowledge graph to obtain a target medical consultation knowledge graph; The target medical inquiry text data is obtained from the target medical inquiry knowledge graph.

6. The method according to any one of claims 1 to 5, characterized in that: Before the target medical consultation text data is diagnosing using the pre-trained medical consultation anomaly detection model to obtain medical consultation reference data, the method further includes: Using the pre-trained medical consultation anomaly detection model to perform word embedding operation on the target medical consultation text data to obtain a medical consultation text matrix; Obtaining an initial weight matrix of the medical inquiry text matrix, and obtaining an initial training parameter matrix of the initial weight matrix; Performing symmetric decomposition on the initial training parameter matrix to obtain a decomposed symmetric matrix; Score the initial weight matrix according to the decomposed symmetric matrix to obtain a weight importance score; The pre-trained medical inquiry anomaly detection model is trained according to the weight importance score to obtain a trained medical inquiry anomaly detection model.

7. The method according to claim 6, characterized in that The pre-trained medical consultation anomaly detection model is trained according to the weight importance score to obtain a trained medical consultation anomaly detection model, including: Performing gradient update on the decomposed symmetric matrix according to a preset loss function and a preset learning rate to obtain an updated decomposed symmetric matrix; The pre-trained medical consultation anomaly detection model is updated according to the updated decomposition symmetric matrix and the weight importance score to obtain the trained medical consultation anomaly detection model.

8. A device for detecting abnormalities in medical consultation, characterized in that: The device comprises: A medical consultation data acquisition module, used to acquire target medical consultation data, wherein the target medical consultation data includes target medical consultation image data and target medical consultation voice data; A speech recognition module, used for performing speech recognition on the target medical consultation speech data to obtain speech medical consultation text data; An image recognition module, used for performing character recognition on the target medical consultation image data to obtain image medical consultation text data; A text enhancement module, used to perform text enhancement processing on the voice consultation text data using the image consultation text data to obtain target consultation text data; A consultation module, used to use a pre-trained consultation anomaly detection model to conduct consultation on the target consultation text data to obtain consultation reference data; The consultation anomaly detection module is used to obtain doctor consultation data, and perform consultation anomaly detection on the doctor consultation data according to the consultation reference data to obtain consultation anomaly data.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method for detecting abnormality in medical consultation according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting abnormality in medical consultation according to any one of claims 1 to 7 is implemented.

Images