Consultation service quality inspection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a consultation service quality inspection method and device, electronic equipment and a storage medium. The method comprises the following steps: in response to a consultation service quality inspection instruction, acquiring target interaction content between a consultation service provider and a consultation service acquirer, and acquiring a deep learning model for realizing consultation service quality inspection; inputting the target interaction content into a deep learning model, and performing retrieval in a target knowledge graph corresponding to the consultation service provided by the consultation service provider according to the deep learning model and based on the target interaction content to obtain a target entity matched with the target interaction content; and performing quality inspection on the consultation service according to the target entity to obtain a quality inspection result of the consultation service. According to the technical scheme provided by the embodiment of the invention, the accuracy rate of quality detection for the consultation service can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of Internet medical technology, and in particular, to a method, device, electronic device, and storage medium for quality inspection of consultation services. Background Art

[0002] Currently, the development of Internet medical care has realized the online interaction between doctors and patients. Remote medical platforms have been established in many provinces across the country relying on the Internet, and online consultation has become the main way of medical consultation.

[0003] It should be noted that medical quality and safety are the core and lifeline of the development of Internet medical care. Therefore, quality inspection of medical consultation services (i.e., quality inspection) is an important means to effectively reduce the occurrence probability of medical risks and ensure the quality of doctors' services.

[0004] In the process of implementing the present invention, the inventor found the following technical problems in the prior art: the precision-recall rate (i.e., accuracy and recall rate) of the currently adopted quality inspection scheme is not high and needs to be improved. Summary of the Invention

[0005] Embodiments of the present invention provide a method, device, electronic device, and storage medium for quality inspection of consultation services to improve the precision-recall rate when performing quality inspection on consultation services.

[0006] According to one aspect of the present invention, a method for quality inspection of consultation services is provided, which may include:

[0007] In response to a consultation service quality inspection instruction, obtain the target interaction content between the consultation service provider and the consultation service acquirer, and obtain a deep learning model for implementing the consultation service quality inspection;

[0008] Input the target interaction content into the deep learning model, and based on the deep learning model and the target interaction content, retrieve in the target knowledge graph corresponding to the consultation service provided by the consultation service provider to obtain a target entity that matches the target interaction content;

[0009] Perform quality inspection on the consultation service according to the target entity to obtain the quality inspection result of the consultation service.

[0010] According to another aspect of the present invention, a device for quality inspection of consultation services is provided, which may include:

[0011] A deep learning model acquisition module, configured to obtain the target interaction content between the consultation service provider and the consultation service acquirer in response to a consultation service quality inspection instruction, and obtain a deep learning model for implementing the consultation service quality inspection;

[0012] A target entity obtaining module, configured to input target interaction content into a deep learning model, and retrieve, based on the deep learning model and the target interaction content, in a target knowledge graph corresponding to the consulting service provided by a consulting service provider, to obtain a target entity that matches the target interaction content;

[0013] A quality inspection result obtaining module, configured to perform quality inspection on the consulting service according to the target entity, to obtain a quality inspection result of the consulting service.

[0014] According to another aspect of the present invention, there is provided an electronic device, which may include:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is caused to implement the consulting service quality inspection method provided in any embodiment of the present invention.

[0018] According to another aspect of the present invention, there is provided a computer-readable storage medium, on which a computer instruction is stored, and when the computer instruction is used to cause a processor to execute, the consulting service quality inspection method provided in any embodiment of the present invention is implemented.

[0019] The technical solution of the embodiment of the present invention, by responding to a consulting service quality inspection instruction, obtains target interaction content between a consulting service provider and a consulting service acquirer, and a deep learning model for implementing consulting service quality inspection; further, inputs the target interaction content into the deep learning model, and based on the deep learning model and the target interaction content, retrieves in a target knowledge graph corresponding to the consulting service provided by the consulting service provider, to obtain a target entity that matches the target interaction content; and obtains a quality inspection result of the consulting service according to the output result of the deep learning model. The above technical solution, by combining the deep learning model with the target knowledge graph, reduces the risk that the deep learning model fails to obtain sufficient vertical domain knowledge (i.e., professional knowledge) corresponding to the consulting service by using the target knowledge graph retrieval, thereby improving the accuracy and recall rate of the consulting service quality inspection.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. Description of the Drawings

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0022] Figure 1 is a flowchart of a consultation service quality inspection method provided according to an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a target knowledge graph in a consultation service quality inspection method provided according to an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of the interface for quality inspection during the doctor's consultation in a consultation service quality inspection method provided according to an embodiment of the present invention;

[0025] Figure 4 is a flowchart of another consultation service quality inspection method provided according to an embodiment of the present invention;

[0026] Figure 5 is a flowchart of yet another consultation service quality inspection method provided according to an embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of a configuration case in yet another consultation service quality inspection method provided according to an embodiment of the present invention;

[0028] Figure 7 is a schematic diagram of a judgment logic in yet another consultation service quality inspection method provided according to an embodiment of the present invention;

[0029] Figure 8 is a flowchart of an optional example in yet another consultation service quality inspection method provided according to an embodiment of the present invention;

[0030] Figure 9 is a structural block diagram of a consultation service quality inspection device provided according to an embodiment of the present invention;

[0031] Figure 10 is a schematic diagram of the structure of an electronic device for implementing the consultation service quality inspection method of the embodiments of the present invention. Detailed implementation manners

[0032] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0033] It should be noted that in the description and claims of the present invention and the above-mentioned accompanying drawings, the terms "first", "second", etc. are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. The same is true for "target", "original", etc., which will not be elaborated here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] It should be noted that in the technical solution of the present invention, the collection, collection, update, analysis, processing, use, transmission, storage, etc. of the user's personal information comply with the provisions of relevant laws and regulations, are used for legal purposes, and do not violate public order and good customs. Necessary measures are taken for the user's personal information to prevent illegal access to the user's personal information data and to safeguard the security of the user's personal information, network security and national security.

[0035] Before introducing the embodiments of the present invention, an exemplary description will be given of the implementation process of the quality inspection scheme currently adopted for medical consultation services and the reasons for the problem of low recall rate, so as to better understand the reasons why the quality inspection scheme proposed in the embodiments of the present invention can improve the recall rate.

[0036] Exemplarily, quality inspection can currently be based on regular matching. However, it has been found through practice that this implementation scheme strongly depends on rule collection and essentially lacks context semantic understanding. This means that when there are differences between the target interaction content in the medical consultation process and the collected rules, even if the semantics are the same, there will still be cases of missed inspection and / or false inspection, thus affecting the recall rate of the quality inspection of medical consultation services.

[0037] In addition, quality inspection can also be performed based on a deep learning model with context semantic understanding ability, thereby solving the problem of low precision and recall rate caused by the lack of context semantic understanding ability in the regular matching scheme. However, through practice, it is found that the deep learning model cannot fully obtain medical professional knowledge, and the lack of this medical professional knowledge will also affect the precision and recall rate of the quality inspection of medical consultation services.

[0038] In view of this, the embodiments of the present invention combine a deep learning model with a target knowledge graph, and reduce the risk of insufficient acquisition of medical professional knowledge by the deep learning model through using the target knowledge graph retrieval, so as to improve the precision and recall rate of the quality inspection of medical consultation services. The following will elaborate on this in detail.

[0039] Figure 1 FIG. is a flowchart of a method for quality inspection of consultation services provided by an embodiment of the present invention. This embodiment is applicable to the situation of detecting the service quality of consultation services, especially applicable to the situation of detecting the service quality of medical consultation services. This method can be executed by a consultation service quality inspection device provided by the embodiments of the present invention. The device can be implemented in software and / or hardware, and the device can be integrated on an electronic device, which can be various user terminals or servers.

[0040] See Figure 1 , the method of the embodiment of the present invention specifically includes the following steps:

[0041] S110. In response to a consultation service quality inspection instruction, obtain the target interaction content between the consultation service provider and the consultation service acquirer, and obtain a deep learning model for implementing the consultation service quality inspection.

[0042] Among them, the consultation service provider can be understood as the party providing the consultation service, and the consultation service acquirer can be understood as the party obtaining the consultation service provided by the consultation service provider. Combined with the application scenarios that the embodiments of the present invention may involve, exemplarily, the consultation service can be a medical consultation service. In this case, the consultation service provider can be a doctor, and the consultation service acquirer can be a patient and / or the patient's family member; exemplarily, the consultation service can be an education consultation service. In this case, the consultation service provider can be a teacher, and the consultation service acquirer can be a student and / or the student's parent; further exemplarily, the consultation service can be a financial consultation service. In this case, the consultation service provider can be a financial planner, and the consultation service acquirer can be a financial manager; and so on. This is related to the actual situation and is not specifically limited here.

[0043] The target interaction content can be understood as the content generated during the interaction between the consulting service provider and the consulting service acquirer. Considering the application scenarios that the embodiments of the present invention may involve, optionally, the interaction process between the consulting service provider and the consulting service acquirer can be implemented, for example, by at least one of text, pictures, voice, and video; additionally, this interaction process can also be referred to as a conversation process. It should be emphasized that, as in the above example, this conversation process does not necessarily need to be implemented by voice and / or video, etc., which require vocalization, but can also be implemented by text and / or pictures, etc., which do not require vocalization, and no specific limitation is made here.

[0044] The consulting service quality inspection instruction can be understood as an instruction used to indicate the quality inspection of the consulting service provided by the consulting service provider based on the target interaction content. Considering the application scenarios that the embodiments of the present invention may involve, here, taking the consulting service as a medical consulting service as an example, during the doctor-patient conversation process, every N rounds of conversations can trigger 1 consulting service quality inspection instruction, so as to achieve in-clinic quality inspection, where N is a positive integer. In this example, optionally, the target interaction content can be the content generated during these N rounds of conversations, or the content generated during these N rounds of conversations and the M rounds of conversations before these N rounds of conversations respectively, where M is a positive integer, etc., and no specific limitation is made here.

[0045] The deep learning model can be understood as a model used to implement the consulting service quality inspection, especially a model used to perform quality inspection on the consulting service provided by the consulting service provider. On this basis, it should be noted that compared with the deep learning model with a small number of parameters, the deep learning model with a large number of parameters (i.e., the large language model, or the large model) has a strong context semantic understanding ability, few-shot learning ability, and keyword learning ability, and can be fine-tuned in a supervised manner (supervised fine-tuning, SFT) through the real interaction content generated during the online consulting service process to strengthen its semantic understanding ability in the consulting service scenario, and can better ensure the precision and recall rate of the consulting service quality inspection. Therefore, in the embodiments of the present invention, optionally, the above deep learning model can be a large language model.

[0046] In response to the consulting service quality inspection instruction, obtain the target interaction content and the deep learning model.

[0047] S120. Input the target interaction content into the deep learning model, and based on the deep learning model, retrieve in the target knowledge graph corresponding to the consulting service provided by the consulting service provider based on the target interaction content, so as to obtain the target entity that matches the target interaction content.

[0048] Among them, the target interaction content is input into a deep learning model, so that the deep learning model can be used to implement the quality detection of the consulting service by performing the following steps: Based on the target interaction content, retrieve in the target knowledge graph corresponding to the consulting service provided by the consulting service provider to obtain a target entity that matches the target interaction content, and perform quality inspection on the consulting service according to the target entity.

[0049] Specifically, the target knowledge graph can be understood as the knowledge graph corresponding to the consulting service, and in particular, it can be understood as the knowledge graph that can reflect the professional knowledge in the consulting scenario where the consulting service is located. Exemplarily, taking the consulting service as a medical consulting service as an example, the target knowledge graph can be the knowledge graph that can reflect the professional knowledge in the medical consulting scenario (i.e., the medical consultation scenario), and in particular, it can be the disease knowledge graph corresponding to the diseases involved in the medical consultation process. Multiple entities are set in the target knowledge graph.

[0050] Using the deep learning model, based on the target interaction content, retrieve in the target knowledge graph to obtain one or more target entities that match the target interaction content from multiple entities.

[0051] In practical applications, optionally, in order to avoid the influence of invalid content (such as prepositions, modal particles, and conjunctions, etc.) in the target interaction content on the quality inspection effect, the invalid content can be removed from the target interaction content first, that is, the key content is extracted from the target interaction content first, and then the key content is input into the deep learning model for processing; of course, the target interaction content can also be directly input into the deep learning model to use the deep learning model to extract the key content from the target interaction content; and so on, which is not specifically limited here.

[0052] S130. Perform quality inspection on the consulting service according to the target entity to obtain the quality inspection result of the consulting service.

[0053] Among them, after obtaining one or more target entities, the consulting service can be quality inspected according to the one or more target entities. For example, the consulting service can be quality inspected according to the matching degree between the one or more target entities; for another example, the consulting service can be quality inspected according to the number of the one or more target entities. Generally, the more the number, the more the professional knowledge in the consulting scenario is involved in the target interaction content, that is, it means that the consulting service provider provides professional consulting services for the consulting service acquirer, and further indicates that the service quality of the consulting service is qualified; and so on, which is not specifically limited here.

[0054] Furthermore, according to the output result of the deep learning model, obtain the quality inspection result of the consulting service.

[0055] In an embodiment of the present invention, optionally, the quality inspection process of the consulting service implemented by retrieving based on the target knowledge graph by using a deep learning model can be regarded as an intension quality inspection process for the consulting service. Exemplarily, taking the medical consulting service as an example, the above-mentioned intension quality inspection process can be used to determine whether there are professional problems in the doctor's consultation, diagnosis, treatment recommendation plan and other links during the doctor-patient conversation process. Therefore, the above-mentioned deep learning model can also be called an intension quality inspection model.

[0056] The technical solution of the embodiment of the present invention is to obtain the target interaction content between the consulting service provider and the consulting service acquirer, and the deep learning model for implementing the quality inspection of the consulting service in response to the consulting service quality inspection instruction; further, input the target interaction content into the deep learning model, and based on the deep learning model, retrieve in the target knowledge graph corresponding to the consulting service provided by the consulting service provider based on the target interaction content to obtain the target entity matching the target interaction content; obtain the quality inspection result of the consulting service according to the output result of the deep learning model. The above technical solution combines the deep learning model with the target knowledge graph to reduce the risk of insufficient acquisition of the vertical domain knowledge (i.e., professional knowledge) corresponding to the consulting service by the deep learning model through the target knowledge graph retrieval, thereby improving the accuracy and recall rate of the consulting service quality inspection.

[0057] An optional technical solution is that the number of target entities can be at least two. Quality inspection of the consulting service according to the target entity includes:

[0058] Quality inspection of the consulting service according to the matching degree between at least two target entities and the preset matching degree ranges for different quality inspection results respectively.

[0059] Among them, the number of target entities can be at least two. On this basis, determine the matching degree between at least two target entities. For example, the matching degree between each target entity in at least two target entities and the remaining target entities can be determined; the matching degree between the target entities in the target entity pairs formed by at least two target entities can be determined; at least two target entities can also be divided into a first entity corresponding to the consulting service provider and a second entity corresponding to the consulting service acquirer, and the matching degree between the first entity and the second entity can be determined; etc., which are not specifically limited here.

[0060] The matching degree range can be understood as the range related to the matching degree preset for a certain quality inspection result, that is, corresponding matching degree ranges can be preset for different quality inspection results respectively. Exemplarily, a matching degree range can be preset for the quality inspection result of passing the quality inspection, and another matching degree range can be preset for the quality inspection result of failing the quality inspection. At this time, there are 2 matching degree ranges.

[0061] Quality check the consulting service according to the matching degree and at least one preset matching degree range. Exemplarily, the quality check result corresponding to the matching degree range that includes the matching degree or is closest to the matching degree in the at least one matching degree range can be used as the quality check result of the consulting service.

[0062] Through the application of the matching degree and at least one matching degree range, the above technical solution achieves the effect of quickly and accurately determining the quality check result.

[0063] Optionally, on this basis, the above consulting service quality check method further includes:

[0064] For each pair of target entities with a preset matching threshold among at least two target entities, obtain the preset matching threshold for the pair of target entities.

[0065] Sum up the matching thresholds corresponding to each pair of target entities, and determine the matching degree between at least two target entities according to the obtained sum result.

[0066] Among them, a pair of target entities can be composed of two or more target entities among at least two target entities, and a matching threshold is preset for the two or more target entities to represent the matching degree between the two or more target entities.

[0067] For each pair of target entities among at least two target entities, obtain the preset matching threshold for the pair of target entities, and thus the matching thresholds corresponding to each pair of target entities among the at least two target entities can be obtained. Further, sum up these matching thresholds to obtain a sum result, and determine the matching degree between the at least two target entities according to the sum result. Exemplarily, the sum result can be directly used as the matching degree; it is also possible to determine the sum level corresponding to the sum result and determine the matching degree based on the sum level; etc., which are not specifically limited here.

[0068] Through presetting pairs of target entities and the matching thresholds corresponding to each pair of target entities, the above technical solution can determine the matching degree according to the sum result of the matching thresholds corresponding to each pair of target entities among at least two target entities, thereby achieving accurate determination of the matching degree.

[0069] On this basis, optionally, the consulting service is a medical consulting service, the target interaction content at least includes the disease diagnosis provided by the consulting service provider and the symptom description provided by the consulting service acquirer, and at least two target entities include the disease entity matching the disease diagnosis and at least two symptom entities matching the symptom description; correspondingly, based on the target interaction content, retrieve in the target knowledge graph corresponding to the consulting service provided by the consulting service provider to obtain the target entity matching the target interaction content, including:

[0070] Extract the disease diagnosis and symptom description from the target interaction content;

[0071] For the target knowledge graph corresponding to the disease diagnosis, retrieve in the target knowledge graph based on the disease diagnosis to obtain the disease entity, and retrieve in the target knowledge graph based on the symptom description to obtain at least two symptom entities;

[0072] Correspondingly, for each target entity pair with a preset matching threshold among at least two target entities, obtain the preset matching threshold for the target entity pair, including:

[0073] For each target entity pair among at least two target entity pairs composed of at least two symptom entities and the disease entity, obtain the preset matching threshold for the target entity pair, where the target entity pair includes the disease entity and at least one symptom entity among at least two symptom entities.

[0074] Among them, the symptom description can be understood as the content provided by the consulting service acquirer in the target interaction content that can be used to describe the symptoms. The disease diagnosis can be understood as the content provided by the consulting service provider in the target interaction content for describing the disease, and the disease is diagnosed at least based on the symptom description.

[0075] On this basis, at least two target entities may include the disease entity matching the disease diagnosis and at least two symptom entities matching the symptom description.

[0076] Extract the disease diagnosis and symptom description from the target interaction content. Then, determine the target knowledge graph corresponding to the disease diagnosis from the knowledge graphs applied in the medical consultation scenario, and retrieve in the target knowledge graph based on the symptom description to obtain at least two symptom entities, and retrieve in the target knowledge graph based on the disease diagnosis to obtain the disease entity.

[0077] Furthermore, for each target entity pair of at least two target entity pairs consisting of at least two symptom entities and disease entities, the target entity pair includes a disease entity and at least one symptom entity of the at least two symptom entities, a matching threshold pre-set for the target entity pair is obtained, and then the degree of matching between the at least two symptom entities and the disease entity is obtained based on the sum of the matching thresholds corresponding to the at least two target entity pairs respectively.

[0078] The above technical solution, in the medical consultation scenario, determines the target knowledge graph corresponding to the disease diagnosis, then retrieves the disease entity corresponding to the disease diagnosis and the symptom entity corresponding to the symptom description in the target knowledge graph, and then determines the degree of match between the disease entity and the symptom entity. The degree of match is the key to quality inspection of medical consulting services based on disease diagnosis and symptom description.

[0079] In order to understand the above technical solution more vividly, the following is an exemplary explanation of it with reference to a specific example. For example, assuming that the disease diagnosis extracted from the target interactive content is leukemia, the target knowledge graph corresponding to leukemia is as follows: Figure 2 As shown, it can include entities such as disease, symptoms, medications, and risk factors, and the disease entity leukemia is related to Figure 2 The matching thresholds (scores) between the symptom entities shown in are shown in Table 1. On this basis, assuming that the symptom descriptions extracted from the target interaction content are high fever, immature cells in the white blood cell classification of the routine blood test and bone marrow puncture, and anemia in the routine blood test, the symptom entities corresponding to the three symptom descriptions are fever, bone marrow puncture of the routine blood test, and anemia, among which the preset matching threshold for the target entity pair of leukemia and fever is 0.1, the preset matching threshold for the target entity pair of leukemia and bone marrow puncture of the routine blood test is 1.0, and the preset matching threshold for the target entity pair of leukemia and anemia is 0.1, then the sum of these three matching thresholds is 1.2.

[0080] Table 1 Examples of matching thresholds between disease entities and symptom entities

[0081] Disease entity Symptom description Symptom entity score Leukemia Fever Fever 0.1 Leukemia High fever Fever 0.1 Leukemia Lymph node enlargement Hepatosplenomegaly and lymphadenopathy 0.1 Leukemia Abnormal white blood cell classification with blasts seen in routine blood test and bone marrow aspiration Routine blood test and bone marrow aspiration 1.0 Leukemia Abnormal white blood cell classification with immature cells seen in routine blood test and bone marrow aspiration Routine blood test and bone marrow aspiration 1.0 Leukemia Routine blood test shows anemia Anemia 0.1

[0082] Furthermore, for disease diagnosis, assuming that the quality inspection results may include misdiagnosis, insufficient diagnostic basis and qualified diagnosis, and the preset matching degree ranges for them are [0,0.3), [0.3,0.7) and [0.7,∞), then 1.2 falls in [0.7,∞), so the quality inspection result this time is a qualified diagnosis.

[0083] Another optional technical solution, the above-mentioned consulting service quality inspection method, further includes:

[0084] During the interaction between the consulting service provider and the consulting service acquirer, when the quality inspection result indicates that there are quality problems with the consulting service, warning information for warning about the quality problems is sent to the user terminal used by the consulting service provider.

[0085] Among them, during the interaction between the consulting service provider and the consulting service acquirer, if the quality inspection result indicates that there are quality problems with the consulting service, warning information for warning about the quality problems can be sent to the user terminal used by the consulting service provider, so that the consulting service provider can timely learn about the quality problems and correct the quality problems, thereby ensuring the service quality of the consulting service.

[0086] Exemplarily, referring to Figure 3 , Pat represents the patient, and Doc represents the doctor. During the online consultation process, through quality inspection, it is found that the doctor did not ask about the allergy history before giving medication advice. Then, the corresponding warning information "Allergy history missing: Please ask about any usage taboos such as allergy history" can be sent to the user terminal used by the doctor to display the warning information on the user terminal. On this basis, if the doctor recognizes the risk reminder represented by the warning information, the risk can be directly canceled by sending the reply content recommended by the system; otherwise, the reminder interface can be closed and the quality control expert can conduct manual review after the consultation.

[0087] Figure 4 It is a flowchart of another consulting service quality inspection method provided by an embodiment of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the above consulting service quality inspection method may further include: performing semantic analysis on the target interaction content to obtain key content; correspondingly, inputting the target interaction content into a deep learning model to retrieve, based on the deep learning model and the target interaction content, a target entity that matches the target interaction content in the target knowledge graph corresponding to the consulting service provided by the consulting service provider, including: inputting the key content into the deep learning model to retrieve, based on the deep learning model and the key content, a target entity that matches the key content in the target knowledge graph corresponding to the consulting service provided by the consulting service provider. Among them, the explanations of the same or corresponding terms as those in the above embodiments are not repeated here.

[0088] Referring to Figure 4 , the method of this embodiment may specifically include the following steps:

[0089] S210. In response to a consulting service quality inspection instruction, obtain the target interaction content between the consulting service provider and the consulting service acquirer, and obtain a deep learning model for implementing consulting service quality inspection.

[0090] S220. Perform semantic analysis on the target interaction content to obtain the key content.

[0091] Among them, as can be seen from the above analysis, there may be invalid content in the target interaction content that affects the quality inspection effect of the consulting service. Therefore, in order to further improve the recall rate of the consulting service quality inspection, semantic analysis can be performed on the target interaction content to obtain the key content, so as to combine with the subsequent steps and input the key content into the deep learning model to carry out the consulting service quality inspection process.

[0092] In practical applications, optionally, the Named Entity Recognition (NER) technology can be used to perform semantic analysis on the target interaction content to identify one or more named entities from the target interaction content, and then use the one or more named entities as the key content; or a semantic analysis model can be used to analyze the target interaction content and obtain the key content according to the output result of the semantic analysis model; of course, the key content can also be obtained through other means, which are not specifically limited here. Optionally, the number of key contents can be one or more, which is related to the actual situation and is not specifically limited here.

[0093] S230. Input the key content into the deep learning model to retrieve in the target knowledge graph corresponding to the consulting service provided by the consulting service provider based on the key content through the deep learning model, and obtain the target entity matching the key content.

[0094] S240. Perform quality inspection on the consulting service according to the target entity to obtain the quality inspection result of the consulting service.

[0095] The technical solution of the embodiment of the present invention performs semantic analysis on the target interaction content to obtain the key content, and then inputs the key content into the deep learning model for consulting service quality inspection. The extraction of the key content further improves the recall rate of the consulting service quality inspection.

[0096] Figure 5It is a flowchart of another consultation service quality inspection method provided in an embodiment of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, semantic analysis is performed on the target interaction content to obtain key content, which may include: obtaining a semantic analysis model, where the semantic analysis model has been pre-learned with configuration cases, and the configuration cases are configured for preset quality inspection items, and the quality inspection items are used for quality inspection of consultation services; inputting the target interaction content into the semantic analysis model, so as to perform semantic analysis on the target interaction content through the semantic analysis model based on the configuration cases, and extract key content corresponding to the quality inspection items from the target interaction content; obtaining the key content according to the output result of the semantic analysis model; correspondingly, performing quality inspection on the consultation service according to the target entity to obtain the quality inspection result of the consultation service, including: performing quality inspection on the consultation service for the quality inspection item according to the target entity to obtain the quality inspection result of the consultation service for the quality inspection item. Among them, the explanations of the same or corresponding terms as those in the above embodiments will not be repeated here.

[0097] See Figure 5 , the method of this embodiment may specifically include the following steps:

[0098] S310. In response to a consultation service quality inspection instruction, obtain the target interaction content between the consultation service provider and the consultation service acquirer, and obtain a deep learning model for implementing consultation service quality inspection.

[0099] S320. Obtain a semantic analysis model;

[0100] Among them, the semantic analysis model has been pre-learned with configuration cases, and the configuration cases are configured for preset quality inspection items, and the quality inspection items are used for quality inspection of the consultation services provided by the consultation service provider.

[0101] Among them, the quality inspection items can be understood as preset items for quality inspection of the consultation services provided by the consultation service provider, and in particular can be understood as items set by the application party using the consultation service quality inspection method described in the embodiments of the present invention according to its own personalized quality inspection requirements. The number of quality inspection items can be one or more, which is related to the actual situation and is not specifically limited here.

[0102] Combined with the application scenarios that the embodiments of the present invention may involve, optionally, from a broad category, the above quality inspection items may include, for example, process quality inspection items and content quality inspection items. On this basis, optionally, the process quality inspection items may be, for example, whether the medical record is written, whether a disease diagnosis is given, whether the disease diagnosis in the doctor-patient conversation is consistent with the disease diagnosis in the medical record, and whether there is a lack of treatment suggestions, etc.; further optionally, the content quality inspection items may be, for example, whether the disease diagnosis is correct and whether the treatment suggestion plan is correct, etc.

[0103] The semantic analysis model can be understood as a model for implementing semantic analysis functions. In the embodiments of the present invention, it should be noted that the semantic analysis model has pre-learned one or more configuration cases, and these one or more configuration cases are configured for quality inspection items. Combining with the subsequent steps, this means that the key content analyzed based on this semantic analysis model corresponds to the quality inspection items, which helps to obtain quality inspection results that match the quality inspection items.

[0104] Exemplarily, referring to Figure 6 , taking the medical consultation service as an example here, the configurator can give the following configuration cases according to personalized quality inspection items:

[0105] Input: I have had prostatitis for many years

[0106] Output: Prostatitis

[0107] In the semantic analysis model, the corresponding prompt information for this configuration case is as follows:

[0108] prompt: The following is a doctor-patient conversation. Please refer to the configuration case to identify Pat_Dis in the following conversation. Among them, the definition of Pat_Dis is the disease that the patient has been diagnosed with before the consultation.

[0109]

[0110] Specific doctor-patient conversation

[0111] That is, input this configuration case into the semantic analysis model so that the semantic analysis model learns this configuration case, and thus can extract key content such as the output from the target interaction content similar to the input.

[0112] In practical applications, optionally, the semantic analysis model and the deep learning model can be the same or different models, and no specific limitation is made here. Additionally, optionally, in order to improve the accuracy of semantic analysis, the semantic analysis model can be a large language model.

[0113] S330. Input the target interaction content into the semantic analysis model, and based on the semantic analysis model and the configuration case, perform semantic analysis on the target interaction content, and extract key content corresponding to the quality inspection items from the target interaction content.

[0114] Among them, after inputting the target interaction content into the semantic analysis model, the semantic analysis model can be used to perform semantic analysis on the target interaction content based on the configuration case, so as to extract key content that matches the quality inspection items corresponding to this configuration case from the target interaction content.

[0115] S340. Input the key content into the deep learning model, and retrieve in the target knowledge graph corresponding to the consulting service based on the key content according to the deep learning model to obtain the target entity that matches the key content.

[0116] S350. Perform quality inspection of the consulting service for the quality inspection items based on the target entity to obtain the quality inspection result of the consulting service for the quality inspection items.

[0117] Among them, since the key content matches the quality inspection item, after the key content is input into the deep learning model, the deep learning model can be used to perform quality inspection on the consulting service for the quality inspection item based on the key content, so as to obtain the quality inspection result of the consulting service for the quality inspection item.

[0118] In the technical solution of the embodiment of the present invention, the configurator can configure corresponding configuration cases according to the quality inspection requirements (i.e., quality inspection items) required by himself (i.e., personalized), and let the semantic analysis model learn these configuration cases. In this way, when the semantic analysis model performs semantic analysis on the target interaction content, the key content corresponding to the quality inspection item can be extracted, so that the deep learning model can perform quality inspection on the consulting service in terms of quality inspection items based on this key content. The above process does not require the development of rules or models, so various quality inspection requirements can be quickly responded to, which helps to promote the large-scale application of the quality inspection solution for consulting services.

[0119] An optional technical solution is that the configuration case includes the configured interaction content for the consulting service and the configured atomic content corresponding to the quality inspection item extracted from the configured interaction content;

[0120] Extracting the key content corresponding to the quality inspection item from the target interaction content includes:

[0121] Extracting the target atomic content corresponding to the quality inspection item from the target interaction content, and using the target atomic content as the key content.

[0122] Among them, the configuration case may include the configured interaction content and the configured atomic content. Specifically, the configured interaction content can be understood as the interaction content configured for the consulting service, that is, the content that may be generated during the interaction between the simulated consulting service provider and the consulting service acquirer. The configured atomic content can be understood as the content with the smallest meaning (i.e., cannot be further segmented) extracted from the configured interaction content, and further can be understood as the content that can be used to perform quality inspection on the consulting service in terms of quality inspection items. The number of the configured atomic content can be one or more, which is related to the actual situation and is not specifically limited here.

[0123] After learning the above configuration cases, the semantic analysis model can analyze the target interaction content to extract the target atomic content from the target interaction content, and use the extracted target atomic content as key content for application. The number of target atomic contents can be one or more, which is related to the actual situation and is not specifically limited here.

[0124] In the above technical solution, configuration is carried out in units of atoms. This fine-grained processing method further improves the accuracy rate of consultation service quality inspection.

[0125] Another alternative technical solution is that the deep learning model includes an intension quality inspection model, the quality inspection result includes the intension quality inspection result, and the quality inspection items include intension quality inspection items and process quality inspection items;

[0126] Obtain the quality inspection result of the consultation service for the quality inspection items, including:

[0127] Obtain the intension quality inspection result of the consultation service for the intension quality inspection items;

[0128] The above consultation service quality inspection method further includes:

[0129] Obtain a process quality inspection model, where the process quality inspection model has pre-learned a judgment logic, and the judgment logic is configured based on the process quality inspection items;

[0130] Input the key content into the process quality inspection model, so that through the process quality inspection model, based on the judgment logic, it is judged whether the key content meets the process quality inspection items;

[0131] According to the output result of the process quality inspection model, the process quality inspection result of the consultation service for the process quality inspection items can be obtained.

[0132] Among them, the quality inspection items described above can include intension quality inspection items for intension quality inspection and process quality inspection items for process quality inspection. Using the intension quality inspection model, based on any of the above technical solutions, the intension quality inspection result of the consultation service for the intension quality inspection items can be obtained. In addition, the process quality inspection result of the consultation service for the process quality inspection items can also be obtained by using the process quality inspection model based on the following steps.

[0133] Specifically, the process quality inspection model can be understood as a model for quality inspection of consulting services based on process quality inspection items. In the embodiments of the present invention, it should be noted that the process quality inspection model has been pre-trained with a judgment logic, which is configured based on the process quality inspection items and can be used to judge whether a certain content meets the process quality inspection items. Moreover, since the key content is extracted based on the process quality inspection items, after the key content is input into the process quality inspection model, the process quality inspection model can judge whether the key content meets the process quality inspection items based on the judgment logic, so as to obtain the process quality inspection result of the consulting service for the process quality inspection items.

[0134] In practical applications, optionally, the above-mentioned process quality inspection model and the content quality inspection model can be the same or different models, which are not specifically limited herein. Additionally, to improve the precision and recall rate of process quality inspection, the process quality inspection model can be a large language model.

[0135] The above technical solution realizes the process quality inspection of consulting services, and the mutual cooperation between process quality inspection and content quality inspection effectively ensures the comprehensive quality inspection of consulting services.

[0136] On this basis, optionally, the key content is represented by the target atomic content, and the judgment logic is also configured based on the candidate source type and the candidate content type, which are preset for the consulting service in advance;

[0137] Judging whether the key content meets the process quality inspection items based on the judgment logic includes:

[0138] Based on the target source type and the target content type of the target atomic content, as well as the judgment logic, judging whether the target atomic content meets the process quality inspection items, where the target source type belongs to the candidate source type and the target content type belongs to the candidate content type.

[0139] Among them, in this technical solution, one or more candidate source types and one or more candidate content types are preset for the consulting service in advance. The candidate source type can represent the source type of the atomic content that may be involved in the consulting service, that is, the target source type of the target atomic content and the configuration source type of the configured atomic content both come from the one or more candidate source types; the candidate content type can represent the content type of the atomic content that may be involved in the consulting service, that is, the target content type of the target atomic content and the configuration content type of the configured atomic content both come from the one or more candidate content types.

[0140] In the case where the configuration case includes configuring atomic content, that is, the key content is represented by the target atomic content, the judgment logic can be configured based on the process quality inspection items, as well as the candidate source type and the candidate content type. On this basis, the process quality inspection model can judge whether the target atomic content can meet the process quality inspection items based on the target source type, the target content type, and the judgment logic.

[0141] The above technical solution configures the judgment logic in units of atoms, ensuring the accuracy rate of process quality inspection.

[0142] To better understand the above-mentioned atomic judgment logic, an exemplary description is given based on a specific example. Exemplarily, here taking the medical consultation service as an example, in the medical consultation scenario, 5 candidate source types as shown in Table 2, namely patients, doctors, medical records, prescriptions, and others, are set, and 12 candidate content types as shown in Table 3, namely diseases, traditional Chinese medicines, Western medicines, nutritional health care, symptoms, severe cases, nursing, surgeries, examinations, treatments, allergy histories, and past histories, are set. According to the above description, the configured source type of the configured atomic content comes from these 5 candidate source types, and the configured content type comes from these 12 candidate content types. On this basis, by combining these candidate source types and candidate content types, 5 * 12 types of combinations can be obtained.

[0143] Table 2 Candidate Source Types and Their English Abbreviations

[0144] Candidate source type English abbreviation Patient Pat Doctor Doc Medical record Rec Prescription Pre Others Oth

[0145] Table 3 Candidate Content Types and Their English Abbreviations

[0146] Candidate content type English abbreviation Candidate content type English abbreviation Candidate content type English abbreviation Disease Dis Traditional Chinese medicine Cmed Western medicine Med Nutrition and health care Nut Symptom Sym Severe illness Ser Nursing Nur Surgery Sur Examination Exa Treatment Tre Allergy history All Previous history Pre

[0147] Based on Tables 2 and 3, Table 4 shows examples of configured interaction content, configured atomic content, and the type combinations corresponding to the configured atomic content.

[0148] Table 4 Examples of Configured Interaction Content, Configured Atomic Content, and Their Corresponding Type Combinations

[0149] Configure interactive content Configure atomic content English abbreviation Patient: I was diagnosed with prostatitis offline Prostatitis Pat_Dis Doctor: Based on your symptoms, it is considered leukemia Leukemia Doc_Dis Patient: I have taken amoxicillin capsules before Amoxicillin capsules Pat_Med Medical record: Allergy history: Penicillin allergy Penicillin Rec_All

[0150] On this basis, exemplarily, see Figure 7 , assuming that the definition of the process quality inspection item for judging whether there is an "allergy history inquiry" in the doctor-patient conversation is the situation where the doctor has a medication recommendation and the patient's allergy history is not clearly stated. Therefore, there are 3 type combinations, namely Doc_Med, Doc_Cmed, and Pat_All, in the judgment logic, as well as logical relationships such as "yes", "no", "and", and "or" between them, and the output results for meeting the requirements (i.e., meeting the process quality inspection items) and not meeting the requirements are configured.

[0151] Specifically, the configuration personnel's judgment logic based on the process quality inspection items is as follows:

[0152] Please determine whether the following situations exist in the doctor-patient conversation

[0153] There is (Doc_Med) or (Doc_Cmed)

[0154] And

[0155] There is no (Pat_All)

[0156] If satisfied, output (No inquiry about allergy history)

[0157] If not satisfied, output (Full score for allergy history).

[0158] The corresponding prompt information of this judgment logic in the process quality inspection model is as follows:

[0159] prompt: Please determine whether there is a situation where Doc_Med or Doc_Cmed exists in the doctor-patient conversation and Pat_All does not exist? If satisfied, output "No inquiry about allergy history"; if not satisfied, output "Full score for allergy history".

[0160] That is, input this judgment logic into the process quality inspection model so that the process quality inspection model can learn this judgment logic, and thus, based on this judgment logic, combined with the target atomic content, conduct quality inspection on the process quality inspection items.

[0161] To better understand the above-mentioned various technical solutions as a whole, the following combines specific examples to illustrate them exemplarily. Exemplarily, see Figure 8 , here taking the semantic analysis model, the process quality inspection model, and the connotation quality inspection model as integrated in a large language model (hereinafter referred to as the large model) as an example. When the in-clinic quality inspection is triggered, extract the target interaction content during the doctor-patient conversation process, and input the target interaction content into the large model, so that through the large model, based on the pre-learned configuration cases, extract one or more target atomic contents from the target interaction content, and output the one or more target atomic contents.

[0162] Further, through traditional rule-based splitting, it is divided into two branches: the process quality inspection branch and the content quality inspection branch. In the process quality inspection branch, the target atomic content is input into a large model, so that through the large model, based on the pre-learned judgment logic, combined with the target source type and target content type corresponding to each of the one or more target atomic contents, it is determined whether the medical consultation service meets the process quality inspection items judged by the judgment logic, and the judged process quality inspection results are output. And, in the content quality inspection branch, the one or more target atomic contents are input into a large model, so that through the large model, based on the pre-learned modular rules, a search is performed in the target knowledge graph to determine whether the medical consultation service meets the content quality inspection items corresponding to the modular rules, and the judged content quality inspection results are output. Among them, the modular rules can represent combining at least some of the one or more target atomic contents for different content quality inspection items to perform a search in the target knowledge graph. According to the process quality inspection results and the content quality inspection results, the in-consultation quality inspection results of the medical consultation service are obtained.

[0163] The above example ensures the accuracy rate of the quality inspection of medical consultation services and is also convenient for popularization and application.

[0164] Figure 9 The following is the structural block diagram of the consultation service quality inspection device provided by the embodiment of the present invention. This device is used to execute the consultation service quality inspection method provided in any of the above embodiments. This device and the consultation service quality inspection methods of the above embodiments belong to the same inventive concept. For the details not described in detail in the embodiment of the consultation service quality inspection device, reference can be made to the embodiments of the above consultation service quality inspection methods. Refer to Figure 9 The device may specifically include: a deep learning model acquisition module 410, a target entity acquisition module 420, and a quality inspection result acquisition module 430.

[0165] Among them, the deep learning model acquisition module 410 is used to, in response to a consultation service quality inspection instruction, acquire the target interaction content between the consultation service provider and the consultation service acquirer, and acquire a deep learning model for implementing the consultation service quality inspection;

[0166] The target entity acquisition module 420 is used to input the target interaction content into the deep learning model, so as to, according to the deep learning model, based on the target interaction content, perform a search in the target knowledge graph corresponding to the consultation service provided by the consultation service provider to obtain a target entity that matches the target interaction content;

[0167] The quality inspection result acquisition module 430 is used to, according to the target entity, perform quality inspection on the consultation service to obtain the quality inspection result of the consultation service.

[0168] Optionally, the number of target entities is at least two, and the quality inspection result obtaining module 430 includes:

[0169] A consulting service quality inspection unit for quality inspecting the consulting service according to the matching degree between at least two target entities and the preset matching degree ranges for different quality inspection results.

[0170] On this basis, optionally, the above-mentioned consulting service quality inspection device further includes:

[0171] A matching threshold obtaining module for obtaining the preset matching threshold for each target entity pair with a preset matching threshold among at least two target entities.

[0172] A matching degree determining module for summing up the matching thresholds corresponding to each target entity pair and determining the matching degree between at least two target entities according to the obtained summation result.

[0173] On this basis, optionally, the consulting service is a medical consulting service, the target interaction content at least includes the disease diagnosis provided by the consulting service provider and the symptom description provided by the consulting service acquirer, and at least two target entities include a disease entity matching the disease diagnosis and at least two symptom entities matching the symptom description; correspondingly, the target entity obtaining module 420 includes:

[0174] A symptom description extraction unit for extracting the disease diagnosis and symptom description from the target interaction content.

[0175] A symptom entity obtaining unit for retrieving the disease entity based on the disease diagnosis in the target knowledge graph corresponding to the disease diagnosis, and retrieving at least two symptom entities based on the symptom description in the target knowledge graph.

[0176] Correspondingly, the matching threshold obtaining module is specifically used for:

[0177] For each target entity pair among at least two target entity pairs composed of at least two symptom entities and a disease entity, obtaining the preset matching threshold for the target entity pair, where the target entity pair includes the disease entity and at least one symptom entity among at least two symptom entities.

[0178] Optionally, the above-mentioned consulting service quality inspection device further includes:

[0179] A key content obtaining module for performing semantic analysis on the target interaction content to obtain the key content.

[0180] Correspondingly, the target entity obtaining module 420 includes:

[0181] The target entity obtaining sub-module is used to input key content into a deep learning model, and based on the deep learning model and the key content, retrieve in a target knowledge graph corresponding to the consulting service provided by the consulting service provider to obtain a target entity that matches the key content.

[0182] On this basis, optionally, the key content obtaining module includes:

[0183] The semantic analysis model acquisition sub-module is used to acquire a semantic analysis model, where the semantic analysis model has pre-learned configuration cases, and the configuration cases are configured for preset quality inspection items, and the quality inspection items are used for quality inspection of consulting services;

[0184] The key content extraction sub-module is used to input the target interaction content into the semantic analysis model, and through the semantic analysis model, based on the configuration cases, perform semantic analysis on the target interaction content, and extract key content corresponding to the quality inspection items from the target interaction content;

[0185] The key content obtaining sub-module is used to obtain key content according to the output result of the semantic analysis model;

[0186] Correspondingly, the quality inspection result obtaining module 430 includes:

[0187] The quality inspection result obtaining unit is used to perform quality inspection of the consulting service for the quality inspection items according to the target entity, and obtain the quality inspection result of the consulting service for the quality inspection items.

[0188] On this basis, optionally, the configuration cases include configured interaction content for the consulting service and configured atomic content corresponding to the quality inspection items extracted from the configured interaction content;

[0189] The key content extraction sub-module includes:

[0190] The key content obtaining unit is used to extract target atomic content corresponding to the quality inspection items from the target interaction content and use the target atomic content as the key content.

[0191] Another option is that the deep learning model may include an intension quality inspection model, the quality inspection result includes an intension quality inspection result, the quality inspection items include intension quality inspection items and process quality inspection items, and the quality inspection result obtaining unit includes:

[0192] The intension quality inspection result obtaining sub-unit can be used to obtain the intension quality inspection result of the consulting service for the intension quality inspection items;

[0193] The above-mentioned consulting service quality inspection device further includes:

[0194] A process quality inspection model acquisition module, which is used to acquire a process quality inspection model. The process quality inspection model has pre-learned judgment logic, and the judgment logic is configured based on process quality inspection items.

[0195] A key content input module, which is used to input key content into the process quality inspection model, so as to judge whether the key content meets the process quality inspection items based on the judgment logic through the process quality inspection model.

[0196] A process quality inspection result acquisition module, which is used to obtain the process quality inspection result of the consulting service for the process quality inspection items according to the output result of the process quality inspection model.

[0197] On this basis, optionally, the key content is represented by target atomic content, and the judgment logic is also configured based on the candidate source type and candidate content type, and the candidate source type and candidate content type are preset for the consulting service.

[0198] The key content input module includes:

[0199] A process quality inspection item judgment unit, which is used to judge whether the target atomic content meets the process quality inspection items based on the target source type and target content type of the target atomic content and the judgment logic, where the target source type belongs to the candidate source type and the target content type belongs to the candidate content type.

[0200] Optionally, the above-mentioned consulting service quality inspection device further includes:

[0201] An early warning information sending module, which is used to send early warning information for warning quality problems to the user terminal used by the consulting service provider during the interaction between the consulting service provider and the consulting service acquirer when the quality inspection result indicates that there are quality problems with the consulting service.

[0202] Optionally, the consulting service includes medical consulting services and / or the deep learning model includes a large language model.

[0203] The consultation service quality inspection device provided by the embodiments of the present invention, through the deep learning model acquisition module, in response to the consultation service quality inspection instruction, acquires the target interaction content between the consultation service provider and the consultation service acquirer, and acquires the deep learning model for implementing the consultation service quality inspection; through the target entity obtaining module, inputs the target interaction content into the deep learning model, and based on the deep learning model, retrieves in the target knowledge graph corresponding to the consultation service provided by the consultation service provider based on the target interaction content to obtain the target entity matching the target interaction content; through the quality inspection result obtaining module, obtains the quality inspection result of the consultation service according to the output result of the deep learning model. The above device, by combining the deep learning model with the target knowledge graph, reduces the risk that the deep learning model fails to obtain sufficient vertical domain knowledge (i.e., professional knowledge) corresponding to the consultation service through the target knowledge graph retrieval, thereby improving the accuracy and recall rate of the consultation service quality inspection.

[0204] The consultation service quality inspection device provided by the embodiments of the present invention can execute the consultation service quality inspection method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0205] It should be noted that in the embodiments of the above consultation service quality inspection device, the various units and modules included are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0206] Figure 10 FIG. 10 shows a schematic structural diagram of an electronic device 10 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described herein and / or claimed.

[0207] As Figure 10As shown, the electronic device 10 includes at least one processor 11 and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0208] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0209] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the consultation service quality inspection method.

[0210] In some embodiments, the consultation service quality inspection method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the consultation service quality inspection method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the consultation service quality inspection method in any other appropriate way (e.g., by means of firmware).

[0211] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0212] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0213] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0214] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0215] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0216] The computing system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0217] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0218] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A consulting service quality inspection method, It is characterized in that include: In response to the consulting service quality inspection instruction, obtaining target interaction content between the consulting service provider and the consulting service acquirer, and obtaining a deep learning model for implementing the consulting service quality inspection; Inputting the target interaction content into the deep learning model, so as to search, according to the deep learning model and based on the target interaction content, in a target knowledge graph corresponding to the consulting service provided by the consulting service provider, to obtain a target entity matching the target interaction content; According to the target entity, the consulting service is quality inspected to obtain a quality inspection result of the consulting service.

2. The method according to claim 1, It is characterized in that The number of the target entities is at least two, and the quality inspection of the consulting service according to the target entities includes: The consulting service is quality checked according to the matching degree between the at least two target entities and the matching degree ranges preset for different quality check results.

3. The method according to claim 2, It is characterized in that Also includes: For each target entity pair with a preset matching threshold among the at least two target entities, obtaining a preset matching threshold for the target entity pair; The matching thresholds corresponding to each pair of target entities are summed, and the matching degree between the at least two target entities is determined according to the summed result.

4. The method according to claim 3, It is characterized in that The consulting service includes a medical consulting service, the target interaction content includes at least a disease diagnosis provided by the consulting service provider and a symptom description provided by the consulting service acquirer, and the at least two target entities include a disease entity matching the disease diagnosis and at least two symptom entities matching the symptom description; Accordingly, based on the target interaction content, searching in the target knowledge graph corresponding to the consulting service provided by the consulting service provider to obtain the target entity matching the target interaction content includes: extracting the disease diagnosis and the symptom description from the target interaction content; For a target knowledge graph corresponding to the disease diagnosis, searching the target knowledge graph based on the disease diagnosis to obtain the disease entity, and searching the target knowledge graph based on the symptom description to obtain the at least two symptom entities; Accordingly, for each target entity pair with a preset matching threshold among the at least two target entities, obtaining the preset matching threshold for the target entity pair includes: For each target entity pair of at least two target entity pairs consisting of the at least two symptom entities and the disease entity, obtaining a matching threshold preset for the target entity pair; Wherein, the target entity pair includes the disease entity and at least one symptom entity of the at least two symptom entities.

5. The method according to claim 1, It is characterized in that Also includes: Performing semantic analysis on the target interactive content to obtain key content; Accordingly, the target interaction content is input into the deep learning model, and a target knowledge graph corresponding to the consulting service provided by the consulting service provider is searched based on the target interaction content according to the deep learning model to obtain a target entity matching the target interaction content, including: The key content is input into the deep learning model, so as to search the target knowledge graph corresponding to the consulting service provided by the consulting service provider based on the key content according to the deep learning model to obtain the target entity matching the key content.

6. The method according to claim 5, It is characterized in that The semantic analysis of the target interactive content to obtain key content includes: Acquire a semantic analysis model, wherein the semantic analysis model has pre-learned configuration cases, the configuration cases are configured for preset quality inspection items, and the quality inspection items are used to quality inspect the consulting service; Inputting the target interaction content into the semantic analysis model, so as to perform semantic analysis on the target interaction content based on the configuration case through the semantic analysis model, and extracting key content corresponding to the quality inspection item from the target interaction content; Obtaining the key content according to the output result of the semantic analysis model; Accordingly, the quality inspection of the consulting service is performed according to the target entity to obtain the quality inspection result of the consulting service, including: According to the target entity, a quality inspection of the consulting service is performed with respect to the quality inspection item, and a quality inspection result of the consulting service with respect to the quality inspection item is obtained.

7. The method according to claim 6, It is characterized in that The configuration case includes configuration interaction content for the consulting service configuration, and configuration atomic content corresponding to the quality inspection item extracted from the configuration interaction content; The extracting key content corresponding to the quality inspection item from the target interactive content includes: Target atomic content corresponding to the quality inspection item is extracted from the target interactive content, and the target atomic content is used as key content.

8. The method according to claim 6 or 7, It is characterized in that The deep learning model includes a connotation quality inspection model, the quality inspection result includes a connotation quality inspection result, the quality inspection items include connotation quality inspection items and process quality inspection items, and the quality inspection results of the consulting service for the quality inspection items include: Obtaining the content quality inspection results of the consulting service for the content quality inspection items; The method further comprises: Acquire a process quality inspection model, wherein the process quality inspection model is pre-learned with judgment logic, and the judgment logic is obtained based on the process quality inspection item configuration; Inputting the key content into the process quality inspection model, so as to determine whether the key content meets the process quality inspection items based on the judgment logic through the process quality inspection model; According to the output result of the process quality inspection model, the process quality inspection result of the consulting service for the process quality inspection item is obtained.

9. The method according to claim 8, It is characterized in that The key content is represented by the target atomic content, and the judgment logic is also configured based on the candidate source type and the candidate content type, and the candidate source type and the candidate content type are pre-set for the consulting service; The determining, based on the determination logic, whether the key content satisfies the process quality inspection item includes: Based on the target source type and target content type of the target atomic content and the judgment logic, determine whether the target atomic content meets the process quality inspection item, wherein the target source type belongs to the candidate source type and the target content type belongs to the candidate content type.

10. The method according to claim 1, It is characterized in that Also includes: During the interaction between the consulting service provider and the consulting service acquirer, if the quality inspection result indicates that the consulting service has quality problems, warning information for warning of the quality problems will be sent to the user terminal used by the consulting service provider.

11. The method according to claim 1, It is characterized in that The consulting service includes a medical consulting service, and / or the deep learning model includes a large language model.

12. A consulting service quality inspection device, It is characterized in that include: A deep learning model acquisition module, used to obtain the target interaction content between the consulting service provider and the consulting service acquirer in response to the consulting service quality inspection instruction, and to obtain a deep learning model for implementing the consulting service quality inspection; A target entity obtaining module, used for inputting the target interaction content into the deep learning model, so as to search in a target knowledge graph corresponding to the consulting service provided by the consulting service provider according to the deep learning model and based on the target interaction content, and obtain a target entity matching the target interaction content; The quality inspection result obtaining module is used to perform a quality inspection on the consulting service according to the target entity to obtain a quality inspection result of the consulting service.

13. An electronic device, It is characterized in that include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the consulting service quality inspection method according to any one of claims 1 to 11.

14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the consulting service quality inspection method as described in any one of claims 1-11 when executed.

Images