Online consultation process quality detection method, online training method and related device

Abstract

The application discloses an online consultation process-based network training method, an online consultation process quality detection method, an online consultation process-based network training device, an online consultation process quality detection device, a first electronic device, a second electronic device and a storage medium. The network training method comprises the following steps: determining a training data set; the training data set comprises consultation dialogue information corresponding to a plurality of online consultation processes and quality detection results; determining a first network and a second network; the first network is used for determining a sub-consultation process included in an online consultation process; the second network is used for determining a necessary sub-consultation process corresponding to the online consultation process; performing joint learning on the first network and the second network by using the training data set, so as to obtain a third network; and the third network is used for determining whether the online consultation process includes the necessary sub-consultation process.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a quality detection method, network training method and related apparatus for an online consultation process. Background Technology

[0002] With strong government support, e-health services such as online medical care and online consultations have grown rapidly in recent years. Unlike traditional offline hospital treatments, online healthcare, especially part-time online care, faces challenges due to the large volume of consultations and the lack of standardized quality control systems. Consequently, the risks associated with online consultations cannot be detected and managed in a timely manner. Therefore, how to quickly and accurately assess the quality of online consultation processes has become an urgent problem to be solved. Summary of the Invention

[0003] To address the related technical issues, embodiments of this application provide a quality detection method, a network training method, and related apparatus for online consultation processes.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a network training method based on an online consultation process, including:

[0006] A training dataset is determined; the training dataset contains consultation dialogue information and quality detection results corresponding to multiple online consultation processes; the consultation dialogue information contains online dialogue statements between doctors and patients in the corresponding online consultation process; the quality detection results indicate whether the corresponding online consultation process contains necessary sub-consultation processes.

[0007] A first network and a second network are determined; the first network is used to determine the sub-consultation processes included in the online consultation process; the second network is used to determine the necessary sub-consultation processes corresponding to the online consultation process.

[0008] Using the training dataset, the first network and the second network are jointly learned to obtain a third network; the third network is used to determine whether the online consultation process includes necessary sub-consultation processes.

[0009] In the above scheme, determining the first network includes:

[0010] The first network is determined based on the first subnetwork, the second subnetwork, the third subnetwork, and the fourth subnetwork; wherein,

[0011] The first sub-network is used to digitize each character of each statement contained in the consultation dialogue information;

[0012] The second sub-network is used to perform semantic expression on each character of each statement contained in the consultation dialogue information based on the output data of the first sub-network;

[0013] The third sub-network is used to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network, and to classify the statements contained in the consultation dialogue information based on the semantic expression of each statement.

[0014] The fourth sub-network is used to determine the sub-consultation processes included in the corresponding online consultation process based on the output data of the third sub-network.

[0015] The method in the above scheme further includes:

[0016] The dictionary matrix maintained by the second sub-network is trained; the second sub-network uses the dictionary matrix to perform semantic representation of each character of each statement contained in the consultation dialogue information.

[0017] In the above scheme, determining the second network includes:

[0018] The second network is determined based on the first subnetwork, the second subnetwork, the fifth subnetwork, and the sixth subnetwork; wherein,

[0019] The first sub-network is used to digitize each character of each statement contained in the consultation dialogue information;

[0020] The second sub-network is used to perform semantic expression on each character of each statement contained in the consultation dialogue information based on the output data of the first sub-network;

[0021] The fifth sub-network is used to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network.

[0022] The sixth sub-network is used to determine the necessary sub-consultation processes corresponding to the corresponding online consultation process based on the output data of the fifth sub-network.

[0023] In the above scheme, when jointly learning the first network and the second network, the method further includes:

[0024] A seventh sub-network is trained using the training dataset, the output data of the first network, and the output data of the second network; the seventh sub-network is used to determine whether the corresponding online consultation process includes necessary sub-consultation processes based on the output data of the first network and the output data of the second network; the third network includes the first network, the second network, and the seventh sub-network.

[0025] This application also provides a quality inspection method for an online consultation process, including:

[0026] Obtain the online consultation dialogue information corresponding to the online consultation process to be tested; the consultation dialogue information includes the online dialogue statements between the doctor and the patient in the online consultation process to be tested;

[0027] Based on the consultation dialogue information, a third network is used to determine whether the online consultation process to be tested contains necessary sub-consultation processes, and to obtain the first quality detection result corresponding to the online consultation process to be tested; the third network is obtained using any of the above network training methods.

[0028] The method in the above scheme further includes:

[0029] If the first quality inspection result indicates that the online consultation process to be inspected contains necessary sub-consultation processes, at least one preset rule is used to determine whether the online consultation process to be inspected contains necessary sub-consultation processes, and a second quality inspection result corresponding to the online consultation process to be inspected is obtained; each preset rule represents the characteristics of a necessary sub-consultation process.

[0030] The method in the above scheme further includes:

[0031] If the first quality inspection result or the second quality inspection result indicates that the online consultation process to be inspected lacks necessary sub-consultation processes, at least one electronic communication address corresponding to the online consultation process to be inspected shall be obtained.

[0032] A warning is issued based on at least one of the electronic communication addresses.

[0033] This application also provides a network training device based on an online consultation process, including:

[0034] The first processing unit is used to determine the training dataset; the training dataset contains consultation dialogue information and quality detection results corresponding to multiple online consultation processes; the consultation dialogue information contains online dialogue statements between doctors and patients in the corresponding online consultation process; the quality detection results indicate whether the corresponding online consultation process contains necessary sub-consultation processes.

[0035] The second processing unit is used to determine the first network and the second network; the first network is used to determine the sub-consultation processes included in the online consultation process; the second network is used to determine the necessary sub-consultation processes corresponding to the online consultation process.

[0036] The third processing unit is used to jointly learn the first network and the second network using the training dataset to obtain a third network; the third network is used to determine whether the online consultation process includes necessary sub-consultation processes.

[0037] This application embodiment also provides a quality detection device for an online consultation process, including:

[0038] The fourth processing unit is used to acquire the online consultation dialogue information corresponding to the online consultation process to be detected; the consultation dialogue information includes online dialogue statements between the doctor and the patient in the online consultation process to be detected;

[0039] The fifth processing unit is used to determine, based on the consultation dialogue information and using the third network, whether the online consultation process to be detected contains necessary sub-consultation processes, and to obtain the first quality detection result corresponding to the online consultation process to be detected; the third network is obtained using any of the above network training methods.

[0040] This application also provides a first electronic device, including: a first processor and a first memory for storing a computer program capable of running on the processor.

[0041] Wherein, when the first processor is used to run the computer program, it executes the steps of any of the above-described network training methods.

[0042] This application also provides a second electronic device, including: a second processor and a second memory for storing a computer program capable of running on the processor.

[0043] Wherein, the second processor is used to execute the steps of the quality detection method for any of the above-mentioned online consultation processes when running the computer program.

[0044] This application embodiment also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the above-described network training methods, or the steps of any of the above-described online consultation process quality detection methods.

[0045] The online consultation process quality detection method, network training method, and related apparatus provided in this application embodiment determine a training dataset; the training dataset includes consultation dialogue information and quality detection results corresponding to multiple online consultation processes; the consultation dialogue information includes online dialogue statements between doctors and patients in the corresponding online consultation process; the quality detection results characterize whether the corresponding online consultation process includes necessary sub-consultation processes; a first network and a second network are determined; the first network is used to determine the sub-consultation processes included in the online consultation process; the second network is used to determine the necessary sub-consultation processes corresponding to the online consultation process; based on the training dataset, the first network and the second network are jointly learned to obtain a third network; the third network is used to determine whether the online consultation process includes necessary sub-consultation processes. The solution in this application embodiment determines a first network for identifying sub-consultation processes included in an online consultation process and a second network for identifying necessary sub-consultation processes corresponding to the online consultation process. Using a training dataset containing online dialogue statements between doctors and patients in the online consultation process and quality inspection results representing whether the online consultation process includes necessary sub-consultation processes, the first and second networks are jointly learned to obtain a third network for determining whether the online consultation process includes necessary sub-consultation processes. Thus, using the third network to perform quality inspection on the online consultation process can quickly and accurately determine whether the online consultation process includes necessary sub-consultation processes, thereby enabling timely alerts in the absence of necessary sub-consultation processes, i.e., timely control of medical risks, and ultimately reducing the probability of medical accidents. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the network training method based on the online consultation process in an embodiment of this application;

[0047] Figure 2 This is a flowchart illustrating the quality inspection method for the online consultation process in an embodiment of this application.

[0048] Figure 3 This is a schematic diagram of the structure of the online medical quality control system in an application embodiment of this application;

[0049] Figure 4 This is a schematic diagram of the workflow of the process quality control module in an application embodiment of this application;

[0050] Figure 5 This is a schematic diagram of the process discrimination model in the application embodiment of this application;

[0051] Figure 6 This is a schematic diagram of the structure of the process necessity determination model for the application embodiments of this application;

[0052] Figure 7This is a schematic diagram of the joint learning model in the application embodiments of this application;

[0053] Figure 8 This is a schematic diagram of the structure of a network training device based on an online consultation process, as described in an embodiment of this application.

[0054] Figure 9 This is a schematic diagram of the structure of the quality inspection device for the online consultation process in an embodiment of this application;

[0055] Figure 10 This is a schematic diagram of the structure of the first electronic device according to an embodiment of this application;

[0056] Figure 11 This is a schematic diagram of the structure of the second electronic device according to an embodiment of this application. Detailed Implementation

[0057] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0058] In related technologies, there are traditional and automated quality control schemes for the consultation process. With traditional quality control schemes, after the consultation process is completed, doctors evaluate each other's work, or professional quality inspectors review the consultation dialogues and medical records. Automated quality control schemes can include offline and online methods. Offline schemes typically employ a case-based approach, extracting the medical records written by doctors after the consultation to assess the accuracy of the diagnosis and the rationality of the treatment, or to assess the appropriateness of prescribed medications, tests, and examinations, thus achieving quality control. Online quality control schemes can use knowledge graphs, deep learning, and other methods to determine the correctness of the final diagnosis and the consultation process based on the doctor's written medical records after the consultation, thereby achieving quality control.

[0059] However, the quality control solutions in related technologies have the following problems:

[0060] 1) Traditional quality control methods require manual quality control, which necessitates a significant investment of human resources. Furthermore, when dealing with a large volume of consultations, quality control can only be achieved through sampling, failing to comprehensively cover all consultations. Additionally, traditional quality control methods cannot detect problems in consultations in real time, resulting in a significant lag in problem identification. By the time serious consultation errors (such as prescribing the wrong medication or making an incorrect diagnosis) are discovered, a considerable amount of time may have passed since the consultation occurred.

[0061] 2) When using an automated quality control solution, the quality control of the consultation is based on the medical records written by the doctor. However, it is impossible to control the data during the doctor-patient dialogue process, that is, it is impossible to control the quality of the consultation in the specific consultation process.

[0062] Based on this, in various embodiments of this application, a first network is determined for identifying sub-consultation processes included in an online consultation process, and a second network is determined for identifying necessary sub-consultation processes corresponding to the online consultation process. A training dataset containing online dialogue statements between doctors and patients in the online consultation process and quality inspection results representing whether the online consultation process includes necessary sub-consultation processes is used to jointly learn the first and second networks, resulting in a third network for determining whether the online consultation process includes necessary sub-consultation processes. Thus, using the third network to perform quality inspection on the online consultation process can quickly and accurately determine whether the online consultation process includes necessary sub-consultation processes, thereby enabling timely alerts in the absence of necessary sub-consultation processes, i.e., timely control of medical risks, and ultimately reducing the probability of medical accidents.

[0063] This application provides a network training method based on an online consultation process, applied to a first electronic device (such as a server), such as... Figure 1 As shown, the method includes:

[0064] Step 101: Determine the training dataset;

[0065] Here, the training dataset contains consultation dialogue information and quality detection results corresponding to multiple online consultation processes; the consultation dialogue information contains online dialogue statements between doctors and patients in the corresponding online consultation process; the quality detection results indicate whether the corresponding online consultation process contains necessary sub-consultation processes;

[0066] Step 102: Determine the first network and the second network;

[0067] Here, the first network is used to determine the sub-consultation processes included in the online consultation process; the second network is used to determine the necessary sub-consultation processes corresponding to the online consultation process.

[0068] Step 103: Using the training dataset, jointly learn the first network and the second network to obtain a third network; the third network is used to determine whether the online consultation process includes necessary sub-consultation processes.

[0069] In step 101, in practical applications, consultation dialogue information between doctors and patients can be collected from online internet hospitals, and the quality detection results corresponding to the online consultation process can be determined through manual methods, knowledge graphs, deep learning, etc., to obtain a training dataset.

[0070] In practical applications, the consultation dialogue information contains online dialogue statements between doctors and patients in the corresponding online consultation process. It can be understood that the statements contained in the consultation dialogue information are associated with role information, that is, it is possible to distinguish between the doctor's statements and the patient's statements.

[0071] In practical applications, the training dataset may also include basic patient information (such as age, gender, allergy history, etc.) and consultation results (such as whether a prescription was issued) for each online consultation process.

[0072] In step 102, in practical application, the function or role of the statements contained in the consultation dialogue information can be understood as a sub-consultation process. For example, if the consultation dialogue information contains the statements "Doctor: Where does it hurt?" and "Patient: Stomach ache, ate mango, mango allergy," since the function of "Doctor: Where does it hurt?" is to inquire about the patient's symptoms, and the function of "Patient: Stomach ache, ate mango, mango allergy" is to describe the symptoms and allergy history, it can be understood that the online consultation process includes a sub-consultation process that inquires about the patient's symptoms and allergy history.

[0073] In practical applications, a necessary sub-diagnosis process may include at least one of the following:

[0074] The sub-diagnosis process involves asking patients about their symptoms, duration of symptoms, severity of symptoms, and whether there are any other accompanying symptoms.

[0075] Sub-consultation process for inquiring about patients' allergy history, medical history, pregnancy and other special circumstances;

[0076] The sub-consultation process for informing patients about their illness;

[0077] The consultation process involves informing patients about the usage and dosage of medications.

[0078] In practical applications, determining the first network and the second network can be understood as constructing the first network and the second network, that is, determining the network architecture of the first network and the second network.

[0079] Based on this, in one embodiment, determining the first network may include:

[0080] The first network is determined based on the first subnetwork, the second subnetwork, the third subnetwork, and the fourth subnetwork; wherein,

[0081] The first sub-network is used to digitize each character of each statement contained in the consultation dialogue information;

[0082] The second sub-network is used to perform semantic expression on each character of each statement contained in the consultation dialogue information based on the output data of the first sub-network;

[0083] The third sub-network is used to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network, and to classify the statements contained in the consultation dialogue information based on the semantic expression of each statement.

[0084] The fourth sub-network is used to determine the sub-consultation processes included in the corresponding online consultation process based on the output data of the third sub-network.

[0085] Here, the first sub-network can convert each character of each statement in the consultation dialogue information into a numerical identifier (ID), that is, convert each character into a character ID, and obtain a two-dimensional vector corresponding to each statement; the second sub-network can use its own maintained dictionary matrix to perform character embedding processing on each character ID of each two-dimensional vector, that is, map each character ID to a one-dimensional vector; the third sub-network can perform multi-layer, multi-scale convolution operations on the output data of the second sub-network to obtain the classification type (i.e., sub-consultation process) corresponding to each statement; the third sub-network can be implemented based on text classification models such as TextCNN (TextConvolutional Neural Network); the fourth sub-network can multiply the vector output by the third sub-network with its own maintained doctor-patient role matrix, filter out (i.e., block) the patient's statements, and then take the maximum value of each classification type (also called classification category) corresponding to each statement to obtain the sub-consultation process corresponding to each statement, in other words, obtain the sub-consultation process contained in the corresponding online consultation process.

[0086] In one embodiment, the method may further include:

[0087] The dictionary matrix maintained by the second sub-network is trained; the second sub-network uses the dictionary matrix to perform semantic representation of each character of each statement contained in the consultation dialogue information.

[0088] Here, the dictionary matrix maintained by the second sub-network is learnable, which can improve the semantic expression of each character of each statement contained in the consultation dialogue information, that is, improve the accuracy of semantic expression.

[0089] In one embodiment, determining the second network may include:

[0090] Based on the first subnetwork, the second subnetwork, the fifth subnetwork, and the sixth subnetwork, the second network is determined; wherein,

[0091] The fifth sub-network is used to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network.

[0092] The sixth sub-network is used to determine the necessary sub-consultation processes corresponding to the corresponding online consultation process based on the output data of the fifth sub-network.

[0093] Here, the fifth sub-network can abstract the semantic expression of each statement into a feature represented by a vector based on the output data of the second sub-network; the sixth sub-network can determine the necessary sub-consultation process corresponding to the corresponding online consultation process by determining the relationship between the feature vectors corresponding to each statement.

[0094] In one embodiment, when jointly learning the first network and the second network, the method may further include:

[0095] A seventh sub-network is trained using the training dataset, the output data of the first network, and the output data of the second network; the seventh sub-network is used to determine whether the corresponding online consultation process includes necessary sub-consultation processes based on the output data of the first network and the output data of the second network; the third network includes the first network, the second network, and the seventh sub-network.

[0096] In practical applications, after the third network is trained, it can be used to perform quality checks on the online consultation process. This allows for quick and accurate determination of whether the online consultation process includes necessary sub-consultation processes, enabling timely alerts in the absence of necessary sub-consultation processes. This timely control of medical risks can reduce the probability of medical accidents.

[0097] Based on this, embodiments of this application also provide a quality detection method for an online consultation process, applied to a second electronic device (such as a server), such as... Figure 2 As shown, the method includes:

[0098] Step 201: Obtain the online consultation dialogue information corresponding to the online consultation process to be tested;

[0099] Here, the consultation dialogue information includes online dialogue statements between the doctor and the patient in the online consultation process to be detected;

[0100] Step 202: Based on the consultation dialogue information, use a third network to determine whether the online consultation process to be tested contains necessary sub-consultation processes, and obtain the first quality detection result corresponding to the online consultation process to be tested;

[0101] Here, the third network is obtained using any of the network training methods provided in the embodiments of this application.

[0102] In step 201, in practical applications, the specific method for obtaining the online consultation process corresponding to the online consultation process to be detected can be set according to the needs of the scenario, and this application embodiment does not limit this.

[0103] In step 202, in practical applications, in order to further reduce the probability of medical accidents, at least one rule can be pre-set to determine whether the online consultation process to be tested contains necessary sub-consultation processes, and the online consultation process can be quality tested in combination with the third network and at least one preset rule.

[0104] Based on this, in one embodiment, the method may further include:

[0105] If the first quality inspection result indicates that the online consultation process to be inspected contains necessary sub-consultation processes, at least one preset rule is used to determine whether the online consultation process to be inspected contains necessary sub-consultation processes, and a second quality inspection result corresponding to the online consultation process to be inspected is obtained; each preset rule represents the characteristics of a necessary sub-consultation process.

[0106] In practical applications, the preset rules can be determined manually or through knowledge graphs, deep learning, or other methods. The specific method for determining the preset rules can be set according to requirements, and this application embodiment does not limit this.

[0107] In practical applications, the first quality detection result, the second quality detection result, and the corresponding consultation dialogue information can be added to the training dataset, i.e., the training dataset can be updated, and the third network can be optimized using the updated training dataset, thereby further improving the accuracy of quality detection of the online consultation process.

[0108] In practical applications, if the online consultation process to be detected lacks necessary sub-consultation processes, an alarm can be triggered, thereby reducing the probability of medical accidents.

[0109] Based on this, in one embodiment, the method may further include:

[0110] If the first quality inspection result or the second quality inspection result indicates that the online consultation process to be inspected lacks necessary sub-consultation processes, at least one electronic communication address corresponding to the online consultation process to be inspected shall be obtained.

[0111] A warning is issued based on at least one of the electronic communication addresses.

[0112] In practical applications, the electronic communication address may include a mobile phone number, email address, etc.; correspondingly, the second electronic device can issue warnings via telephone, SMS or email.

[0113] The network training method and quality detection method based on the online consultation process provided in this application embodiment determine a training dataset; the training dataset contains consultation dialogue information and quality detection results corresponding to multiple online consultation processes; the consultation dialogue information contains online dialogue statements between doctors and patients in the corresponding online consultation process; the quality detection results characterize whether the corresponding online consultation process contains necessary sub-consultation processes; a first network and a second network are determined; the first network is used to determine the sub-consultation processes contained in the online consultation process; the second network is used to determine the necessary sub-consultation processes corresponding to the online consultation process; based on the training dataset, the first network and the second network are jointly learned to obtain a third network; the third network is used to determine whether the online consultation process contains necessary sub-consultation processes. The solution in this application embodiment determines a first network for identifying sub-consultation processes included in an online consultation process and a second network for identifying necessary sub-consultation processes corresponding to the online consultation process. Using a training dataset containing online dialogue statements between doctors and patients in the online consultation process and quality inspection results representing whether the online consultation process includes necessary sub-consultation processes, the first and second networks are jointly learned to obtain a third network for determining whether the online consultation process includes necessary sub-consultation processes. Thus, using the third network to perform quality inspection on the online consultation process can quickly and accurately determine whether the online consultation process includes necessary sub-consultation processes, thereby enabling timely alerts in the absence of necessary sub-consultation processes, i.e., timely control of medical risks, and ultimately reducing the probability of medical accidents.

[0114] The present application will be further described in detail below with reference to application examples.

[0115] This application example provides an online medical quality control system for performing medical-related quality control on the online consultation process of online medical services, such as... Figure 3As shown, the system includes a consultation dialogue collection module (also known as a consultation information acquisition module), a process quality control module, an early warning module (also known as an alarm module), and a manual quality control module. The consultation dialogue collection module collects online consultation dialogue records between doctors and patients. The process quality control module then scores the consultation process based on the model's output and judges the quality control results. If the quality control result indicates a serious problem in the consultation process, the early warning module is immediately invoked to notify the relevant doctor or quality control personnel to verify the issue and contact the patient promptly. If the quality control result indicates a general problem in the consultation process (the conditions for general problems can be set according to needs) or no problem exists, the manual quality control module is invoked for manual sampling verification. Simultaneously, the manual verification results are used as training data to optimize the performance of the model in the process quality control module.

[0116] In this application embodiment, the online medical quality control system's medical-related quality control of the online consultation process of online medical services refers to: determining whether the doctor's consultation process (i.e., the online consultation process to be tested mentioned above) includes necessary consultation procedures (i.e., the sub-consultation procedures mentioned above). Necessary consultation procedures include, but are not limited to, the following:

[0117] 1) Ask the patient in detail about their symptoms, duration of symptoms, severity of symptoms, and whether there are any other accompanying symptoms;

[0118] 2) Before prescribing medication, inquire about or have already learned about the patient's allergy history, medical history, pregnancy, and other special circumstances;

[0119] 3) Clearly inform the patient of their medical condition;

[0120] 4) When prescribing medication, clearly inform the patient of the medication usage and dosage.

[0121] In this application embodiment, the process quality control module can perform medical-related quality control on the online consultation process of online medical services, obtain the quality control results of the consultation process, identify problems existing in the consultation process, and subsequently perform the following processing:

[0122] 1) Quality control personnel with professional medical backgrounds will conduct sampling reviews of quality control reports through the manual quality control module, which can assess the performance of the current system and continuously provide data for subsequent model optimization;

[0123] 2) For consultations with significant quality issues (such as prescribing penicillin to a patient with a penicillin allergy), the warning module should be used to promptly alert relevant operations, quality control, and doctors to ensure the safety and health of patients.

[0124] 3) The online medical quality control system will regularly summarize and provide feedback on quality control to doctors, thereby improving the quality of doctors' future consultations.

[0125] The functions of each module in the online medical quality control system are described in detail below.

[0126] First, let's explain the functions of the consultation dialogue collection module.

[0127] In this application embodiment, the consultation dialogue collection module needs to collect information required for process quality control from the online internet hospital, specifically the following information needs to be extracted:

[0128] Basic patient information, such as age and gender;

[0129] Consultation dialogue information, that is, the record of the conversation between the doctor and the patient during the consultation. Each sentence needs to carry role information, that is, it should be able to distinguish between the patient's sentences and the doctor's sentences.

[0130] Consultation results information, such as whether a prescription was issued.

[0131] Secondly, the functions of the process quality control module will be explained.

[0132] In this application example, such as Figure 4 As shown, when the process quality control module performs medical-related quality control on the online consultation process of online medical services, it needs to perform process identification and process necessity identification. Process necessity identification determines which processes are required for the current consultation, while process identification determines which processes are present in the current consultation. These two identification processes jointly determine the final process quality control result.

[0133] In this application example, because the quality control physician does not distinguish between process identification and process necessity identification when scoring the online consultation process, but directly gives the final conclusion, namely whether the online consultation process includes the necessary consultation process, process identification and process necessity identification are not independent of each other in the calculation.

[0134] For example, in a general consultation, the doctor needs to clearly inform the patient of their illness:

[0135] Doctor: Hello, based on your described symptoms, you may have chronic obstructive pulmonary disease (COPD).

[0136] However, if the patient has already described their illness, for example,

[0137] Patient: Hello doctor, I have been diagnosed with COPD at the hospital. What treatment options are available?

[0138] At this point, the doctor does not need to explicitly repeat the patient's illness, and the quality control doctor will not think that the necessary consultation process is missing.

[0139] Therefore, the probability distribution for determining whether a certain step is missing in the online consultation process can be viewed as a joint probability distribution of step determination and step necessity determination, and can be expressed by the following formula:

[0140] P1(CORRECT A ) = P2(NEED A )×P3(HAVE A )+P2(NOT_NEED A (1)

[0141] Among them, P1(CORRECT) A P2(NEED) represents the probability that process A is not missing in the final determination; A P2(NOT_NEED) represents the probability that process A is indeed necessary during the consultation; A P3(HAVE) represents the probability that process A is not required during the consultation; A ) represents the probability that process A is included in the consultation.

[0142] Therefore, process discrimination and process necessity discrimination cannot be trained using independent models. This application example draws on the multi-task learning approach, training process discrimination and process necessity discrimination through joint learning. That is, process discrimination and process necessity discrimination are treated as two learning tasks, placed in the same network, and their parameters are shared for model training.

[0143] The independent process discrimination model (i.e., the first network mentioned above), the independent process necessity discrimination model (i.e., the second network mentioned above), and the joint learning model (i.e., the third network mentioned above) will be explained in detail below.

[0144] Independent process discrimination model

[0145] In this application example, such as Figure 5 As shown, the process discrimination model includes: a sentence input layer (which can be simply referred to as the Sentence layer, i.e. the first sub-network mentioned above), an embedding layer (i.e. the second sub-network mentioned above), a TextCNN layer (i.e. the third sub-network mentioned above), and a maximum value layer (i.e. the fourth sub-network mentioned above).

[0146] The Sentence layer serves as the input layer for the training data. It converts each character of each sentence (statement) in the consultation dialogue into a numeric ID for easier computer processing. Each statement can be represented by a vector whose length is its length; in other words, each statement can be represented by a vector whose length is the statement length. Through the processing of the Sentence layer, each consultation dialogue can be represented by a two-dimensional vector with the shape `[number of statements in the consultation dialogue, statement length]`. Here, the vector shape refers to the dimension of the quantity. For example, assuming the consultation dialogue contains three statements s1, s2, and s3, placing each statement in an array can form a vector [s1, s2, s3]. Each statement (represented as s) represents a combination of multiple words (represented as w), i.e., s = [w1, w2, w3, w4...]. Then the entire consultation dialogue can constitute vector A:

[0147] A=[[w11,w12,w13,w14],[w21,w22,w23,w24],[w31,w32,w33,w34]].

[0148] Vector A has a shape of 3*4, where 3 represents three statements and 4 represents that each statement has four characters (i.e., statement length). In this way, the entire consultation dialogue can be represented by a single mathematical vector.

[0149] For example, when the Sentence layer converts each character of each statement in the consultation dialogue into a numeric ID, it can count all the characters that have appeared in the entire document, filter them according to frequency, discard characters that have appeared less than 3 times, build a dictionary, and then assign IDs (such as 2, 3, 4, etc.) to the characters in the dictionary; at the same time, ID=1 is used to indicate that there is no character in the dictionary, and ID=0 is used to indicate a placeholder, that is, there is no character in reality, but a 0 can be added to the end to unify the length.

[0150] The Embedding layer is a word embedding layer that maps each character ID in the Sentence layer to a one-dimensional vector to represent the character in vector space. Therefore, the processing in the Embedding layer can also be called word embedding processing. The Embedding layer maintains a dictionary matrix V. vocab_size×emb_dim The dictionary matrix can be used to determine the vector V[Char_id] that maps character IDs. The dictionary matrix of the embedding layer can be maintained and trained, and the values ​​in the matrix are continuously corrected during model training to achieve better mapping results. The shape of the output vector of the embedding layer is `[number of sentences in the consultation dialogue, sentence length, embedding vector length]`.

[0151] The TextCNN layer can perform multi-layer, multi-scale convolution operations on the output of the Embedding layer to obtain the classification output of each statement in the same consultation dialogue (i.e., the classification of the consultation process). The shape of the TextCNN layer output vector is `[number of statements in the consultation dialogue, number of categories]`. Here, the output of the TextCNN layer can be the probability of each category, that is, the probability that a statement belongs to a certain type (i.e., the consultation process).

[0152] The MAX layer multiplies the output vector of the TextCNN layer with a preset role matrix, and then takes the maximum value of each category value for each statement to obtain the flow classification of the entire consultation dialogue; the shape of the MAX layer output vector is `[number of categories]`. If it is determined that a statement belongs to a consultation flow, then the consultation dialogue to which this statement belongs naturally also exists in that consultation flow; in addition, the quality control of the doctor's consultation is actually unrelated to what the patient says, therefore, the role matrix is ​​used to indicate whether a sentence is sent by the doctor (value 1) or the patient (value 0). After multiplying the value of the role matrix with the category value, the value of the patient is always 0, thereby achieving the purpose of blocking patient messages.

[0153] In practical applications, the calculation process of the process discrimination model can be represented by the following formula:

[0154] input = List[<role,sent> (2)

[0155] sent_vec i =[v1, v2, v3,...]=Embedding(input_sent i (3)

[0156] have_res=TextCNN(MiniBatch(sent_vec)) (4)

[0157] have_res=have_res*role_map (5)

[0158] final_have=max(have_res, axis=1) (6)

[0159] Formula (2) indicates that the input of the model is a list of dialogue information from the consultation, and each list element contains the speaker's role (doctor or patient) and the content of the statement (sent).

[0160] Formula (3) represents the use of the Embedding layer to process the i-th statement input_sent in the consultation dialogue. iAfter processing, the matrix representation of the i-th statement, sent_vec, is obtained. i ,sent_vec i It consists of word vectors v.

[0161] Formula (4) indicates that the TextCNN layer performs multi-layer, multi-scale convolution operations on the output of the Embedding layer. In order to speed up the calculation, all sent_vec obtained from Formula (3) need to be combined into a batch of data, that is, multiple sentences are input into the TextCNN layer at the same time. have_res represents the classification result of the TextCNN layer (whether each sentence contains a specific consultation process).

[0162] Formula (5) means multiplying the output vector of the TextCNN layer with the role matrix role_map (the value of the patient's statement), thereby covering the processing result of the patient's statement and only retaining the processing result of the doctor's statement.

[0163] Formula (6) represents taking the maximum value for the classification of all statements. Thus, if the classification result of a statement is "there is a process" (value 1), the result of the entire consultation is also 1, indicating that the consultation has a corresponding process. Here, final_have represents the output vector of the MAX layer, where the values ​​are floating-point decimals (i.e., the probability of the existence of a corresponding consultation process). For these floating-point decimals, values ​​greater than 0.5 need to be converted to 1 to indicate the existence of a corresponding consultation process, and values ​​less than or equal to 0.5 need to be converted to 0 to indicate the absence of a corresponding consultation process.

[0164] Independent process necessity judgment model

[0165] In this application example, such as Figure 6 As shown, the process necessity discrimination model includes: Sentence layer (i.e., the first sub-network mentioned above), Embedding layer (i.e., the second sub-network mentioned above), TextCNN_insent layer (i.e., the fifth sub-network mentioned above), and TextCNN_crosssent layer (i.e., the sixth sub-network mentioned above).

[0166] The processing of the Sentence and Embedding layers is the same as that of the process discrimination model, and will not be elaborated here. The TextCNN_insent layer is used to abstract the semantic expression of each statement in the consultation dialogue into a feature represented by a vector. In other words, it performs feature extraction on the output data of the Embedding layer again to obtain the semantic expression vector of each statement. The shape of the output vector of the TextCNN_insent layer is `[number of statements in the consultation dialogue, size of hidden state]`. Since the process necessity discrimination needs to refer to the messages sent by the patient, and also needs to be judged based on the dialogue content between the doctor and the patient, and since the TextCNN_insent layer only obtains the vector of a single statement, the TextCNN_crosssent layer needs to further determine the relationship between statements, that is, to calculate the output data of the TextCNN_insent layer, and the shape of the output vector is `[number of categories]`, that is, whether the current consultation requires the corresponding consultation process.

[0167] Here, the processing of the TextCNN_insent layer can be represented by the following formula:

[0168] have res =TextCNN_insent(MiniBatch(sent_vec)) (7)

[0169] Where MiniBatch(sent_vec) represents the output data of the Embedding layer; have res This represents the output data of the TextCNN_insent layer.

[0170] The processing of the TextCNN_crosssent layer can be represented by the following formula:

[0171] final_need=TextCNN_crosssent(have_res) (8)

[0172] Here, final_need represents the output data of the TextCNN_crosssent layer, where the values ​​are floating-point decimals (i.e., the probability of whether the corresponding consultation process is needed). For these floating-point decimals, values ​​greater than 0.5 need to be converted to 1 to indicate that the corresponding consultation process is needed, and values ​​less than or equal to 0.5 need to be converted to 0 to indicate that the corresponding consultation process is not needed.

[0173] Joint learning model

[0174] In this application embodiment, when the process quality control module scores the online consultation process, it can award full marks regardless of whether a consultation process actually exists, provided that the process is not required. Points are only deducted when a consultation process is required but does not actually exist. If the process discrimination model and the process necessity discrimination model are trained separately, the system may have the following problems:

[0175] 1) Due to a lack of training data, online doctors will only provide the final quality control score, instead of indicating whether a certain process is needed and providing a score for that process.

[0176] 2) Model losses overlap, preventing joint optimization; when two independent models are trained, gradients cannot be propagated, leading to the overlap of losses.

[0177] Therefore, in this application embodiment, the process discrimination model and the process necessity discrimination model are jointly learned, merging identical operations in the two models to reduce computational load. For example... Figure 7 As shown, the joint learning model includes a Have*Need layer (i.e., the seventh sub-network mentioned above), which is a fully connected network used to concatenate the output data of the MAX layer and the output data of the TextCNN_crosssent layer before outputting the result.

[0178] In this application example, such as Figure 4 As shown, the process quality control module can also combine fixed rules for quality control when performing medical-related quality control on the online consultation process of online medical services. Specifically, quality control personnel can manually organize feature rules for consultation statements based on the historical dialogue data between doctors and patients and the corresponding quality control data. For all statements made by the doctor, the rules are used to determine whether the corresponding statement belongs to a specific consultation process.

[0179] For example, suppose the statement "Doctor: Do you have a history of drug allergies?" appears frequently in the consultation dialogue. Quality control personnel can manually extract rules for this statement to obtain the rule "Do you have a history of drug allergies?" When a statement matching this rule exists in the consultation dialogue, it can be determined that the consultation dialogue contains a process of asking about allergy history.

[0180] The process quality control module can first use a joint learning model to detect the consultation process. When the detection result indicates that the consultation process does not lack the necessary consultation procedures, the pre-organized rules can be used again to extract sentences (i.e., detect). The detection results of the joint learning model and the rule detection results are used as the final quality control results of the consultation process.

[0181] Third, the functions of the early warning module will be explained.

[0182] In this application embodiment, the process quality control module may detect multiple process defects in a consultation process, i.e., the absence of several necessary consultation procedures. Based on the quality control results of the process quality control module, on the one hand, important procedures can be pre-configured. For example, if a consultation process lacks a procedure for inquiring about allergy history, this procedure can be configured as an important procedure, i.e., a necessary consultation procedure. On the other hand, when the process quality control module determines that a consultation process lacks a procedure for inquiring about allergy history, it can invoke the early warning module. The early warning module will notify doctors, patients, quality control personnel, and other relevant personnel via email, consultation desk messages, etc., thereby enabling timely control of medical risks and reducing the probability of medical accidents.

[0183] Fourth, the functions of the manual quality control module are explained.

[0184] In cases where algorithm quality control may contain errors, quality control experts need to sample and review the algorithm's output quality control results, providing manual quality control findings. After a period of manual quality control (e.g., a week), the week's manual quality control data can be used to retrain the quality control algorithm, i.e., optimize the joint learning model. This can, on the one hand, appropriately alleviate model drift; on the other hand, it can continuously provide high-quality training data for the joint learning model, gradually improving performance through accumulation.

[0185] The solution provided in this application embodiment has the following advantages:

[0186] 1) Quality control of online consultation messages between doctors and patients can determine whether some necessary key processes are missing during the consultation process, thereby improving the quality of online medical consultations;

[0187] 2) Using a computer system to perform quality control on online doctor-patient dialogues can cover 100% of consultations, provided that computing resources are sufficient, thus avoiding huge manual input or assisting quality control personnel in improving work efficiency.

[0188] 3) Using a computer system to perform quality control on online doctor-patient dialogues provides high real-time performance, enabling timely detection and alerts of problems, which to some extent prevents medical accidents from occurring.

[0189] To implement the network training method based on the online consultation process in this application embodiment, this application embodiment also provides a network training device based on the online consultation process, such as... Figure 8 As shown, the device includes:

[0190] The first processing unit 801 is used to determine a training dataset; the training dataset contains consultation dialogue information and quality detection results corresponding to multiple online consultation processes; the consultation dialogue information contains online dialogue statements between doctors and patients in the corresponding online consultation process; the quality detection results indicate whether the corresponding online consultation process contains necessary sub-consultation processes.

[0191] The second processing unit 802 is used to determine a first network and a second network; the first network is used to determine the sub-consultation processes included in the online consultation process; the second network is used to determine the necessary sub-consultation processes corresponding to the online consultation process.

[0192] The third processing unit 803 is used to jointly learn the first network and the second network using the training dataset to obtain a third network; the third network is used to determine whether the online consultation process includes necessary sub-consultation processes.

[0193] In one embodiment, the second processing unit 802 is specifically used to determine the first network based on the first sub-network, the second sub-network, the third sub-network, and the fourth sub-network; wherein,

[0194] The first sub-network is used to digitize each character of each statement contained in the consultation dialogue information;

[0195] The second sub-network is used to perform semantic expression on each character of each statement contained in the consultation dialogue information based on the output data of the first sub-network;

[0196] The third sub-network is used to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network, and to classify the statements contained in the consultation dialogue information based on the semantic expression of each statement.

[0197] The fourth sub-network is used to determine the sub-consultation processes included in the corresponding online consultation process based on the output data of the third sub-network.

[0198] In one embodiment, the second processing unit 802 is further configured to train the dictionary matrix maintained by the second sub-network; the second sub-network uses the dictionary matrix to perform semantic expression on each character of each statement contained in the consultation dialogue information.

[0199] In one embodiment, the second processing unit 802 is specifically used to determine the second network based on the first sub-network, the second sub-network, the fifth sub-network, and the sixth sub-network; wherein,

[0200] The first sub-network is used to digitize each character of each statement contained in the consultation dialogue information;

[0201] The second sub-network is used to perform semantic expression on each character of each statement contained in the consultation dialogue information based on the output data of the first sub-network;

[0202] The fifth sub-network is used to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network.

[0203] The sixth sub-network is used to determine the necessary sub-consultation processes corresponding to the corresponding online consultation process based on the output data of the fifth sub-network.

[0204] In one embodiment, when jointly learning the first network and the second network, the third processing unit 803 is specifically used to train a seventh sub-network using the training dataset, the output data of the first network, and the output data of the second network; the seventh sub-network is used to determine whether the corresponding online consultation process includes a necessary sub-consultation process based on the output data of the first network and the output data of the second network; the third network includes the first network, the second network, and the seventh sub-network.

[0205] In practical applications, the first processing unit 801, the second processing unit 802, and the third processing unit 803 can be implemented by a processor in the network training device combined with a communication interface.

[0206] It should be noted that the network training device provided in the above embodiments is only illustrated by the division of the above program modules during network training. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the network training device and the network training method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0207] To implement the quality detection method for the online consultation process in this application embodiment, this application embodiment also provides a quality detection device for the online consultation process, such as... Figure 9 As shown, the device includes:

[0208] The fourth processing unit 901 is used to acquire the online consultation dialogue information corresponding to the online consultation process to be detected; the consultation dialogue information includes online dialogue statements between the doctor and the patient in the online consultation process to be detected;

[0209] The fifth processing unit 902 is used to determine, based on the consultation dialogue information and using a third network, whether the online consultation process to be detected contains necessary sub-consultation processes, and to obtain a first quality detection result corresponding to the online consultation process to be detected; the third network is obtained using any of the network training methods provided in the embodiments of this application.

[0210] In one embodiment, the fifth processing unit 902 is further configured to, when the first quality detection result indicates that the online consultation process to be detected contains necessary sub-consultation processes, use at least one preset rule to determine whether the online consultation process to be detected contains necessary sub-consultation processes, and obtain a second quality detection result corresponding to the online consultation process to be detected; each preset rule represents the characteristics of a necessary sub-consultation process.

[0211] In one embodiment, the fifth processing unit 902 is further configured to:

[0212] If the first quality inspection result or the second quality inspection result indicates that the online consultation process to be inspected lacks necessary sub-consultation processes, at least one electronic communication address corresponding to the online consultation process to be inspected shall be obtained.

[0213] A warning is issued based on at least one of the electronic communication addresses.

[0214] In practical applications, the fourth processing unit 901 and the fifth processing unit 902 can be implemented by the processor in the quality inspection device combined with the communication interface.

[0215] It should be noted that the quality inspection device provided in the above embodiments is only illustrated by the division of the above-described program modules when performing quality inspection on the online consultation process. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the quality inspection device and the quality inspection method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0216] Based on the hardware implementation of the above program modules, and in order to implement the network training method based on the online consultation process in this application embodiment, this application embodiment also provides a first electronic device, such as... Figure 10 As shown, the first electronic device 1000 includes:

[0217] The first communication interface 1001 is capable of exchanging information with other electronic devices;

[0218] The first processor 1002 is connected to the first communication interface 1001 to enable information interaction with other electronic devices and to execute the network training method based on the online consultation process provided by one or more of the above technical solutions when running a computer program.

[0219] The first memory 1003 stores computer programs that can run on the first processor 1002.

[0220] Specifically, the first processor 1002 is used for:

[0221] A training dataset is determined; the training dataset contains consultation dialogue information and quality detection results corresponding to multiple online consultation processes; the consultation dialogue information contains online dialogue statements between doctors and patients in the corresponding online consultation process; the quality detection results indicate whether the corresponding online consultation process contains necessary sub-consultation processes.

[0222] A first network and a second network are determined; the first network is used to determine the sub-consultation processes included in the online consultation process; the second network is used to determine the necessary sub-consultation processes corresponding to the online consultation process.

[0223] Using the training dataset, the first network and the second network are jointly learned to obtain a third network; the third network is used to determine whether the online consultation process includes necessary sub-consultation processes.

[0224] In one embodiment, the first processor 1002 is specifically configured to determine the first network based on a first sub-network, a second sub-network, a third sub-network, and a fourth sub-network; wherein,

[0225] The first sub-network is used to digitize each character of each statement contained in the consultation dialogue information;

[0226] The second sub-network is used to perform semantic expression on each character of each statement contained in the consultation dialogue information based on the output data of the first sub-network;

[0227] The third sub-network is used to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network, and to classify the statements contained in the consultation dialogue information based on the semantic expression of each statement.

[0228] The fourth sub-network is used to determine the sub-consultation processes included in the corresponding online consultation process based on the output data of the third sub-network.

[0229] In one embodiment, the first processor 1002 is further configured to train a dictionary matrix maintained by the second sub-network; the second sub-network uses the dictionary matrix to perform semantic representation of each character of each statement contained in the consultation dialogue information.

[0230] In one embodiment, the first processor 1002 is specifically configured to determine the second network based on the first sub-network, the second sub-network, the fifth sub-network, and the sixth sub-network; wherein,

[0231] The first sub-network is used to digitize each character of each statement contained in the consultation dialogue information;

[0232] The second sub-network is used to perform semantic expression on each character of each statement contained in the consultation dialogue information based on the output data of the first sub-network;

[0233] The fifth sub-network is used to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network.

[0234] The sixth sub-network is used to determine the necessary sub-consultation processes corresponding to the corresponding online consultation process based on the output data of the fifth sub-network.

[0235] In one embodiment, when jointly learning the first network and the second network, the first processor 1002 is further configured to train a seventh sub-network using the training dataset, the output data of the first network, and the output data of the second network; the seventh sub-network is configured to determine whether the corresponding online consultation process includes a necessary sub-consultation process based on the output data of the first network and the output data of the second network; the third network includes the first network, the second network, and the seventh sub-network.

[0236] It should be noted that the specific processing procedure of the first processor 1002 can be understood by referring to the above method.

[0237] Of course, in practical applications, the various components in the first electronic device 1000 are coupled together via a bus system 1004. It can be understood that the bus system 1004 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 1004 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 10 The general labeled all buses as Bus System 1004.

[0238] The first memory 1003 in this embodiment is used to store various types of data to support the operation of the first electronic device 1000. Examples of such data include any computer program used to operate on the first electronic device 1000.

[0239] The methods disclosed in the above embodiments of this application can be applied to the first processor 1002, or implemented by the first processor 1002. The first processor 1002 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the first processor 1002. The first processor 1002 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The first processor 1002 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the first memory 1003. The first processor 1002 reads the information in the first memory 1003 and completes the steps of the aforementioned method in combination with its hardware.

[0240] In an exemplary embodiment, the first electronic device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0241] Based on the hardware implementation of the above program modules, and in order to implement the quality detection method for the online consultation process in this application embodiment, this application embodiment also provides a second electronic device, such as... Figure 11 As shown, the second electronic device 1100 includes:

[0242] The second communication interface 1101 is capable of exchanging information with other electronic devices;

[0243] The second processor 1102 is connected to the second communication interface 1101 to enable information interaction with other electronic devices and to execute the quality detection method of the online consultation process provided by one or more of the above technical solutions when running a computer program.

[0244] The second memory 1103 stores computer programs that can run on the second processor 1102.

[0245] Specifically, the second processor 1102 is used for:

[0246] Obtain the online consultation dialogue information corresponding to the online consultation process to be tested; the consultation dialogue information includes the online dialogue statements between the doctor and the patient in the online consultation process to be tested;

[0247] Based on the consultation dialogue information, a third network is used to determine whether the online consultation process to be tested contains necessary sub-consultation processes, and a first quality detection result corresponding to the online consultation process to be tested is obtained; the third network is obtained using any of the network training methods provided in the embodiments of this application.

[0248] In one embodiment, the second processor 1102 is further configured to, when the first quality detection result indicates that the online consultation process to be detected contains necessary sub-consultation processes, use at least one preset rule to determine whether the online consultation process to be detected contains necessary sub-consultation processes, and obtain a second quality detection result corresponding to the online consultation process to be detected; each preset rule represents the feature of a necessary sub-consultation process.

[0249] In one embodiment, the second processor 1102 is further configured to:

[0250] If the first quality inspection result or the second quality inspection result indicates that the online consultation process to be inspected lacks necessary sub-consultation processes, at least one electronic communication address corresponding to the online consultation process to be inspected shall be obtained.

[0251] A warning is issued based on at least one of the electronic communication addresses.

[0252] It should be noted that the specific processing procedure of the second processor 1102 can be understood by referring to the above method.

[0253] Of course, in practical applications, the various components in the second electronic device 1100 are coupled together via a bus system 1104. It is understood that the bus system 1104 is used to achieve communication between these components. In addition to a data bus, the bus system 1104 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 11 The general designated all buses as Bus System 1104.

[0254] The second memory 1103 in this embodiment is used to store various types of data to support the operation of the second electronic device 1100. Examples of such data include any computer program used to operate on the second electronic device 1100.

[0255] The methods disclosed in the embodiments of this application can be applied to the second processor 1102, or implemented by the second processor 1102. The second processor 1102 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the second processor 1102. The second processor 1102 may be a general-purpose processor, a digital signal processor, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The second processor 1102 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the second memory 1103. The second processor 1102 reads the information in the second memory 1103 and completes the steps of the aforementioned method in combination with its hardware.

[0256] In an exemplary embodiment, the second electronic device 1100 may be implemented by one or more ASICs, digital signal processors, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0257] It is understood that the memories (first memory 1003, second memory 1103) in the embodiments of this application can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0258] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a first memory 1003 storing a computer program, which can be executed by the first processor 1002 of the first electronic device 1000 to complete the steps of the aforementioned network training method based on the online consultation process. Another example is a second memory 1103 storing a computer program, which can be executed by the second processor 1102 of the second electronic device 1100 to complete the steps of the aforementioned quality detection method for the online consultation process. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0259] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0260] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0261] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A network training method based on an online consultation process, characterized in that, include: Determine the training dataset; The training dataset contains consultation dialogue information and quality detection results corresponding to multiple online consultation processes; The online consultation dialogue information includes online dialogue statements between doctors and patients in the corresponding online consultation process; the quality inspection result indicates whether the corresponding online consultation process includes necessary sub-consultation processes. A first network is determined based on a first subnetwork, a second subnetwork, a third subnetwork, and a fourth subnetwork. The first subnetwork is used to digitize each character of each statement in the online consultation dialogue information. The second subnetwork is used to semantically represent each character of each statement in the online consultation dialogue information based on the output data of the first subnetwork. The third subnetwork is used to semantically represent each statement in the online consultation dialogue information based on the output data of the second subnetwork, and to classify the statements in the online consultation dialogue information based on the semantic representation of each statement. The fourth subnetwork is used to determine the sub-consultation processes included in the corresponding online consultation process based on the output data of the third subnetwork. The first network is used to determine the sub-consultation processes included in the online consultation process. A second network is determined based on the first sub-network, the second sub-network, the fifth sub-network, and the sixth sub-network; wherein, the fifth sub-network is used to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network; the sixth sub-network is used to determine the necessary sub-consultation processes corresponding to the corresponding online consultation process based on the output data of the fifth sub-network; the second network is used to determine the necessary sub-consultation processes corresponding to the online consultation process. Using the training dataset, the first network and the second network are jointly learned to obtain a third network; the third network is used to determine whether the online consultation process includes necessary sub-consultation processes.

2. The method according to claim 1, characterized in that, The method further includes: The dictionary matrix maintained by the second sub-network is trained; the second sub-network uses the dictionary matrix to perform semantic representation of each character of each statement contained in the consultation dialogue information.

3. The method according to claim 1, characterized in that, When jointly learning the first network and the second network, the method further includes: A seventh sub-network is trained using the training dataset, the output data of the first network, and the output data of the second network; the seventh sub-network is used to determine whether the corresponding online consultation process includes necessary sub-consultation processes based on the output data of the first network and the output data of the second network; the third network includes the first network, the second network, and the seventh sub-network.

4. A quality inspection method for an online consultation process, characterized in that, include: Obtain the online consultation dialogue information corresponding to the online consultation process to be tested; The consultation dialogue information includes online dialogue statements between the doctor and the patient in the online consultation process to be detected; Based on the consultation dialogue information, a third network is used to determine whether the online consultation process to be tested contains necessary sub-consultation processes, and a first quality detection result corresponding to the online consultation process to be tested is obtained; the third network is obtained using the method described in any one of claims 1 to 3.

5. The method according to claim 4, characterized in that, The method further includes: If the first quality inspection result indicates that the online consultation process to be inspected contains necessary sub-consultation processes, at least one preset rule is used to determine whether the online consultation process to be inspected contains necessary sub-consultation processes, and a second quality inspection result corresponding to the online consultation process to be inspected is obtained; each preset rule represents the characteristics of a necessary sub-consultation process.

6. The method according to claim 5, characterized in that, The method further includes: If the first quality inspection result or the second quality inspection result indicates that the online consultation process to be inspected lacks necessary sub-consultation processes, at least one electronic communication address corresponding to the online consultation process to be inspected shall be obtained. A warning is issued based on at least one of the electronic communication addresses.

7. A network training device based on an online consultation process, characterized in that, include: The first processing unit is used to determine the training dataset; The training dataset contains consultation dialogue information and quality detection results corresponding to multiple online consultation processes; The online consultation dialogue information includes online dialogue statements between doctors and patients in the corresponding online consultation process; the quality inspection result indicates whether the corresponding online consultation process includes necessary sub-consultation processes. The second processing unit is used to determine the first network based on the first sub-network, the second sub-network, the third sub-network, and the fourth sub-network; wherein, the first sub-network is used to digitize each character of each statement contained in the consultation dialogue information; the second sub-network is used to semantically express each character of each statement contained in the consultation dialogue information based on the output data of the first sub-network; the third sub-network is used to semantically express each statement contained in the consultation dialogue information based on the output data of the second sub-network, and to classify the statements contained in the consultation dialogue information based on the semantic expression of each statement; the fourth sub-network is used to determine the sub-consultation processes contained in the corresponding online consultation process based on the output data of the third sub-network; the first network is used to determine the sub-consultation processes contained in the online consultation process; The second processing unit is further configured to determine a second network based on the first sub-network, the second sub-network, the fifth sub-network, and the sixth sub-network; wherein, the fifth sub-network is configured to perform semantic expression on each statement contained in the consultation dialogue information based on the output data of the second sub-network; the sixth sub-network is configured to determine the necessary sub-consultation processes corresponding to the corresponding online consultation process based on the output data of the fifth sub-network; and the second network is configured to determine the necessary sub-consultation processes corresponding to the online consultation process. The third processing unit is used to jointly learn the first network and the second network using the training dataset to obtain a third network; the third network is used to determine whether the online consultation process includes necessary sub-consultation processes.

8. A quality inspection device for an online consultation process, characterized in that, include: The fourth processing unit is used to acquire the online consultation dialogue information corresponding to the online consultation process to be detected; The consultation dialogue information includes online dialogue statements between the doctor and the patient in the online consultation process to be detected; The fifth processing unit is used to determine, based on the consultation dialogue information and using a third network, whether the online consultation process to be tested contains necessary sub-consultation processes, and to obtain a first quality detection result corresponding to the online consultation process to be tested; the third network is obtained using the method described in any one of claims 1 to 3.

9. A first electronic device, characterized in that, include: A first processor and a first memory for storing computer programs capable of running on the processor. Wherein, when the first processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 3.

10. A second electronic device, characterized in that, include: A second processor and a second memory for storing computer programs that can run on the processor. Wherein, when the second processor is used to run the computer program, it performs the steps of the method according to any one of claims 4 to 6.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3, or the steps of the method according to any one of claims 4 to 6.

Images