Intelligent doctor-patient conflict mediation method based on multi-mode technology

By using multimodal technology and deep learning models, combined with text, image and audio data, a database of doctor-patient conflicts is constructed, which solves the problems of high cost and poor traceability in traditional methods, and achieves more efficient mediation of doctor-patient conflicts and improved satisfaction.

CN118969208APending Publication Date: 2024-11-15NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410988607.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Traditional intelligent methods for handling doctor-patient conflicts are costly, time-consuming, and have poor traceability. They mainly process single-modal data and cannot fully capture both linguistic and non-linguistic information.

Method used

By employing multimodal technology and combining text, image, and audio data, a multimodal audiovisual recognition model based on deep learning is used to construct a database of doctor-patient conflicts, extract and evaluate multimodal features, and generate intelligent mediation solutions.

Benefits of technology

It enables more comprehensive capture of doctor-patient conflict information, improves identification performance and mediation efficiency, provides personalized and effective mediation solutions, and enhances the satisfaction of both doctors and patients.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118969208A_ABST
    Figure CN118969208A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of artificial intelligence, and discloses an intelligent doctor-patient conflict mediation method based on a multi-modal technology, which captures text data, image data and audio data through the multi-modal technology, can capture language information and non-language information more comprehensively, and improves the mediation efficiency. According to the method, the text data, the image data and the audio data are subjected to multi-modal feature fusion, and a solution case library is constructed, so that a mediation scheme can be more properly and intelligently recommended according to the possibility of generating conflicts and a prediction result of properties. According to the method, the limitation of single-mode characteristics is made up, data information can be more fully utilized, the characteristics of data of each mode can be more fully learned, the recognition performance is effectively improved, and effective and fair doctor-patient conflict resolution is facilitated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to an intelligent doctor-patient conflict mediation method based on multimodal technology. Background Art

[0003] Traditional intelligent methods for resolving doctor-patient conflicts are not only costly, time-consuming, labor-intensive, and have poor traceability, but also mainly deal with single-modal data. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an intelligent doctor-patient conflict mediation method based on multimodal technology. The method captures text data, image data and audio data through multimodal technology, and can more comprehensively capture language information and non-language information, thereby achieving more adequate utilization of data information. The construction of a multimodal audio-visual recognition model based on deep learning can more fully learn the characteristics of each modal data and effectively improve the recognition performance. The intelligent recommendation of mediation plans is conducive to the effective and fair resolution of doctor-patient conflicts.

[0005] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0006] The present invention is an intelligent doctor-patient conflict mediation method based on multimodal technology, which specifically comprises the following steps:

[0007] Step 1: Build a doctor-patient conflict database: The database contains text data, image data, audio data, and a conflict mediation solution library, that is, all tables are combined together;

[0008] Step 2, receiving complaints about doctor-patient conflicts: complaints about doctor-patient conflicts are received through two channels. One is the conventional channel, which receives complaints through the complaint platform, and the other is the channel that this patent autonomously identifies based on multimodal data analysis. The latter obtains modal features: based on deep learning and sentiment dictionaries, the text data, image data, and audio data obtained in step 1 are analyzed and feature extracted to obtain modal features;

[0009] Step 3: Compare the conflict mediation solution library and evaluate the medical conflict. Abstract the subject and object in the medical conflict and their relationship, and construct a multimodal medical conflict assessment model to evaluate the medical conflict: first, abstract the subject in the medical conflict, namely doctors and patients, and the object, namely medical problems, communication process, conflict mediation solutions, etc. and their relationship; secondly, integrate the collected text data, image data, and audio data to construct a multimodal medical conflict assessment model; and compare the results of the analysis and identification of the multimodal medical conflict assessment model with the conflict solution library to evaluate the medical conflict and determine the nature and severity of the conflict;

[0010] Step 4: Provide a mediation plan, and automatically generate a solution to the medical dispute corresponding to the conflict based on the evaluation results of step 3;

[0011] Step 5: Mediation records and tracking: Record the entire process of medical conflict mediation and build a mediation database, including the opinions of all parties and the consensus reached, for subsequent conflict tracking and evaluation of the effectiveness of mediation results.

[0012] Step 6: Update and iterate the conflict mediation solution library: If you encounter a new mediation problem, summarize and abstract the mediation process into logic, iterate the solution library, and provide useful guidance for the next mediation of the same case.

[0013] A further improvement of the present invention is that the construction of the database in step 1 includes collecting medical records, patient satisfaction surveys, and doctor evaluations, wherein the medical records include

[0014] Patient’s information: name, gender, age, contact information;

[0015] Medical history: family medical history, past illnesses, drug allergies;

[0016] Treatment records: dosage and frequency of medications, surgical information, and treatment outcomes;

[0017] Follow-up records: frequency of follow-up visits and follow-up results;

[0018] Medical reports: surgical reports;

[0019] The patient satisfaction survey includes: diagnosis and treatment services, hospital environment, medical process, and treatment effect;

[0020] Doctor evaluation includes: diagnosis accuracy, treatment effect and service attitude.

[0021] A further improvement of the present invention is that the construction of other types of information databases includes the following steps:

[0022] Step 1.1, construct language data: collect language texts of doctor-patient communication, including written records and oral communication, use scanners to convert the texts of doctor-patient communication into text data, use ORC technology to convert the image texts of doctor-patient communication into text data, and use sound recognition software to realize online speech-to-text conversion to convert the audio data of recorded doctor-patient communication into text;

[0023] Step 1.2, construct image data: obtain visual information of facial expressions, body language, and postures of doctor-patient communication through video recording or image acquisition equipment;

[0024] Step 1.3: Construct audio data: Capture the emotions during the communication process through voice sentiment analysis.

[0025] A further improvement of the present invention is that the step 1.2 specifically includes the following steps:

[0026] Step 1.2.1, use a camera to record image data of doctor-patient communication;

[0027] Step 1.2.2: Use high-precision sensors to capture details and facial expressions of doctor-patient communication;

[0028] Step 1.2.3, converting the image data collected during the doctor-patient communication in step 1.2.1 into a digital signal for computer processing;

[0029] Step 1.2.4: Build an image database to store the collected image data of doctor-patient communication.

[0030] A further improvement of the present invention is that the construction of the audio data in step 1.3 is specifically as follows:

[0031] Step 1.3.1. Use a microphone to capture the sound of doctor-patient communication;

[0032] Step 1.3.2, use recording equipment to record the voice of doctor-patient communication;

[0033] Step 1.3.3: Use speech recognition software to achieve online speech-to-text conversion to convert the audio data of the recorded doctor-patient communication into text.

[0034] A further improvement of the present invention is that: the conflict mediation solution library in step 1 specifically includes

[0035] Step 1.4.1. Collection of medical conflict mediation cases: Collect medical conflict mediation cases that have been handled from hospitals, courts, arbitration institutions, and mediation institutions, and invite medical conflict mediation experts to provide cases to build a mediation foundation;

[0036] Step 1.4.3, Organize medical conflict mediation cases: Classify the types, procedures and methods of mediation according to the background, process, method, result and agreement, organize the collected cases and store them in the solution library;

[0037] Step 1.4.3, Doctor-patient conflict mediation plan template: Based on mediation theory, objectives, strategies, procedures, methods and other factors, formulate a doctor-patient conflict mediation plan template;

[0038] Step 1.4.4: Solution library maintenance: Maintain the mediation solution library to ensure the robustness of the solution library.

[0039] A further improvement of the present invention is that the text data analysis in step 2 includes text data collection and text sentiment value calculation, which specifically includes the following steps:

[0040] Step 2.1, text data collection and conversion: Write a crawler based on Python to collect text materials of doctor-patient communication, and collect the language text of doctor-patient communication based on speech-to-text technology;

[0041] Step 2.2: Structured text collection: Based on RPA technology, automatically collect structured texts of medical records, patient satisfaction surveys, and doctor evaluations when seeking medical treatment;

[0042] Step 2.3, based on the language text collected in step 2.1 and the structured text collected in step 2.2, construct a medical sentiment dictionary and sentiment rules, and calculate the sentiment value;

[0043] Step 2.4: Perform sentiment polarity classification based on the BERT-TextRCNN model, where the BERT-TextRCNN model consists of four parts: BERT language pre-training model, bi-LSTM layer, pooling layer and softmax layer.

[0044] The further improvement of the present invention is that: in step 2.3, constructing the medical sentiment dictionary is: constructing the medical sentiment dictionary based on the HowNet sentiment dictionary, the NTSUSD simplified Chinese sentiment polarity dictionary and manually adding professional vocabulary in the medical field, and constructing the sentiment rule to calculate the sentiment value specifically includes the following steps:

[0045] Step 2.3.1, use the jieba package to segment the language text and structured text;

[0046] Step 2.3.2, search for sentiment words in the sentence, and mark the sentiment polarity and position;

[0047] Step 2.3.3, weight the sentiment value according to the degree of the sentiment word. If a negative word appears before the sentiment word, the sentiment polarity is reversed;

[0048] Step 2.3.4, calculate the sentiment polarity value of each sentence respectively;

[0049] Step 2.3.5: Subtract the negative value from the positive value to get the emotional value of each sentence.

[0050] A further improvement of the present invention is that the step 2.4 specifically includes the following steps:

[0051] Step 2.4.1, build a BERT language pre-training model;

[0052] Step 2.4.2: Input the tensor obtained by the BERT language pre-training model into the bi-LSTM layer;

[0053] Step 2.4.3: The tensor obtained by the bi-LSTM layer is transferred to the maximum pooling layer. The information extracted by the bi-LSTM layer is filtered and reduced in dimension through the maximum pooling strategy to retain the most significant features.

[0054] Step 2.4.4, use the softmax layer to implement text sentiment classification, and map the result prediction to the [0.1] interval through softmax, where the largest value is the predicted classification.

[0055] A further improvement of the present invention is that the analysis of the image data in step 2 is specifically: using a camera to shoot the patient's behavior during medical treatment, and using the collected image to predict behavior based on a convolutional neural network, wherein the convolutional neural network includes a convolutional layer, a pooling layer, a fully connected layer and a behavior prediction classifier:

[0056] First, construct the convolution layer: the convolution layer abstracts and extracts image information through (2, 3, 4) multi-scale convolution kernels;

[0057] Secondly, the information extracted by the convolution layer is input into the pooling layer. The pooling layer uses the maximum pooling strategy to filter and reduce the convolution features extracted by the convolution layer, retaining the most significant features.

[0058] Again, the tensor features extracted by the pooling layer are transmitted to the fully connected layer. The fully connected layer expands and concatenates the vector features of the pooling layer.

[0059] Finally, the result prediction is mapped to the [0.1] interval through softmax, where the largest value is the predicted classification.

[0060] A further improvement of the present invention is that the analysis of the audio data in step 2 is specifically to collect the audio of the conversation during the medical treatment process through a recording device, and perform audio sentiment analysis on the collected data based on a bi-LSTM-attention mechanism network, wherein the bi-LSTM-attention mechanism network includes a bi-LSTM layer, a multi-head attention mechanism and an over-prediction classification:

[0061] First: Use the bi-LSTM layer to extract features from the sound and process the sequence data;

[0062] Secondly: the sequence data processed by the bi-LSTM layer is fed into the multi-head attention mechanism;

[0063] Finally: Use attention output for over-prediction classification.

[0064] A further improvement of the present invention is that: the step 3 of constructing a multimodal doctor-patient conflict assessment model specifically includes the following steps:

[0065] Step 3.1, feature fusion: text data, image data and audio data are fused in series strategy to obtain multimodal fusion feature E:

[0066] E=[T,V,S]

[0067] Among them, T is text data, V is image data, and S is audio data;

[0068] Step 3.2: Use the fusion features obtained in step 3.1 to predict the possibility of conflict, and output the probability P of predicted conflict through the fully connected layer:

[0069] P = linear(E);

[0070] Step 3.3: Predict the nature of the conflict: Use softmax to map the result prediction to the [0.1] interval, where the largest value is the predicted classification:

[0071] I = Softmax(E).

[0072] The further improvement of the present invention is that: the step 4 is specifically: according to the evaluation result of the multimodal doctor-patient conflict evaluation model in step 3, the solution to alleviate the dispute includes communication, negotiation, and intervention, and a solution satisfactory to both the doctor and the patient is reached through communication, negotiation and intervention. The specific method includes the following steps:

[0073] Step 4.1, determination of the nature of the conflict: determine the nature of the doctor-patient conflict according to the evaluation results of the multimodal doctor-patient conflict evaluation model, and call the corresponding conditional solution library according to its nature;

[0074] Step 4.2, mediation plan model: according to the background, process, method, result, and content of the agreement of the medical conflict, call the corresponding condition type, procedure and method.

[0075] A further improvement of the present invention is that the step 4.2 is specifically:

[0076] Step 4.2.1, data collection: organize the time, place, people, and process of past medical incidents into a structured data format;

[0077] Step 4.2.2, model training: Use the decision tree and SVM machine learning algorithms to learn the features and train the intelligent recommendation model for doctor-patient mediation solutions;

[0078] Step 4.2.3, case analysis: structure the relevant information of the current event and input it into the multimodal doctor-patient conflict assessment model;

[0079] Step 4.2.4, solution selection: The multimodal doctor-patient conflict assessment model intelligently generates corresponding mediation solutions based on the input cases;

[0080] Step 4.2.5, solution feedback: Based on the mediation results, evaluate and provide feedback on the results generated by the multimodal doctor-patient conflict assessment model.

[0081] A further improvement of the present invention is that the step 5 specifically comprises:

[0082] Step 5.1, process record: Make a structured record of the background, cause, time, place, person, method, process, result, and effect of the mediation of doctor-patient conflict, and record and mark important text materials, audio materials, and image materials in the mediation process;

[0083] Step 5.2, record analysis: Sort out and summarize the records in step 5.1, ask professionals to analyze their processes and nature, make archives of the analysis results, store them in the mediation database, and analyze the results of the mediation, including the success rate of mediation, mediation cycle, and mediation effect, in order to evaluate the effectiveness and shortcomings of the mediation work;

[0084] Step 5.3: Develop a cycle, method, and database for tracking mediation effects, conduct regular tracking, and record the tracking process, including the background, reasons, time, location, people, methods, processes, results, and effects in a structured manner. Organize and summarize them, and ask professionals to analyze and evaluate the mediation effects. According to the tracking results, revise the model and solution library to improve the mediation effect.

[0085] A further improvement of the present invention is that the step 6 is specifically:

[0086] Step 6.1, Theoretical update: Regularly absorb the latest academic theoretical results, optimize the structure of the solution library, and supplement the missing dimensions;

[0087] Step 6.2, Plan Update: Regularly summarize and add mediation plan prototypes to increase the plan reserve;

[0088] Step 6.3, Tracking feedback: Optimize the solution library based on the mediation tracking results;

[0089] Step 6.4: Usage guidance: Provide guidance to employees on how to use the solution library and absorb their opinions and suggestions in a timely manner.

[0090] The beneficial effects of the present invention are:

[0091] The present invention is a new means of mediating doctor-patient conflicts by combining multimodal information such as text information, image information and audio information, and is novel.

[0092] In the field of medical conflict mediation, the present invention combines multimodal technology, deep learning methods, sentiment analysis technology and big data to achieve intelligent recommendation of mediation plans, innovates the mediation method, and is innovative.

[0093] The intelligent doctor-patient conflict mediation method based on multimodal technology proposed in the present invention can realize the early warning of doctor-patient conflicts and provide intelligent mediation solutions, which can solve conflicts in a timely and effective manner and improve the satisfaction of doctors and patients. Therefore, the invention is expected to become an important tool for doctor-patient conflict mediation and has practicality.

[0094] The present invention provides useful guidance for the future intelligent mediation of doctor-patient conflicts, and is of great significance for resolving doctor-patient disputes and alleviating doctor-patient conflicts. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 It is a flow chart of the mediation method of the present invention.

[0096] Figure 2 It is a schematic diagram of intelligent recommendation of the mediation scheme of the present invention.

[0097] Figure 3 It is a detailed flow chart of the adjustment method of the present invention. DETAILED DESCRIPTION

[0098] The following will disclose the embodiments of the present invention with drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.

[0099] like Figure 1 , 3 As shown, the present invention is an intelligent medical conflict mediation method based on multimodal technology. The intelligent medical conflict mediation method captures text data, image data and audio data through multimodal technology, and can capture language information and non-language information more comprehensively; by integrating multimodal features such as image features, text features and audio features, it makes up for the limitations of single modal features and can achieve more comprehensive utilization of data information. In addition, compared with traditional deep learning models, the multimodal audio-visual recognition model based on deep learning constructed by the present invention can more fully learn the characteristics of each modal data and effectively improve the recognition performance. In addition, by constructing a solution case library, the present invention can more appropriately and intelligently recommend mediation plans based on the predicted results of the possibility and nature of the conflict, which is conducive to the effective and fair resolution of medical conflicts. Specifically including the following steps:

[0100] Step 1: Build a doctor-patient conflict database: Build a structured database, an unstructured database, and a mediation solution library, and obtain text data, image data, and audio data.

[0101] 1. Database construction includes the collection of medical records, patient satisfaction surveys, and physician evaluations.

[0102] (1) Medical records:

[0103] project content Patient Information Name, gender, age, contact information Medical history Family history, past illnesses, drug allergies Treatment Records Dosage and frequency of medications, surgical information, and effectiveness of treatment Follow-up records Frequency of follow-up visits and follow-up results Medical Report Surgical Report

[0104] Classify at one level at a time, then subdivide at the next level to construct a scale.

[0105] (2) Patient satisfaction survey:

[0106] project content Medical services Very dissatisfied, Somewhat dissatisfied, Satisfied, Somewhat satisfied, Very satisfied Hospital environment Very dissatisfied, Somewhat dissatisfied, Satisfied, Somewhat satisfied, Very satisfied Medical treatment process Very dissatisfied, Somewhat dissatisfied, Satisfied, Somewhat satisfied, Very satisfied Treatment Effects Very dissatisfied, Somewhat dissatisfied, Satisfied, Somewhat satisfied, Very satisfied

[0107] Classify at one level at a time, then subdivide at the next level to construct a scale.

[0108] (3) Doctor’s evaluation

[0109] project content Accurate diagnosis Is the diagnosis accurate? Treatment Effects Is the treatment effective? Service attitude Is the attitude friendly?

[0110] Classify at one level at a time, then subdivide at the next level to construct a scale.

[0111] 2. The construction of multimodal data in the database includes the following steps:

[0112] Step 1.1, construct language data: collect language texts of doctor-patient communication, including written records and oral communication, use scanners to convert the texts of doctor-patient communication into text data, use ORC technology to convert the image texts of doctor-patient communication into text data, and use sound recognition software to realize online speech-to-text conversion to convert the audio data of recorded doctor-patient communication into text;

[0113] Step 1.2: Construct image data: Obtain visual information of facial expressions, body language, and postures during doctor-patient communication through video recording or image acquisition equipment. The specific steps include:

[0114] Step 1.2.1, use a camera to record image data of doctor-patient communication;

[0115] Step 1.2.2: Use high-precision sensors to capture details and facial expressions of doctor-patient communication;

[0116] Step 1.2.3, converting the image data collected during the doctor-patient communication in step 1.2.1 into a digital signal for computer processing;

[0117] Step 1.2.4: Build an image database to store the collected image data of doctor-patient communication.

[0118] Step 1.3, construct audio data: Through voice sentiment analysis, capture the emotions in the communication process, such as anger, anxiety, satisfaction, etc. The construction of step 1.3 audio data is as follows:

[0119] Step 1.3.1. Use a microphone to capture the sound of doctor-patient communication;

[0120] Step 1.3.2, use recording equipment to record the voice of doctor-patient communication;

[0121] Step 1.3.3: Use speech recognition software to achieve online speech-to-text conversion to convert the audio data of the recorded doctor-patient communication into text.

[0122] In step 1, the mediation solution library is constructed under the guidance of experts, summarizing the general patterns of medical conflict mediation solutions in various situations, including the construction of the solution library and the composition of the solution library. The construction of the solution library is based on the Mediation Management Information System (MMIS) as the main body, and Eviews, SPSS and Excel are used as auxiliary tools.

[0123] The composition of the conflict mediation solution library specifically includes:

[0124] Step 1.4.1. Collection of medical conflict mediation cases: Collect medical conflict mediation cases that have been handled from hospitals, courts, arbitration institutions, and mediation institutions, and invite medical conflict mediation experts to provide cases to build a mediation foundation;

[0125] Step 1.4.3, Organize medical conflict mediation cases: Classify the types, procedures and methods of mediation according to the background, process, method, result and agreement, organize the collected cases and store them in the solution library;

[0126] Step 1.4.3, Doctor-patient conflict mediation plan template: Based on mediation theory, objectives, strategies, procedures, methods and other factors, formulate a doctor-patient conflict mediation plan template;

[0127] Step 1.4.4: Solution library maintenance: Maintain the mediation solution library to ensure the robustness of the solution library.

[0128] The maintenance of the adjustment scheme library is as follows:

[0129] Regular review: Regularly review and update the mediation plans in the medical conflict mediation plan library to ensure their effectiveness, match them with laws and regulations, and ensure their effective and smooth implementation.

[0130] Clean up outdated plans: Regularly clean up outdated mediation plans based on actual conditions and laws and regulations.

[0131] Regular updates: Regularly collect new mediation solutions from the medical-patient conflict mediation solution library to maintain the timeliness and diversity of the solution library. The sources are mediators, legal experts, university researchers, and government agencies.

[0132] Maintain documentation: Build detailed documentation for the mediation plan, clearly listing the steps and examples, and including relevant regulations.

[0133] Feedback mechanism: Build a user feedback mechanism to timely absorb user suggestions to improve the content and functions of the solution library. Usage training: Encourage personnel to use the solution library and provide them with usage training.

[0134] Step 1.4.5, conflict mediation solution library structure: Maintain the mediation solution library to ensure the robustness of the solution library.

[0135]

[0136] Step 2: Receiving complaints about medical disputes: First, the regular platform complaint channel, and second, the autonomous recognition channel that analyzes and extracts features from the text data, image data, and audio data obtained in step 1 based on deep learning and sentiment dictionaries. Multimodal data analysis specifically includes text data analysis, image data analysis, and audio data analysis.

[0137] The text data analysis in step 2 includes text data collection and text sentiment value calculation, which specifically includes the following steps:

[0138] Step 2.1, text data collection and conversion: Write a crawler based on Python to collect text materials of doctor-patient communication, and collect the language text of doctor-patient communication based on speech-to-text technology;

[0139] Step 2.2: Structured text collection: Based on RPA technology, automatically collect structured texts of medical records, patient satisfaction surveys, and doctor evaluations when seeking medical treatment;

[0140] Step 2.3: Based on the language text collected in step 2.1 and the structured text collected in step 2.2, a medical sentiment dictionary and sentiment rules are constructed, and the sentiment value is calculated.

[0141] In this step, the medical sentiment dictionary is constructed by: constructing a medical sentiment dictionary based on the HowNet sentiment dictionary, the NTSUSD simplified Chinese sentiment polarity dictionary, and manually adding professional vocabulary in the medical field. Constructing sentiment rules to calculate sentiment values ​​specifically includes the following steps:

[0142] Step 2.3.1, use the jieba package to segment the language text and structured text;

[0143] Step 2.3.2: Search for sentiment words in the sentence and mark the sentiment polarity and position. The marking of sentiment polarity and position is achieved through the following process:

[0144] Importing libraries

[0145] Importing the Natural Language Processing Library

[0146] Importing the Tokenizer

[0147] Import stop word list

[0148] Importing the Part-of-Speech Tagger

[0149] Define the function to extract sentiment words

[0150] Participle

[0151] Part-of-speech tagging

[0152] Initialize the sentiment word list

[0153] Iterate over each word and its part of speech

[0154] Add to sentiment word list

[0155] Returns the list of sentiment words

[0156] Specific implementation

[0157] Define the sentences to be processed

[0158] Calling a function

[0159] Print sentiment words.

[0160] Step 2.3.3, weight the sentiment value according to the degree of the sentiment word. If a negative word appears before the sentiment word, the sentiment polarity is reversed;

[0161] Step 2.3.4: Calculate the sentiment polarity value of each sentence separately. The process is as follows:

[0162] Split sentences into lists

[0163] For each sentence in the list

[0164] Use jieba to segment words without full-mode segmentation to get the word list of positive and negative words to initialize

[0165] For each word in the word list

[0166] If the word is in the active dictionary

[0167] Positive score 1

[0168] For each word in the current word signature

[0169] If the word is in the most value dictionary

[0170] Multiply the positive score by 5

[0171] If the word is in the non-common dictionary

[0172] Multiply the positive score by 4

[0173] If the word is in the slightly higher-level dictionary

[0174] Multiply the positive score by 3

[0175] If the word is in the fine-grained dictionary

[0176] Multiply the positive score by 0.7

[0177] If the word is in the less advanced dictionary

[0178] Multiply the positive score by -0.2

[0179] If the word is in the transitional dictionary

[0180] Multiply the positive score by -0.6

[0181] If the word is in the reverse dictionary

[0182] Reverse polarity

[0183] Calculate the sentiment value of all words in the sentence and sum them up

[0184] Or if the word is in the negative dictionary

[0185] Perform the same logic as for the positive correlation, but use the negative score and its associated variable.

[0186] Step 2.3.5: Subtract the negative value from the positive value to get the emotional value of each sentence.

[0187] Step 2.4: Perform sentiment polarity classification based on the BERT-TextRCNN model, where the BERT-TextRCNN model includes four parts: BERT language pre-training model, bi-LSTM layer, pooling layer and softmax layer. Specifically, the following steps are included:

[0188] Step 2.4.1. Build a BERT language pre-training model. The BERT model is a pre-trained language representation model that can generate bidirectional language representations, thereby better realizing the vectorized representation of text.

[0189] Step 2.4.2: Input the tensor obtained by the BERT language pre-training model into the bi-LSTM layer.

[0190] Bidirectional long short-term memory network is a type of recurrent neural network, which is a special type of RNN and is good at learning long sequence information.

[0191] i t =σ(A ii x t +m ii +A hi h t-1 +m hi )

[0192] f t =σ(A if x t +m if +A hf h t-1 +m hf )

[0193] g t =tanh(A ig x t +m ig +A hg h t-1 +m hg )

[0194] o t =σ(A io x t +m io +A ho h t-1 +m ho )

[0195] c t =f t ⊙c t-1 +i t ⊙g t

[0196] h t =o t ⊙tanh(c t )

[0197] Step 2.4.3: The tensor obtained by the bi-LSTM layer is transferred to the maximum pooling layer. The information extracted by the bi-LSTM layer is filtered and reduced in dimension through the maximum pooling strategy to retain the most significant features.

[0198] M=Maxpool(H)

[0199] Step 2.4.4, use the softmax layer to implement text sentiment classification, and map the result prediction to the [0.1] interval through softmax, where the largest value is the predicted classification.

[0200] T = Softmax(M)

[0201] The specific analysis of image data is as follows: use a camera to shoot the patient's behavior during medical treatment, and use the collected images to predict behavior based on a convolutional neural network, where the convolutional neural network includes a convolution layer, a pooling layer, a fully connected layer, and a behavior prediction classification:

[0202] First, construct the convolution layer: The convolution layer abstracts and extracts image information through (2, 3, 4) multi-scale convolution kernels.

[0203] C=Cov(X)

[0204] Secondly, the information extracted by the convolution layer is input into the pooling layer. The pooling layer uses the maximum pooling strategy to filter and reduce the convolution features extracted by the convolution layer to retain the most significant features.

[0205] M=Maxpool(C)

[0206] Again, the tensor features extracted by the pooling layer are transmitted to the fully connected layer. The fully connected layer expands and concatenates the vector features of the pooling layer.

[0207] Finally, the result prediction is mapped to the [0.1] interval through softmax, where the largest value is the predicted classification.

[0208] V = Softmax(M)

[0209] The analysis of audio data is specifically to collect the audio of the conversation during the medical treatment process through a recording device, and perform audio sentiment analysis on the collected data based on a bi-LSTM-attention mechanism network, wherein the bi-LSTM-attention mechanism network includes a bi-LSTM layer, a multi-head attention mechanism and an over-prediction classification:

[0210] First: Use the bi-LSTM layer to extract features from the sound and process sequence data. The bidirectional long short-term memory network is good at processing time series data, and use bi-LSTM to extract features from the sound.

[0211] i t =σ(A ii x t +m ii +A hi h t-1 +m hi )

[0212] f t =σ(A if x t +m if +A hf ht-1 +m hf )

[0213] g t =tanh(A ig x t +m ig +A hg h t-1 +m hg )

[0214] o t =σ(A io x t +m io +A ho h t-1 +m ho )

[0215] c t =f t ⊙c t-1 +i t ⊙g t

[0216] h t =o t ⊙tanh(c t )

[0217] Secondly: the sequence data processed by the bi-LSTM layer is fed into the multi-head attention mechanism, which has better performance and can better give greater weight to important information.

[0218] Multi(q,k,v)=[h 1 ,…,h h ]A O

[0219] h i =Attention(qA iQ ,kA iK ,vA iV ).

[0220] Finally: Use the attention output to perform over-prediction classification. Use softmax to map the result prediction to the [0.1] interval, where the largest value is the predicted classification.

[0221] S=Softmax(M).

[0222] Step 3: Compare the conflict mediation solution library and evaluate the doctor-patient conflict. The subject, object and relationship in the doctor-patient conflict are abstracted, and on this basis, a multimodal doctor-patient conflict evaluation model is constructed to evaluate the doctor-patient conflict: the collected text data, image data and audio data are integrated to construct a multimodal doctor-patient conflict evaluation model, and based on the analysis and identification results of the multimodal doctor-patient conflict evaluation model, the doctor-patient conflict is evaluated to determine the nature and severity of the conflict.

[0223] The construction of a multimodal doctor-patient conflict assessment model specifically includes the following steps:

[0224] Step 3.1, first abstract the subject, object and the relationship between them in the doctor-patient conflict. The subject includes doctors and patients. The doctor includes three parts: medical services, explanation of diagnosis and treatment plan, and the patient includes three parts: condition, emotions and needs. The object includes medical problems, communication process and conflict mediation plan. Medical problems include the patient's condition, diagnosis and treatment plan; the communication process includes communication methods, communication feelings and communication results; the conflict mediation plan includes the type of conflict and the corresponding mediation plan (this part does not need to be abstracted, it is the output result, and the rest is the input).

[0225] Step 3.2: Perform feature fusion: Perform serial strategy fusion on the text data, image data and audio data extracted based on deep learning and sentiment feature methods to obtain multimodal fusion feature E:

[0226] E = [T, V, S]

[0227] Among them, T is text data, V is image data, and S is audio data;

[0228] Step 3.3: Use the fusion features obtained in step 3.1 to predict the possibility of conflict, and output the probability P of predicted conflict through the fully connected layer:

[0229] P = linear(E);

[0230] Step 3.4: Predict the nature of the conflict: Use softmax to map the result prediction to the [0.1] interval, where the largest value is the predicted classification:

[0231] I = Softmax(E).

[0232] Based on the analysis and identification results of the multimodal early warning model, the doctor-patient conflict is evaluated. If a conflict is determined to have occurred, a complaint is generated and an early warning is sent to the relevant platform. On this basis, the nature and severity of the conflict are determined.

[0233] Nature of conflict Conflict Level Serious violations Level 1: Law Dissatisfaction Second level: third party Slight contradiction Level 3: Doctors and patients

[0234] Step 4: Provide a mediation plan. Based on the evaluation results of step 3, a customized solution to the medical dispute is automatically generated.

[0235] like Figure 2 As shown, according to the evaluation results of the multimodal doctor-patient conflict evaluation model in step 3, the generated dispute resolution solution includes communication, negotiation, and intervention. Through communication, negotiation, and intervention, a solution satisfactory to both the doctor and the patient is reached. The specific method includes the following steps:

[0236] Step 4.1, determination of the nature of the conflict: determine the nature of the doctor-patient conflict based on the evaluation results of the multimodal doctor-patient conflict evaluation model, and call the corresponding conditional solution library based on its nature.

[0237] Specifically:

[0238] Step 4.2.1, data collection: organize the time, place, people, and process of past medical incidents into a structured data format; as shown in the following table:

[0239] Serial number name describe 1 Event event 2 ID Event Identifier 3 Time Event (year / month / day / time) 4 Location Place 5 Doctor ID Physician Identifier 6 Patient ID Patient Identifiers 7 Content Event content 8 Type Nature of the incident 9 Action Treatment measures 10 Result result 11 Witnesses eyewitness

[0240] Step 4.2.2, model training: Use the decision tree and SVM machine learning algorithms to learn the features and train the intelligent recommendation model for medical mediation solutions. The learning process of the decision tree and SVM machine learning algorithms is as follows:

[0241] (1) Decision Tree

[0242] Import the libraries required for decision tree and SVM classifiers

[0243] Loading the medical dataset

[0244] -Specify the dataset path

[0245] - Assigning values ​​to the features of the dataset

[0246] -Assign the target variable of the dataset

[0247] Divide the dataset into training and testing sets

[0248] - Divide the dataset into X_train, X_test, y_train, y_test

[0249] -Test set size is set to 20%

[0250] Decision Tree Classifier

[0251] - Create a decision tree classifier

[0252] - Train the model using X_train and y_train

[0253] -Use X_test to make predictions and get y_pred

[0254] -Calculation accuracy

[0255] SVM Classifier

[0256] -Create SVM classifier

[0257] - Train the model using X_train and y_train

[0258] -Use X_test to make predictions and get y_pred

[0259] -Calculate accuracy.

[0260] Step 4.2.3, case analysis: structure the relevant information of the current event and input it into the multimodal doctor-patient conflict assessment model;

[0261] Step 4.2.4, solution selection: The multimodal doctor-patient conflict assessment model generates corresponding mediation solutions based on the input cases and training based on the deep learning model;

[0262] Step 4.2.5, solution feedback: Based on the mediation results, evaluate and provide feedback on the results generated by the multimodal doctor-patient conflict assessment model.

[0263] Step 4.2, mediation plan model: according to the background, process, method, result, and content of the agreement of the medical conflict, call the corresponding condition type, procedure and method.

[0264] Step 5. Mediation record and tracking: The multimodal doctor-patient conflict assessment model records the entire process of doctor-patient conflict mediation and builds a mediation database, including the opinions of all parties and the consensus reached, which is used for subsequent conflict tracking and evaluation of the effectiveness of mediation results.

[0265] Specifically include:

[0266] Step 5.1, process record: Make a structured record of the background, cause, time, place, person, method, process, result, and effect of the mediation of doctor-patient conflict, and manually record and mark the important text materials, audio materials, and image materials in the mediation process;

[0267] Step 5.2, record analysis: Sort out and summarize the records in step 5.1, ask professionals, i.e. relevant scholars, to analyze their processes and nature, make archives of the analysis results, store them in the mediation database, and analyze the results of the mediation, including the success rate of mediation, mediation cycle, and mediation effect, in order to evaluate the effectiveness and shortcomings of the mediation work;

[0268] Step 5.3: Develop a cycle, method, and database for tracking mediation effects, conduct regular tracking, and record the tracking process, including the background, reasons, time, location, people, methods, processes, results, and effects in a structured manner. Organize and summarize them, and ask professionals to analyze and evaluate the mediation effects. According to the tracking results, revise the model and solution library to improve the mediation effect.

[0269] Step 6: Update and iterate the conflict mediation solution library: If you encounter a new mediation problem, summarize the mediation process into logic, iterate the solution library, and provide useful guidance for the next mediation of the same case. Specifically:

[0270] Step 6.1, Theoretical update: Regularly absorb the latest academic theoretical results, optimize the structure of the solution library, and supplement the missing dimensions;

[0271] Step 6.2, plan update: Regularly manually summarize and add mediation plan prototypes to increase the plan reserve;

[0272] Step 6.3, Tracking feedback: Based on the mediation tracking results, use deep learning algorithms to optimize the solution library;

[0273] Step 6.4: Usage guidance: Provide guidance to employees on how to use the solution library and absorb their opinions and suggestions in a timely manner.

[0274] The present invention can provide more effective and personalized medical mediation solutions, and can also analyze and evaluate medical conflicts in a more in-depth manner. By integrating multimodal information channels and data, this patent can more timely and accurately grasp the trends and development trends of both parties in medical conflicts, timely warn of possible conflicts, and provide intelligent conditional solutions, thereby promoting more reasonable conflict resolution.

[0275] In order to verify the present invention, the patent content of the present invention was applied in practice. One day, a patient had a conflict with a doctor in a hospital in Nanjing due to the treatment of ear disease. Through the mediation of the present invention, the conflict was successfully mediated. The specific process is as follows: Step 1: Collect doctor-patient conflict data text data: collect doctors' diagnosis reports and suggestions, medical records, patient satisfaction surveys, doctor evaluations, patients' feedback and questions, and language texts of doctor-patient communication, including written records and oral communication.

[0276] Image data: Obtain visual information about facial expressions, body language, and postures during doctor-patient communication through video recording or image acquisition equipment. Use cameras to record image data of doctor-patient communication, use high-precision sensors to capture details and changes in expression during doctor-patient communication, and convert the image data collected during doctor-patient communication into digital signals for computer processing.

[0277] Audio data: Through voice sentiment analysis, the emotions during the communication process, such as anger, anxiety, satisfaction, etc. Use a microphone to capture the sound of the doctor-patient communication, and use speech recognition software to achieve online speech-to-text conversion to convert the audio data of the recorded doctor-patient communication into text.

[0278] Step 2: Receive complaints about conflicts between doctors and patients. The patient believes that the doctor did not fully explain the ear problem and feels that communication is not smooth, but did not submit a complaint through the regular channel (complaint platform) to express dissatisfaction with the doctor. However, this patent can use multimodal data analysis channels to analyze text, image and audio data through deep learning and sentiment dictionary. Through the prediction results of the model, it was found that the patient expressed dissatisfaction in the voice recording. The occurrence of conflicts between doctors and patients was discovered in time, and complaints were generated for reception.

[0279] Step 3: Compare with the conflict mediation program library and evaluate the doctor-patient conflict. Compare the form and content of the ear patient's complaint with the conflict mediation program library to find similar conflict situations, that is, conflicts of the inappropriate interpretation type, and preliminarily evaluate the nature and severity of the conflict.

[0280] Step 4: Provide a mediation plan Based on the results of the conflict and misunderstanding assessment, the system automatically generates a corresponding mediation plan, including suggestions for improving communication, clarifying the explanation of the diagnosis, and possible treatment plans. The plan aims to respect the feelings of both doctors and patients and improve the effectiveness of communication. Based on this plan, the doctor-patient conflict was effectively resolved.

[0281] Step 5: Mediation Record and Tracking Record the entire mediation process, including the doctor's explanation to the patient, the patient's feedback, and the final consensus. These records will be used to monitor conflicts and evaluate effectiveness.

[0282] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. An intelligent doctor-patient conflict mediation method based on multimodal technology, characterized by: The intelligent doctor-patient conflict mediation method specifically comprises the following steps: Step 1: Build a doctor-patient conflict database: build a structured database, an unstructured database, and a mediation solution library, and obtain text data, image data, and audio data; Step 2: Analyze multimodal data to obtain modal features: Analyze and extract features of the text data, image data, and audio data obtained in step 1 based on deep learning and sentiment dictionary to obtain modal features; Step 3: Construct a multimodal doctor-patient conflict assessment model and evaluate the doctor-patient conflict: Integrate the collected text data, image data, and audio data to construct a multimodal doctor-patient conflict assessment model, and evaluate the doctor-patient conflict based on the analysis and identification results of the multimodal doctor-patient conflict assessment model to determine the nature and severity of the conflict; Step 4: According to the evaluation results of step 3, a customized solution to the medical dispute is automatically generated; Step 5: Mediation Record and Tracking: The multimodal doctor-patient conflict assessment model records the entire process of doctor-patient conflict mediation and builds a mediation database, including the opinions of all parties and the consensus reached, for subsequent conflict tracking and evaluation of the effectiveness of mediation results; Step 6: Update and iterate the mediation solution library: If you encounter a new mediation problem, summarize and abstract the mediation process into logic, iterate the solution library, and provide useful guidance for the next mediation of the same case.

2. According to claim 1, an intelligent doctor-patient conflict mediation method based on multimodal technology is characterized by: The step 3 of constructing a multimodal doctor-patient conflict assessment model specifically includes the following steps: Step 3.1, feature fusion: text data, image data and audio data are fused in series strategy to obtain multimodal fusion feature E: E=[T,V,S] Among them, T is text data, V is image data, and S is audio data; Step 3.2: Use the fusion features obtained in step 3.1 to predict the possibility of conflict, and output the probability P of predicted conflict through the fully connected layer: P = linear(E); Step 3.3: Predict the nature of the conflict: Use softmax to map the result prediction to the [0.1] interval, where the largest value is the predicted classification: I = Softmax(E).

3. According to claim 1, the intelligent doctor-patient conflict mediation method based on multimodal technology is characterized by: The step 4 is specifically: according to the evaluation results of the multimodal doctor-patient conflict evaluation model in step 3, a solution to alleviate the dispute is generated, including communication, negotiation, and intervention, and a solution satisfactory to both the doctor and the patient is reached through communication, negotiation, and intervention. The specific method includes the following steps: Step 4.1, determination of the nature of the conflict: determine the nature of the doctor-patient conflict according to the evaluation results of the multimodal doctor-patient conflict evaluation model, and call the corresponding conditional solution library according to its nature; Step 4.2, mediation plan model: according to the background, process, method, result, and content of the agreement of the medical conflict, call the corresponding condition type, procedure and method.

4. According to claim 3, the intelligent doctor-patient conflict mediation method based on multimodal technology is characterized by: The step 4.2 is specifically as follows: Step 4.2.1, data collection: organize the time, place, people, and process of past medical incidents into a structured data format; Step 4.2.2, model training: Use the decision tree and SVM machine learning algorithms to learn the features and train the intelligent recommendation model for doctor-patient mediation solutions; Step 4.2.3, case analysis: structure the relevant information of the current event and input it into the multimodal doctor-patient conflict assessment model; Step 4.2.4, solution selection: The multimodal doctor-patient conflict assessment model intelligently generates corresponding mediation solutions based on the input cases; Step 4.2.5, solution feedback: Based on the mediation results, evaluate and provide feedback on the results generated by the multimodal doctor-patient conflict assessment model.

5. According to claim 1, the intelligent doctor-patient conflict mediation method based on multimodal technology is characterized by: The step 5 specifically includes: Step 5.1, process record: Make a structured record of the background, cause, time, place, person, method, process, result, and effect of the mediation of doctor-patient conflict, and record and mark important text materials, audio materials, and image materials in the mediation process; Step 5.2, record analysis: Sort out and summarize the records in step 5.1, ask professionals to analyze their processes and nature, make archives of the analysis results, store them in the mediation database, and analyze the results of the mediation, including the success rate of mediation, mediation cycle, and mediation effect, in order to evaluate the effectiveness and shortcomings of the mediation work; Step 5.3: Develop a cycle, method, and database for tracking mediation effects, conduct regular tracking, and record the tracking process, including the background, reasons, time, location, people, methods, processes, results, and effects in a structured manner. Organize and summarize them, and ask professionals to analyze and evaluate the mediation effects. According to the tracking results, revise the model and solution library to improve the mediation effect.

6. The intelligent doctor-patient conflict mediation method based on multimodal technology according to claim 1 is characterized by: The step 6 is specifically as follows: Step 6.1, Theoretical update: Regularly absorb the latest academic theoretical results, optimize the structure of the solution library, and supplement the missing dimensions; Step 6.2, Plan Update: Regularly summarize and add mediation plan prototypes to increase the plan reserve; Step 6.3, Tracking feedback: Optimize the solution library based on the mediation tracking results; Step 6.4: Usage guidance: Provide guidance to employees on how to use the solution library and absorb their opinions and suggestions in a timely manner.

7. An intelligent doctor-patient conflict mediation method based on multimodal technology according to any one of claims 1 to 6, characterized in that: The text data analysis in step 2 includes text data collection and text sentiment value calculation, which specifically includes the following steps: Step 2.1, text data collection and conversion: Write a crawler based on Python to collect text materials of doctor-patient communication, and collect the language text of doctor-patient communication based on speech-to-text technology; Step 2.2: Structured text collection: Based on RPA technology, automatically collect structured texts of medical records, patient satisfaction surveys, and doctor evaluations when seeking medical treatment; Step 2.3, based on the language text collected in step 2.1 and the structured text collected in step 2.2, construct a medical sentiment dictionary and sentiment rules, and calculate the sentiment value; Step 2.4: Perform sentiment polarity classification based on the BERT-TextRCNN model, where the BERT-TextRCNN model consists of four parts: BERT language pre-training model, bi-LSTM layer, pooling layer and softmax layer.

8. The intelligent doctor-patient conflict mediation method based on multimodal technology according to claim 7 is characterized by: In step 2.3, the medical sentiment dictionary is constructed by building a medical sentiment dictionary based on the HowNet sentiment dictionary, the NTSUSD simplified Chinese sentiment polarity dictionary, and manually adding professional medical vocabulary. The construction of sentiment rules to calculate sentiment values ​​specifically includes the following steps: Step 2.3.1, use the jieba package to segment the language text and structured text; Step 2.3.2, search for sentiment words in the sentence, and mark the sentiment polarity and position; Step 2.3.3, weight the sentiment value according to the degree of the sentiment word. If a negative word appears before the sentiment word, the sentiment polarity is reversed; Step 2.3.4, calculate the sentiment polarity value of each sentence respectively; Step 2.3.5: Subtract the negative value from the positive value to get the emotional value of each sentence.

9. The intelligent doctor-patient conflict mediation method based on multimodal technology according to claim 7 is characterized by: The step 2.4 specifically includes the following steps: Step 2.4.1, build a BERT language pre-training model; Step 2.4.2: Input the tensor obtained by the BERT language pre-training model into the bi-LSTM layer; Step 2.4.3: The tensor obtained by the bi-LSTM layer is transferred to the maximum pooling layer. The information extracted by the bi-LSTM layer is filtered and reduced in dimension through the maximum pooling strategy to retain the most significant features. Step 2.4.4, use the softmax layer to implement text sentiment classification, and map the result prediction to the [0.1] interval through softmax, where the largest value is the predicted classification.

10. The intelligent doctor-patient conflict mediation method based on multimodal technology according to claim 1 is characterized by: The analysis of the image data in step 2 is specifically as follows: using a camera to shoot the patient's behavior during the medical treatment process, and using the collected images to predict behavior based on a convolutional neural network, wherein the convolutional neural network includes a convolutional layer, a pooling layer, a fully connected layer and a behavior prediction classification: First, construct the convolution layer: the convolution layer abstracts and extracts image information through (2, 3, 4) multi-scale convolution kernels; Secondly, the information extracted by the convolution layer is input into the pooling layer. The pooling layer uses the maximum pooling strategy to filter and reduce the convolution features extracted by the convolution layer, retaining the most significant features. Again, the tensor features extracted by the pooling layer are transmitted to the fully connected layer, and the fully connected layer expands the vector features of the pooling layer for splicing; Finally, the result prediction is mapped to the [0.1] interval through softmax, where the largest value is the predicted classification.

Citation Information

Cited By

  • Intelligent management and control method and system for whole process of contradiction regulation

    CN120182051A