An analysis system for MDT consultation data

Abstract

The present application provides an analysis system for MDT consultation data, which relates to the field of data processing technology. The system includes a data analysis end and a doctor end. The data analysis end is used to determine the diagnosis result of each subject expert for the disease of the target patient based on the speech voice data of at least one subject expert in the MDT consultation of the target patient for the disease entity of the target patient. The data analysis end is also used to send the diagnosis result to the doctor end corresponding to the subject expert, so as to form a consultation report based on the diagnosis result confirmed by the subject expert, and use deep learning to analyze the confidence level and semantic analysis of the subject expert's speech to determine the diagnosis result of the subject expert. A subject expert review link is also set to improve the accuracy of the generation of automated consultation reports.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an analysis system for MDT consultation data. Background Art

[0002] Different from the treatment pathways of common diseases, the diagnosis and treatment process of rare diseases usually involves the cooperation and evaluation of multidisciplinary teams. Currently, a number of remote multidisciplinary team (MDT) consultation systems have been applied to the diagnosis and treatment of rare diseases, aiming to integrate the resources of different medical centers and professional fields to implement joint diagnosis and treatment of patients with rare diseases.

[0003] After the consultation, a written consultation report is usually generated, which records the discussion and consensus in detail. If this information is manually summarized, it will take a lot of time and effort. In addition, given the complexity and difficulty of diagnosing rare diseases, doctors may need to explore the possibility of multiple potential diseases during the consultation process, which to some extent increases the challenge of determining the final opinions of experts in various fields. Summary of the invention

[0004] The purpose of the embodiment of the present application is to provide an analysis system for MDT consultation results, which uses deep learning to analyze the confidence level and semantic analysis of the subject experts' speeches to determine the diagnosis results of the subject experts. A subject expert review link is also set up to improve the accuracy of the generation of automated consultation reports.

[0005] In a first aspect, the present application provides an analysis system for MDT consultation data, the system comprising a data analysis end and a doctor end, the data analysis end is used to determine the diagnosis result of each subject expert for the disease of the target patient based on the speech data of at least one subject expert in the MDT consultation of the target patient for the disease entity of the target patient. And the data analysis end is also used to send the diagnosis result to the doctor end corresponding to the subject expert, so as to form a consultation report based on the diagnosis result confirmed by the subject expert. Among them, for each subject expert, the data analysis end obtains the diagnosis result of the subject expert on the disease suffered by the target patient in the following way: according to the speech data of the subject expert, a plurality of first target voice segments and second target voice segments are extracted, the first target voice segment is the voice segment where the disease entity is located, and the second target voice segment is the voice segment adjacent to the first target voice segment; for each disease entity, at least one first target voice segment corresponding to the subject expert is input into a pre-trained emotion polarity analysis model, and the emotion category of each first target voice segment output by the emotion polarity analysis model is obtained, and the emotion polarity is used to indicate the emotion polarity of the first target voice segment published by the target subject expert for the disease entity; for each disease entity, at least one second target voice segment corresponding to the disease entity is input into a pre-trained emotion polarity analysis model. The emotion fluctuation analysis model is used to obtain the emotion fluctuation value output by the emotion fluctuation analysis model, and the emotion fluctuation value is used to indicate the degree of emotion change during the subject expert's speech; for each disease entity, based on the emotion stability category and corresponding emotion fluctuation value of the subject expert for the disease entity, the speech emotion score of the subject expert for the disease entity is calculated, and the speech emotion score is used to indicate the degree of confidence of the subject expert in speaking for the disease entity, so as to determine the first target disease entity; for each disease entity, based on the first target speech segment corresponding to the disease entity, the corresponding speech text data is generated; the speech text data corresponding to each disease entity is associated and input into the pre-trained semantic recognition model to obtain the second target disease entity output by the semantic recognition model; the first target disease entity and the second target disease entity are determined as the diagnosis result of the subject expert for the disease suffered by the target patient.

[0006] Preferably, for each subject expert, the data analysis end determines the first target disease entity that the subject expert should target by the following method:

[0007] The disease entities are sorted in ascending order of speech emotion scores to determine the disease entity corresponding to the minimum speech emotion score as the first target disease entity.

[0008] Preferably, for each disease entity, the data analysis end calculates the target subject expert's speech emotion score for the disease entity in the following way: :

[0009] ;

[0010] in, The first target speech segment published by the target subject expert on the disease entity The emotion category, is the number of the first target speech segments, The first target speech segment The corresponding second target speech segment The emotional fluctuation value.

[0011] Preferably, the emotion polarity analysis model includes a first feature extractor and a first classifier, and the data analysis end obtains the emotion category of each first target speech segment output by the emotion polarity analysis model in the following manner: inputting each first target speech segment into the first feature extractor to extract rhythmic features and quality features, and obtaining a first feature vector sequence corresponding to the output of the first feature extractor; inputting the first feature vector sequence into the first classifier, and obtaining the emotion category label value of each first target speech segment output by the first classifier, wherein the emotion category label value includes one of +1, 0, and -1, wherein +1 is used to indicate positive emotions, -1 is used to indicate negative emotions, and 0 is used to indicate stable emotions.

[0012] Preferably, the data analysis end obtains the first feature vector sequence corresponding to the output of the first feature extractor in the following manner: extracting the root mean square energy feature, MFCC feature and fundamental frequency feature of each first target speech segment as a prosodic feature vector through the first feature extractor; extracting the loudness feature and zero-crossing rate feature of each first target speech segment as a quality feature vector through the first feature extractor; and generating a first feature vector sequence based on the prosodic feature vector and the quality feature vector.

[0013] Preferably, the emotion fluctuation analysis model includes a second feature extractor and a second classifier, and the step of obtaining the emotion fluctuation value output by the emotion fluctuation analysis model by the data analysis end in the following manner specifically includes:

[0014] At least one second target speech segment corresponding to the first target speech segment is input into a second feature extractor to extract emotion fluctuation features, and a second feature vector sequence corresponding to the output of the second feature extractor is obtained; the second feature vector sequence is input into a second classifier, and the emotion fluctuation value output by the second classifier is obtained.

[0015] Preferably, the data analysis end obtains the second feature vector sequence corresponding to the output of the second feature extractor in the following manner:

[0016] The sharpness feature and the loudness feature of each second target speech segment are extracted by the second feature extractor as an emotion fluctuation feature vector; based on the emotion fluctuation feature vectors of all the second target speech segments, a second feature vector sequence is generated.

[0017] Preferably, for each first target voice segment, at least one second target voice segment corresponding to the first target voice segment and a pre-trained emotion fluctuation analysis model associated with the first target voice segment are input to obtain the emotion fluctuation value output by the emotion fluctuation analysis model.

[0018] Preferably, the emotion fluctuation analysis model includes a second feature extractor and a second classifier, and the data analysis end obtains the emotion fluctuation value output by the emotion fluctuation analysis model in the following manner: at least one second target speech segment corresponding to the first target speech segment and the first target speech segment are sequentially input into the second feature extractor to extract emotion fluctuation features, and a second feature vector sequence corresponding to the output of the second feature extractor is obtained; the second feature vector sequence is input into the second classifier, and the emotion fluctuation value output by the second classifier is obtained.

[0019] Preferably, the data analysis end obtains the second feature vector sequence corresponding to the output of the second feature extractor in the following manner: extracting the sharpness feature and the loudness feature of the first target speech segment and each second target speech segment as an emotion fluctuation feature vector through the second feature extractor; generating a second feature vector sequence based on the emotion fluctuation feature vectors of the first target speech segment and all second target speech segments.

[0020] The present application provides an analysis system for MDT consultation data, the system includes a data analysis end and a doctor end, the data analysis end is used to extract the speech voice data of multiple different subject experts participating in the MDT consultation on the disease entity of the target patient based on the acquired MDT consultation recording, so as to determine the diagnosis result of each subject expert on the disease of the target patient; and the data analysis end is also used to send the diagnosis result to the doctor end corresponding to the subject expert, so as to form a consultation report based on the diagnosis result confirmed by the subject expert; wherein, for each subject expert, the data analysis end obtains the diagnosis result of the subject expert on the disease of the target patient in the following manner: according to the speech voice data of the subject expert, extract a plurality of first target voice segments and second target voice segments, the first target voice segment is the voice segment where the disease entity is located, and the second target voice segment is the voice segment adjacent to the first target voice segment; for each disease entity, at least one first target voice segment corresponding to the subject expert is respectively input into a pre-trained emotion polarity analysis model, and the emotion category of each first target voice segment output by the emotion polarity analysis model is obtained, and the emotion polarity analysis model is used to obtain the emotion category of each first target voice segment output by the emotion polarity analysis model. The emotion polarity is used to indicate the emotion polarity of the disease entity in the first target speech segment published by the target subject expert; for each disease entity, the disease entity and at least one second target speech segment corresponding to the disease entity are respectively input into the pre-trained emotion fluctuation analysis model to obtain the emotion fluctuation value output by the emotion fluctuation analysis model, and the emotion fluctuation value is used to indicate the degree of emotion change during the subject expert's speech; for each disease entity, based on the emotion stability category and corresponding emotion fluctuation value of the subject expert for the disease entity, the subject expert's speech emotion score for the disease entity is calculated, and the speech emotion score is used to indicate the degree of confidence of the subject expert in speaking about the disease entity, so as to determine the first target disease entity; for each disease entity, based on the first target speech segment and the second target speech segment corresponding to the disease entity, the corresponding speech text data is generated; the speech text data corresponding to each disease entity is associated and input into the pre-trained semantic recognition model to obtain the second target disease entity output by the semantic recognition model; according to the shown first target disease entity and the second target disease entity, the diagnosis result of the subject expert for the disease suffered by the target patient is determined. Deep learning is used to analyze the confidence level and semantic analysis of the subject experts' speeches to determine the subject experts' diagnosis results. A subject expert review process is also set up to improve the accuracy of the generation of automated consultation reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of the structure of an MDT consultation data analysis system provided in an embodiment of the present application;

[0023] Figure 2 A flow chart of the personal diagnosis results of a subject expert provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] First, the application scenario of the present application is described. The technical solution of the present application can be applied to the automatic generation of a consultation report for an MDT consultation in a rare disease MDT consultation system.

[0025] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0026] Figure 1 A schematic diagram of the structure of an analysis system for MDT consultation data provided in an embodiment of the present application. An embodiment of the present application provides an MDT consultation platform for supporting MDT consultation for rare diseases for patients both inside and outside the hospital. The MDT consultation platform may include an analysis system for MDT consultation data. Figure 1 As shown, the analysis system of MDT consultation data includes a data analysis end and a doctor end. The doctor end here can be set up in different medical institutions and departments and belong to different subject experts (doctors).

[0027] For example, for a target patient, doctor terminal 1 can issue an MDT consultation request, and doctor terminal 2, doctor terminal 3, doctor terminal 4, etc. can respond to form an MDT expert group corresponding to the target patient. The corresponding subject experts in the MDT expert group can conduct MDT consultations based on the doctor terminal through text group chats, video conferences, on-site meetings, etc., and the doctor terminal or service center can collect audio, video, text, pictures and other data of the MDT consultation, and summarize them to the analysis system of the MDT consultation data.

[0028] The MDT consultation data analysis system can automatically generate a corresponding consultation report for each MDT consultation and send it to each doctor in the MDT expert group. After the corresponding specialist on the doctor side confirms that the content is correct, the final version of the consultation report is generated and synchronized to each doctor. It can also be sent to the patient client of the target patient.

[0029] The consultation report here may include the target patient's medical data, personal data, personal diagnosis results of experts in various disciplines, MDT consultation diagnosis results, etc.

[0030] The target patient's medical data may include the target patient's medical records and various physical examination data. The target patient's personal data may include personal information such as gender and age, as well as private data such as living habits and medication information.

[0031] In one embodiment of the present application, when the MDT consultation is conducted in the form of a video conference, a consultation report can be automatically generated in the following manner:

[0032] The present application provides a data analysis terminal, which is used to determine the diagnosis result of each subject expert for the disease suffered by the target patient based on the speech voice data of at least one subject expert in the MDT consultation of the target patient for the disease entity suffered by the target patient. And the data analysis terminal is also used to send the diagnosis result to the doctor end corresponding to the subject expert, so as to form a consultation report based on the diagnosis result confirmed by the subject expert, wherein the MDT consultation recording obtained by the data analysis terminal can be a full recording of the meeting, or the speech semantic data of a single subject expert.

[0033] The data analysis end can obtain the video and audio recording of the entire MDT consultation, and can also obtain the partial video and audio recording of the target subject experts during the MDT consultation uploaded by each doctor. For the first case, the data analysis end can extract the speech data corresponding to the experts in different disciplines from the video and audio recording of the entire MDT consultation.

[0034] The data analysis end can analyze the personal diagnosis results of each subject expert. The diagnosis results here include at least one disease entity suffered by the target patient, and are displayed to the subject expert in the application interface through the corresponding doctor end. The subject expert selects or modifies the first target disease entity and the second target disease entity to obtain the final diagnosis result to be formed in the consultation report.

[0035] Figure 2 A flowchart of the personal diagnosis results of a subject expert provided in an embodiment of the present application. Figure 2 As shown, for each subject expert, the data analysis end can obtain the diagnosis result of the subject expert for the disease of the target patient in the following ways:

[0036] S1. Extract multiple first target voice segments and second target voice segments based on the speech data of subject experts. The first target voice segment is the voice segment where the disease entity is located, and the second target voice segment is the voice segment adjacent to the first target voice segment.

[0037] In step S1, the first target speech segment and the second target speech segment may be extracted by the following steps:

[0038] For each subject expert's speech data, noise is removed to remove background noise and environmental interference, and improve the clarity and readability of the text. Speech recognition technology is then used to convert the subject expert's speech data into speech text data. The accurate conversion of text can be completed through audio sampling, signal processing, and training or calling of text generation models.

[0039] The entire text is segmented according to a certain logic, such as by sentence. Specifically, the segmentation is carried out according to the commas in the speech text data. For example, the speech text data of the subject expert can be "Because the target patient has pain in XX part, accompanied by fever symptoms, this is a common manifestation of disease entity A, but the target patient's blood test results do not show relevant manifestations", then the first target text segment W1 "This is a common manifestation of disease entity A" and the second target text segment W2 "Because the target patient has pain in XX part, accompanied by fever symptoms" W3 "But the target patient's blood test results do not show relevant manifestations" can be extracted.

[0040] Finally, based on the text data, the start and end positions of the speech segment containing the disease entity and its adjacent context can be accurately located in the speech data, and then the corresponding first target speech segment (the speech segment where the disease entity is located) and the second target speech segment (the speech segment adjacent to the first target speech segment) can be segmented. The relevant speech recognition technology has been widely used in the prior art and will not be described here.

[0041] S2. For each disease entity, at least one first target voice segment corresponding to the subject expert is input into a pre-trained emotion polarity analysis model to obtain the emotion category of each first target voice segment output by the emotion polarity analysis model. The emotion polarity is used to indicate the emotion polarity of the first target voice segment published by the target subject expert for the disease entity.

[0042] S3. For each disease entity, the disease entity and at least one second target speech segment corresponding to the disease entity are respectively input into a pre-trained emotion fluctuation analysis model to obtain an emotion fluctuation value output by the emotion fluctuation analysis model, where the emotion fluctuation value is used to indicate the degree of emotion change during the subject expert's speech.

[0043] S4. For each disease entity, based on the emotional stability category and corresponding emotional fluctuation value of the subject expert for the disease entity, calculate the subject expert's speech emotion score for the disease entity. The speech emotion score is used to indicate the subject expert's confidence level in speaking about the disease entity, so as to determine the first target disease entity.

[0044] In steps S2 and S3, emotion analysis is performed on the first target speech segment and the second target speech segment of the subject expert. The emotion category in step S2 can indicate the emotion polarity of the target subject expert when delivering the first target speech segment, and in step S3, the emotion fluctuation value when the target subject expert delivers adjacent speech segments can be indicated, thereby analyzing the degree of confidence of the subject expert in speaking about the disease entity. In this way, the degree of confidence of the subject expert in speaking about different disease entities can be determined to determine a first target disease entity.

[0045] S5. For each disease entity, generate corresponding speech text data based on the first target voice segment corresponding to the disease entity.

[0046] S6. Associate the speech text data corresponding to each disease entity and input it into a pre-trained semantic recognition model to obtain a second target disease entity output by the semantic recognition model.

[0047] In order to further improve the accuracy, in steps S5 and S6, a second target disease entity is determined by performing text analysis on the speech text data of the subject experts.

[0048] S7. Determine the first target disease entity and the second target disease entity as the diagnosis results of the subject experts on the disease suffered by the target patient.

[0049] In this embodiment, AI technology is used to analyze speech in both voice and text modes to determine the diagnosis results of each subject expert for the disease suffered by the target patient. The analysis results are more accurate and can improve the accuracy of automatic generation of consultation reports.

[0050] In one embodiment of the present application, the emotion polarity analysis model includes a first feature extractor and a first classifier. The emotion polarity analysis model can be trained and generated in the following manner:

[0051] Collect the first sample voice clip. The first sample voice clip here can be extracted from the historical MDT consultation video conference. The first sample voice clip includes the disease entity.

[0052] The first feature extractor extracts features from the sample speech segment to extract prosodic features and quality features. Among them, the prosodic features are composed of root mean square energy features, MFCC features and fundamental frequency features, which can indicate changes in speech frequency, speech speed, pitch changes, etc., and reflect the speaker's sense of rhythm and emotional intensity.

[0053] The quality features are composed of loudness features and zero-crossing rate features, which can indicate the quality indicators of sound, such as noise level, clarity, sound pressure level, etc., reflecting the physical properties of speech and environmental influences.

[0054] The first feature extractor may use an algorithm (such as Fourier transform, waveform analysis, MFCC, etc.) to extract the above features from the sample speech segment.

[0055] The first feature extractor converts the prosodic feature vector and the quality characteristic vector , combined into the first sample feature vector sequence ,in, The first sample speech segment The rhythmic feature vector of The first sample speech segment The first sample feature vector sequence is then labeled with an emotion category label, where the emotion category label value includes one of +1, 0, and -1, where +1 is used to indicate positive emotion, -1 is used to indicate negative emotion, and 0 is used to indicate stable emotion. Here, for sample speech segments that contain real disease entities, they are labeled as 0, and other sample speech segments that do not contain real disease entities are mostly labeled as +1, and some are labeled as -1 (such as when subject experts quarrel).

[0056] The first sample feature vector sequence is input into the first classifier for training. After receiving the feature vector sequence, the classifier attempts to predict the emotion category corresponding to each sequence by learning the pattern in the training data. The classifier here can be a neural network such as CNN, or SVM, random forest, etc., without limitation.

[0057] Specifically, the data analysis end may obtain the emotion category of each first target voice segment output by the emotion polarity analysis model in the following manner:

[0058] A1. Input each first target speech segment into a first feature extractor to extract prosodic features and quality features, and obtain a first feature vector sequence corresponding to the output of the first feature extractor.

[0059] The data analysis end can obtain the first feature vector sequence corresponding to the output of the first feature extractor in the following manner:

[0060] A11. Extracting, by a first feature extractor, root mean square energy features, MFCC features, and fundamental frequency features of each first target speech segment as a prosodic feature vector.

[0061] A12. Extracting loudness features and zero-crossing rate features of each first target speech segment through a first feature extractor as quality feature vectors.

[0062] A13. Generate a first feature vector sequence based on the rhythm feature vector and the quality feature vector.

[0063] A2. Input the first feature vector sequence into the first classifier, and obtain the emotion category label value of each first target speech segment output by the first classifier, where the emotion category label value includes one of +1, 0, and -1, where +1 is used to indicate positive emotion, -1 is used to indicate negative emotion, and 0 is used to indicate stable emotion.

[0064] In this way, through voice emotion analysis, the confidence level of the subject expert in delivering the first target voice segment is analyzed. The more stable the emotion, the higher the confidence level, which is more likely to be the actual diagnosis result of the subject expert.

[0065] In one embodiment of the present application, unlike the first target voice segment, the second target voice segment is no longer analyzed for emotional polarity, but for the emotional fluctuations during the speech. This is because joy and happiness can indicate the transition of the speaker's inner confidence from an unstable state to a stable state. Therefore, when there are large emotional fluctuations in the second target voice segment adjacent to the first target voice segment, it can also indirectly reflect the level of confidence in the speech of the first target voice segment.

[0066] The emotion fluctuation analysis model includes a second feature extractor and a second classifier. The emotion fluctuation analysis model can be trained and generated in the following ways:

[0067] Similarly, collect second sample voice segments. The second sample voice segments here can be extracted from the historical MDT consultation video conference and are adjacent to the first sample voice segments. Use a second feature extractor to process these second sample voice segments. The second feature extractor is used to extract the emotion fluctuation feature vector from the voice segment.

[0068] The emotion fluctuation feature vector here is composed of sharpness feature and loudness feature. Among them, the sharpness feature reflects the change of sound frequency and can reflect the tension of emotion. The loudness feature is an indicator of sound intensity and is related to the intensity of emotion. It should be noted that the emotion fluctuation feature vector here can be the emotion fluctuation characteristic vector corresponding to multiple emotion fluctuation features sampled at intervals. , that is, each second sample voice segment can extract multiple emotion fluctuation feature vectors arranged in chronological order ,in The sampling time point The corresponding emotion fluctuation feature vector.

[0069] Then, the emotion fluctuation feature vector corresponding to each second sample speech segment is annotated with the emotion fluctuation value. Here, for the second sample speech segment adjacent to the first sample speech segment including the actual disease entity, the emotion fluctuation value is usually small and can be annotated in the range of 0 to 0.4, and the remaining second sample speech segments can be annotated in the range of 0.41 to 1.

[0070] It should be noted that the emotion fluctuation value can be obtained by performing spectrum conversion on the corresponding emotion fluctuation feature vector, and then extracting the amplitude, and taking the difference between the maximum value and the minimum value as the emotion fluctuation amplitude. The emotion fluctuation amplitudes corresponding to all the second sample voice segments are normalized to the range of 0 to 1, and then the emotion fluctuation value corresponding to the second sample voice segment is obtained.

[0071] For each first sample speech segment, the corresponding second sample speech segment is formed into a second feature vector sequence ,in, The first sample speech segment The corresponding second sample speech segment The second feature vector sequence is used to train the second classifier so that it outputs the corresponding emotion fluctuation value, and is tuned by cross-validation and other methods to improve the generalization ability and prediction accuracy of the model.

[0072] Specifically, the data analysis end obtains the emotion fluctuation value output by the emotion fluctuation analysis model in the following manner, which specifically includes:

[0073] B1. Input at least one second target speech segment corresponding to the first target speech segment into a second feature extractor to extract emotion fluctuation features, and obtain a second feature vector sequence corresponding to the output of the second feature extractor.

[0074] Here, the data analysis end can obtain the second feature vector sequence corresponding to the output of the second feature extractor in the following manner:

[0075] B11. Using a second feature extractor, extract the sharpness feature and the loudness feature of each second target speech segment as an emotion fluctuation feature vector.

[0076] B12. Generate a second feature vector sequence based on the emotion fluctuation feature vectors of all second target speech segments.

[0077] B2. Input the second feature vector sequence into the second classifier, and obtain the emotion fluctuation value output by the second classifier.

[0078] In this way, the obtained emotion fluctuation value can reflect whether the subject expert's emotion changes when speaking about the disease entity. The smaller the emotion change, the higher the confidence of the subject expert's speech.

[0079] Furthermore, the scope of emotion change analysis can be improved. That is, for each first target voice segment, at least one second target voice segment corresponding to the first target voice segment and the first target voice segment are associated with an input of a pre-trained emotion fluctuation analysis model to obtain the emotion fluctuation value output by the emotion fluctuation analysis model.

[0080] In this case, the emotion fluctuation analysis model includes a second feature extractor and a second classifier, and its training steps can be similar to the aforementioned training steps, which will not be repeated here. The data analysis end can obtain the emotion fluctuation value output by the emotion fluctuation analysis model in the following way:

[0081] C1. Input at least one second target speech segment corresponding to the first target speech segment and the first target speech segment into a second feature extractor in sequence to extract emotion fluctuation features, and obtain a second feature vector sequence corresponding to the output of the second feature extractor.

[0082] Different from step B1, here only the second target voice segment is input, but the corresponding first voice segment, that is, the entire voice segment is analyzed for emotion changes.

[0083] The data analysis end can obtain the second feature vector sequence corresponding to the output of the second feature extractor in the following way:

[0084] C11. Using a second feature extractor, extract the sharpness feature and the loudness feature of the first target speech segment and each second target speech segment as an emotion fluctuation feature vector.

[0085] C12. Generate a second feature vector sequence based on the emotion fluctuation feature vectors of the first target speech segment and all second target speech segments.

[0086] It should be noted that the first target speech segment and the second target speech segment here can be arranged according to their time sequence in the speech text data to generate a second feature vector sequence.

[0087] C2. Input the second feature vector sequence into the second classifier, and obtain the emotion fluctuation value output by the second classifier.

[0088] In one embodiment of the present application, for each subject expert, the data analysis end determines the first target disease entity that the subject expert is targeting by:

[0089] The disease entities are sorted in ascending order of speech emotion scores to determine the disease entity corresponding to the minimum speech emotion score as the first target disease entity.

[0090] Among them, for each disease entity, the data analysis terminal can calculate the speech emotion score of the target discipline expert for the disease entity through the following methods :

[0091] ;

[0092] Among them, is the emotion category of the first target speech segment published by the target discipline expert for the disease entity , is the number of the first target speech segments , is the first target speech segment corresponding to the second target speech segment of the emotional fluctuation value .

[0093] In this way, the smaller the speech emotion score, the higher the confidence level of the discipline expert in the current disease entity, and the more likely it is to become the final diagnosis result of the discipline expert and serve as the first target disease entity

[0094] In an embodiment of the present application, in order to further analyze the final diagnosis result of the discipline expert, in addition to performing emotion analysis from the speech modality, semantic analysis is also performed from the text modality

[0095] The semantic recognition model here can be trained and generated through the following methods

[0096] First, for the sample, the first sample speech segment can be directly converted into the first sample text segment, and then multiple first sample text segments can be spliced into the speech sample data. The first sample text segment contains the disease entity, which can be a rare disease entity or a common disease entity

[0097] Next, remove the modal particles from the first sample text, such as the words "de", "le", etc., and perform word segmentation. For example, for the first sample text "This is the common manifestation of disease entity A", it can be segmented into "This is", "disease entity A", "common", "manifestation". After word segmentation, the part-of-speech classification and annotation of each word can be performed. The word categories here can include disease entity words and opinion words. The disease entity word is the disease name, such as "disease entity A". Opinion words can be divided into affirmative words, interrogative words, and neutral words. For example, "This is", "prove" are affirmative words. "May", "speculate" are interrogative words

[0098] Next, each segmented word needs to be converted into the corresponding word vector to form the word vector set of each first sample text. The word vector here can be extracted through a neural network such as the pre-trained Word2Vec, or can be extracted using the BERT model, etc

[0099] The word vector set of the first sample text where the actual disease entity is located can be marked as 1, and the word vector sequence of the first sample text where the non-actual disease entity is located can be marked as 0.

[0100] Finally, the labeled speech sample data is input into the third classifier, which outputs the confidence of the disease entity in the speech sample data. The confidence here is used to indicate the probability that the disease entity in the speech sample data is the actual disease entity. The third classifier here can be a support vector machine (SVM), logistic regression, random forest, gradient boosting decision tree (GBDT), deep neural network (DNN), etc.

[0101] The data analysis end can obtain the second target disease entity output by the semantic recognition model in the following ways:

[0102] For each disease entity, the first target speech segment corresponding to the disease entity is determined. It should be noted that when a disease entity corresponds to multiple first target speech segments, the first target speech segment published last can be selected according to its time and input into the semantic recognition model.

[0103] Each first target speech segment is converted into corresponding text data, and the speech text data is generated by concatenation.

[0104] The third feature extractor extracts word vectors from the speech text data to form a corresponding word vector sequence ,in The word vector set of the first target speech segment corresponding to the nth disease entity is input into the third classifier. The third classifier outputs the confidence corresponding to each disease entity, and takes the disease entity with the largest confidence as the second target disease entity.

[0105] In this way, by performing semantic analysis on multiple disease entities mentioned in the MDT consultation by subject experts, the disease entities that the subject experts believe the target patient actually suffers from are predicted for use in the formation of the consultation report.

[0106] The present application provides an analysis system for MDT consultation data. Based on the speech of each subject expert, in addition to analyzing the second target disease entity from the text modal semantics, it also performs sentiment analysis from the voice modal, taking into account the confidence level of the subject expert, and then determines the first target disease entity. The first target disease entity and the second target disease entity are provided to the subject expert for manual determination, and the final diagnosis result is formed in the consultation report, thereby realizing the accurate generation of automated consultation reports.

[0107] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0108] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0109] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0110] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0111] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0112] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An analysis system for MDT consultation data, characterized in that: The system includes a data analysis end and a doctor end. The data analysis end is used to determine the diagnosis result of each subject expert on the disease suffered by the target patient based on the speech data of at least one subject expert on the disease entity suffered by the target patient in the MDT consultation of the target patient; The data analysis end is also used to send the diagnosis result to the doctor end corresponding to the subject expert, so as to form a consultation report based on the diagnosis result confirmed by the subject expert; Among them, for each subject expert, the data analysis end obtains the diagnosis result of the subject expert on the disease suffered by the target patient in the following way: Extract multiple first target voice segments and second target voice segments based on the speech data of the subject experts, wherein the first target voice segment is the voice segment where the disease entity is located, and the second target voice segment is the voice segment adjacent to the first target voice segment; For each disease entity, at least one first target speech segment corresponding to the subject expert is input into a pre-trained emotion polarity analysis model to obtain the emotion category of each first target speech segment output by the emotion polarity analysis model, and the emotion polarity is used to indicate the emotion polarity of the first target speech segment published by the target subject expert for the disease entity; For each disease entity, the disease entity and at least one second target speech segment corresponding to the disease entity are respectively input into a pre-trained emotion fluctuation analysis model to obtain an emotion fluctuation value output by the emotion fluctuation analysis model, wherein the emotion fluctuation value is used to indicate the degree of emotion change during the subject expert's speech; For each disease entity, based on the subject expert's emotion category and corresponding emotion fluctuation value for the disease entity, the subject expert's speech emotion score for the disease entity is calculated, and the speech emotion score is used to indicate the subject expert's confidence level in speaking for the disease entity, so as to determine the first target disease entity; For each disease entity, generating corresponding speech text data based on the first target speech segment corresponding to the disease entity; Associating the speech text data corresponding to each disease entity into a pre-trained semantic recognition model to obtain a second target disease entity output by the semantic recognition model; Determine the first target disease entity and the second target disease entity as the diagnosis result of the subject expert on the disease suffered by the target patient; Among them, for each subject expert, the data analysis end determines the first target disease entity that the subject expert should target through the following methods: Sort the disease entities in ascending order of speech emotion scores to determine that the disease entity corresponding to the minimum speech emotion score is the first target disease entity; For each disease entity, the data analysis end calculates the target subject expert's speech sentiment score for the disease entity in the following way: : ； in, The first target speech segment published by the target subject expert on the disease entity The emotion category, is the number of the first target speech segments, The first target speech segment The corresponding second target speech segment Mood swing value.

2. The system according to claim 1, characterized in that The emotion polarity analysis model includes a first feature extractor and a first classifier. The data analysis end obtains the emotion category of each first target voice segment output by the emotion polarity analysis model in the following manner: Inputting each first target speech segment into a first feature extractor to extract prosodic features and quality features, and obtaining a first feature vector sequence corresponding to the output of the first feature extractor; The first feature vector sequence is input into a first classifier, and an emotion category label value of each first target speech segment output by the first classifier is obtained, wherein the emotion category label value includes one of +1, 0, and -1, where +1 is used to indicate a positive emotion, -1 is used to indicate a negative emotion, and 0 is used to indicate a stable emotion.

3. The system according to claim 2, characterized in that The data analysis end obtains the first feature vector sequence corresponding to the output of the first feature extractor in the following manner: Extracting, by a first feature extractor, a root mean square energy feature, an MFCC feature, and a fundamental frequency feature of each first target speech segment as a prosodic feature vector; Extracting loudness features and zero-crossing rate features of each first target speech segment as quality feature vectors through a first feature extractor; A first feature vector sequence is generated based on the prosody feature vector and the quality feature vector.

4. The system according to claim 1, characterized in that The emotion fluctuation analysis model includes a second feature extractor and a second classifier. The step of obtaining the emotion fluctuation value output by the emotion fluctuation analysis model by the data analysis end in the following manner specifically includes: Inputting at least one second target speech segment corresponding to the first target speech segment into a second feature extractor to extract emotion fluctuation features, and obtaining a second feature vector sequence corresponding to the output of the second feature extractor; The second feature vector sequence is input into a second classifier, and the emotion fluctuation value output by the second classifier is obtained.

5. The system according to claim 4, characterized in that The data analysis end obtains the second feature vector sequence corresponding to the output of the second feature extractor in the following manner: Extracting the sharpness feature and the loudness feature of each second target speech segment as an emotion fluctuation feature vector by a second feature extractor; A second feature vector sequence is generated based on the emotion fluctuation feature vectors of all second target speech segments.

6. The system according to claim 1, characterized in that For each first target voice segment, at least one second target voice segment corresponding to the first target voice segment and the first target voice segment are associated with an input into a pre-trained emotion fluctuation analysis model to obtain an emotion fluctuation value output by the emotion fluctuation analysis model.

7. The system according to claim 6, characterized in that The emotion fluctuation analysis model includes a second feature extractor and a second classifier. The data analysis end obtains the emotion fluctuation value output by the emotion fluctuation analysis model in the following manner: Inputting at least one second target speech segment corresponding to the first target speech segment and the first target speech segment into a second feature extractor in sequence to extract emotion fluctuation features, and obtaining a second feature vector sequence corresponding to the output of the second feature extractor; The second feature vector sequence is input into a second classifier, and the emotion fluctuation value output by the second classifier is obtained.

8. The system according to claim 7, characterized in that The data analysis end obtains the second feature vector sequence corresponding to the output of the second feature extractor in the following manner: Extracting the sharpness feature and the loudness feature of the first target speech segment and each second target speech segment as an emotion fluctuation feature vector by a second feature extractor; A second feature vector sequence is generated based on the emotion fluctuation feature vectors of the first target speech segment and all second target speech segments.

Images