Text connotation quality evaluation method and device, equipment and storage medium

Abstract

The application relates to the field of artificial intelligence and discloses a text connotation quality evaluation method for improving the accuracy of text connotation quality evaluation and improving model training efficiency. The text connotation quality evaluation method comprises the following steps: obtaining an initial text from a preset medical record text; processing the initial text through a natural language processing algorithm to obtain a target text; performing text coding on the target text through a preset bag-of-words model and a preset automatic coding model to obtain first text features; performing feature extraction on the target text to obtain second text features, wherein the second text features comprise text complexity features, text grammar style features and medical semantic features; performing evaluation processing on the first text features and the second text features through a trained logistic regression model to obtain an evaluation result, wherein the evaluation result is used for identifying the connotation quality grade of the preset medical record text. The application also relates to the blockchain technology, and the target text is stored in the blockchain.

Description

Technical Field

[0001] The present invention relates to the field of machine learning of artificial intelligence, and in particular to a method, apparatus, device and storage medium for evaluating the quality of text content. Background Art

[0002] Medical record systems document the onset, progression, diagnosis, and treatment of patients' illnesses. Medical record quality control is a crucial component of a hospital's healthcare quality management. Medical record quality control is typically performed manually by professional quality control personnel. However, there are widespread issues with insufficient professional quality control personnel, and manual medical record quality control is labor-intensive and inefficient.

[0003] With the popularization of electronic medical record systems in hospitals, electronic medical records have gradually replaced manual medical records, making the collection of medical record information more convenient and faster. However, existing electronic medical record systems generally only check the form of medical records and cannot check the quality of medical records.

[0004] Traditional text content quality control systems typically assess the quality of medical records by examining their content based on manually defined rules. However, these traditional text content quality control systems fail to consider the entire medical record text, resulting in low accuracy in assessing text content quality. Meanwhile, some current text content quality control systems utilize natural language processing and deep learning technologies to build models based on large amounts of medical record data. However, these systems suffer from the time-consuming model training process. Summary of the Invention

[0005] The main purpose of the present invention is to solve the technical problems that the accuracy of text connotation quality assessment is low due to the content inspection based on manually set rules, and the model training in the existing text connotation quality control system is time-consuming.

[0006] To achieve the above-mentioned purpose, the first aspect of the present invention provides a method for evaluating the quality of text connotation, comprising: obtaining an initial text from a preset medical record text, the initial text including chief complaint information, existing medical history information, physical examination information, first medical record information, medical record information, ward round record information and surgical record information; performing text preprocessing on the initial text through a natural language processing algorithm to obtain a target text; performing text encoding on the target text through a preset bag-of-words model and a preset automatic encoding model to obtain a first text feature; performing feature extraction on the target text to obtain a second text feature, the second text feature including a text complexity feature, a text grammatical style feature and a medical semantic feature, the feature extraction including calculating the number of each type of words, the ratio of each type of symbols and the ratio of each type of words; performing evaluation processing on the first text feature and the second text feature through a trained logistic regression model to obtain an evaluation result, and the evaluation result is used to identify the connotation quality level of the preset medical record text.

[0007] Optionally, in a first implementation manner of the first aspect of the present invention, the target text is stored in a blockchain, and the text preprocessing of the initial text by a natural language processing algorithm to obtain a target text includes: segmenting the initial text by a natural language processing algorithm to obtain a segmented text; performing part-of-speech tagging on the segmented text to obtain a part-of-speech tagged text; performing medical named entity recognition on the part-of-speech tagged text to obtain an entity recognition text, wherein the entity recognition text includes k words, where k is a positive integer; and setting the segmented text, the part-of-speech tagged text, and the entity recognition text as the target text.

[0008] Optionally, in a second implementation of the first aspect of the present invention, the target text is encoded by a preset bag-of-words model and a preset automatic encoding model to obtain a first text feature, including: extracting multiple target words from the target text, and converting the multiple target words into an m-dimensional feature vector through a preset bag-of-words model, where m is a positive integer; converting the m-dimensional feature vector into an n-dimensional feature vector through a preset automatic encoding model, where n is a positive integer and n is less than m; setting the n-dimensional feature vector as a first text feature, and storing the first text feature in a preset data table.

[0009] Optionally, in a third implementation of the first aspect of the present invention, before obtaining the initial text from the preset medical record text, wherein the initial text includes chief complaint information, existing medical history information, physical examination information, first medical record information, medical record information, ward round record information and surgical record information, the method for evaluating the text content quality also includes: obtaining a training feature vector Z, wherein the dimension of the training feature vector is x, and x is a positive integer; encoding the training feature vector through a preset encoder to obtain a target feature vector, wherein the dimension of the target feature vector is y, and y is a positive integer, and x>y; decoding the y-dimensional target feature vector through a preset decoder to obtain the x-dimensional feature vector Z'; calculating the cross-entropy loss function of Z and Z', and using the gradient descent algorithm to iteratively update each parameter in the initial automatic encoding model until the training of the initial automatic encoding model converges, thereby obtaining a preset automatic encoding model.

[0010] Optionally, in a fourth implementation of the first aspect of the present invention, the target text is subjected to feature extraction to obtain a second text feature, the second text feature including a text complexity feature, a text grammatical style feature and a medical semantic feature, the feature extraction including calculating the number of each type of words, the ratio of each type of symbols and the ratio of each type of words, including: extracting text complexity features from the word segmentation text, the text complexity features including the number of each type of words and the number of sentences; extracting text grammatical style features from the part-of-speech text, the text grammatical style features including the ratio of each type of symbols and the ratio of each type of words; determining the number of named entities associated with preset medical care from the entity recognition text to obtain a medical semantic feature; and setting the text complexity feature, the text grammatical style feature and the medical semantic feature as the second text feature.

[0011] Optionally, in a fifth implementation of the first aspect of the present invention, before obtaining the initial text from the preset medical record text, wherein the initial text includes chief complaint information, existing medical history information, physical examination information, first medical record information, medical record information, ward round record information and surgical record information, the method for evaluating the text connotation quality also includes: obtaining preset text feature samples; selecting a training data set and a test data set from the preset text feature samples according to a preset ratio, wherein the preset text feature samples are text feature samples that have been manually marked with scores in advance; calculating the loss function corresponding to the initial logistic regression model based on the training data set; updating the target model parameters in the initial logistic regression model according to the loss function to obtain a trained logistic regression model; inputting the test data set into the trained logistic regression model to perform connotation quality grade classification to obtain a classification result, and iteratively updating the trained logistic regression model based on the classification result.

[0012] Optionally, in a sixth implementation of the first aspect of the present invention, obtaining preset text feature samples includes: obtaining multiple electronic medical record samples, the electronic medical record samples being used to indicate samples for which connotation quality assessment has been performed using a percentage score system; performing text encoding on the multiple electronic medical record samples and extracting text features to obtain a first feature set and a second feature set; constructing a feature matrix of the multiple electronic medical record samples using the second feature set in a preset feature order; extracting multiple connotation quality scores from the multiple labeled electronic medical record samples, and forming the multiple connotation quality scores into a column vector to obtain a scoring matrix; calculating the feature matrix and the scoring matrix according to a preset formula to obtain multiple Pearson correlation coefficients, wherein the preset formula is: Among them, X i For each text feature in the second feature set, Y i For each of the connotation quality scores, r is used to indicate the reflection of the two variables X i and the Y i the degree of linear correlation; determining whether each Pearson coefficient is greater than a preset threshold; if each Pearson correlation coefficient is greater than the preset threshold, setting the corresponding text feature as the target text feature, and setting the selected target text feature and the first feature set as the preset text feature sample.

[0013] The second aspect of the present invention provides a device for evaluating the connotation quality of text, including: a first acquisition module, used to obtain an initial text from a preset medical record text, the initial text including chief complaint information, existing medical history information, physical examination information, first medical record information, medical record information, ward round record information and surgical record information; a preprocessing module, used to perform text preprocessing on the initial text through a natural language processing algorithm to obtain a target text; an encoding module, used to perform text encoding on the target text through a preset bag-of-words model and a preset automatic encoding model to obtain a first text feature; an extraction module, used to perform feature extraction on the target text to obtain a second text feature, the second text feature including a text complexity feature, a text grammatical style feature and a medical semantic feature, the feature extraction including calculating the number of each type of words, the ratio of each type of symbols and the ratio of each type of words; an evaluation module, used to evaluate the first text feature and the second text feature through a trained logistic regression model to obtain an evaluation result, and the evaluation result is used to identify the connotation quality level of the preset medical record text.

[0014] Optionally, in a first implementation manner of the second aspect of the present invention, the target text is stored in a blockchain, and the preprocessing module is specifically used to: perform word segmentation on the initial text through a natural language processing algorithm to obtain a word segmentation text; perform part-of-speech tagging on the word segmentation text to obtain a part-of-speech tagged text; perform medical named entity recognition on the part-of-speech tagged text to obtain an entity recognition text, wherein the entity recognition text includes k words, where k is a positive integer; and set the word segmentation text, the part-of-speech tagged text, and the entity recognition text as the target text.

[0015] Optionally, in a second implementation of the second aspect of the present invention, the first encoding module includes: a first conversion unit, used to extract target words from the target text, and convert the multiple target words into an m-dimensional feature vector through a preset word bag model, where m is a positive integer; a second conversion unit, used to convert the m-dimensional feature vector into an n-dimensional feature vector through a preset automatic encoding model, where n is a positive integer and n is less than m; a setting unit, used to set the n-dimensional feature vector as a first text feature, and store the first text feature in a preset data table.

[0016] Optionally, in a third implementation of the second aspect of the present invention, the device for evaluating the quality of text content further includes: a second acquisition module, used to acquire a training feature vector Z, the dimension of the training feature vector is x, and x is a positive integer; a second encoding module, used to encode the training feature vector through a preset encoder to obtain a target feature vector, the dimension of the target feature vector is y, and y is a positive integer, and x>y; a decoding module, used to decode the y-dimensional target feature vector through a preset decoder to obtain the x-dimensional feature vector Z'; a processing module, used to calculate the cross-entropy loss function of Z and Z', and use a gradient descent algorithm to iteratively update each parameter in the initial automatic encoding model until the training of the initial automatic encoding model converges, and a preset automatic encoding model is obtained.

[0017] Optionally, in a fourth implementation of the second aspect of the present invention, the extraction module is specifically used to: extract text complexity features from the word segmentation text, the text complexity features including the number of each type of words and the number of sentences; extract text grammatical style features from the part-of-speech tagged text, the text grammatical style features including the ratio of each type of symbols and the ratio of each type of words; determine the number of named entities associated with preset medical care from the entity recognition text to obtain medical semantic features; set the text complexity features, the text grammatical style features and the medical semantic features as second text features.

[0018] Optionally, in a fifth implementation of the second aspect of the present invention, the device for evaluating the text connotation quality further includes: a third acquisition module for acquiring preset text feature samples; a selection module for selecting a training data set and a test data set from the preset text feature samples according to a preset ratio, wherein the preset text feature samples are text feature samples that have been manually marked with scores in advance; a calculation module for calculating the loss function corresponding to the initial logistic regression model based on the training data set; an update module for updating the target model parameters in the initial logistic regression model according to the loss function to obtain a trained logistic regression model; and a testing module for inputting the test data set into the trained logistic regression model to perform connotation quality grade classification to obtain a classification result, and iteratively updating the trained logistic regression model based on the classification result.

[0019] Optionally, in a sixth implementation of the second aspect of the present invention, the third acquisition module is specifically used to: acquire multiple electronic medical record samples, the electronic medical record samples being used to indicate samples for which connotation quality assessment has been performed using a percentage score system; perform text encoding on the multiple electronic medical record samples and extract text features to obtain a first feature set and a second feature set; construct a feature matrix of the multiple electronic medical record samples using the second feature set according to a preset feature order; extract multiple connotation quality scores from the multiple labeled electronic medical record samples, and form the multiple connotation quality scores into a column vector to obtain a scoring matrix; calculate the feature matrix and the scoring matrix according to a preset formula to obtain multiple Pearson correlation coefficients, and the preset formula is: Among them, X i For each text feature in the second feature set, Y i For each of the connotation quality scores, r is used to indicate the reflection of the two variables X i and the Y i the degree of linear correlation; determining whether each Pearson coefficient is greater than a preset threshold; if each Pearson correlation coefficient is greater than the preset threshold, setting the corresponding text feature as the target text feature, and setting the selected target text feature and the first feature set as the preset text feature sample.

[0020] The third aspect of the present invention provides a device for evaluating the quality of text content, comprising: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a line; the at least one processor calls the instructions in the memory so that the device for evaluating the quality of text content executes the above-mentioned method for evaluating the quality of text content.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, enable the computer to execute the above-mentioned method for evaluating the quality of textual content.

[0022] In the technical solution provided by the present invention, an initial text is obtained from a preset medical record text, wherein the initial text includes chief complaint information, existing medical history information, physical examination information, first medical record information, medical record information, ward round record information and surgical record information; the initial text is preprocessed by a natural language processing algorithm to obtain a target text; the target text is encoded by a preset bag-of-words model and a preset automatic encoding model to obtain a first text feature; feature extraction is performed on the target text to obtain a second text feature, wherein the second text feature includes a text complexity feature, a text grammatical style feature and a medical semantic feature, and the feature extraction includes calculating the number of each type of words, the ratio of each type of symbols and the ratio of each type of words; the first text feature and the second text feature are evaluated by a trained logistic regression model to obtain an evaluation result, and the evaluation result is used to identify the connotation quality level of the preset medical record text. In an embodiment of the present invention, features obtained by encoding medical record texts and features of text connotation quality extracted from medical record texts are combined as overall features of text connotation quality. A classification model is trained based on the overall features of text connotation quality and a logistic regression algorithm to improve the accuracy of text connotation quality assessment and the efficiency of text connotation quality assessment model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic diagram of an embodiment of a method for evaluating text connotation quality in an embodiment of the present invention;

[0024] Figure 2 Schematic diagram of another embodiment of a method for evaluating text connotation quality in an embodiment of the present invention;

[0025] Figure 3 Schematic diagram of an embodiment of a device for evaluating text connotation quality according to an embodiment of the present invention;

[0026] Figure 4 Schematic diagram of another embodiment of a device for evaluating text connotation quality according to an embodiment of the present invention;

[0027] Figure 5 Schematic diagram of an embodiment of a device for evaluating text content quality in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The embodiments of the present invention provide a method, apparatus, device and storage medium for evaluating text connotation quality, which are used to obtain features by encoding medical record text and extract features of text connotation quality from medical record text, and combine the two as the overall features of text connotation quality. A classification model is trained based on the overall features of text connotation quality and a logistic regression algorithm to improve the accuracy of text connotation quality evaluation and the efficiency of text connotation quality evaluation model training.

[0029] The terms "first," "second," "third," "fourth," and the like (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0030] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 An embodiment of a method for evaluating text connotation quality in an embodiment of the present invention includes:

[0031] 101. Obtain an initial text from a preset medical record text, the initial text including chief complaint information, current medical history information, physical examination information, first medical record information, medical record information, ward round record information, and surgical record information.

[0032] The pre-installed medical record text is a digital medical record that is stored, managed, transmitted, and reproduced electronically in advance. It replaces handwritten paper medical records. The pre-installed medical record text includes all the information of the paper medical record. In other words, the pre-installed medical record text and the paper medical record are identical at the text level. Furthermore, the server reads the initial text from the pre-installed database according to the different dimensional information of the pre-installed medical record text. The different dimensional information is used to indicate the different content of the initial text.

[0033] 102. The initial text is preprocessed using a natural language processing algorithm to obtain a target text.

[0034] The server performs text preprocessing on the initial text using a preset natural language processing algorithm to obtain a preprocessed target text. The preprocessing includes word segmentation, part-of-speech tagging, and medical named entity recognition. The preprocessed target text includes word segmentation text, part-of-speech tagging text, and entity recognition text.

[0035] It should be noted that natural language processing algorithms are algorithms that intelligently analyze, understand, and extract data from human language. These include forward maximum matching algorithms, reverse maximum matching algorithms, maximum probability word segmentation algorithms, and conditional random field algorithms. It should be emphasized that to further ensure the privacy and security of the target text, the target text can also be stored on a blockchain node.

[0036] 103. Perform text encoding on the target text using a preset bag-of-words model and a preset auto-encoding model to obtain a first text feature.

[0037] The server performs text encoding on the target text. Specifically, the server first converts the preprocessed target text into a high-dimensional sparse feature vector through a preset bag-of-words model. The server then converts the high-dimensional sparse feature vector into a low-dimensional compact feature vector through a preset auto-encoding model. The server sets the low-dimensional compact feature vector as the text encoding of the target text, which is the first text feature. Among them, the preset bag-of-words model is a text representation algorithm commonly used in natural language processing when modeling text. The preset auto-encoding model compresses and encodes the input data, that is, the high-dimensional original data is represented by a low-dimensional vector, so that the compressed low-dimensional vector retains the typical characteristics of the input data, thereby making it easier to restore the original data.

[0038] 104. Feature extraction is performed on the target text to obtain a second text feature. The second text feature includes text complexity feature, text grammatical style feature and medical semantic feature. Feature extraction includes calculating the number of each type of words, the ratio of each type of symbols and the ratio of each type of words.

[0039] The server extracts secondary text features from the preprocessed target text. These secondary text features include text complexity features, text grammatical style features, and medical semantic features. The preprocessed target text includes word segmentation text, part-of-speech tagged text, and entity recognition text. Features are extracted by calculating the frequency of occurrence of certain types of words or symbols, such as the number of words of each type, the ratio of each type of symbol, and the ratio of each type of word. These secondary text features can, to a certain extent, reflect the quality of the medical record's content.

[0040] 105. The first text feature and the second text feature are evaluated and processed by the trained logistic regression model to obtain an evaluation result, which is used to identify the connotation quality level of the preset medical record text.

[0041] The first and second features are features that combine the text encoding and text features of the target text. A logistic regression model is trained to assess the connotation quality of the medical record text. Training a logistic regression model is more efficient than training other machine learning or deep learning models. The server evaluates the first and second text features using the trained logistic regression model to obtain an evaluation result, which is used to identify the connotation quality level of the preset medical record text. Specifically, the server merges the first and second text features and inputs the merged text features into the trained logistic regression model. The server then performs the evaluation using the trained logistic regression model to obtain an evaluation result, which is used to identify the connotation quality level of the preset medical record text. The connotation quality level uses 2, 1, and 0 to represent the level of the preset medical record text, i.e., high, medium, and low. For example, after classification of preset medical record text A using the trained logistic regression model, the model output value is 2, which is also the evaluation result, indicating that the connotation quality level of the preset medical record text is high.

[0042] It should be noted that personalized assessment of textual content quality is achieved through different selections of medical record text features. Different hospitals have different requirements for textual content quality assessment. For example, Hospital A has higher requirements for the grammar of medical record texts and will focus on features that reflect the grammar of medical record texts; Hospital B has higher requirements for medical terminology that appears in medical record texts and will focus on features related to these medical entities. The specific requirements are not limited here.

[0043] In an embodiment of the present invention, features obtained by encoding medical record texts and features of text connotation quality extracted from medical record texts are combined as overall features of text connotation quality. A classification model is trained based on the overall features of text connotation quality and a logistic regression algorithm to improve the accuracy of text connotation quality assessment and the efficiency of text connotation quality assessment model training.

[0044] See also Figure 2 Another embodiment of the method for evaluating text connotation quality in the embodiment of the present invention includes:

[0045] 201. Obtaining an initial text from a preset medical record text, the initial text including chief complaint information, current medical history information, physical examination information, first medical record information, medical record information, ward round record information, and surgical record information.

[0046] Specifically, the server reads the initial text from a pre-set database based on the different dimensions of the pre-set medical record text. The different dimensions indicate different contents of the initial text, including the chief complaint, current medical history, physical examination, initial medical record, medical history, ward rounds, and surgical records. The pre-set medical record text is a digitized medical record that is pre-stored, managed, transmitted, and reproduced using electronic devices and is used to replace handwritten paper medical records.

[0047] 202. Perform text preprocessing on the initial text using a natural language processing algorithm to obtain a target text.

[0048] Specifically, the server first uses a natural language processing algorithm to segment the initial text into words, generating segmented text. Segmentation is the process of recombining consecutive characters into word sequences according to preset rules. The target text includes sentences and paragraphs, and the characters and words between sentences and paragraphs are consecutive characters. Specifically, the server uses a forward maximum matching algorithm or a reverse maximum matching algorithm to segment the target text, resulting in a word as the smallest unit of granularity.

[0049] Furthermore, the server segments the initial text according to the maximum probability segmentation algorithm based on contextual semantics, so as to facilitate ambiguity identification and new word identification for the initial text. Ambiguity refers to the existence of multiple segmentation results for the same character string to be segmented, and new words refer to words that are not included in the dictionary. Specifically, the server obtains multiple strings from the initial text, and for each string, the server extracts multiple candidate words in order from left to right; the server finds the probability value of each candidate word from the preset dictionary and records all the left neighbors of each candidate word; the server calculates the cumulative probability of each candidate word and compares all the left neighbors of each candidate word to obtain the best left neighbor word of each candidate word; if the current word is the last word of the string and the cumulative probability is the maximum value, the server sets the current word as the end word of the string; the current word is set as the starting point, and the server outputs the best left neighbor word of each word in order from right to left to obtain the segmentation result of the string of the target text, and repeats the above process until all the segmentation results of the target text are obtained.

[0050] The server then performs part-of-speech tagging on the segmented text to produce part-of-speech tagged text. Part-of-speech tagging is the process of determining the grammatical category of each word in the sentence, determining its part of speech, and then tagging it. Parts of speech include content words and function words. Optionally, the server uses a conditional random field algorithm to perform part-of-speech tagging on the segmented text to produce part-of-speech tagged text.

[0051] Secondly, the server performs medical named entity recognition on the part-of-speech tagged text to obtain entity recognition text, which includes k words, where k is a positive integer. Among them, medical named entity recognition is the process of identifying and classifying medical-related proper nouns in the part-of-speech tagged text. For example, proper nouns include cold, upper respiratory tract, infection, fever, improvement, and cure. Optionally, the server performs medical named entity recognition on the part-of-speech tagged text based on the conditional random field algorithm to obtain entity recognition text; finally, the server sets the word segmentation text, part-of-speech tagged text, and entity recognition text as the target text.

[0052] 203. Perform text encoding on the target text using a preset bag-of-words model and a preset auto-encoding model to obtain a first text feature.

[0053] Specifically, first, the server extracts multiple target words from the target text, and converts the multiple target words into m-dimensional feature vectors through the preset bag-of-words model, where m is a positive integer. Furthermore, the server counts the number of times each word appears in the preprocessed target text through the preset bag-of-words model. After the server merges all the words and the corresponding number of times, it normalizes the merged data to obtain an m-dimensional feature vector. The m-dimensional feature vector is a high-dimensional sparse feature vector, wherein the position corresponding to each element in the m-dimensional feature vector represents the number of times the corresponding word appears in the preset medical record text, wherein most of the features of the high-dimensional sparse feature vector are 0. It should be noted that the preset bag-of-words model does not consider the contextual relationship between words in the text, but only considers the weight of all words in the text, and the weight is related to the frequency of the word in the text.

[0054] Secondly, the server converts the m-dimensional feature vector obtained by the preset bag-of-words model into an n-dimensional feature vector through the preset auto-encoding model, where n is a positive integer and n is less than m. Specifically, the server system inputs the m-dimensional feature vector into the preset auto-encoding model, and converts the m-dimensional feature vector into an n-dimensional feature vector through the preset auto-encoding model, wherein the n-dimensional feature vector is a low-dimensional and compact feature vector. The preset auto-encoding model is a pre-trained auto-encoding network model, and the specific process of training the preset auto-encoding model further includes: the server obtains a training feature vector Z, the dimension of the training feature vector is x, and x is a positive integer; the server encodes the training feature vector x through the preset encoder to obtain a target feature vector, the dimension of the target feature vector is y, and y is a positive integer, and x>y; the server decodes the y-dimensional target feature vector through the preset decoder to obtain an x-dimensional feature vector Z'; the server calculates the cross-entropy loss function of Z and Z', and uses the gradient descent algorithm to iteratively update the various parameters in the initial auto-encoding model until the initial auto-encoding model training converges, thereby obtaining a trained preset auto-encoding model. Finally, the server sets the n-dimensional feature vector as the first text feature and stores the n-dimensional feature vector in a preset data table.

[0055] 204. Extract text complexity features from the segmented text. The text complexity features include the number of each type of words and the number of sentences in the target text.

[0056] Text complexity features include the number of each type of words and the number of sentences in the target text, for example, the number of characters appearing in the segmented text, the number of words appearing in the segmented text, the number of sentences appearing in the segmented text, the average number of characters per word in the segmented text, the average number of characters per sentence in the segmented text, and the average number of words per sentence in the segmented text.

[0057] 205. Extract text grammatical style features from the part-of-speech tagged text, where the text grammatical style features include the ratio of each type of symbol and the ratio of each type of word.

[0058] Among them, the ratio of each type of symbols and the ratio of each type of words, that is, by calculating the frequency of occurrence of a certain type of words or symbols, for example, the ratio of content words appearing in the part-of-speech tagged text to the total number of words, content words include nouns, verbs, adjectives, numerals, quantifiers and pronouns; the ratio of function words appearing in the part-of-speech tagged text to the total number of words, among which function words include adverbs, prepositions, conjunctions, auxiliary words, interjections, and onomatopoeia; the number of all punctuation marks appearing in the part-of-speech tagged text, punctuation marks include punctuation marks, signs and symbols, the ratio of the number of punctuation marks appearing in the part-of-speech tagged text to the number of all punctuation marks, the ratio of the number of signs appearing in the part-of-speech tagged text to the number of all punctuation marks, and the ratio of the number of signs appearing in the part-of-speech tagged text to the number of all punctuation marks.

[0059] 206. Determine named entities associated with preset medical conditions from entity recognition texts and obtain medical semantic features.

[0060] Furthermore, the server matches and reads named entities associated with preset medical conditions from the entity recognition text. For example, the medical semantic features include the number of medical entities appearing in the entity recognition text, the number of medical entities belonging to symptoms appearing in the entity recognition text, the number of medical entities belonging to diseases appearing in the entity recognition text, the number of medical entities belonging to tests and examinations appearing in the entity recognition text, and the number of medical entities belonging to drugs appearing in the entity recognition text.

[0061] 207. Set the text complexity feature, text grammatical style feature and medical semantic feature as the second text feature.

[0062] That is, the server assembles and merges the text complexity features, text grammatical style features and medical semantic features into the second text features. It can be understood that the text complexity features extracted from the word segmentation text processed by word segmentation are used to indicate the attribute features of characters, words and sentences in the text; the text grammatical style features extracted from the part-of-speech tagged text processed by part-of-speech tagging are used to indicate the attribute features of words and punctuation marks in the text; and the medical semantic features are used to indicate the attribute features of medical entities such as diseases, symptoms, tests and examinations, and drugs.

[0063] 208. Evaluate the first text feature and the second text feature using the trained logistic regression model to obtain an evaluation result, which is used to identify the connotation quality level of the preset medical record text.

[0064] Among them, training a logistic regression model is more efficient than training other machine learning or deep learning models. Specifically, the server merges the first text feature and the second text feature, and inputs the merged text feature into the trained logistic regression model; the server evaluates the merged feature through the trained logistic regression model to obtain an evaluation result, which is used to identify the connotation quality level of the preset medical record text. Among them, the connotation quality level uses A, B and C to represent the level of the preset medical record text, that is, high level, medium level and low level. For example, the preset medical record text A is classified and processed by the trained logistic regression model to obtain a model output value of A, which is also the evaluation result. A indicates that the connotation quality level of the preset medical record text is high, and C indicates that the connotation quality level of the preset medical record text is low.

[0065] The server trains the initial logistic regression model to obtain a trained logistic regression model. The sample feature vector is then input into the trained initial logistic regression model to learn the relationship between the feature vectors reflecting the medical record text content and the text content quality score, generating the model output. It should be noted that the proposed text content quality control is evaluated using test data, calculating accuracy, precision, and recall. Logistic regression models are relatively fast to train among machine learning models. Specifically, the server obtains a preset text feature sample; the server selects a training data set and a test data set from the preset text feature samples according to a preset ratio, and the preset text feature sample is a text feature sample that has been manually marked with scores in advance; the server obtains the loss function corresponding to the initial logistic regression model based on the training data set; the server updates the target model parameters in the initial logistic regression model according to the loss function to obtain a trained logistic regression model. Furthermore, the server solves the minimum value of the loss function through the gradient descent algorithm, and calculates the estimated values ​​of the model parameters, and then substitutes the estimated values ​​into the target logistic regression model function. After multiple iterations, the model converges to obtain a trained logistic regression model; the server inputs the test data set into the trained logistic regression model for connotation quality grade classification, obtains the classification results, and iteratively updates the trained logistic regression model based on the classification results.

[0066] Furthermore, the server obtains a plurality of electronic medical record samples, where the electronic medical record samples are used to indicate samples for which connotation quality has been evaluated using a percentage score system; the server performs text encoding on the plurality of electronic medical record samples and extracts text features to obtain a first feature set and a second feature set; the server constructs a feature matrix of the plurality of electronic medical record samples using the second feature set according to a preset feature order; the server extracts a plurality of connotation quality scores from the plurality of labeled electronic medical record samples, and forms the plurality of connotation quality scores into a column vector to obtain a score matrix; the server calculates the feature matrix and the score matrix according to a preset formula to obtain a plurality of Pearson correlation coefficients, where the preset formula is: Among them, X i For each text feature in the second feature set, Y i For each connotation quality score, r is used to indicate the reflection of two variables X i and Y i The server determines whether each Pearson coefficient is greater than a preset threshold; if each Pearson correlation coefficient is greater than the preset threshold, the server sets the corresponding text feature as the target text feature, and sets the selected target text feature and the first feature set as the preset text feature sample.

[0067] In an embodiment of the present invention, features obtained by encoding medical record texts and features of text connotation quality extracted from medical record texts are combined as overall features of text connotation quality. A classification model is trained based on the overall features of text connotation quality and a logistic regression algorithm to improve the accuracy of text connotation quality assessment and the efficiency of text connotation quality assessment model training.

[0068] The above describes the method for evaluating the text connotation quality in the embodiment of the present invention. The following describes the device for evaluating the text connotation quality in the embodiment of the present invention. Figure 3 An embodiment of a device for evaluating text connotation quality in an embodiment of the present invention includes:

[0069] The first acquisition module 301 is used to acquire an initial text from a preset medical record text, where the initial text includes chief complaint information, current medical history information, physical examination information, first medical record information, medical record information, ward round record information, and surgical record information;

[0070] A preprocessing module 302 is configured to perform text preprocessing on the initial text using a natural language processing algorithm to obtain a target text;

[0071] A first encoding module 303 is configured to perform text encoding on the target text using a preset bag-of-words model and a preset automatic encoding model to obtain a first text feature;

[0072] An extraction module 304 is configured to perform feature extraction on the target text to obtain second text features, where the second text features include text complexity features, text grammatical style features, and medical semantic features. Feature extraction includes calculating the number of words of each type, the ratio of symbols of each type, and the ratio of words of each type.

[0073] The evaluation module 305 is used to evaluate the first text feature and the second text feature using a trained logistic regression model to obtain an evaluation result, which is used to identify the connotation quality level of the preset medical record text.

[0074] In an embodiment of the present invention, features obtained by encoding medical record texts and features of text connotation quality extracted from medical record texts are combined as overall features of text connotation quality. A classification model is trained based on the overall features of text connotation quality and a logistic regression algorithm to improve the accuracy of text connotation quality assessment and the efficiency of text connotation quality assessment model training.

[0075] See also Figure 4 Another embodiment of the text content quality assessment device in the embodiment of the present invention includes:

[0076] The first acquisition module 301 is used to acquire an initial text from a preset medical record text, where the initial text includes chief complaint information, current medical history information, physical examination information, first medical record information, medical record information, ward round record information, and surgical record information;

[0077] A preprocessing module 302 is configured to perform text preprocessing on the initial text using a natural language processing algorithm to obtain a target text;

[0078] A first encoding module 303 is configured to perform text encoding on the target text using a preset bag-of-words model and a preset automatic encoding model to obtain a first text feature;

[0079] An extraction module 304 is configured to perform feature extraction on the target text to obtain second text features, where the second text features include text complexity features, text grammatical style features, and medical semantic features. Feature extraction includes calculating the number of words of each type, the ratio of symbols of each type, and the ratio of words of each type.

[0080] The evaluation module 305 is used to evaluate the first text feature and the second text feature using a trained logistic regression model to obtain an evaluation result, which is used to identify the connotation quality level of the preset medical record text.

[0081] Optionally, the pre-processing module 302 may also be specifically configured to:

[0082] Segment the initial text using a natural language processing algorithm to obtain segmented text;

[0083] Perform part-of-speech tagging on the segmented text to obtain a part-of-speech tagged text;

[0084] Perform medical named entity recognition on the part-of-speech tagged text to obtain entity recognition text, which includes k words, where k is a positive integer;

[0085] Set the word segmentation text, part-of-speech tagging text, and entity recognition text as the target text.

[0086] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned target text, the above-mentioned target text can also be stored in a node of a blockchain.

[0087] Optionally, the first encoding module 303 is specifically configured to:

[0088] Extract multiple target words from the target text and convert them into m-dimensional feature vectors using a preset bag-of-words model, where m is a positive integer.

[0089] The m-dimensional feature vector is converted into an n-dimensional feature vector by a preset automatic encoding model, where n is a positive integer and n is less than m;

[0090] The n-dimensional feature vector is set as the first text feature, and the first text feature is stored in a preset data table.

[0091] Optionally, the text content quality assessment device further includes:

[0092] The second acquisition module 306 is used to obtain a training feature vector Z, where the dimension of the training feature vector is x, where x is a positive integer;

[0093] A second encoding module 307 is configured to encode the training feature vector using a preset encoder to obtain a target feature vector, where the dimension of the target feature vector is y, where y is a positive integer and x>y;

[0094] A decoding module 308 is configured to decode the y-dimensional target feature vector using a preset decoder to obtain an x-dimensional feature vector Z';

[0095] The processing module 309 is used to calculate the cross entropy loss function of Z and Z', and iteratively update the parameters of the initial auto-encoding model using a gradient descent algorithm until the initial auto-encoding model training converges to obtain a preset auto-encoding model.

[0096] Optionally, the extraction module 304 may also be specifically configured to:

[0097] Extract text complexity features from the segmented text. The text complexity features include the number of words of each type and the number of sentences in the target text.

[0098] Extracting text grammatical style features from part-of-speech tagged text, the text grammatical style features including the ratio of each type of symbol and the ratio of each type of word;

[0099] Determine the number of named entities associated with preset medical treatments from entity recognition texts to obtain medical semantic features;

[0100] The text complexity feature, text grammatical style feature and medical semantic feature are set as the second text feature.

[0101] Optionally, the text content quality assessment device further includes:

[0102] The third acquisition module 310 is used to obtain a preset text feature sample;

[0103] A selection module 311 is configured to select a training data set and a test data set from preset text feature samples according to a preset ratio, wherein the preset text feature samples are text feature samples that have been manually marked in advance;

[0104] A calculation module 312 is used to calculate the loss function corresponding to the initial logistic regression model based on the training data set;

[0105] An updating module 313 is configured to update target model parameters in the initial logistic regression model according to the loss function to obtain a trained logistic regression model;

[0106] The testing module 314 is used to input the test data set into the trained logistic regression model to perform connotation quality grade classification, obtain the classification result, and iteratively update the trained logistic regression model based on the classification result.

[0107] Optionally, the third obtaining module 310 may also be specifically configured to:

[0108] Obtain multiple electronic medical record samples, where the electronic medical record samples are used to indicate samples that have been evaluated for connotation quality using a percentage score;

[0109] Performing text encoding on multiple electronic medical record samples and extracting text features to obtain a first feature set and a second feature set;

[0110] Constructing a feature matrix of multiple electronic medical record samples using the second feature set according to a preset feature order;

[0111] Extracting multiple connotation quality scores from multiple labeled electronic medical record samples, and forming the multiple connotation quality scores into column vectors to obtain a score matrix;

[0112] The feature matrix and the rating matrix are calculated according to the preset formula to obtain multiple Pearson correlation coefficients. The preset formula is: Among them, X i For each text feature in the second feature set, Y i For each connotation quality score, r is used to indicate the reflection of two variables X i and Y i The degree of linear correlation;

[0113] Determine whether each Pearson coefficient is greater than a preset threshold;

[0114] If each Pearson correlation coefficient is greater than a preset threshold, the corresponding text feature is set as the target text feature, and the selected target text feature and the first feature set are set as preset text feature samples.

[0115] In an embodiment of the present invention, features obtained by encoding medical record texts and features of text connotation quality extracted from medical record texts are combined as overall features of text connotation quality. A classification model is trained based on the overall features of text connotation quality and a logistic regression algorithm to improve the accuracy of text connotation quality assessment and the efficiency of text connotation quality assessment model training.

[0116] above Figure 3 and Figure 4The text content quality evaluation device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The text content quality evaluation device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0117] Figure 5 1 is a schematic diagram of the structure of a device for evaluating textual content quality provided by an embodiment of the present invention. The device 500 for evaluating textual content quality may vary significantly due to different configurations or performances, and may include one or more processors (central processing units, CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing application programs 533 or data 532. The memory 520 and the storage medium 530 may be either short-term storage or persistent storage. The program stored in the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations in the device 500 for evaluating textual content quality. Furthermore, the processor 510 may be configured to communicate with the storage medium 530 to execute the series of instruction operations in the storage medium 530 on the device 500 for evaluating textual content quality.

[0118] The text content quality assessment device 500 may further include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The illustrated structure of the text content quality assessment device does not limit the text content quality assessment device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0119] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the text content quality evaluation method.

[0120] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0121] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0122] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program code.

[0123] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating the quality of text content, characterized in that: The evaluation method of the text connotation quality includes: Obtaining an initial text from a preset medical record text, wherein the initial text includes chief complaint information, current medical history information, physical examination information, first medical record information, medical record information, ward round record information, and surgical record information; The initial text is preprocessed by a natural language processing algorithm to obtain a target text, which includes a segmented text, a part-of-speech tagged text and an entity recognition text, which includes: segmenting the initial text by a natural language processing algorithm to obtain a segmented text; performing part-of-speech tagging on the segmented text to obtain a part-of-speech tagged text; performing medical named entity recognition on the part-of-speech tagged text to obtain an entity recognition text, and the entity recognition text includes word, is a positive integer; setting the word segmentation text, the part-of-speech tagging text and the entity recognition text as the target text; Performing text encoding on the target text by using a preset bag-of-words model and a preset auto-encoding model to obtain a first text feature; Performing feature extraction on the target text to obtain a second text feature, the second text feature includes a text complexity feature, a text grammatical style feature and a medical semantic feature, the feature extraction includes calculating the number of each type of words, the ratio of each type of symbols and the ratio of each type of words, including: extracting a text complexity feature from the word segmentation text, the text complexity feature includes the number of each type of words and the number of sentences in the target text; extracting a text grammatical style feature from the part-of-speech tagged text, the text grammatical style feature includes the ratio of each type of symbols and the ratio of each type of words; determining the number of named entities associated with preset medical care from the entity recognition text to obtain a medical semantic feature; setting the text complexity feature, the text grammatical style feature and the medical semantic feature as the second text feature; The first text feature and the second text feature are evaluated and processed by a trained logistic regression model to obtain an evaluation result, and the evaluation result is used to identify the connotation quality level of the preset medical record text.

2. The method for evaluating the textual content quality according to claim 1, wherein: The step of encoding the target text by using a preset bag-of-words model and a preset auto-encoding model to obtain a first text feature includes: Extract multiple target words from the target text, and convert the multiple target words into dimensional feature vector, the is a positive integer; By presetting the automatic encoding model dimensional feature vector is converted to dimensional feature vector, the is a positive integer, and the Less than ; The The dimension feature vector is set as the first text feature, and the first text feature is stored in a preset data table.

3. The method for evaluating the textual content quality according to claim 1, wherein: Before obtaining the initial text from the preset medical record text, wherein the initial text includes chief complaint information, current medical history information, physical examination information, first medical record information, medical record information, ward round record information, and surgical record information, the text content quality assessment method further includes: Get training feature vector , the dimension of the training feature vector is , is a positive integer; The training feature vector is encoded by a preset encoder to obtain a target feature vector, the dimension of which is , is a positive integer, and ; By presetting the decoder dimensional target feature vector is decoded to obtain the dimensional feature vector ; Calculate the and stated The cross entropy loss function is used, and the gradient descent algorithm is used to iteratively update the parameters in the initial automatic encoding model until the initial automatic encoding model training converges to obtain the preset automatic encoding model.

4. The method for evaluating the textual content quality according to any one of claims 1 to 3, characterized in that: Before obtaining the initial text from the preset medical record text, wherein the initial text includes chief complaint information, current medical history information, physical examination information, first medical record information, medical record information, ward round record information, and surgical record information, the text content quality assessment method further includes: Get preset text feature samples; Selecting a training data set and a test data set from the preset text feature samples according to a preset ratio, wherein the preset text feature samples are text feature samples that have been manually marked in advance; Calculate the loss function corresponding to the initial logistic regression model based on the training data set; Update the target model parameters in the initial logistic regression model according to the loss function to obtain a trained logistic regression model; The test data set is input into the trained logistic regression model to perform connotation quality grade classification to obtain a classification result, and the trained logistic regression model is iteratively updated based on the classification result.

5. The method for evaluating the text content quality according to claim 4, characterized in that: The step of obtaining a preset text feature sample includes: obtaining a plurality of electronic medical record samples, wherein the electronic medical record samples are used to indicate samples that have been subjected to connotation quality assessment using a percentage score; Performing text encoding on the plurality of electronic medical record samples and extracting text features to obtain a first feature set and a second feature set; Constructing a feature matrix of the plurality of electronic medical record samples using the second feature set according to a preset feature order; Extracting multiple connotation quality scores from multiple labeled electronic medical record samples, and forming the multiple connotation quality scores into column vectors to obtain a score matrix; The feature matrix and the scoring matrix are calculated according to a preset formula to obtain multiple Pearson correlation coefficients. The preset formula is: ,in, For each text feature in the second feature set, Score the quality of each of the said connotations, Used to indicate the reflection of two variables and stated The degree of linear correlation; Determine whether each Pearson coefficient is greater than a preset threshold; If each of the Pearson correlation coefficients is greater than the preset threshold, the corresponding text feature is set as the target text feature, and the selected target text feature and the first feature set are set as the preset text feature sample.

6. A device for evaluating the quality of text content, characterized in that: The text content quality evaluation device includes: A first acquisition module is used to acquire an initial text from a preset medical record text, wherein the initial text includes chief complaint information, current medical history information, physical examination information, first medical record information, medical record information, ward round record information, and surgical record information; The preprocessing module is used to perform text preprocessing on the initial text through a natural language processing algorithm to obtain a target text, and the target text includes a segmented text, a part-of-speech tagged text and an entity recognition text, which includes: performing segmentation on the initial text through a natural language processing algorithm to obtain a segmented text; performing part-of-speech tagging on the segmented text to obtain a part-of-speech tagged text; performing medical named entity recognition on the part-of-speech tagged text to obtain an entity recognition text, and the entity recognition text includes word, is a positive integer; setting the word segmentation text, the part-of-speech tagging text and the entity recognition text as the target text; a first encoding module, configured to perform text encoding on the target text using a preset bag-of-words model and a preset automatic encoding model to obtain a first text feature; An extraction module is used to perform feature extraction on the target text to obtain a second text feature, wherein the second text feature includes a text complexity feature, a text grammatical style feature, and a medical semantic feature, and the feature extraction includes calculating the number of each type of words, the ratio of each type of symbols, and the ratio of each type of words, including: extracting a text complexity feature from the word segmentation text, wherein the text complexity feature includes the number of each type of words and the number of sentences in the target text; extracting a text grammatical style feature from the part-of-speech tagged text, wherein the text grammatical style feature includes the ratio of each type of symbols and the ratio of each type of words; determining the number of named entities associated with preset medical treatments from the entity recognition text to obtain a medical semantic feature; and setting the text complexity feature, the text grammatical style feature, and the medical semantic feature as the second text feature; The evaluation module is used to evaluate the first text feature and the second text feature through a trained logistic regression model to obtain an evaluation result, and the evaluation result is used to identify the connotation quality level of the preset medical record text.

7. A device for evaluating the quality of text content, characterized in that: The text connotation quality assessment device includes: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a circuit; The at least one processor calls the instructions in the memory to enable the text content quality assessment device to execute the text content quality assessment method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for evaluating the text content quality according to any one of claims 1 to 5 is implemented.

Images