An intelligent checking method and system for esophageal cancer data entry link

Abstract

The present application relates to the technical field of data processing, and discloses an intelligent checking method and system for esophageal cancer data entry link, comprising: obtaining an esophageal cancer medical report from a database; preprocessing the esophageal cancer medical report to obtain training data; making type labels of the training data set; respectively constructing a medical image data feature extraction network, a diagnosis text data feature extraction network and a feature vector dynamic splicing network, and training the medical image data feature extraction network, the diagnosis text data feature extraction network and the feature vector dynamic weighted splicing network, and saving the trained network; using the network to extract medical image data features and diagnosis text data features, dynamically weighting and splicing the two kinds of data features and inputting them into a softmax to output a medical record category probability, and after calculating a divergence value, judging whether the entry type is correct; this method can greatly improve the work efficiency of the operator and improve the accuracy and reliability of the entered data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to an intelligent verification method and system for esophageal cancer data entry. Background Technology

[0002] The clinical diagnosis process for esophageal cancer generates a massive amount of heterogeneous medical data, mainly including medical images and corresponding text reports. This data usually comes from diverse sources and has different formats.

[0003] Currently, the mainstream processing method still relies heavily on manual input and integration. Operators need to frequently switch between data from different sources, in different formats, and in different modalities, and perform tedious manual matching and association operations. This method is inefficient, time-consuming, and seriously affects the speed of the diagnostic process. Furthermore, human operation is very prone to introducing errors in the matching and association process, resulting in inconsistencies between image and report data, which affects the accuracy and reliability of the data.

[0004] The aforementioned deficiencies not only increase the workload of medical personnel, but more importantly, low-quality or inconsistent data may mislead subsequent diagnostic analysis, treatment planning, and prognostic assessment, posing a potential risk to patient safety and treatment outcomes.

[0005] Therefore, how to provide a solution that can overcome the above problems is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to improve upon the shortcomings of existing technologies and provide an intelligent verification method for esophageal cancer data entry. The method comprises the following steps: S1, retrieving historical esophageal cancer medical reports from a database to train a medical image data feature extraction network, a diagnostic text data feature extraction network, and a feature vector dynamic concatenation network; S2, preprocessing the patient's esophageal cancer medical report to obtain medical image data I and corresponding diagnostic text data T, using the trained medical image data feature extraction network to extract features from medical image data I to obtain feature vector V1, and using the trained diagnostic text data feature extraction network to extract features from diagnostic text data T to obtain feature vector V2; S3, dynamically weighting feature vectors V1 and V2 using the feature vector dynamic concatenation network and concatenating them into feature vector V; S4, using feature vector V to calculate the probability of the category to which the esophageal cancer medical report belongs, and verifying the correctness of the pathological type of the entered esophageal cancer data.

[0007] Further, in step S1, the medical image data feature extraction network includes: a first dilated convolutional layer with a dilation rate of 1, a second dilated convolutional layer with a dilation rate of 12, a third dilated convolutional layer with a dilation rate of 16, a fourth dilated convolutional layer with a dilation rate of 18, a global average pooling layer, and a feature concatenation layer; the inputs of the first dilated convolutional layer, the second dilated convolutional layer, the third dilated convolutional layer, the fourth dilated convolutional layer, and the global average pooling layer are all medical image data I, and the outputs of the first dilated convolutional layer, the second dilated convolutional layer, the third dilated convolutional layer, the fourth dilated convolutional layer, and the global average pooling layer are all connected to the input of the feature concatenation layer; the diagnostic text data feature extraction network includes: a Jieba word segmentation layer, a NER processing layer, and a Bi layer. The system consists of a BioBERT embedding layer, a GGNN network layer, a GRU layer, and a dimensionality compression layer. The Jieba word segmentation layer takes diagnostic text data T as input, and its output is connected to the input of the NER processing layer. The NER processing layer's output is connected to the input of the BioBERT embedding layer, which in turn is connected to the input of the GGNN network layer. The GGNN network layer's output is connected to the input of the GRU layer and the dimensionality compression layer. The feature vector dynamic weighted concatenation network includes a weight generation layer and a weighted concatenation layer. The outputs of the feature concatenation layer and the dimensionality compression layer are connected to the weight generation layer, and also to the weighted concatenation layer.

[0008] Further, in step S4, the trained medical image data feature extraction network is used to extract features from the medical image data I to obtain feature vector V1, as follows:

[0009] Sq1. The medical image data I is fed into the four pre-constructed dilated convolutional layers. The following convolution formula applies to any dilated convolutional layer:

[0010]

[0011] Among them, F k The feature map output by the k-th dilated convolutional layer is I, where k is the index of the dilated convolutional layer (k=1,2,3,4); the height of I is H. in Width is W in ; and Representing respectively and The result is an integer; m is the coordinate of the height of the dilated convolution kernel, m∈(0,F). h -1), where n is the coordinate of the dilated convolution kernel width, n∈(0,F) w -1); F h F is the height of the dilated convolution kernel. wis the width of the dilated convolution kernel; i is the height index of the output feature map, j is the width index of the output feature map, i∈(0,H) in -1), j∈(0, W in -1); r is the dilation rate of the dilated convolution kernel, r=1,12,16,18; SAME is used to fill the kernel during convolution so that the height and width of the output feature map are consistent with the input.

[0012] This step generates four feature maps: F1, F2, F3, and F4.

[0013] Sq2, the global average pooling layer, performs global average pooling on the input medical image data I to obtain medical image data g, as follows:

[0014]

[0015] Where u is the height position index in medical image data I during global average pooling, u∈(0,H) in -1), v is the width position index, v∈(0,W in -1);

[0016] Sq3. Perform channel concatenation on feature maps F1, F2, F3, F4 and medical image data g in the feature concatenation layer to obtain feature vector V1:

[0017]

[0018] Concat() is the concatenation operation.

[0019] Further, in step S4, the trained diagnostic text data feature extraction network is used to extract features from the diagnostic text data T to obtain feature vector V2, as follows:

[0020] Sp1. Use the Jieba word segmentation layer to segment the diagnostic text data T, obtaining the word sequence G:

[0021]

[0022] Where Jieba() represents the Jieba word segmentation layer operation; o n Let N be the nth word in the word sequence G, and N be the number of words in the word sequence G, where n∈(0,N);

[0023] Sp2. The NER processing layer is used to perform named entity recognition on the N words in the word sequence G to obtain the extracted medical entities. The nth entity E n Represented as:

[0024]

[0025] Where NER() represents the NER processing layer operation;

[0026] Sp3, Create an adjacency matrix If entity With entity When they are related or can coexist, then ,otherwise ;in , ;

[0027] Sp4, Generating Medical Entities Using BioBERT Embedding Layers a single entity in Embedding matrix:

[0028]

[0029] in, represents the embedding matrix of the nth entity; t represents the time step, which is fixed at 0 in step Sp4; BioBERT() represents the BioBERT embedding layer operation;

[0030] Sp5. Concatenate the embedding matrices of all entities to obtain the node feature matrix X. (t=0) :

[0031]

[0032] Sp6 and GGNN network layers generate graph propagation messages M based on node feature matrices. (t) :

[0033]

[0034] Among them, X (t-1) W is the node feature matrix from the previous time step. msg With B msg These are learnable parameters;

[0035] Sp7. Feed the graph propagation message M(t) into the GRU layer for updating, and obtain the updated feature vector X. (t) :

[0036]

[0037]

[0038]

[0039]

[0040] Among them, Z(t) For updating the gate; X (t-1) This represents the hidden state of the previous time step; R (t) To reset the door; Let W be a candidate hidden state; t∈(0,N-1); z W r W h All are d×d hid A matrix of dimension U; z U r U h All are d hid ×d hid A matrix of dimension b z b r b h All are d hid A vector of dimension, where the values ​​in the above matrix are all learnable parameters;

[0041] Sp8, repeat Sp6-Sp7, stop when t=N-1, and extract X. (t=N-1) The data is then fed into a dimension compression layer for dimensional compression, calculated as follows:

[0042]

[0043] Where V2 is the feature vector of the diagnostic text data T; ReLU() is the activation function; W c With b c These are learnable parameters.

[0044] Further, step S3 specifically includes:

[0045] V1 and V2 are fed into a feature vector dynamic concatenation network to obtain the weights w1 of V1 and w2 of V2, which are calculated as follows:

[0046]

[0047] Among them, W w With b w The parameters are learnable; softmax() is the activation function;

[0048] Multiply the obtained weights w1 and V1, and multiply the weights w2 and V, then concatenate the results to obtain the feature vector V:

[0049]

[0050] Concat() is the concatenation operation.

[0051] Further, step S4 includes the following steps:

[0052] Step 4-1: After processing the feature vector V through a fully connected layer, obtain the fraction vector L. Use the fraction vector L to calculate the list of category probability values ​​β of the medical report on esophageal cancer to be tested.

[0053] Step 4-2: Calculate the probability values ​​of pathological types in the training dataset, and calculate the category probability value of the medical report on esophageal cancer to be tested and the KL divergence value of the probability values ​​of pathological types in the training dataset.

[0054] Step 4-3: Compare the KL divergence value with the first threshold. Compare and verify whether the pathological type of the entered esophageal cancer data is correct.

[0055] Further, step 4-1 specifically includes:

[0056] The feature vector V is processed through a fully connected layer to obtain the score vector L:

[0057] L=Full(V)=[z1,z2,z3];

[0058] Full(V) means inputting the feature vector V into the fully connected layer for processing; The score is the output of the fully connected layer. ;

[0059] Fractions in fraction vector L The input to the softmax function is represented as:

[0060]

[0061]

[0062] in, The probability values ​​for the medical reports of the esophageal cancer to be tested to belong to the three categories.

[0063] Furthermore, step 4-2 specifically includes:

[0064] Statistical analysis of the probability values ​​of three pathological types in the training dataset ;

[0065] according to and Calculate the KL divergence value D KL :

[0066] .

[0067] Furthermore, step 4-3 specifically includes:

[0068] (1) When the divergence value D KL ≤ When the time is right, it means that the pathological type of the esophageal cancer data entered is correct;

[0069] (2) When the divergence value D KL > If this happens, it means the pathology type of the entered esophageal cancer data is incorrect.

[0070] Furthermore, based on an intelligent verification method for esophageal cancer data entry, an intelligent verification system for esophageal cancer data entry is also disclosed, comprising:

[0071] The multimodal data acquisition module is used to collect esophageal cancer data during the esophageal cancer data entry database process and to preprocess the esophageal cancer data.

[0072] The cascaded intelligent recognition module is used to extract features from the preprocessed data, assign weights to the extracted feature vectors and concatenate them, and calculate the probability of the pathological type to which the concatenated feature vectors belong.

[0073] The dynamic verification module is used to calculate the pathological type probability calculated by the cascaded intelligent recognition module and the pathological type probability of the data in the database to verify whether the pathological type of the entered esophageal cancer data is correct.

[0074] An adaptive interactive interface is used to remind users to further review or correct esophageal cancer data when the pathology type is incorrect.

[0075] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention extracts medical image data and diagnostic text data from esophageal cancer medical reports, and inputs the medical image data and diagnostic text data into a medical image feature extraction network and a diagnostic text feature extraction network, respectively, to obtain two corresponding feature vectors V1 and V2. Then, a feature vector dynamic concatenation network is used to dynamically weight the two feature vectors and concatenate them into a single feature vector V. After processing feature vector V, the probability of the esophageal cancer medical report belonging to one of the three pathological types is obtained. The probability distribution of the three pathological types in all esophageal cancer medical reports in the database was statistically analyzed. ,calculate and divergence value D KL The method uses the divergence value to determine whether the pathological type of the entered esophageal cancer data is correct. This method can help operators verify the correctness of the entered pathological type when entering esophageal cancer medical reports into the esophageal cancer database. At the same time, due to its multimodal data processing capabilities, it can greatly improve the work efficiency of operators and enhance the accuracy and reliability of the entered data. Attached Figure Description

[0076] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a flowchart of an intelligent verification method for esophageal cancer data entry, as described in the embodiment.

[0078] Figure 2 This is a diagram of the medical image data feature extraction network structure involved in the embodiment.

[0079] Figure 3 This is a diagram of the network structure for feature extraction of diagnostic text data involved in the embodiment.

[0080] Figure 4 This is a diagram of the feature vector dynamic weighted splicing network structure involved in the embodiment. Detailed Implementation

[0081] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0082] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance, or suggesting any such actual relationship or order between these entities or operations. Additionally, the terms "connected," "linked," etc., can refer to a direct connection between elements or an indirect connection via other elements.

[0083] See Figure 1 This embodiment provides an intelligent verification method for esophageal cancer data entry, specifically including steps S1 to S4:

[0084] S1. Obtain historical esophageal cancer medical reports from the database to train the medical image data feature extraction network, the diagnostic text data feature extraction network, and the feature vector dynamic splicing network.

[0085] First, the medical reports of esophageal cancer are preprocessed to obtain a training dataset. Esophageal cancer medical reports are retrieved from the database, and medical imaging data and corresponding diagnostic text data are extracted from them. After preprocessing the medical imaging data and corresponding diagnostic text data, they are packaged into a training dataset for subsequent processing.

[0086] Then, the data in the training dataset is divided into three types: malignant esophageal cancer, benign esophageal cancer, and uncertain. The data types are represented by one-hot vectors. When the data type is malignant esophageal cancer, the data label is [1,0,0], when the data type is benign, the data label is [0,1,0], and when the data type is uncertain, the data label is [0,0,1].

[0087] Next, the training dataset was used to train the medical image data feature extraction network, the diagnostic text data feature extraction network, and the feature vector dynamic weighted concatenation network. Finally, the trained networks were saved.

[0088] See Figure 2 The medical image data feature extraction network includes: a first dilated convolutional layer with a dilation rate of 1, a second dilated convolutional layer with a dilation rate of 12, a third dilated convolutional layer with a dilation rate of 16, a fourth dilated convolutional layer with a dilation rate of 18, a global average pooling layer, and a feature splicing layer.

[0089] The inputs to the first, second, third, and fourth dilated convolutional layers and the global average pooling layer are all medical image data I. The outputs of the first, second, third, and fourth dilated convolutional layers and the global average pooling layer are all connected to the input of the feature splicing layer.

[0090] See Figure 3 The diagnostic text data feature extraction network includes: a Jieba segmentation layer, a NER processing layer, a BioBERT embedding layer, a GGNN network layer, a GRU layer, and a dimensionality compression layer. The input to the Jieba segmentation layer is the diagnostic text data T. The output of the Jieba segmentation layer is connected to the input of the NER processing layer, the output of the NER processing layer is connected to the input of the BioBERT embedding layer, the output of the BioBERT embedding layer is connected to the input of the GGNN network layer, and the output of the GGNN network layer is connected to the input of the GRU layer and the input of the dimensionality compression layer.

[0091] See Figure 4The feature vector dynamic weighted concatenation network includes a weight generation layer and a weighted concatenation layer. The feature concatenation layer of the medical image data feature extraction network is connected to both the weight generation layer and the weighted concatenation layer, while the dimensionality compression layer of the diagnostic text data feature extraction network is connected to both the weight generation layer and the weighted concatenation layer.

[0092] S2. After preprocessing the patient's esophageal cancer medical report, medical image data I and corresponding diagnostic text data T are obtained. The trained medical image data feature extraction network is used to extract features from medical image data I to obtain feature vector V1. The trained diagnostic text data feature extraction network is used to extract features from diagnostic text data T to obtain feature vector V2.

[0093] Sq1. The medical image data I is fed into the four pre-constructed dilated convolutional layers. The following convolution formula applies to any dilated convolutional layer:

[0094]

[0095] Among them, F k The feature map output by the k-th dilated convolutional layer is I, where k is the index of the dilated convolutional layer (k=1,2,3,4); the height of I is H. in Width is W in ; and Representing respectively and The result is an integer; m is the coordinate of the height of the dilated convolution kernel, m∈(0,F). h -1), where n is the coordinate of the dilated convolution kernel width, n∈(0,F) w -1); F h F is the height of the dilated convolution kernel. w is the width of the dilated convolution kernel; i is the height index of the output feature map, j is the width index of the output feature map, i∈(0,H) in -1), j∈(0, W in -1); r is the dilation rate of the dilated convolution kernel, r=1,12,16,18; SAME is used to fill the kernel during convolution so that the height and width of the output feature map are consistent with the input.

[0096] This step generates four feature maps: F1, F2, F3, and F4.

[0097] Sq2, the global average pooling layer, performs global average pooling on the input medical image data I to obtain medical image data g, as follows:

[0098]

[0099] Where u is the height position index in medical image data I during global average pooling, u∈(0,H) in -1), v is the width position index, v∈(0,W in -1);

[0100] Sq3. Perform channel concatenation on feature maps F1, F2, F3, F4 and medical image data g in the feature concatenation layer to obtain feature vector V1:

[0101]

[0102] Concat() is the concatenation operation.

[0103] This study utilizes four dilated convolutional layers with different dilation rates and a global average pooling layer to extract features from medical image data I. This approach can extract information contained in medical image data I from different scales. The dilated convolutional layers with smaller dilation rates are good at extracting local details, while the dilated convolutional layers with larger dilation rates are good at extracting the correlation information between various details. The global average pooling layer can further extract the content of the overall medical image data. The medical image data feature extraction network can take into account both local and global information, and the extraction of image data is relatively comprehensive.

[0104] Simultaneously, the trained diagnostic text data feature extraction network is used to extract features from the diagnostic text data T to obtain feature vector V2, specifically:

[0105] Sp1. Use the Jieba word segmentation layer to segment the diagnostic text data T, obtaining the word sequence G:

[0106]

[0107] Where Jieba() represents the Jieba word segmentation layer operation; o n Let N be the nth word in the word sequence G, and N be the number of words in the word sequence G, where n∈(0,N);

[0108] Sp2. The NER processing layer is used to perform named entity recognition on the N words in the word sequence G to obtain the extracted medical entities. The nth entity E n Represented as:

[0109]

[0110] Where NER() represents the NER processing layer operation;

[0111] Sp3, Create an adjacency matrix If entity With entity When they are related or can coexist, then ,otherwise ;in , ;

[0112] Sp4, Generating Medical Entities Using BioBERT Embedding Layers a single entity in Embedding matrix:

[0113]

[0114] in, represents the embedding matrix of the nth entity; t represents the time step, which is fixed at 0 in step Sp4; BioBERT() represents the BioBERT embedding layer operation;

[0115] Sp5. Concatenate the embedding matrices of all entities to obtain the node feature matrix X. (t=0) :

[0116]

[0117] Sp6 and GGNN network layers generate graph propagation messages M based on node feature matrices. (t) :

[0118]

[0119] Among them, X (t-1) W is the node feature matrix from the previous time step. msg With B msg These are learnable parameters;

[0120] Sp7. Feed the graph propagation message M(t) into the GRU layer for updating, and obtain the updated feature vector X. (t) :

[0121]

[0122]

[0123]

[0124]

[0125] Among them, Z (t) For updating the gate; X (t-1) This represents the hidden state of the previous time step; R (t) To reset the door; Let W be a candidate hidden state; t∈(0,N-1); z W r W hAll are d×d hid A matrix of dimension U; z U r U h All are d hid ×d hid A matrix of dimension b z b r b h All are d hid A vector of dimension, where the values ​​in the above matrix are all learnable parameters;

[0126] In the diagnostic text data feature extraction network, the pre-trained BioBERT embedding layer is used to perform word embedding on individual entities in medical entities. Since the BioBERT embedding layer is a model developed for the medical field, it can efficiently perform word embedding on medical entities. In the graph propagation message update process of the GGNN network layer, the GRU gating mechanism is introduced, which can take into account the features of the previous time step (t-1) and the current time step (t).

[0127] Sp8, repeat Sp6-Sp7, stop when t=N-1, and extract X. (t=N-1) The data is then fed into a dimension compression layer for dimensional compression, calculated as follows:

[0128]

[0129] Where V2 is the feature vector of the diagnostic text data T; ReLU() is the activation function; W c With b c These are learnable parameters.

[0130] S3. Dynamically weight feature vectors V1 and V2 and concatenate them into feature vector V using a feature vector dynamic concatenation network.

[0131] V1 and V2 are fed into a feature vector dynamic concatenation network to obtain the weights w1 of V1 and w2 of V2, which are calculated as follows:

[0132]

[0133] Among them, W w With b w The parameters are learnable; softmax() is the activation function;

[0134] Multiply the obtained weights w1 and V1, and multiply the weights w2 and V, then concatenate the results to obtain the feature vector V:

[0135]

[0136] Concat() is the concatenation operation.

[0137] The dynamic feature vector concatenation network can dynamically assign weights to feature vectors V1 and V2, enabling the network to consider the importance of text data and image data to decision-making with different weights when processing different data.

[0138] S4. Calculate the probability values ​​of the esophageal cancer medical report to be tested belonging to the three categories using the feature vector V. And calculate the probability values ​​of the three pathological types in the training dataset. and KL divergence value D KL Using D KL Verify the accuracy of the pathological type in the entered esophageal cancer data.

[0139] The feature vector V is processed through a fully connected layer to obtain the score vector L:

[0140] L=Full(V)=[z1,z2,z3];

[0141] Full(V) means inputting the feature vector V into the fully connected layer for processing; The score is the output of the fully connected layer. ;

[0142] Fractions in fraction vector L The input to the softmax function is represented as:

[0143]

[0144]

[0145] in, The probability values ​​for the medical reports of the esophageal cancer to be tested to belong to the three categories.

[0146] Input the values ​​from the list of category probability values ​​β into the argmax function to obtain the pathological type of the medical report on the esophageal cancer to be tested.

[0147] Statistical analysis of the probability values ​​of three pathological types in the training dataset .

[0148] according to and Calculate the KL divergence value D KL :

[0149]

[0150] Based on the divergence value D KL Verify that the pathology type of the entered esophageal cancer data is correct, specifically:

[0151] (1) When the divergence value D KL ≤ When the time is right, it means that the pathological type of the esophageal cancer data entered is correct;

[0152] (2) When the divergence value D KL > If this happens, it means the pathology type of the entered esophageal cancer data is incorrect;

[0153] In this embodiment =0.3; The value is calculated based on actual data from the hospital database.

[0154] When the pathology type of the esophageal cancer data entered is correct, the operator will continue to enter data based on the current esophageal cancer medical report and pathology type. When the pathology type of the esophageal cancer medical report entered is incorrect, the operator needs to further check whether the pathology type of the currently entered esophageal cancer medical report is correct.

[0155] Based on the above-mentioned intelligent verification method for esophageal cancer data entry, this embodiment provides an intelligent verification system for esophageal cancer data entry, including a multimodal data acquisition module, a cascaded intelligent recognition module, a dynamic verification module, and an adaptive interactive interface, wherein:

[0156] The multimodal data acquisition module is used to collect esophageal cancer medical reports during the esophageal cancer medical report database entry process, and to preprocess the esophageal cancer medical reports to obtain medical imaging data and corresponding diagnostic text data.

[0157] The cascaded intelligent recognition module receives medical image data and corresponding diagnostic text data from the multimodal data acquisition module. It uses the medical image data feature extraction network and the diagnostic text data feature extraction network to extract feature vectors from the medical image data and the diagnostic text, and then feeds the two feature vectors into the feature vector dynamic splicing network for splicing. Finally, it calculates the pathological type of the esophageal cancer medical report.

[0158] The dynamic verification module is used to calculate the pathological type of the esophageal cancer medical report obtained by the cascaded intelligent recognition module and the pathological type distribution of the data in the database. Based on this, it verifies whether the pathological type of the entered esophageal cancer data is correct and transmits the result to the adaptive interactive interface.

[0159] The adaptive interactive interface presents the operator with the correctness of the entered esophageal cancer medical report and the corresponding pathology type. When the entered esophageal cancer medical report and the corresponding pathology type are correct, the operator can continue to enter the next esophageal cancer medical report. When they are incorrect, the operator receives a prompt from the adaptive interactive interface, indicating that the entered esophageal cancer medical report and pathology type need to be further checked and then re-entered.

[0160] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An intelligent verification method for esophageal cancer data entry, characterized in that, Includes the following steps: S1. Retrieve historical esophageal cancer medical reports from the database to train a medical image data feature extraction network, a diagnostic text data feature extraction network, and a feature vector dynamic concatenation network. The medical image data feature extraction network includes: a first dilated convolutional layer with a dilation rate of 1, a second dilated convolutional layer with a dilation rate of 12, a third dilated convolutional layer with a dilation rate of 16, a fourth dilated convolutional layer with a dilation rate of 18, a global average pooling layer, and a feature concatenation layer; the first dilated convolutional layer, the second dilated convolutional layer, the third dilated convolutional layer, the fourth dilated convolutional layer, and the global average pooling layer... The input to the pooling layers is medical image data I. The outputs of the first, second, third, and fourth dilated convolutional layers, as well as the global average pooling layer, are all connected to the input of the feature concatenation layer. The diagnostic text data feature extraction network includes: a Jieba word segmentation layer, a NER processing layer, a BioBERT embedding layer, a GGNN network layer, a GRU layer, and a dimensionality compression layer. The input to the Jieba word segmentation layer is diagnostic text data T. The output of the Jieba word segmentation layer is connected to the input of the NER processing layer, and the output of the NER processing layer... The system is as follows: The input of the BioBERT embedding layer is connected to the input of the GGNN network layer, and the output of the GGNN network layer is connected to the input of the GRU layer and the input of the dimensionality compression layer. The feature vector dynamic weighted concatenation network includes a weight generation layer and a weighted concatenation layer. The outputs of the feature concatenation layer and the dimensionality compression layer are connected to the weight generation layer, and also to the weighted concatenation layer. S2: After preprocessing the patient's esophageal cancer medical report, medical image data I and corresponding diagnostic text data T are obtained. The trained medical image data feature extraction network is used to extract features from medical image data I to obtain feature vector V1, and the trained diagnostic text data feature extraction network is used to extract features from diagnostic text data T to obtain feature vector V2. S3: The feature vectors V1 and V2 are dynamically weighted and concatenated into feature vector V using the feature vector dynamic concatenation network. S4: The probability of the category of the esophageal cancer medical report to be tested is calculated using feature vector V, and the correctness of the pathological type of the entered esophageal cancer data is verified.

2. The intelligent verification method for esophageal cancer data entry according to claim 1, characterized in that, In step S2, the trained medical image data feature extraction network is used to extract features from medical image data I to obtain feature vector V1, as follows: Sq1. The medical image data I is fed into the four pre-constructed dilated convolutional layers. The following convolution formula applies to any dilated convolutional layer: Among them, F k The feature map output by the k-th dilated convolutional layer is I, where k is the index of the dilated convolutional layer (k=1,2,3,4); the height of I is H. in Width is W in ; and Representing respectively and The result is an integer; m is the coordinate of the height of the dilated convolution kernel, m∈(0,F). h -1), where n is the coordinate of the dilated convolution kernel width, n∈(0,F) w -1); F h F is the height of the dilated convolution kernel. w is the width of the dilated convolution kernel; i is the height index of the output feature map, j is the width index of the output feature map, i∈(0,H) in -1), j∈(0, W in -1); r is the dilation rate of the dilated convolution kernel, r=1,12,16,18; SAME is used to fill the kernel during convolution so that the height and width of the output feature map are consistent with the input. This step generates four feature maps: F1, F2, F3, and F4. Sq2, the global average pooling layer, performs global average pooling on the input medical image data I to obtain medical image data g, as follows: Where u is the height position index in medical image data I during global average pooling, u∈(0,H) in -1), v is the width position index, v∈(0,W in -1); Sq3. Perform channel concatenation on feature maps F1, F2, F3, F4 and medical image data g in the feature concatenation layer to obtain feature vector V1: Concat() is the concatenation operation.

3. The intelligent verification method for esophageal cancer data entry according to claim 1, characterized in that, In step S2, the trained diagnostic text data feature extraction network is used to extract features from the diagnostic text data T to obtain feature vector V2, as follows: Sp1. Use the Jieba word segmentation layer to segment the diagnostic text data T, obtaining the word sequence G: Where Jieba() represents the Jieba word segmentation layer operation; o n Let N be the nth word in the word sequence G, and N be the number of words in the word sequence G, where n∈(0,N); Sp2. The NER processing layer is used to perform named entity recognition on the N words in the word sequence G to obtain the extracted medical entities. The nth entity E n Represented as: Where NER() represents the NER processing layer operation; Sp3, Create an adjacency matrix If entity With entity When they are related or can coexist, then ,otherwise ;in , ; Sp4, Generating Medical Entities Using BioBERT Embedding Layers a single entity in Embedding matrix: in, represents the embedding matrix of the nth entity; t represents the time step, which is fixed at 0 in step Sp4; BioBERT() represents the BioBERT embedding layer operation; Sp5. Concatenate the embedding matrices of all entities to obtain the node feature matrix X. (t=0) : Sp6 and GGNN network layers generate graph propagation messages M based on node feature matrices. (t) : Among them, X (t-1) W is the node feature matrix from the previous time step. msg With B msg These are learnable parameters; Sp7, Transmitting the message M from the image (t) The updated feature vector X is fed into the GRU layer for updating. (t) : Among them, Z (t) For updating the gate; X (t-1) This represents the hidden state of the previous time step; R (t) To reset the door; Let W be a candidate hidden state; t∈(0,N-1); z W r W h All are d×d hid A matrix of dimension U; z U r U h All are d hid ×d hid A matrix of dimension b z b r b h All are d hid A vector of dimension, where the values ​​in the above matrix are all learnable parameters; Sp8, repeat Sp6-Sp7, stop when t=N-1, and extract X. (t=N-1) The data is then fed into a dimension compression layer for dimensional compression, calculated as follows: Where V2 is the feature vector of the diagnostic text data T; ReLU() is the activation function; W c With b c These are learnable parameters.

4. The intelligent verification method for esophageal cancer data entry according to claim 3, characterized in that, Step S3 specifically involves: V1 and V2 are fed into a feature vector dynamic concatenation network to obtain the weights w1 of V1 and w2 of V2, which are calculated as follows: Among them, W w With b w The parameters are learnable; softmax() is the activation function; Multiply the obtained weights w1 and V1, and multiply the weights w2 and V, then concatenate the results to obtain the feature vector V: Concat() is the concatenation operation.

5. The intelligent verification method for esophageal cancer data entry according to claim 4, characterized in that, Step S4 includes the following steps: Step 4-1: After processing the feature vector V through a fully connected layer, obtain the fraction vector L. Use the fraction vector L to calculate the list of category probability values ​​β of the medical report on esophageal cancer to be tested. Step 4-2: Calculate the probability values ​​of pathological types in the training dataset, and calculate the category probability value of the medical report on esophageal cancer to be tested and the KL divergence value of the probability values ​​of pathological types in the training dataset. Step 4-3: Compare the KL divergence value with the first threshold. Compare and verify whether the pathological type of the entered esophageal cancer data is correct.

6. The intelligent verification method for esophageal cancer data entry according to claim 5, characterized in that, Step 4-1 specifically involves: The feature vector V is processed through a fully connected layer to obtain the score vector L: L=Full(V)=[z1,z2,z3]; Full(V) means inputting the feature vector V into the fully connected layer for processing; The score is the output of the fully connected layer. ; Fractions in fraction vector L The input to the softmax function is represented as: in, The probability values ​​for the medical reports of the esophageal cancer to be tested to belong to the three categories.

7. The intelligent verification method for esophageal cancer data entry according to claim 6, characterized in that, Step 4-2 specifically involves: Statistical analysis of the probability values ​​of three pathological types in the training dataset ; according to and Calculate the KL divergence value D KL : 。 8. The intelligent verification method for esophageal cancer data entry according to claim 7, characterized in that, Step 4-3 specifically involves: (1) When the divergence value D KL ≤ When the time is right, it means that the pathological type of the esophageal cancer data entered is correct; (2) When the divergence value D KL > If this happens, it means the pathology type of the entered esophageal cancer data is incorrect.

9. An intelligent verification system for esophageal cancer data entry, applied to the intelligent verification method for esophageal cancer data entry as described in any one of claims 1-8, characterized in that, include: The multimodal data acquisition module is used to collect esophageal cancer data during the esophageal cancer data entry database process and to preprocess the esophageal cancer data. The cascaded intelligent recognition module is used to extract features from the preprocessed data, assign weights to the extracted feature vectors and concatenate them, and calculate the probability of the pathological type to which the concatenated feature vectors belong. The dynamic verification module is used to calculate the pathological type probability calculated by the cascaded intelligent recognition module and the pathological type probability of the data in the database to verify whether the pathological type of the entered esophageal cancer data is correct. An adaptive interactive interface is used to remind users to further review or correct esophageal cancer data when the pathology type is incorrect.

Images