Pathological section feature fusion method and system based on background guidance attention mechanism

Through a pathological section feature fusion method based on the background-guided attention mechanism, combined with the lesion feature grouping and assignment of fine-tuning index, the problem of ignoring background information in the prior art is solved, and a more accurate and stable pathological image analysis is achieved.

CN119006966AInactive Publication Date: 2024-11-22SHIJIAZHUANG ZHANGSHENG TECHNOLOGY CO LTD

Patent Information

Application Number
CN202411205410.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing pathological image processing methods ignore the background information of the lesion area, which makes it difficult to accurately identify and analyze the lesion in complex pathological images, especially when the lesion area is small or the characteristics are not obvious, and pathological monitoring is not fully utilized. Information is characterized and analyzed.

Method used

The pathological section feature fusion method based on the background-guided attention mechanism is adopted. By collecting and preprocessing pathological data, key features are extracted and background-guided assignment model is constructed. The assignment fine-tuning index is used to dynamically adjust the background-guided assignment, combining the lesion feature grouping strategy and background attention. The force mechanism performs feature weighting fusion to output high-level features.

Benefits of technology

It improves the accuracy and stability of pathological analysis, ensures the flexibility and adaptability of feature fusion results, and provides a more reliable basis for pathological analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006966A_ABST
    Figure CN119006966A_ABST
Patent Text Reader

Abstract

The invention provides a pathological section feature fusion method and system based on a background guidance attention mechanism, and relates to the technical field of automatic diagnosis of histopathology image.According to the method, feature fusion is more flexible and high in adaptability by constructing an assignment fine adjustment index and dynamically adjusting background guidance assignment; the assignment fine tuning strategy can adjust the background guidance assignment according to the actual situation, ensures the accuracy and stability of the feature fusion result, combines the segmentation processing result with the lesion feature grouping strategy, and uses the background guidance attention mechanism to carry out weighted fusion on the extracted features, thereby improving the accuracy and stability of the feature fusion result. The finally output fused high-level features can not only improve the accuracy of pathological analysis, but also provide a more reliable basis for subsequent pathological analysis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of automatic diagnosis of tissue pathology images, and specifically to a pathology slice feature fusion method and system based on a background-guided attention mechanism. Background Art

[0002] Pathological image processing mainly relies on technical means such as image segmentation, feature extraction and classification. Among them, image segmentation methods based on deep learning (such as U-Net, etc.) and feature extraction methods (such as convolutional neural networks) have made significant progress in accuracy and efficiency. However, these methods often only rely on the characteristics of the image itself and ignore other valuable pathological monitoring information, such as disease examination results and diagnosis results, which will lead to incomplete and inaccurate identification and analysis of lesion areas.

[0003] In the prior art, the publication number is CN114118258A, and the name is a pathological slice feature fusion method based on background-guided attention mechanism, which includes: S1: using a pre-trained deep neural network to extract the features of the pathological image slice, and using a linear transformation to convert it into a d-dimensional feature; S2: randomly generating a d-dimensional initial background vector according to a d-dimensional normal distribution; S3: starting from the initial background vector generated in step 2, through a loop module, obtaining the background vector of the pathological image; S4: using the background vector obtained in step 3, calculating the attention weight of each slice feature; S5: weighting and summing the features according to their attention weights to obtain the fused pathological image features;

[0004] Existing pathological image processing methods have certain limitations in feature fusion. Traditional methods mainly use the features extracted from the image directly for classification or regression models, ignoring the important influence of background information of different lesion areas on feature extraction and fusion. This single feature processing method will make it difficult to accurately identify and analyze lesions when processing complex pathological images, especially when the lesion area is small or the features are not obvious. In addition, the existing technology does not make full use of pathological monitoring information, resulting in the inability to fully utilize the potential information in these data for feature fusion and analysis of pathological images.

[0005] The above information disclosed in the above Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a pathological slice feature fusion method and system based on a background-guided attention mechanism to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A pathological slice feature fusion method based on background-guided attention mechanism is applied to a pathological image processing system. The specific steps include:

[0009] After collecting the pathological data of several patients, they are sequentially numbered to form a number set {1,2,...,h,...,H}, where h represents the index of the h-th patient, H represents the total number of patients, and the pathological data includes pathological slice images collected from the patient and pathological monitoring information obtained from the examination, and several pathological slice images of the h-th patient are sequentially numbered to form a number set {h1,h2,...,hi,...,hn}, where hi represents the i-th pathological slice image of the h-th patient, hn represents the n-th pathological slice image of the h-th patient, and n represents the total number of pathological slice images of the patient;

[0010] After receiving the pathological slice image and the pathological monitoring information, preprocessing is performed, the preprocessing includes denoising, normalizing and segmenting the pathological slice image, and structuring the pathological monitoring information, and the pathological monitoring information further includes disease examination result data and disease diagnosis result data, and the structuring processing is to convert the disease examination result data and the disease diagnosis result data into a structured data format;

[0011] Obtaining disease examination result data and disease diagnosis result data converted into a structured data format, then analyzing and processing the disease examination result data to generate a first influence coefficient for evaluating the severity of the disease examination result, and analyzing and processing the disease diagnosis result data to generate a second influence coefficient for evaluating the severity of the diagnosis result;

[0012] Obtain the first influence coefficient and the second influence coefficient, adopt the lesion feature grouping strategy, group the label set {h1,h2,...,hi,...,hn} according to the severity of the lesion in the pathological slice image to form {1,2,...,j,...,m}, where j represents the jth pathological slice image group, and m represents the total number of pathological slice image groups;

[0013] Extract key features from pathological monitoring information, and use the extracted key features as training data to construct a background guidance assignment model, and the output of the background guidance assignment model is the background guidance assignment for each group of pathological slice images;

[0014] Obtaining a first influence coefficient and a second influence coefficient, analyzing and processing them, and constructing an assignment fine-tuning index, the assignment fine-tuning index being used to provide a fine-tuning strategy for background guidance assignment of each group of pathological slice images;

[0015] Obtain a fine-tuning strategy, and determine the background guidance assignment for each group of pathological slice images according to the fine-tuning strategy, and finally use the background guidance attention mechanism to perform weighted fusion on the extracted features and output the fused high-level features;

[0016] The output fused high-level features are applied to subsequent pathological analysis.

[0017] Furthermore, the fine-tuning index of the hth patient is defined as , the calculation formula is as follows:

[0018]

[0019] in, is the fine-tuning index assigned to the hth patient, r1 is the weight of the first influence coefficient, 0.12≤r1≤0.89; r2 is the weight of the second influence coefficient, 0.08≤r1≤0.79, r1>r2, and r1+r2=1, is the first influence coefficient, is the second influence coefficient;

[0020] Set the value fine-tuning index The value range of is (0, Z), where Z is a positive number;

[0021] when The closer it is to 0, the more the inspection and diagnosis results of the overall pathological slice image deviate from the normal range, and the first level of fine-tuning of the background guidance assignment is required;

[0022] when When it is closer to Z, it means that the inspection and diagnosis results of the overall pathological slice image are within the normal range, and the background guidance assignment is fine-tuned to the second level;

[0023] And the fine-tuning amount of the first level is greater than the fine-tuning amount of the second level.

[0024] Furthermore, the first level of fine-tuning amount and the second level of fine-tuning amount specifically include:

[0025] Set MQ to the value fine-tuning index The comparison threshold is 0.69≤MQ≤0.96, MQ<Z; MQ is used to divide different fine-tuning strategies; the adjusted background guidance assignment is set to ;

[0026] 1) When assigning fine-tuning index When it is greater than the threshold MQ, it means Approaching Z, at this time, and The values ​​are all in the interval [0.69, 1), indicating that the inspection and diagnosis results of the overall pathological slice image are good, and the background guidance assignment Perform second level fine adjustment;

[0027] The second level corresponding fine-tuning strategy is to reduce the background guidance assignment of the jth group of pathological slice images of the current hth patient. , The formula is as follows:

[0028]

[0029] 2) When assigning fine-tuning index When it is equal to the threshold MQ, and The values ​​are all in the interval (0.45, 0.69), indicating that the examination and diagnosis results of the overall pathological section images are critical;

[0030] The fine-tuning strategy is to keep the background guidance assignment constant, The mathematical expression is as follows:

[0031]

[0032] 3) When assigning fine-tuning index When it is less than the threshold MQ, it means Approaching 0, and The values ​​are all in the interval (0, 0.45], indicating that the inspection and diagnosis results of the overall pathological section image deviate greatly from the normal range, and the background guidance assignment needs to be fine-tuned at the first level;

[0033] The first level corresponding fine-tuning strategy is to increase the background guidance assignment of the jth group of pathological slice images of the current hth patient ; The mathematical expression is as follows:

[0034] .

[0035] A pathological slice feature fusion system based on background-guided attention mechanism, the system is used to execute the pathological slice feature fusion method based on background-guided attention mechanism, comprising:

[0036] Sequential labeling module: used to collect several patients' pathological data and then label them sequentially to form a label set {1,2,...,h,...,H}, where h represents the index of the h-th patient, H represents the total number of patients, and the pathological data includes pathological slice images collected from the patient and pathological monitoring information obtained from the examination, and sequentially number several pathological slice images of the h-th patient to form a label set {h1,h2,...,hi,...,hn}, where hi represents the i-th pathological slice image of the h-th patient, hn represents the n-th pathological slice image of the h-th patient, and n represents the total number of pathological slice images of the patient;

[0037] Data preprocessing module: used for receiving pathological slice images and pathological monitoring information and performing preprocessing, the preprocessing includes denoising, normalizing and segmenting the pathological slice images, and structuring the pathological monitoring information, and the pathological monitoring information further includes disease examination result data and disease diagnosis result data, and the structuring processing is to convert the disease examination result data and disease diagnosis result data into a structured data format;

[0038] An influence coefficient generation module: used to obtain the disease examination result data and the disease diagnosis result data converted into a structured data format, and then analyze and process the disease examination result data to generate a first influence coefficient for evaluating the severity of the disease examination result, and analyze and process the disease diagnosis result data to generate a second influence coefficient for evaluating the severity of the diagnosis result;

[0039] Grouping module: used to obtain the first influence coefficient and the second influence coefficient, adopt the lesion feature grouping strategy, group the label set {h1,h2,...,hi,...,hn} according to the severity of the lesions in the pathological slice image, and form {1,2,...,j,...,m}, where j represents the jth pathological slice image group, and m represents the total number of pathological slice image groups;

[0040] The assignment model construction module is used to extract key features from the pathological monitoring information and use the extracted key features as training data to construct a background guidance assignment model, and the output of the background guidance assignment model is the background guidance assignment for each group of pathological slice images;

[0041] Index generation module: used to obtain the first influence coefficient and the second influence coefficient, and perform analysis and processing to construct an assignment fine-tuning index, which is used to provide a fine-tuning strategy for the background guidance assignment of each group of pathological slice images;

[0042] Feature fusion module: used to obtain the fine-tuning strategy and determine the background guidance assignment of each group of pathological slice images according to the fine-tuning strategy. Finally, the background guidance attention mechanism is used to perform weighted fusion on the extracted features and output the fused high-level features.

[0043] Application module: used to apply the output fusion high-level features to subsequent pathological analysis.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows: by constructing an assignment fine-tuning index, the background guidance assignment is dynamically adjusted to make feature fusion more flexible and adaptable; the assignment fine-tuning strategy can adjust the background guidance assignment according to actual conditions to ensure the accuracy and stability of the feature fusion results; by combining the segmentation processing results with the lesion feature grouping strategy and using the background guidance attention mechanism to perform weighted fusion of the extracted features, the final output of the fused high-level features can not only improve the accuracy of pathological analysis, but also provide a more reliable basis for subsequent pathological analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0046] Figure 2 It is a system module block diagram of the present invention. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0048] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0049] Embodiment 1:

[0050] See also Figure 1 , the present invention provides a technical solution:

[0051] A pathological slice feature fusion method based on background-guided attention mechanism is applied to a pathological image processing system. The specific steps include:

[0052] Step S1: The pathological image processing system collects pathological data of several patients and sequentially labels them to form a label set {1, 2, ..., h, ..., H}, where h represents the index of the h-th patient, H represents the total number of patients, and the pathological data includes pathological slice images collected from the patient and pathological monitoring information obtained from the examination. Several pathological slice images of the h-th patient are sequentially numbered to form a label set {h1, h2, ..., hi, ..., hn}, where hi represents the i-th pathological slice image of the h-th patient, hn represents the n-th pathological slice image of the h-th patient, and n represents the total number of pathological slice images of the patient;

[0053] Step S2: the pathological image processing system performs preprocessing after receiving the pathological slice image and the pathological monitoring information, the preprocessing includes denoising, normalizing and segmenting the pathological slice image, and structuring the pathological monitoring information, and the pathological monitoring information further includes disease examination result data and disease diagnosis result data, and the structuring processing is to convert the disease examination result data and the disease diagnosis result data into a structured data format;

[0054] Step S3: The pathological image processing system obtains the disease examination result data and the disease diagnosis result data converted into a structured data format, and then analyzes and processes the disease examination result data to generate a first influence coefficient for evaluating the severity of the disease examination result, and analyzes and processes the disease diagnosis result data to generate a second influence coefficient for evaluating the severity of the diagnosis result;

[0055] Step S4: The pathological image processing system obtains the first influence coefficient and the second influence coefficient, adopts the lesion feature grouping strategy, and groups the label set {h1, h2, ..., hi, ..., hn} according to the severity of the lesions in the pathological slice image to form {1, 2, ..., j, ..., m}, where j represents the jth pathological slice image group, and m represents the total number of pathological slice image groups;

[0056] Step S5: The pathological image processing system extracts key features from the pathological monitoring information, and uses the extracted key features as training data to construct a background guidance assignment model, and the output of the background guidance assignment model is the background guidance assignment for each group of pathological slice images;

[0057] Step S6: The pathological image processing system obtains the first influence coefficient and the second influence coefficient, performs analysis and processing, and constructs an assignment fine-tuning index, which is used to provide a fine-tuning strategy for background guidance assignment of each group of pathological slice images;

[0058] Step S7: The pathological image processing system extracts and fuses features based on the background guidance attention mechanism, obtains a fine-tuning strategy, and determines the background guidance assignment for each group of pathological slice images according to the fine-tuning strategy, and finally uses the background guidance attention mechanism to perform weighted fusion on the extracted features and output the fused high-level features;

[0059] Step S8: The pathological image processing system applies the output fused high-level features to subsequent pathological analysis.

[0060] Further explanation: after receiving the pathological slice image and pathological monitoring information, the pathological image processing system performs preprocessing, specifically including:

[0061] 1) Receive data:

[0062] The pathological image processing system receives a number of pathological slice images {h1, h2, ..., hi, ..., hn} of the hth patient and corresponding pathological monitoring information from the database;

[0063] 2) Denoising:

[0064] A Gaussian filter is used to perform denoising on each received pathological slice image to eliminate the noise generated during the image acquisition process;

[0065] 3) Normalization processing:

[0066] Each denoised pathological slice image is normalized to normalize the image pixel value to the range of (0,1) using the following formula:

[0067]

[0068] in, is the normalized pixel value of the i-th pathological slice image of the h-th patient, is the pixel value of the denoised image of the i-th pathological section of the h-th patient, and are the minimum and maximum pixel values ​​of the i-th pathological slice image of the h-th patient, respectively;

[0069] 4) Segmentation processing:

[0070] The normalized image is segmented using a convolutional neural network (CNN) to extract the lesion area. The specific steps are as follows:

[0071] 4.1) Data preparation:

[0072] Get the normalized image pixel value ;

[0073] 4.2) Load the pre-trained CNN model:

[0074] Select the U-Net model and load the pre-trained model weight file;

[0075] 4.3) Image preprocessing:

[0076] Will Convert to grayscale and resize the image to 256x256 pixels, and finally convert the image to tensor format to increase the batch dimension;

[0077] 4.4) Perform segmentation prediction:

[0078] Input the preprocessed image into the U-Net model, and then obtain the probability map output by the CNN model ;

[0079] 4.5) Processing segmentation mask:

[0080] The probability map is binarized using a threshold of 0.5 to obtain a binary mask. :

[0081]

[0082] 4.6) Map back to original image size:

[0083] The binary mask Resize back to original image size ;

[0084] 4.7) Extract the lesion area:

[0085] Create an all-zero matrix For storing the lesion area;

[0086] The pixel value of the lesion area is changed from Copy to :

[0087]

[0088] 4.8) Save the segmentation results:

[0089] The diseased area Save as image file and store in database;

[0090] Further explanation: the disease examination result data and disease diagnosis result data are structured, specifically including:

[0091] After receiving the disease examination result data and disease diagnosis result data, structured processing is performed and the structured data is defined as and , which has the following form:

[0092]

[0093] in, and represent the uth examination item and the hth examination item of the hth patient respectively. diagnostic items, U represents the total number of examination items, Indicates the total number of diagnostic items, and Respectively represent the corresponding inspection value and diagnosis result, Structured data representing disease test result data, Structured data representing disease diagnosis result data;

[0094] The specific steps are as follows:

[0095] 1) Data extraction and classification:

[0096] Extracting original text data of disease examination results and diagnosis results from the patient's electronic medical record system or other medical records; and dividing the extracted data into examination result data and diagnosis result data;

[0097] 2) Text cleaning and information extraction:

[0098] Clean the original text data and remove irrelevant information;

[0099] Extract key information from the cleaned text to form structured data and ;

[0100] 3) Define the scoring criteria:

[0101] Disease examination result scoring criteria: setting and The normal range values ​​are and ;set up The rating value is express, The rating value is express; and They need to be determined according to corresponding medical standards or medical experts, and the determination method is existing technology and will not be elaborated on;

[0102] when The value is When it is within the range, the corresponding output score value is 1;

[0103] when The value is When it is outside the range, the corresponding output score value is 0;

[0104] when The value is When it is within the range, the corresponding output score value is 1;

[0105] when The value is When it is outside the range, the corresponding output score value is 0;

[0106] According to the above scoring criteria, the scoring set of U inspection items is obtained:

[0107]

[0108] as well as A collection of scores for diagnostic items:

[0109]

[0110] Define the first influence coefficient of the hth patient as , the calculation formula is as follows:

[0111]

[0112] in, is the adjustment factor, , U is the total number of inspection items, c1 is the positive adjustment item, and 0.03≤c1≤0.87, ; The specific value of c1 is determined by the expert group through experimental data and will not be elaborated here;

[0113] is the score value of the u-th examination item of the h-th patient;

[0114] is the actual examination value of the u-th examination item of the h-th patient;

[0115] is the normal range value of the u-th examination item for the h-th patient;

[0116] The second influence coefficient of the hth patient is defined as , the calculation formula is as follows:

[0117]

[0118] in, is the adjustment factor, , U' is the total number of diagnostic items, c2 is the adjusted positive item, and 0.12≤c2≤0.96, ; The specific value of c2 is determined by the expert group through experimental data and will not be elaborated here;

[0119] and The value range of is in the range of (0,1);

[0120] when =0, indicating that all test results are beyond the normal range. The closer it is to 0, the more test results are outside the normal range;

[0121] when =1, indicating that all test results are within the normal range. The closer it is to 1, the more test results are within the normal range;

[0122] when =0, indicating that all diagnostic results are very serious. The closer it is to 0, the more diagnostic results are beyond the normal range;

[0123] when =1, indicating that all diagnostic results are within the acceptable range. The closer it is to 1, the more diagnostic results are within the normal range;

[0124] In pathology, the worse the disease examination results, the worse the diagnosis results are. This consistency is highly consistent. This consistency stems from logical reasoning, clinical experience, diagnostic criteria and pathological correlation. The following exceptions are not within the scope of protection of this application:

[0125] Theoretically, there are cases where the disease examination results are inconsistent with the diagnosis results, but this is very rare in actual clinical practice for the following reasons:

[0126] Test errors or discrepancies: Occasionally, test results may be inaccurate, resulting in discrepancies between the actual condition and the actual condition. This is usually confirmed and corrected through re-examination or further examinations.

[0127] Early or latent nature of the disease: Some diseases are not obvious in early examination results, but as the disease progresses, the examination results and diagnosis results will tend to be consistent;

[0128] Individual differences: Different individuals react differently to the disease, but the general trend is that the test results are consistent with the diagnosis;

[0129] Further explanation: The lesion feature grouping strategy includes:

[0130] Define the lesion severity value of the i-th pathological section image of the h-th patient as , the calculation formula is as follows:

[0131]

[0132] in, and is the weight coefficient, , n is the total number of pathological slice images; according to the severity of the lesion Cluster analysis is performed on the pathological slice images. Specifically, the K-means clustering algorithm is used to divide the pathological slice images into different groups:

[0133]

[0134] Wherein, k3 is the preset number of groups, and J is the grouping result;

[0135] The steps of K-means clustering algorithm are as follows:

[0136] 1) Initialization parameters:

[0137] Determine the number of cluster centers k;

[0138] Set the maximum number of iterations and convergence conditions;

[0139] 2) Select the initial cluster center:

[0140] Randomly select k pathological slice images with the lesion severity values As the initial cluster center ,in, Indicates Cluster centers, represents the kth cluster;

[0141] 3) Assign samples to the nearest cluster center:

[0142] Calculate the distance between each pathological section image and all cluster centers, and assign it to the cluster where the nearest cluster center is located;

[0143] 4) Update cluster centers:

[0144] The new cluster center in each cluster is calculated as the mean of the lesion severity values ​​of all pathological section images in the cluster;

[0145] 5) Evaluate convergence conditions:

[0146] Calculate the change between the new cluster center and the old cluster center; if the change of all cluster centers is less than the preset convergence condition, or the maximum number of iterations is reached, the algorithm terminates; otherwise, return to step 3);

[0147] 6) According to the final cluster center, each pathological slice image is assigned to the corresponding cluster to obtain the final grouping result {1, 2, ..., j, ..., m}, and the pathological slice image of each group is output;

[0148] Further explanation: a background guidance assignment model is constructed, and the output of the background guidance assignment model is the background guidance assignment for each group of pathological slice images, specifically including:

[0149] 1) Feature extraction:

[0150] Extracting key features from structured pathology monitoring information , including but not limited to the patient's age, gender, medical history, treatment plan, disease examination results and disease diagnosis results; these characteristics are closely related to the analysis of pathological slice images;

[0151] The extracted key features are normalized to ensure that they have the same scale during model training; the Z-score normalization method is used:

[0152]

[0153] in, is the normalized feature vector, is the mean of the feature, is the standard deviation of the feature;

[0154] 2) Training data preparation:

[0155] The key features after standardization Scoring set with inspection items and the score set of diagnostic items Combine them to form the following training data set:

[0156]

[0157] in, and Represent the uth inspection item and the The score of each diagnostic item, represents the training data set;

[0158] 3) Model training:

[0159] Select the RandomForest machine learning model for training;

[0160] Define the loss function and use the cross entropy loss function to optimize the model parameters:

[0161]

[0162] in, is the true label, is the predicted value, is the total number of training samples, For the The index of the training samples;

[0163] Using the training dataset Perform model training and iteratively update model parameters through gradient descent method until the loss function converges;

[0164] 4) Model verification and optimization:

[0165] The cross-validation method is used to evaluate the model performance. The data set is divided into a training set and a validation set. Multiple iterations of training and validation are performed to calculate the accuracy, precision, and recall indicators of the model.

[0166] Adjust the model's hyperparameters based on the validation results to improve the model's generalization ability;

[0167] 5) Generation of background guidance assignment:

[0168] Input the structured pathology monitoring information and scoring values ​​into the trained model to obtain the background guidance assignment for each set of pathology slice images:

[0169]

[0170] in, represents the background guidance assignment of the jth group of pathological slice images;

[0171] Further explanation: the fine-tuning index of the hth patient is defined as , the calculation formula is as follows:

[0172]

[0173] in, is the fine-tuning index assigned to the hth patient, r1 is the weight of the first influence coefficient, 0.12≤r1≤0.89; r2 is the weight of the second influence coefficient, 0.08≤r1≤0.79, r1>r2, and r1+r2=1, is the first influence coefficient, is the second influence coefficient;

[0174] Used in or The closer it is to 0, the greater the contribution to the result;

[0175] Used in or When it is close to 1, the contribution to the result is increased;

[0176] Set the value fine-tuning index The value range of is (0, Z), where Z is a positive number. In this embodiment, Z is 1.1. The specific value of Z is determined by the expert group through experimental data.

[0177] when The closer it is to 0, the more the inspection and diagnosis results of the overall pathological slice image deviate from the normal range, and the background guidance assignment needs to be fine-tuned to the first level;

[0178] when When it is closer to Z, it means that the inspection and diagnosis results of the overall pathological slice image are within the normal range, and the background guidance assignment is fine-tuned to the second level;

[0179] and the fine-tuning amount of the first level is greater than the fine-tuning amount of the second level;

[0180] Set MQ to the value fine-tuning index The comparison threshold is 0.69≤MQ≤0.96, MQ<Z, and the specific value of MQ is determined by the expert group through experimental data. The determination method is the existing technology and will not be described in detail. MQ is used to divide different fine-tuning strategies;

[0181] Set the adjusted background guidance assignment to ;

[0182] 1) When assigning fine-tuning index When it is greater than the threshold MQ, and The values ​​are all in the interval [0.69, 1), indicating that the inspection and diagnosis results of the overall pathological slice image are good, and the background guidance assignment Perform second level fine adjustment;

[0183] The fine-tuning strategy is to reduce the background guidance assignment of the jth group of pathological slice images of the current hth patient , The formula is as follows:

[0184]

[0185] 2) When assigning fine-tuning index When it is equal to the threshold MQ, and The values ​​are all in the interval (0.45, 0.69), indicating that the inspection and diagnosis results of the overall pathological section image are critical, and the first level of fine-tuning is performed on the background guidance assignment;

[0186] The fine-tuning strategy is to keep the background guidance assignment constant, The mathematical expression is as follows:

[0187]

[0188] 3) When assigning fine-tuning index When it is less than the threshold MQ, and The values ​​are all in the interval (0, 0.45], indicating that the inspection and diagnosis results of the overall pathological section image deviate greatly from the normal range, and the background guidance assignment needs to be fine-tuned at the first level;

[0189] The fine-tuning strategy is to increase the background guidance assignment of the jth group of pathological slice images of the current hth patient ; The mathematical expression is as follows:

[0190] .

[0191] To further explain, the high-level features after output fusion include:

[0192] Weighted feature fusion:

[0193] Using the adjusted background to guide the assignment , perform weighted fusion on the extracted features, the specific formula is as follows:

[0194]

[0195] in, is the extracted features of the jth group of pathological slice images, is the high-level feature after fusion;

[0196] The fused high-level features are used as the final output of the pathology image processing system for pathology analysis and diagnosis.

[0197] Embodiment 2:

[0198] See also Figure 2 , a pathological slice feature fusion system based on background-guided attention mechanism, the system is used to perform a pathological slice feature fusion method based on background-guided attention mechanism, including:

[0199] Sequential labeling module: used to collect several patients' pathological data and then label them sequentially to form a label set {1,2,...,h,...,H}, where h represents the index of the h-th patient, H represents the total number of patients, and the pathological data includes pathological slice images collected from the patient and pathological monitoring information obtained from the examination, and sequentially number several pathological slice images of the h-th patient to form a label set {h1,h2,...,hi,...,hn}, where hi represents the i-th pathological slice image of the h-th patient, hn represents the n-th pathological slice image of the h-th patient, and n represents the total number of pathological slice images of the patient;

[0200] Data preprocessing module: used for receiving pathological slice images and pathological monitoring information and performing preprocessing, the preprocessing includes denoising, normalizing and segmenting the pathological slice images, and structuring the pathological monitoring information, and the pathological monitoring information further includes disease examination result data and disease diagnosis result data, and the structuring processing is to convert the disease examination result data and disease diagnosis result data into a structured data format;

[0201] An influence coefficient generation module: used to obtain the disease examination result data and the disease diagnosis result data converted into a structured data format, and then analyze and process the disease examination result data to generate a first influence coefficient for evaluating the severity of the disease examination result, and analyze and process the disease diagnosis result data to generate a second influence coefficient for evaluating the severity of the diagnosis result;

[0202] Grouping module: used to obtain the first influence coefficient and the second influence coefficient, adopt the lesion feature grouping strategy, group the label set {h1,h2,...,hi,...,hn} according to the severity of the lesions in the pathological slice image, and form {1,2,...,j,...,m}, where j represents the jth pathological slice image group, and m represents the total number of pathological slice image groups;

[0203] The assignment model construction module is used to extract key features from the pathological monitoring information and use the extracted key features as training data to construct a background guidance assignment model, and the output of the background guidance assignment model is the background guidance assignment for each group of pathological slice images;

[0204] Index generation module: used to obtain the first influence coefficient and the second influence coefficient, and perform analysis and processing to construct an assignment fine-tuning index, which is used to provide a fine-tuning strategy for the background guidance assignment of each group of pathological slice images;

[0205] Feature fusion module: used to obtain the fine-tuning strategy and determine the background guidance assignment of each group of pathological slice images according to the fine-tuning strategy. Finally, the background guidance attention mechanism is used to perform weighted fusion on the extracted features and output the fused high-level features.

[0206] Application module: used to apply the output fusion high-level features to subsequent pathological analysis.

[0207] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0208] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A pathological slice feature fusion method based on background-guided attention mechanism, which is applied to pathological image processing system and is characterized by: The specific steps include: After collecting the pathological data of several patients, they are sequentially numbered to form a number set {1,2,...,h,...,H}, where h represents the index of the h-th patient, H represents the total number of patients, and the pathological data includes pathological slice images collected from the patient and pathological monitoring information obtained from the examination, and several pathological slice images of the h-th patient are sequentially numbered to form a number set {h1,h2,...,hi,...,hn}, where hi represents the i-th pathological slice image of the h-th patient, hn represents the n-th pathological slice image of the h-th patient, and n represents the total number of pathological slice images of the patient; After receiving the pathological slice image and the pathological monitoring information, preprocessing is performed, the preprocessing includes denoising, normalizing and segmenting the pathological slice image, and structuring the pathological monitoring information, and the pathological monitoring information further includes disease examination result data and disease diagnosis result data, and the structuring processing is to convert the disease examination result data and the disease diagnosis result data into a structured data format; Obtaining disease examination result data and disease diagnosis result data converted into a structured data format, then analyzing and processing the disease examination result data to generate a first influence coefficient for evaluating the severity of the disease examination result, and analyzing and processing the disease diagnosis result data to generate a second influence coefficient for evaluating the severity of the diagnosis result; Obtain the first influence coefficient and the second influence coefficient, adopt the lesion feature grouping strategy, group the label set {h1,h2,...,hi,...,hn} according to the severity of the lesion in the pathological slice image to form {1,2,...,j,...,m}, where j represents the jth pathological slice image group, and m represents the total number of pathological slice image groups; Extract key features from pathological monitoring information, and use the extracted key features as training data to construct a background guidance assignment model, and the output of the background guidance assignment model is the background guidance assignment for each group of pathological slice images; Obtaining a first influence coefficient and a second influence coefficient, analyzing and processing them, and constructing an assignment fine-tuning index, the assignment fine-tuning index being used to provide a fine-tuning strategy for background guidance assignment of each group of pathological slice images; Obtain a fine-tuning strategy, and determine the background guidance assignment for each group of pathological slice images according to the fine-tuning strategy, and finally use the background guidance attention mechanism to perform weighted fusion on the extracted features and output the fused high-level features; The output fused high-level features are applied to subsequent pathological analysis.

2. The pathological section feature fusion method based on background-guided attention mechanism according to claim 1 is characterized in that: The receiving of the pathological slice image and the pathological monitoring information is followed by preprocessing, specifically including: The pathological image processing system receives the pathological slice image and corresponding pathological monitoring information of the hth patient from the database; A Gaussian filter is used to perform denoising on each received pathological slice image; Each denoised pathological slice image is normalized to normalize the image pixel value to the range of (0,1); The normalized image is segmented using a convolutional neural network to extract the lesion area.

3. The pathological section feature fusion method based on background-guided attention mechanism according to claim 2 is characterized in that: The structural processing of the disease examination result data and the disease diagnosis result data specifically includes: After receiving the disease examination result data and disease diagnosis result data, structured processing is performed and the structured data is defined as and , which has the following form: in, and represent the uth examination item and the hth examination item of the hth patient respectively. diagnostic items, U represents the total number of examination items, Indicates the total number of diagnostic items, and Respectively represent the corresponding inspection value and diagnosis result, Structured data representing disease test result data, Structured data representing disease diagnosis result data.

4. The pathological section feature fusion method based on background-guided attention mechanism according to claim 3 is characterized in that: set up and The normal range values ​​are and ;set up The corresponding rating value is express, The corresponding rating value is express; when Value in When it is within the range, the corresponding output score value is 1; when Value in When it is outside the range, the corresponding output score value is 0; when Value in When it is within the range, the corresponding output score value is 1; when Value in When it is outside the range, the corresponding output score value is 0; According to the above scoring criteria, the scoring set of U inspection items is obtained: as well as A collection of scores for diagnostic items: Define the first influence coefficient of the hth patient as , the calculation formula is as follows: in, is the adjustment factor, , U is the total number of inspection items, c1 is the positive adjustment item, and 0.03≤c1≤0.87, ; is the score value of the u-th examination item of the h-th patient; is the actual examination value of the u-th examination item of the h-th patient; is the normal range value of the u-th examination item for the h-th patient; The second influence coefficient of the hth patient is defined as , the calculation formula is as follows: in, is the adjustment factor, , U' is the total number of diagnostic items, c2 is the adjusted positive item, and 0.12≤c2≤0.96, ; and The value range of is in the range of (0,1); when =0, indicating that all test results are beyond the normal range. The closer it is to 0, the more test results are outside the normal range; when =1, indicating that all test results are within the normal range. The closer it is to 1, the more test results are within the normal range; when =0, indicating that all diagnostic results are very serious. The closer it is to 0, the more diagnostic results are beyond the normal range; when =1, indicating that all diagnostic results are within the acceptable range. The closer it is to 1, the more diagnostic results are within the normal range.

5. The pathological section feature fusion method based on background-guided attention mechanism according to claim 4 is characterized in that: The lesion feature grouping strategy includes: The lesion severity value of the i-th pathological slice image of the h-th patient is defined as , the calculation formula is as follows: in, and is the weight coefficient, , n is the total number of pathological slice images; according to the severity of the lesion Cluster analysis is performed on the pathological slice images. Specifically, the K-means clustering algorithm is used to divide the pathological slice images into different groups: Wherein, k3 is the preset number of groups, and J is the grouping result; According to the final clustering center, each pathological slice image is assigned to the corresponding cluster to obtain the final grouping result {1, 2, ..., j, ..., m}, and the pathological slice image of each group is output.

6. The pathological section feature fusion method based on background-guided attention mechanism according to claim 5 is characterized in that: The background guidance assignment model is constructed, and the output of the background guidance assignment model is the background guidance assignment for each group of pathological slice images, which specifically includes: Extracting key features from structured pathology monitoring information , key features Including but not limited to the patient's disease examination results and disease diagnosis results; The key features to be extracted Use the Z-score standardization method to perform standardization and obtain the standardized feature vector : The key features after standardization Scoring set with inspection items and the score set of diagnostic items Combine them to form the following training data set: in, and Represent the uth inspection item and the The score of each diagnostic item, represents the training data set; Select the random forest machine learning model for training; Input the structured pathology monitoring information and scoring values ​​into the trained model to obtain the background guidance assignment for each set of pathology slice images: in, represents the background guidance assignment of the jth group of pathological slice images, Represents a random forest model.

7. The pathological section feature fusion method based on background-guided attention mechanism according to claim 6 is characterized in that: Define the fine-tuning index for the hth patient as , the calculation formula is as follows: in, is the fine-tuning index assigned to the hth patient, r1 is the weight of the first influence coefficient, 0.12≤r1≤0.89; r2 is the weight of the second influence coefficient, 0.08≤r1≤0.79, r1>r2, and r1+r2=1, is the first influence coefficient, is the second influence coefficient; Set the value fine-tuning index The value range of is (0, Z), where Z is a positive number; when The closer it is to 0, the more the inspection and diagnosis results of the overall pathological slice image deviate from the normal range, and the background guidance assignment needs to be fine-tuned to the first level; when When it is closer to Z, it means that the inspection and diagnosis results of the overall pathological slice image are within the normal range, and the background guidance assignment is fine-tuned to the second level; And the fine-tuning amount of the first level is greater than the fine-tuning amount of the second level.

8. The pathological section feature fusion method based on background-guided attention mechanism according to claim 7 is characterized in that: The first level of fine-tuning amount and the second level of fine-tuning amount specifically include: Set MQ to the value fine-tuning index The comparison threshold is 0.69≤MQ≤0.96, MQ<Z; MQ is used to divide different fine-tuning strategies; the adjusted background guidance assignment is set to ; 1) When assigning fine-tuning index When it is greater than the threshold MQ, it means Approaching Z, at this time, and The values ​​are all in the interval [0.69, 1), indicating that the inspection and diagnosis results of the overall pathological slice image are good, and the background guidance assignment Perform second level fine adjustment; The second level corresponding fine-tuning strategy is to reduce the background guidance assignment of the jth group of pathological slice images of the current hth patient. , The formula is as follows: 2) When assigning fine-tuning index When it is equal to the threshold MQ, and The values ​​are all in the interval (0.45, 0.69), indicating that the examination and diagnosis results of the overall pathological section images are critical; The fine-tuning strategy is to keep the background guidance assignment constant, The mathematical expression is as follows: 3) When assigning fine-tuning index When it is less than the threshold MQ, it means Approaching 0, and The values ​​are all in the interval (0, 0.45], indicating that the inspection and diagnosis results of the overall pathological section image deviate greatly from the normal range, and the background guidance assignment needs to be fine-tuned at the first level; The first level corresponding fine-tuning strategy is to increase the background guidance assignment of the jth group of pathological slice images of the current hth patient ; The mathematical expression is as follows: 。 9. The pathological section feature fusion method based on background-guided attention mechanism according to claim 8, characterized in that: The high-level features after the output fusion specifically include: Using the adjusted background to guide the assignment , perform weighted fusion on the extracted features, the specific formula is as follows: in, is the extracted features of the jth group of pathological slice images, is the high-level feature after fusion; The fused high-level features are used as the final output of the pathology image processing system for pathology analysis and diagnosis.

10. A pathological slice feature fusion system based on background-guided attention mechanism, characterized by: The system is used to execute the pathological section feature fusion method based on the background-guided attention mechanism according to any one of claims 1 to 9, comprising: Sequential labeling module: used to collect several patients' pathological data and then label them sequentially to form a label set {1,2,...,h,...,H}, where h represents the index of the h-th patient, H represents the total number of patients, and the pathological data includes pathological slice images collected from the patient and pathological monitoring information obtained from the examination, and sequentially number several pathological slice images of the h-th patient to form a label set {h1,h2,...,hi,...,hn}, where hi represents the i-th pathological slice image of the h-th patient, hn represents the n-th pathological slice image of the h-th patient, and n represents the total number of pathological slice images of the patient; Data preprocessing module: used for receiving pathological slice images and pathological monitoring information and performing preprocessing, the preprocessing includes denoising, normalizing and segmenting the pathological slice images, and structuring the pathological monitoring information, and the pathological monitoring information further includes disease examination result data and disease diagnosis result data, and the structuring processing is to convert the disease examination result data and disease diagnosis result data into a structured data format; An influence coefficient generation module: used to obtain the disease examination result data and the disease diagnosis result data converted into a structured data format, and then analyze and process the disease examination result data to generate a first influence coefficient for evaluating the severity of the disease examination result, and analyze and process the disease diagnosis result data to generate a second influence coefficient for evaluating the severity of the diagnosis result; Grouping module: used to obtain the first influence coefficient and the second influence coefficient, adopt the lesion feature grouping strategy, group the label set {h1,h2,...,hi,...,hn} according to the severity of the lesions in the pathological slice image, and form {1,2,...,j,...,m}, where j represents the jth pathological slice image group, and m represents the total number of pathological slice image groups; The assignment model construction module is used to extract key features from the pathological monitoring information and use the extracted key features as training data to construct a background guidance assignment model, and the output of the background guidance assignment model is the background guidance assignment for each group of pathological slice images; Index generation module: used to obtain the first influence coefficient and the second influence coefficient, and perform analysis and processing to construct an assignment fine-tuning index, which is used to provide a fine-tuning strategy for the background guidance assignment of each group of pathological slice images; Feature fusion module: used to obtain the fine-tuning strategy and determine the background guidance assignment of each group of pathological slice images according to the fine-tuning strategy. Finally, the background guidance attention mechanism is used to perform weighted fusion on the extracted features and output the fused high-level features. Application module: used to apply the output fusion high-level features to subsequent pathological analysis.

Citation Information

Patent Citations

  • Pathological section feature fusion method based on background guidance attention mechanism

    CN114118258A

Cited By

  • Coal bed gas productivity prediction method and system combined with artificial intelligence

    CN120258328A

  • Case allocation method and device based on sub-specialized matching, terminal and medium

    CN120544819A