Prediction methods, systems, devices, and media based on feature map similarity weighting

Abstract

This invention discloses a prediction method, system, device, and medium based on feature map similarity weighting. The method includes: constructing an integrated prediction model, which is composed of multiple optimal prediction models, each of which is trained using a corresponding training set and validation set; inputting the corresponding validation set into the corresponding optimal prediction model to obtain a first feature map for each category; inputting the corresponding test set into the corresponding optimal prediction model to obtain a prediction probability and a second feature map; calculating the cosine similarity between the first and second feature maps of the same optimal prediction model; weighting the prediction probabilities of the optimal prediction models based on the cosine similarity to obtain a weighted prediction probability; calculating the mean of the weighted prediction probabilities of all optimal prediction models; and using the mean as the prediction probability of the integrated prediction model. This improves the prediction performance, robustness, and generalization ability of the model.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a prediction method, system, device and medium based on feature map similarity weighting. Background Technology

[0002] In the past few years, with the rise of big data and the improvement of computing power, deep learning has achieved great success in the field of vision. New models and methods are proposed every year, and the accuracy of classification, detection, and segmentation is constantly being refreshed. The application of deep learning has also expanded from traditional natural images to the field of medical images. However, compared with natural images, the application of deep learning in the field of medical images faces greater challenges: less data, high annotation costs, and more complex scenarios.

[0003] Medical image data is collected from different hardware devices and different hospitals, resulting in significant differences in image data distribution. Using simple models to extract and fit features to image data leads to slow convergence, or even failure to converge, resulting in underfitting and poor predictive performance. Conversely, using complex models for feature extraction and fitting results in overfitting to the personalized features of the training set, leading to poor generalization. Summary of the Invention

[0004] This invention aims to at least address the technical problems existing in the prior art. To this end, this invention proposes a prediction method, system, device, and medium based on feature map similarity weighting, which can improve the predictive performance of the model, as well as enhance the model's robustness and generalization ability.

[0005] In a first aspect, embodiments of the present invention provide a prediction method based on feature map similarity weighting, comprising the following steps:

[0006] Step S100: Divide the acquired MR images into multiple training sets, validation sets, and test sets;

[0007] Step S200: Construct an ensemble prediction model, which consists of a set of multiple optimal prediction models. Each optimal prediction model is trained using the corresponding training set and validation set, and is obtained through training in the following manner:

[0008] Step S201: Construct a first prediction model based on deep learning, train the first prediction model according to the training set and verify the first prediction model according to the validation set to obtain a second prediction model;

[0009] Step S202: Based on the second prediction model, select a high-confidence training set and a high-confidence validation set from the training set and the validation set; the difference between the MR image prediction probability and the MR image category in the high-confidence training set and the high-confidence validation set is lower than a threshold.

[0010] Step S203: Adjust the second prediction model according to the high confidence training set, and verify the second prediction model according to the high confidence validation set to obtain the optimal prediction model;

[0011] Step S300: Input the corresponding validation set into the corresponding optimal prediction model to obtain the first feature map corresponding to each category;

[0012] Step S400: Input the corresponding test set into the corresponding optimal prediction model to obtain the prediction probability and the second feature map;

[0013] Step S500: Calculate the cosine similarity between the first feature map and the second feature map of the same optimal prediction model, and weight the prediction probability of the optimal prediction model according to the cosine similarity to obtain the weighted prediction probability.

[0014] Step S600: Calculate the mean of the weighted prediction probabilities of all the optimal prediction models, use the mean as the prediction probability of the integrated prediction model, and obtain the classification result based on the prediction probability of the integrated prediction model.

[0015] According to embodiments of the present invention, at least the following technical effects are achieved:

[0016] By filtering the training and validation sets to obtain high-confidence training and validation sets, and then fine-tuning the model using these sets, this coarse-to-fine dataset cleaning method reduces the difficulty of model fitting, improves the model's prediction performance on independent test sets, and enhances the model's robustness. Furthermore, by calculating the cosine similarity between the first and second feature maps to weight the model's prediction probabilities, the confidence and generalization of the predictions are improved. Finally, the mean of the prediction probabilities is applied to further enhance the model's robustness.

[0017] According to some embodiments of the present invention, adjusting the second prediction model based on the high-confidence training set includes:

[0018] Freeze the weight parameters of all layers of the second prediction model except the classification layer;

[0019] The parameters of the classification layer are updated during gradient backpropagation based on the high-confidence training set.

[0020] According to some embodiments of the present invention, the optimal prediction model is evaluated using an accuracy index. When the high-confidence validation set has the highest accuracy index in the second prediction model, the second prediction model is the optimal prediction model.

[0021] According to some embodiments of the present invention, the first feature map is obtained by average pooling of all feature maps of each category in each of the validation sets, and the second feature map is obtained by the feature maps of each of the optimal prediction models in the test sets.

[0022] According to some embodiments of the present invention, the formula for calculating the cosine similarity is as follows:

[0023]

[0024] Wherein, the f i The first feature map representing the i-th model, the F ij The second feature map representing the j-th category of the i-th model, wherein cos_simi(f i ,F ij ) represents cosine similarity.

[0025] According to some embodiments of the present invention, the calculation formula for weighting the prediction probability of the optimal prediction model based on the cosine similarity is as follows:

[0026] p ij =p ij *cos_simi(f i ,F ij )

[0027] Where, P on the left side of the equation ij P represents the weighted predicted probability, and P on the right side of the equation. ij This represents the predicted probability before weighting.

[0028] According to some embodiments of the present invention, the formula for calculating the mean of the weighted prediction probabilities of all the optimal prediction models is as follows:

[0029]

[0030] Wherein, p j The denot represents the prediction probability of the integrated prediction model, m represents the number of the optimal prediction models, and n represents the number of categories of the MR image.

[0031] Secondly, embodiments of the present invention provide a prediction system based on feature map similarity weighting, comprising:

[0032] The image acquisition module is used to divide the acquired MR images into multiple training sets, validation sets, and test sets;

[0033] A model building module is used to construct an integrated prediction model, which is composed of a set of multiple optimal prediction models. Each optimal prediction model is trained using a corresponding training set and validation set. The optimal prediction model is obtained through the following training method: constructing a first prediction model based on deep learning; training the first prediction model using the training set and validating the first prediction model using the validation set to obtain a second prediction model; selecting a high-confidence training set and a high-confidence validation set from the training set and the validation set based on the second prediction model; the difference between the MR image prediction probability and the MR image category in the high-confidence training set and the high-confidence validation set is lower than a threshold; adjusting the second prediction model based on the high-confidence training set and validating the second prediction model using the high-confidence validation set to obtain the optimal prediction model.

[0034] The first feature map acquisition module is used to input the corresponding validation set into the corresponding optimal prediction model to obtain the first feature map corresponding to each category;

[0035] The second feature map acquisition module inputs the corresponding test set into the corresponding optimal prediction model to obtain the prediction probability and the second feature map;

[0036] The probability weighting module calculates the cosine similarity between the first feature map and the second feature map of the same optimal prediction model, and weights the prediction probability of the optimal prediction model according to the cosine similarity to obtain the weighted prediction probability.

[0037] The probability mean module calculates the mean of the weighted prediction probabilities of all the optimal prediction models, uses the mean as the prediction probability of the integrated prediction model, and obtains the classification result based on the prediction probability of the integrated prediction model.

[0038] Thirdly, embodiments of the present invention provide a prediction device based on feature map similarity weighting, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform any of the feature map similarity weighting prediction methods described in the first aspect.

[0039] Fourthly, embodiments of the present invention provide a computer-readable storage medium for a prediction method based on feature map similarity weighting, the computer-readable storage medium storing computer-executable instructions for causing a computer to perform any of the feature map similarity weighting prediction methods described in the first aspect.

[0040] It should be noted that the beneficial effects of the second to fourth aspects of the present invention compared with the prior art are the same as the beneficial effects of the feature map similarity-weighted prediction method of the first aspect, and will not be described in detail here.

[0041] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0042] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0043] Figure 1 This is a flowchart of a prediction method based on feature map similarity weighting provided in one embodiment of the present invention;

[0044] Figure 2 This is a flowchart of a method for training an optimal prediction model according to an embodiment of the present invention;

[0045] Figure 3 This is a structural diagram of a prediction system based on feature map similarity weighting provided in one embodiment of the present invention;

[0046] Figure 4 This is a structural diagram of an electronic device provided in one embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of a prediction method based on feature map similarity weighting provided in an embodiment of the present invention. Detailed Implementation

[0048] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0049] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "axial," "radial," and "circumferential," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, features defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0050] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0051] Reference Figure 1 and Figure 2 In some embodiments of the present invention, a prediction method based on feature map similarity weighting is provided, comprising the following steps:

[0052] Step S100: Divide the acquired MR images into multiple training sets, validation sets, and test sets;

[0053] Step S200: Construct an ensemble prediction model. The ensemble prediction model consists of a set of multiple optimal prediction models. Each optimal prediction model is trained using its corresponding training and validation sets. The optimal prediction model is obtained through training in the following manner:

[0054] Step S201: Construct a first prediction model based on deep learning. Train the first prediction model according to the training set and validate the first prediction model according to the validation set to obtain a second prediction model.

[0055] Step S202: Based on the second prediction model, select a high-confidence training set and a high-confidence validation set from the training set and the validation set; the difference between the MR image prediction probability and the MR image category in the high-confidence training set and the high-confidence validation set is lower than the threshold.

[0056] Step S203: Adjust the second prediction model based on the high-confidence training set, and validate the second prediction model based on the high-confidence validation set to obtain the optimal prediction model;

[0057] Step S300: Input the corresponding validation set into the corresponding optimal prediction model to obtain the first feature map for each category;

[0058] Step S400: Input the corresponding test set into the corresponding optimal prediction model to obtain the prediction probability and the second feature map;

[0059] Step S500: Calculate the cosine similarity between the first feature map and the second feature map of the same optimal prediction model, and weight the prediction probability of the optimal prediction model according to the cosine similarity to obtain the weighted prediction probability.

[0060] Step S600: Calculate the mean of the weighted prediction probabilities of all optimal prediction models, use the mean as the prediction probability of the ensemble prediction model, and obtain the classification result based on the prediction probability of the ensemble prediction model.

[0061] According to the feature map similarity-weighted prediction method of this invention, a high-confidence training set and a high-confidence validation set are obtained by filtering the training set and validation set, and the model is adjusted using the high-confidence training set and the high-confidence validation set. This coarse-to-fine dataset cleaning method reduces the difficulty of model fitting, improves the model's prediction performance on the independent test set, and improves the model's robustness. By calculating the cosine similarity between the first feature map and the second feature map to weight the model's prediction probability, the confidence and generalization of the model's prediction are improved. Finally, the mean of the prediction probability is processed to improve the model's robustness.

[0062] In some embodiments of the present invention, adjusting the second prediction model based on a high-confidence training set includes:

[0063] Freeze the weight parameters of all layers in the second prediction model except the classification layer;

[0064] The parameters of the classification layer are updated during gradient backpropagation based on the high-confidence training set.

[0065] Freeze the weight parameters of all layers except the classification layer to prevent errors when adjusting the second prediction model. Modifying the parameters of other layers would affect the accuracy of the second prediction model. Secondly, only modify the classification layer, and change the parameters of the classification layer through a high-confidence training set to make the second prediction model more reliable.

[0066] In some embodiments of the present invention, the optimal prediction model is evaluated using the accuracy metric. When the high-confidence validation set has the highest accuracy metric in the second prediction model, the second prediction model is the optimal prediction model. Accuracy is the sum of recall rates for each class divided by the total number of samples. Using the accuracy metric is the simplest and most intuitive evaluation metric in sample classification problems, with low time complexity.

[0067] In some embodiments of the present invention, the first feature map is obtained by average pooling all feature maps of each category in each optimal prediction model from the validation set, and the second feature map is obtained by the feature maps of each optimal prediction model from the test set. The feature maps are located in the layer preceding the classification layer of the model's network structure. The first feature map is a reference feature map obtained by average pooling all feature maps of each category in each optimal prediction model from the validation set, with each category of each optimal prediction model having a corresponding first feature map. The second feature map is a comparison feature map obtained by the test set in each category of each optimal prediction model, with each category of each optimal prediction model having a corresponding first and second feature map.

[0068] In some embodiments of the present invention, the formula for calculating cosine similarity is as follows:

[0069]

[0070] Where, f i F represents the first feature map of the i-th model. ij Let cos_simi(f) represent the second feature map of the j-th class in the i-th model. i ,F ij The symbol ) represents cosine similarity. Cosine similarity is not sensitive to absolute numerical values, thus correcting the problem of inconsistent measurement standards that may exist between feature maps.

[0071] In some embodiments of the present invention, the formula for weighting the prediction probability of the optimal prediction model based on cosine similarity is as follows:

[0072] p ij =p ij *cos_simi(f i ,F ij )

[0073] Where, P on the left side of the equation ij P represents the weighted predicted probability, and P on the right side of the equation. ij This represents the predicted probability before weighting. The predicted probabilities are weighted according to the cosine similarity and the category of the corresponding optimal prediction model, making the predicted probabilities more accurate, the prediction error smaller, and thus the model more robust.

[0074] In some embodiments of the present invention, the formula for calculating the mean of the weighted prediction probabilities of all optimal prediction models is as follows:

[0075]

[0076] Where, p j The ensemble represents the prediction probability of the ensemble prediction model, where m represents the number of optimal prediction models and n represents the number of MR image categories. Integrating all optimal prediction models into a single ensemble improves the model's robustness, and the resulting prediction probabilities are derived from all MR images, leading to more accurate predictions and enhancing the overall model's precision.

[0077] Reference Figure 5 To facilitate understanding by those skilled in the art, a specific embodiment of a prediction method based on feature map similarity weighting is provided, including the following steps:

[0078] The first step is to divide the 188 MR cases into training set / validation set / test set in a ratio of 4:2:1, and denoted as D_Train / D_Val / D_Test respectively.

[0079] The second step involves training a deep learning model using D_Train, with the chosen model architecture being EfficientNetB3 (but not limited to this model). The model's performance on D_Val is evaluated at different training stages, and the best-performing model is selected as model A1. Simultaneously, model A1 is used to predict the samples in D_Train. Samples with predicted probabilities and class differences below a given threshold (set to 0.2 here, but thresholds between 0 and 0.5 are acceptable) are considered high-confidence samples and selected as D_Train1 (high-confidence training set). Samples in D_Train with predicted probabilities and class differences above the given threshold are selected as low-confidence training sets and selected as D_Train2. Model A1 is used to predict samples in D_Val. Samples with predicted probabilities and class differences below a given threshold (set to 0.2 in this patent) are considered high-confidence samples and selected as D_Val1 (high-confidence validation set). Samples in D_Val with predicted probabilities and class differences above the given threshold are selected as low-confidence samples and selected as D_Val2. Model A is fine-tuned using D_Train1 (the earlier layers of the model network structure are fixed, and only the last classification layer is trained). The model with the highest prediction performance in D_Val1 is selected to replace the original model A1 as the latest model A1. The accuracy metric is used to evaluate the model's performance, which is the sum of recall rates for each class divided by the total number of samples. The model's performance on the prediction dataset is evaluated. When the accuracy on the validation set is the highest, the model is considered to have the best performance on the validation set. Models EfficientNetB2, EfficientNetB1, and EfficientNetB0 are constructed sequentially. The models are trained using their respective training datasets that have been selected sequentially, and the optimal models A2, A3, and A4 are selected using their respective validation sets.

[0080] The third step involves using models A1, A2, A3, and A4 to extract feature maps of the layer preceding the classification layer of the network structure from the images in their respective validation sets D_Val1, D_Val2, D_Val3, and D_Val4. These feature maps are denoted as F111, F112, ..., F1KN, where the first dimension of the label represents the model labels A1, A2, A3, and A4; the second dimension represents the class label of the data; and the third dimension represents the image sequence label in their respective validation sets. For example, Fijk represents the feature map predicted by model Ai for image sequence k of class j in D_Val1.

[0081] For feature maps F111, F112, ..., F1KN, perform average pooling on all feature maps belonging to the same category within each model A1, A2, A3, and A4 to obtain the support feature maps (first feature maps) for each category of each model A1, A2, A3, and A4 as F. ij , where i is the model index and j is the corresponding category index.

[0082] For images on the test set D_Test, predictions are made using models A1, A2, A3, and A4 to obtain feature maps (second feature maps) and class probabilities for each model. The cosine similarity between the feature maps and the supporting feature maps for different classes under each model is calculated. This cosine similarity is then used to weight the class probabilities predicted by A1, A2, A3, and A4 to obtain the final probability predicted by each model. The specific algorithm flow is as follows:

[0083] The data x from D_Test is input as parameters into models A1, A2, A3, and A4 respectively to obtain the predicted probability P. ij , and the corresponding feature map f i .

[0084] Calculate f i and F ij The cosine similarity is calculated using the following formula:

[0085]

[0086] f i Represents the feature map, F ij Represents the supporting feature map, cos_simi(f i ,F ij ) represents cosine similarity.

[0087] The obtained cosine similarity is then used to weight the prediction probability, as shown below:

[0088] p ij =p ij *cos_simi(f i ,F ij )

[0089] Where, P on the left side of the equation ij P represents the weighted predicted probability, and P on the right side of the equation. ij This represents the predicted probability before weighting.

[0090] The mean of the predicted probabilities of A1, A2, A3, and A4 is calculated, and then the class probabilities are normalized to obtain the predicted probabilities of the ensemble prediction model, as shown below:

[0091]

[0092] Where, p j represents the prediction probability of the ensemble prediction model, m represents the number of optimal prediction models, and n represents the number of categories of the MR image.

[0093] Reference Figure 3 One embodiment of the present invention provides a prediction system 1000 based on feature map similarity weighting, including an image acquisition module 1001, a model building module 1002, a first feature map acquisition module 1003, a second feature map acquisition module 1004, a probability weighting module 1005, and a probability mean module 1006, wherein:

[0094] Image acquisition module 1001 is used to divide the acquired MR images into multiple training sets, validation sets and test sets;

[0095] Model building module 1002 is used to build an integrated prediction model. The integrated prediction model consists of a set of multiple optimal prediction models. Each optimal prediction model is trained using a corresponding training set and validation set. The optimal prediction model is obtained by training as follows: a first prediction model based on deep learning is constructed; the first prediction model is trained using the training set and validated using the validation set to obtain a second prediction model; a high-confidence training set and a high-confidence validation set are selected from the training set and validation set based on the second prediction model; the difference between the MR image prediction probability and the MR image category in the high-confidence training set and the high-confidence validation set is lower than a threshold; the second prediction model is adjusted based on the high-confidence training set and validated using the high-confidence validation set to obtain the optimal prediction model.

[0096] The first feature map acquisition module 1003 is used to input the corresponding validation set into the corresponding optimal prediction model to obtain the first feature map corresponding to each category.

[0097] The second feature map acquisition module 1004 is used to input the corresponding test set into the corresponding optimal prediction model to obtain the prediction probability and the second feature map.

[0098] The probability weighting module 1005 is used to calculate the cosine similarity between the first feature map and the second feature map of the same optimal prediction model, and to weight the prediction probability of the optimal prediction model according to the cosine similarity to obtain the weighted prediction probability.

[0099] The probability mean module 1006 is used to calculate the mean of the weighted prediction probabilities of all optimal prediction models, and use the mean as the prediction probability of the integrated prediction model to obtain the classification result.

[0100] It should be noted that since the prediction system based on feature map similarity weighting in this embodiment is based on the same inventive concept as the prediction method based on feature map similarity weighting described above, the corresponding content in the method embodiment is also applicable to this device embodiment, and will not be described in detail here.

[0101] Reference Figure 4 In another embodiment of the present invention, an electronic device 6000 is also provided, which can be any type of smart terminal, such as a mobile phone, tablet computer, personal computer, etc.

[0102] Specifically, the electronic device 6000 includes: one or more control processors 6001 and memory 6002. Figure 4 Taking a control processor 6001 and a memory 6002 as an example, the control processor 6001 and the memory 6002 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0103] The memory 6002, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to an electronic device in an embodiment of the present invention.

[0104] The control processor 6001 executes various functional applications and data processing of a feature map similarity-weighted prediction method by running non-transient software programs, instructions, and modules stored in the memory 6002, thereby implementing a feature map similarity-weighted prediction method according to the above method embodiment.

[0105] The memory 6002 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created using a feature map similarity-weighted prediction method, etc. Furthermore, the memory 6002 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 6002 may optionally include memory remotely located relative to the control processor 6001, and these remote memories can be connected to the electronic device 6000 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0106] One or more modules are stored in memory 6002. When executed by one or more control processors 6001, the feature map similarity-weighted prediction method in the above method embodiments is executed, such as the method described above. Figure 1 and Figure 2 The method and steps.

[0107] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0108] It should be noted that since the electronic device in this embodiment is based on the same inventive concept as the prediction method based on feature map similarity weighting described above, the corresponding content in the method embodiment is also applicable to this device embodiment, and will not be described in detail here.

[0109] In one embodiment of the present invention, a computer-readable storage medium is also provided, storing computer-executable instructions for performing: the feature map similarity-weighted prediction method as described in the above embodiments.

[0110] It should be noted that since the computer-readable storage medium in this embodiment is based on the same inventive concept as the prediction method based on feature map similarity weighting described above, the corresponding content in the method embodiment is also applicable to this device embodiment, and will not be described in detail here.

[0111] Those skilled in the art will understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing data (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired data and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any data delivery medium.

[0112] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0113] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A prediction method based on feature map similarity weighting, characterized in that, Includes the following steps: Step S100: Divide the acquired MR images into multiple training sets, validation sets, and test sets; Step S200: Construct an ensemble prediction model, which consists of a set of multiple optimal prediction models. Each optimal prediction model is trained using its corresponding training set and validation set, and is obtained through training in the following manner: Step S201: Construct a first prediction model based on deep learning, train the first prediction model according to the training set and verify the first prediction model according to the validation set to obtain a second prediction model; Step S202: Based on the second prediction model, select a high-confidence training set and a high-confidence validation set from the training set and the validation set; the difference between the MR image prediction probability and the MR image category in the high-confidence training set and the high-confidence validation set is lower than a threshold. Step S203: Adjust the second prediction model according to the high confidence training set, and verify the second prediction model according to the high confidence validation set to obtain the optimal prediction model; Step S300: Input the corresponding validation set into the corresponding optimal prediction model to obtain a first feature map for each category; the first feature map is obtained by average pooling of all feature maps of each category in each optimal prediction model from the validation set; each category of each optimal prediction model has a one-to-one corresponding first feature map; Step S400: Input the corresponding test set into the corresponding optimal prediction model to obtain the prediction probability and the second feature map; the second feature map refers to the feature map of the test set that serves as a comparison in each optimal prediction model; each category of each optimal prediction model has a corresponding second feature map. Step S500: Calculate the cosine similarity between the first feature map and the second feature map of the same optimal prediction model, and weight the prediction probability of the optimal prediction model according to the cosine similarity to obtain the weighted prediction probability. Step S600: Calculate the mean of the weighted prediction probabilities of all the optimal prediction models, use the mean as the prediction probability of the integrated prediction model, and obtain the classification result based on the prediction probability of the integrated prediction model.

2. The prediction method based on feature map similarity weighting according to claim 1, characterized in that, The step of adjusting the second prediction model based on the high-confidence training set includes: Freeze the weight parameters of all layers of the second prediction model except the classification layer; The parameters of the classification layer are updated during gradient backpropagation based on the high-confidence training set.

3. The prediction method based on feature map similarity weighting according to claim 1, characterized in that, The optimal prediction model is evaluated using the accuracy index. When the high-confidence validation set has the highest accuracy index in the second prediction model, the second prediction model is the optimal prediction model.

4. The prediction method based on feature map similarity weighting according to claim 1, characterized in that, The formula for calculating the cosine similarity is as follows: Wherein, the Indicates the first The model in the first The first feature map of each category, Indicates the first The model of the first The second feature map of each category, This represents the cosine similarity.

5. The prediction method based on feature map similarity weighting according to claim 4, characterized in that, The formula for weighting the prediction probability of the optimal prediction model based on the cosine similarity is as follows: Among them, the left side of the equation This represents the weighted predicted probability, and the right side of the equation... This represents the predicted probability before weighting.

6. A prediction system based on feature map similarity weighting, characterized in that, include: The image acquisition module is used to divide the acquired MR images into multiple training sets, validation sets, and test sets; A model building module is used to construct an integrated prediction model, which consists of a set of multiple optimal prediction models. Each optimal prediction model is trained using its corresponding training set and validation set. The optimal prediction model is obtained through the following training method: constructing a first prediction model based on deep learning; training the first prediction model using the training set and validating it using the validation set to obtain a second prediction model; selecting a high-confidence training set and a high-confidence validation set from the training set and validation set based on the second prediction model; ensuring that the difference between the MR image prediction probability and the MR image category in the high-confidence training set and the high-confidence validation set is below a threshold; adjusting the second prediction model based on the high-confidence training set and validating it using the high-confidence validation set to obtain the optimal prediction model. The first feature map acquisition module is used to input the corresponding validation set into the corresponding optimal prediction model to obtain a first feature map for each category; the first feature map is obtained by average pooling of all feature maps of each category in each optimal prediction model from the validation set. The second feature map acquisition module inputs the corresponding test set into the corresponding optimal prediction model to obtain the prediction probability and the second feature map; the second feature map refers to the feature map of the test set that serves as a comparison for each category in each optimal prediction model. The probability weighting module calculates the cosine similarity between the first feature map and the second feature map of the same optimal prediction model, and weights the prediction probability of the optimal prediction model according to the cosine similarity to obtain the weighted prediction probability. The probability mean module calculates the mean of the weighted prediction probabilities of all the optimal prediction models, uses the mean as the prediction probability of the integrated prediction model, and obtains the classification result based on the prediction probability of the integrated prediction model.

7. An electronic device, characterized in that: It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform the feature map similarity-weighted prediction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the feature map similarity-weighted prediction method as described in any one of claims 1 to 5.

Images