A method and device for evaluating the malignant risk of a mass in the multimodal attachment area

Abstract

The present application discloses a method and device for evaluating the malignant risk of multi-modal adnexal mass. The method for evaluating the malignant risk of multi-modal adnexal mass includes: obtaining an image to be recognized; obtaining basic information of the person to be evaluated; extracting image features of the image to be recognized; generating structured basic information features according to the basic information of the person to be evaluated; splicing the structured basic information features and the image features to obtain features to be used; obtaining a plurality of trained xgboost models; obtaining a plurality of initial prediction results according to the features to be used and each trained xgboost model; obtaining a drift diffusion model; and obtaining a final prediction result according to each initial prediction result and the drift diffusion model. By cooperating the xgboost model with the drift diffusion model, the present application can increase the AUC to 0.957, improve the model ACC from 0.892 to 0.910, and significantly improve the sensitivity (from 0.749 to 0.814) while increasing the specificity (from 0.933 to 0.941).

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a multimodal adnexal mass malignancy risk assessment method and a multimodal adnexal mass malignancy risk assessment device. Background Art

[0002] Ovarian cancer has a poor prognosis and is the gynecological malignancy with the highest mortality rate. Correctly distinguishing benign and malignant lesions can provide conservative treatment for physiological and partially benign masses, preserve the patient's fertility, and avoid surgical complications; for borderline and malignant tumors, correct differentiation can help patients be referred to hospitals with corresponding surgical capabilities in a timely manner, and at the same time ensure that doctors are adequately prepared before surgery.

[0003] Ultrasound is a conventional, non-invasive examination method. It is one of the important imaging examination methods in the diagnosis of ovarian cancer because of its low examination cost, lack of radioactivity, and ease of repeated examination.

[0004] Currently, the grading techniques for adnexal masses using ultrasound imaging mainly include the Orads grading method and the IOTA ADNEX model.

[0005] Although the Orads grading standard can achieve a relatively accurate classification of mass grades after a period of training for doctors, it is a purely manual judgment method that consumes manpower and has a slow identification speed. In addition, its accuracy is limited by the doctor's level and proficiency and is affected by the doctor's personal condition.

[0006] The IOTAADNEX model collected a small amount of data and only collected one tumor marker, CA125, in the experiment. In addition, the indicator decreased significantly when the tumor marker was missing.

[0007] Therefore, it is hoped that a technical solution can be provided to solve or at least alleviate the above-mentioned deficiencies of the prior art. Summary of the invention

[0008] The object of the present invention is to provide a multimodal adnexal mass malignancy risk assessment method to solve at least one of the above-mentioned technical problems.

[0009] The present invention provides the following scheme:

[0010] According to one aspect of the present invention, a multi-modal adnexal mass malignancy risk assessment method is provided, and the multi-modal adnexal mass malignancy risk assessment method comprises:

[0011] Obtain an image to be recognized;

[0012] Obtain basic information of the person to be evaluated;

[0013] Extracting image features of the image to be identified;

[0014] Generate basic information structured features according to the basic information of the person to be evaluated;

[0015] splicing the basic information structured features and the image features to obtain features to be used;

[0016] Get multiple trained xgboost models;

[0017] Get multiple initial prediction results based on the features to be used and each trained xgboost model;

[0018] Get the drift-diffusion model;

[0019] The final prediction result is obtained according to each initial prediction result and the drift diffusion model.

[0020] Optionally, the basic information of the person to be evaluated includes:

[0021] Family history information, medical history information, hormone treatment history information, the presence or absence of blood flow signals, whether menopausal, basic information on tumor markers, and mass size information;

[0022] The generating of basic information structured features according to the basic information of the person to be evaluated includes:

[0023] Obtain screening conditions for benign and malignant correlation;

[0024] Screening the basic information of the person to be evaluated according to the benign and malignant correlation screening condition, thereby obtaining the screened basic information of the person to be evaluated;

[0025] Generate basic information structured features based on the screened basic information of the person to be evaluated.

[0026] Optionally, before extracting the image features of the image to be identified, the multimodal adnexal mass malignancy risk assessment method further comprises:

[0027] Preprocessing the image to be identified, thereby obtaining preprocessed image information;

[0028] The extracting the image features of the image to be identified comprises:

[0029] The image features are extracted according to the preprocessed image information.

[0030] Optionally, extracting the image features according to the preprocessed image information includes:

[0031] Get the trained image feature extractor;

[0032] The preprocessed image information is input into the image feature extractor to obtain image features.

[0033] Optionally, the image feature extractor comprises:

[0034] A spatially local convolution;

[0035] A spatial long-range convolution;

[0036] One channel convolution;

[0037] The loss function uses the following formula:

[0038] FL(p t )=-α t (1-p t ) γ log(p t );in,

[0039] P t is the probability that the model predicts that the sample belongs to category t; the weight factor α∈[0,1], when it is a positive sample, the weight factor is α, when it is a negative sample, the weight factor is 1-α; the modulation factor (1-p t ) γ , γ ranges from [0,5].

[0040] Optionally, the step of combining the basic information structured features and the image features to obtain features to be used includes:

[0041] The image features are concatenated with the structural features and processed using the Lasso method to obtain a data feature matrix.

[0042] Optionally, obtaining a final prediction result according to each initial prediction result and the drift diffusion model includes:

[0043] Randomly sort the initial prediction results, and obtain the top N initial prediction results as the initial prediction results to be used;

[0044] Each initial prediction result to be used is input into the drift diffusion model to obtain the final prediction result and response time.

[0045] Optionally, in the process of inputting each initial prediction result to be used into the drift diffusion model to obtain the final prediction result and response time, for each initial prediction result to be used, first initialize an array with a length equal to the randomly selected number and uniform distribution, in which each element represents the posterior probability, and loop through each time step, multiply the probability output by the neural network by the posterior of the previous step at each step, and perform a normalization operation to obtain the posterior probability of each category.

[0046] The present application also provides a multi-modal adnexal mass malignancy risk assessment device, the multi-modal adnexal mass malignancy risk assessment device comprising:

[0047] An image acquisition module to be identified, wherein the image acquisition module to be identified is used to acquire the image to be identified;

[0048] A basic information acquisition module for the person to be evaluated, wherein the basic information acquisition module for the person to be evaluated is used to acquire basic information of the person to be evaluated;

[0049] An image feature acquisition module, the image feature acquisition module is used to extract image features of the image to be identified;

[0050] A structured feature generation module, the structured feature generation module is used to generate basic information structured features according to the basic information of the person to be evaluated;

[0051] A feature splicing module to be used, the feature splicing module to be used is used to splice the basic information structured features and the image features to obtain the feature to be used;

[0052] An xgboost model acquisition module, wherein the xgboost model acquisition module is used to acquire multiple trained xgboost models;

[0053] An initial prediction result acquisition module, wherein the initial prediction result acquisition module is used to obtain multiple initial prediction results according to the data feature matrix and each trained xgboost model;

[0054] A drift diffusion model acquisition module, wherein the drift diffusion model acquisition module is used to acquire a drift diffusion model;

[0055] A prediction module is used to obtain a final prediction result based on each initial prediction result and the drift diffusion model.

[0056] This application uses the xgboost model in conjunction with the drift diffusion model to enable the image model to achieve an AUC of 0.911. The xgboost multiple group results are directly averaged to achieve an AUC of 0.946. In addition, through the screening of the brain-like accumulation module, the AUC of the model determination part is further improved to 0.957, and the model ACC is increased from 0.892 to 0.910. The specificity is improved (0.933 to 0.941) while the sensitivity is greatly improved (0.749 to 0.814). BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flowchart of a multimodal adnexal mass malignancy risk assessment method in one embodiment of the present application.

[0058] Figure 2 4 is a schematic diagram of a multimodal adnexal mass malignancy risk assessment device in one embodiment of the present application.

[0059] Figure 3 It is a detailed flowchart of a multimodal adnexal mass malignancy risk assessment method in one embodiment of the present application. DETAILED DESCRIPTION

[0060] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Glossary:

[0062] AUC: Area under the ROC curve (or area under the curve, in English, area under the curve).

[0063] ACC: Accuracy.

[0064] Figure 1 It is a flowchart of a multimodal adnexal mass malignancy risk assessment method in one embodiment of the present application.

[0065] like Figure 1 The multimodal approach to assessing the risk of malignancy for adnexal masses presented here includes:

[0066] Step 1: Get the image to be recognized;

[0067] Step 2: Obtain basic information of the person to be evaluated;

[0068] Step 3: Extracting image features of the image to be identified;

[0069] Step 4: Generate basic information structured features based on the basic information of the person to be evaluated;

[0070] Step 5: splicing the basic information structured features and the image features to obtain features to be used;

[0071] Step 6: Get multiple trained xgboost models;

[0072] Step 7: Obtain multiple initial prediction results based on the features to be used and each trained xgboost model;

[0073] Step 8: Obtain the drift diffusion model;

[0074] Step 9: Obtain the final prediction result based on each initial prediction result and the drift diffusion model.

[0075] This application uses the xgboost model in conjunction with the drift diffusion model to enable the image model to achieve an AUC of 0.911. The xgboost multiple group results are directly averaged to achieve an AUC of 0.946. In addition, through the screening of the brain-like accumulation module, the AUC of the model determination part is further improved to 0.957, and the model ACC is increased from 0.892 to 0.910. The specificity is improved (0.933 to 0.941) while the sensitivity is greatly improved (0.749 to 0.814).

[0076] In this embodiment, the basic information of the person to be evaluated includes:

[0077] Family history information, medical history information, hormone treatment history information, the presence or absence of blood flow signals, whether menopausal, basic tumor marker information, and mass size information.

[0078] In this embodiment, family history (assessing genetic risk) refers to whether close relatives have (or have had) related ovarian diseases, including ovarian tumors, polycystic ovary syndrome, and neoplastic cysts.

[0079] Medical history (to assess the risk of recurrence and metastasis) refers to whether the individual has (or has had) related ovarian disease or other cancers.

[0080] The basic information structured features generated according to the basic information of the person to be evaluated include:

[0081] Obtain screening conditions for benign and malignant correlation;

[0082] Screening the basic information of the person to be evaluated according to the benign and malignant correlation screening condition, thereby obtaining the screened basic information of the person to be evaluated;

[0083] Generate basic information structured features based on the screened basic information of the person to be evaluated.

[0084] Specifically, the basic information includes a discrete part and a continuous part. The discrete part (family history, medical history, history of hormone treatment, presence or absence of blood flow, whether menopause) is represented by 0 or 1 respectively (no / no) or (yes / yes), and the continuous part is represented by a number greater than 0. Because xgboost supports null values, all unobtained information is represented by the null value "nan", thus forming the structured features of the basic information.

[0085] In this embodiment, before extracting the image features of the image to be identified, the multimodal adnexal mass malignancy risk assessment method further includes:

[0086] Preprocessing the image to be identified, thereby obtaining preprocessed image information;

[0087] The extracting the image features of the image to be identified comprises:

[0088] The image features are extracted according to the preprocessed image information.

[0089] In this embodiment, extracting the image features according to the preprocessed image information includes:

[0090] Get the trained image feature extractor;

[0091] The preprocessed image information is input into the image feature extractor to obtain image features.

[0092] In this embodiment, the image feature extractor includes:

[0093] A spatially local convolution;

[0094] A spatial long-range convolution;

[0095] One channel convolution.

[0096] It is understandable that the VAN model mentioned here is only one of the implementation processes. This application is not limited to a specific deep neural network model, and algorithm models such as CNN and Transformers are all acceptable.

[0097] The loss function uses the following formula:

[0098] FL(p t )=-α t (1-p t ) γ log(p t );in,

[0099] P t is the probability that the model predicts that the sample belongs to category t; the weight factor α∈[0,1], when it is a positive sample, the weight factor is α, when it is a negative sample, the weight factor is 1-α; the modulation factor (1-p t ) γ , γ ranges from [0,5].

[0100] In this embodiment, the step of combining the basic information structured features and the image features to obtain features to be used includes:

[0101] The image features are concatenated with the structural features and processed using the Lasso method to obtain a data feature matrix.

[0102] In this embodiment, the data feature matrix is ​​input into each xgboost model respectively, so as to obtain the initial prediction results output by each xgboost model respectively.

[0103] In this embodiment, obtaining the final prediction result according to each initial prediction result and the drift diffusion model includes:

[0104] Randomly sort the initial prediction results, and obtain the top N initial prediction results as the initial prediction results to be used;

[0105] Each initial prediction result to be used is input into the drift diffusion model to obtain the final prediction result and response time.

[0106] In this embodiment, when inputting each initial prediction result to be used into the drift diffusion model to obtain the final prediction result and response time, for each initial prediction result to be used, an array with a length equal to the randomly selected number and uniform distribution is first initialized, in which each element represents the posterior probability, and each time step is looped through, and the probability output by the neural network is multiplied by the posterior of the previous step at each step, and a normalization operation is performed to obtain the posterior probability of each category.

[0107] The present application is further described in detail below by way of examples. It should be understood that the examples do not constitute any limitation to the present application.

[0108] In this embodiment, the xgboost model of the present application needs to be trained to obtain the trained xgboost model.

[0109] In this embodiment, the training process of the feature extraction network is as follows:

[0110] Collect the images taken by ultrasound doctors to obtain training image formats including jpg, png, bmp and dcm, and convert the images into png format for use.

[0111] In the training phase, the dataset is first balanced, the input image is downsampled according to the number of malignant samples in the minority class, the coordinates are generated according to the doctor's box selection and cropped, and finally the cropped image is normalized. The final size of all images is unified to 352*352.

[0112] Four different feature extraction networks are trained (for example, two van_base and van_large networks of VAN and two resnet56 networks), where the VAN network uses a large kernel attention mechanism that divides the large convolution kernel into three parts:

[0113] 1. A spatial local convolution (depth-wise convolution)

[0114] 2. A spatial long-range convolution (depth-wise dilated convolution)

[0115] 3. One channel convolution (1×1 convolution)

[0116] The overall attention process is as follows:

[0117] Attention=Conv 1*1 (DW-D-Conv(DW-Conv(F)))

[0118]

[0119] where F∈R C×H×W ,Attention∈R C×H×W Based on this mechanism, VAN configures four networks of different sizes (Tiny, Small, Base, Large). The present invention mainly uses the van_base and van_large networks for feature extraction, and uses focal loss as the loss function. The formula is as follows:

[0120] FL(p t )=-α t (1-p t ) γ log(p t );in,

[0121] P t is the probability that the model predicts that the sample belongs to category t, the weight factor α∈[0,1], when it is a positive sample, the weight factor is α, when it is a negative sample, the weight factor is 1-α; the modulation factor (1-p t ) γ , γ ranges from [0,5].

[0122] This loss function can change the loss weight of the side with more or fewer samples by adjusting two hyperparameters, thus better solving the problem of data imbalance.

[0123] All lesion images were preprocessed without data balancing, and the four trained image feature extractors (such as visual attention network VAN and resnet56, etc.) were used to extract features from the preprocessed data. By turning on dropout during prediction, a total of 16 groups of 512 image features were extracted (each feature extraction network extracted four groups of features, and dropout was set to 0, 0.1, 0.2, and 0.2 respectively).

[0124] The basic information of the person to be evaluated (family history information, medical history information, hormone treatment history information, the presence or absence of blood flow signals, whether menopausal or not, basic information on tumor markers, and mass size information) is initially screened based on the correlation with benign or malignant tumors, and converted into discrete or continuous structured features based on the data distribution.

[0125] In this embodiment, the step of combining the basic information structured features and the image features to obtain features to be used includes:

[0126] The four sets of image features obtained from four different networks with dropout of 0 were concatenated with the preprocessed structured features, and the Lasso method was used to delete the items that were not helpful for judgment. Finally, the four related tumor marks and the maximum size of the mass were retained, including whether menopause, medical history / family history, blood flow, hormone treatment history. The 521 features were screened using the lasso method, and the feature items with non-zero results were retained, so as to simplify the features and obtain the data feature matrix.

[0127] In this embodiment, the xgboost model is trained using a data feature matrix. Specifically, for each set of data feature matrices, two different data balance ratios are used to downsample the training set data, and for all spliced ​​features, the missing values ​​are retained without additional filling, and then all data columns are subjected to random fluctuations within the range of (95% upper bound -5% lower bound) * 0.03, and finally trained using the xgboost model. Finally, 8 trained xgboost models were obtained (4 sets of different features were extracted from the training set images using the 4 feature extraction networks mentioned above, and each set of features was spliced ​​with its corresponding structured features to form 4 sets of data feature matrices; 2 different data balance ratios were used for downsampling to obtain 8 sets of training data; each set of training data was used to train an xgboost model, thereby obtaining 8 trained xgboost models). In this way, the output difference will be relatively large, and the effect of the drift diffusion model will be better. That is to say, in this step, for each preprocessed image, we get 4*4*2 (4 dropout settings, 4 feature extraction networks, 2 data balances) a total of 32 prediction values ​​(initial prediction results).

[0128] In this embodiment, obtaining the final prediction result according to each initial prediction result and the drift diffusion model includes:

[0129] Multiple groups of predicted values ​​are randomly sorted, and the first 15 items are intercepted and sequentially input into the drift diffusion model of the present application for cumulative calculation.

[0130] Different from the way of independent accumulation of each element in the traditional DDM (each input of the drift-diffusion model, that is, each predicted value), we use the model probability as the posterior for the next step, and through continuous multiplication and normalization operations, the samples with more consistent model conclusions quickly approach the target threshold. Specifically, for each sample, first initialize an array with a length of the randomly drawn number and a uniform distribution, where each element represents the posterior probability. By looping through each time step

[0131] (0 < t < 16), at each step, multiply the probability output by the neural network with the posterior of the previous step and perform a normalization operation to obtain the posterior probability of each category. The formula is as follows:

[0132] P t (x i ) = Norm(pred i_t * P t-1 (x i ))

[0133] Where pred i_t represents the probability of the model prediction value read at the t-th time step under the given label i, and P t (x i ) represents the joint posterior probability under the condition of the given label i in the previous t time steps of observation, and Norm represents the normalization operation. Then, by comparing a set of obtained posterior probabilities with the selected threshold (0.9999), the direction (prediction result) and speed (response time) of accumulation to the threshold are obtained. If the threshold is still not reached finally, the prediction result is recorded as the label with a larger probability in the last round, and the response time is recorded as 16. Finally, the uncertainty of the model result is jointly evaluated according to the response time and the previous image confidence score, and the predictions with higher comprehensive uncertainty are marked as model uncertain and handed over to the ultrasound doctor for secondary reading.

[0134] For example, first initialize a uniform distribution (set an array with the same length as each group of predicted values according to the number of categories n, and each value in the array is initially 1 / n), then multiply it with the first group of predictions and normalize it, and then multiply it with each subsequent group of predictions and normalize it in turn until a certain category reaches the threshold.

[0135] In this embodiment, random permutation means that the purpose of intercepting and orderly input is random non-repetitive sampling, and the result input to the drift-diffusion model is composed of any prediction sequence. Mainly through this randomness, the diversity of combinations is simulated.

[0136] The application of the present invention in the specific diagnosis and treatment process is as follows:

[0137] 1. The ultrasound doctor will take a screenshot of the suspicious area scanned and mark it, and input it into the system together with the basic information of the patient obtained through questioning and the tumor markers obtained from the patient's laboratory test.

[0138] 2. The system uses algorithms to preprocess data, extract and concatenate features, and input them into the xgboost model to generate multiple sets of prediction results.

[0139] 3. The prediction results and response time are obtained through the brain-like accumulation module, and the model uncertainty is evaluated based on the response time to decide whether it is necessary to hand over the film to an experienced doctor for a second review.

[0140] 4. When the model uncertainty is low, the doctor can choose whether to directly let the model make the decision and use the model evaluation as the final conclusion; when the model uncertainty is high, the doctor is prompted to read the film again. At this time, the doctor will combine his own evaluation with the model prediction to make risk adjustments and give the final result.

[0141] The present invention provides a multi-modal adnexal mass malignant risk assessment system based on uncertainty. The system includes a data preprocessing module for data processing; an image feature extraction module for extracting feature information of data to form structured image features; a Lasso model simplification module for feature simplification to obtain a data feature matrix; a malignant risk prediction module to obtain a malignant risk prediction value of an adnexal mass; and a brain-like accumulation module to integrate multiple groups of outputs, evaluate model uncertainty and give a final conclusion.

[0142] The present invention can be used to conduct risk assessment on ultrasound images of outpatients to avoid missed diagnosis of malignant patients. At the same time, AI can assist doctors in making judgments, improve efficiency and avoid the risk of over-medicalization to a certain extent; and give more accurate prompts when information such as tumor marks is added. The image model alone achieved an AUC of 0.911, and the xgboost multi-group results directly took the mean to calculate an AUC of 0.946. In addition, through the screening of the brain-like accumulation module, the model-determined part AUC was further improved to 0.957, and the model ACC was increased from 0.892 to 0.910. The specificity was slightly improved (0.933 to 0.941) while the sensitivity was greatly improved (0.749 to 0.814).

[0143] The present application also provides a multimodal attachment area mass image recognition module training device, which includes an image acquisition module to be identified, a basic information acquisition module for a person to be evaluated, an image feature acquisition module, a structured feature generation module, a feature splicing module to be used, an xgboost model acquisition module, an initial prediction result acquisition module, a drift diffusion model acquisition module and a prediction module, wherein:

[0144] The to-be-recognized image acquisition module is used to acquire the to-be-recognized image;

[0145] The basic information acquisition module of the person to be evaluated is used to obtain the basic information of the person to be evaluated;

[0146] The image feature acquisition module is used to extract the image features of the image to be identified;

[0147] The structured feature generation module is used to generate basic information structured features according to the basic information of the person to be evaluated;

[0148] The feature splicing module to be used is used to splice the basic information structured features and the image features to obtain the features to be used;

[0149] The xgboost model acquisition module is used to obtain multiple trained xgboost models;

[0150] The initial prediction result acquisition module is used to obtain multiple initial prediction results based on the data feature matrix and each trained xgboost model;

[0151] The drift diffusion model acquisition module is used to acquire the drift diffusion model;

[0152] The prediction module is used to obtain the final prediction result according to each initial prediction result and the drift diffusion model.

[0153] The above explanation of the method also applies to the explanation of the device.

[0154] Figure 2 It is a structural block diagram of an electronic device provided by one or more embodiments of the present invention.

[0155] like Figure 2 As shown, the present application also discloses an electronic device (i.e., the master controller in the present application), comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the multimodal accessory area mass malignancy risk assessment method.

[0156] The present application also provides a computer-readable storage medium, which stores a computer program executable by an electronic device. When the computer program runs on the electronic device, the steps of the multimodal accessory area mass malignancy risk assessment method can be implemented.

[0157] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0158] The electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory. The operating system can be any one or more computer operating systems that implement electronic device control through a process, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. In addition, in an embodiment of the present invention, the electronic device can be a handheld device such as a smart phone or a tablet computer, or can be an electronic device such as a desktop computer or a portable computer, which is not particularly limited in the embodiment of the present invention.

[0159] The execution subject of the electronic device control in the embodiment of the present invention may be an electronic device, or a functional module in the electronic device that can call and execute a program. The electronic device may obtain the firmware corresponding to the storage medium. The firmware corresponding to the storage medium is provided by the supplier. The firmware corresponding to different storage media may be the same or different, which is not limited here. After the electronic device obtains the firmware corresponding to the storage medium, the firmware corresponding to the storage medium may be written into the storage medium, specifically, the firmware corresponding to the storage medium may be burned into the storage medium. The process of burning the firmware into the storage medium may be implemented using existing technology, which will not be described in detail in the embodiment of the present invention.

[0160] The electronic device may also obtain a reset command corresponding to the storage medium. The reset command corresponding to the storage medium is provided by the supplier. The reset commands corresponding to different storage media may be the same or different, and are not limited here.

[0161] At this time, the storage medium of the electronic device is a storage medium in which the corresponding firmware is written, and the electronic device can respond to the reset command corresponding to the storage medium in the storage medium in which the corresponding firmware is written, so that the electronic device resets the storage medium in which the corresponding firmware is written according to the reset command corresponding to the storage medium. The process of resetting the storage medium according to the reset command can be implemented by the existing technology and will not be described in detail in the embodiments of the present invention.

[0162] For the convenience of description, the above devices are described in terms of functions and are divided into various units and modules. Of course, when implementing the present application, the functions of each unit and module can be implemented in the same or multiple software and / or hardware.

[0163] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined.

[0164] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0165] It can be known from the description of the above implementation modes that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute the methods described in the various implementation modes of the present application or certain parts of the implementation modes.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal method for assessing the risk of malignancy of adnexal masses, characterized in that: The multimodal adnexal mass malignancy risk assessment method comprises: Obtain an image to be recognized; Obtain basic information of the person to be evaluated; Extracting image features of the image to be identified; Generate basic information structured features according to the basic information of the person to be evaluated; splicing the basic information structured features and the image features to obtain features to be used; Get multiple trained xgboost models; Get multiple initial prediction results based on the features to be used and each trained xgboost model; Get the drift-diffusion model; Obtaining a final prediction result based on each initial prediction result and the drift diffusion model; The obtaining of the final prediction result according to each initial prediction result and the drift diffusion model comprises: Randomly sort the initial prediction results, and obtain the top N initial prediction results as the initial prediction results to be used; Input each initial prediction result to be used into the drift diffusion model to obtain the final prediction result and response time; In the process of inputting each initial prediction result to be used into the drift diffusion model to obtain the final prediction result and response time, for each initial prediction result to be used, first initialize an array with a length of the randomly selected number and uniform distribution, in which each element represents the posterior probability, and loop through each time step. At each step, the probability output by the neural network is multiplied by the posterior of the previous step, and normalized, to obtain the posterior probability of each category.

2. The multimodal adnexal mass malignancy risk assessment method according to claim 1, characterized in that: The basic information of the person to be evaluated includes: Family history information, medical history information, hormone treatment history information, the presence or absence of blood flow signals, whether menopausal, basic information on tumor markers, and mass size information; The generating of basic information structured features according to the basic information of the person to be evaluated includes: Obtain screening conditions for benign and malignant correlation; Screening the basic information of the person to be evaluated according to the benign and malignant correlation screening condition, thereby obtaining the screened basic information of the person to be evaluated; Generate basic information structured features based on the screened basic information of the person to be evaluated.

3. The multimodal adnexal mass malignancy risk assessment method according to claim 1, characterized in that: Before extracting the image features of the image to be identified, the multimodal adnexal mass malignancy risk assessment method further comprises: Preprocessing the image to be identified, thereby obtaining preprocessed image information; The extracting the image features of the image to be identified comprises: The image features are extracted according to the preprocessed image information.

4. The multimodal adnexal mass malignancy risk assessment method according to claim 3, characterized in that: The extracting the image features according to the preprocessed image information comprises: Get the trained image feature extractor; The preprocessed image information is input into the image feature extractor to obtain image features.

5. The multimodal adnexal mass malignancy risk assessment method according to claim 3, characterized in that: The step of combining the basic information structured features and the image features to obtain features to be used includes: The image features are concatenated with the structural features and processed using the Lasso method to obtain a data feature matrix.

6. A multi-modal adnexal mass malignancy risk assessment device, characterized in that: The multimodal adnexal mass malignancy risk assessment device comprises: An image acquisition module to be identified, wherein the image acquisition module to be identified is used to acquire the image to be identified; A basic information acquisition module for the person to be evaluated, wherein the basic information acquisition module for the person to be evaluated is used to acquire basic information of the person to be evaluated; An image feature acquisition module, the image feature acquisition module is used to extract image features of the image to be identified; A structured feature generation module, the structured feature generation module is used to generate basic information structured features according to the basic information of the person to be evaluated; A feature splicing module to be used, the feature splicing module to be used is used to splice the basic information structured features and the image features to obtain the features to be used; An xgboost model acquisition module, wherein the xgboost model acquisition module is used to acquire multiple trained xgboost models; An initial prediction result acquisition module, wherein the initial prediction result acquisition module is used to obtain multiple initial prediction results according to the data feature matrix and each trained xgboost model; A drift diffusion model acquisition module, wherein the drift diffusion model acquisition module is used to acquire a drift diffusion model; A prediction module, the prediction module is used to obtain a final prediction result based on each initial prediction result and the drift diffusion model; The obtaining of the final prediction result according to each initial prediction result and the drift diffusion model comprises: Randomly sort the initial prediction results, and obtain the top N initial prediction results as the initial prediction results to be used; Input each initial prediction result to be used into the drift diffusion model to obtain the final prediction result and response time; In the process of inputting each initial prediction result to be used into the drift diffusion model to obtain the final prediction result and response time, for each initial prediction result to be used, first initialize an array with a length of the randomly selected number and uniform distribution, in which each element represents the posterior probability, and loop through each time step. At each step, the probability output by the neural network is multiplied by the posterior of the previous step, and normalized, to obtain the posterior probability of each category.

Images