Image detection method, target prediction model training method, device and equipment

Abstract

This application relates to an image detection method, a training method for a target prediction model, an apparatus, and a device. The method includes: acquiring a retinal image, wherein the retinal image is obtained by image acquisition of the eye region of a target object; inputting the retinal image into a target prediction model to obtain a prediction result output by the target prediction model; wherein the prediction result is used to characterize the presence of a lesion region in the retinal image and the predicted probability of the lesion category corresponding to the lesion region; the target prediction model is obtained by multi-stage training of an initial prediction model, wherein the sample weights of each sample retinal image in each training stage are at least partially different, and the sample weights are determined based on the lesion category corresponding to the sample retinal image, the prediction accuracy corresponding to the sample retinal image, or a default sample weight. This method can simplify the diagnostic process and improve diagnostic efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image detection method, a training method for a target prediction model, an apparatus, and a device. Background Technology

[0002] Systemic lupus erythematosus (SLE) is a systemic autoimmune disease that can affect multiple organ systems, including the skin, joints, central nervous system, eyes, and kidneys. For SLE patients, screening for SLE and its complications is equally important. Early detection and treatment of SLE and its complications can enhance personalized care and improve the long-term prognosis of SLE patients.

[0003] Currently, the clinical diagnosis of SLE still mainly relies on methods such as molecular marker detection, immunological examination, and comprehensive assessment of clinical symptoms. However, in large-scale SLE screening scenarios, these diagnostic methods suffer from problems such as complex diagnostic processes and low diagnostic efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide an image detection method, target prediction model training method, apparatus, and equipment that can simplify the diagnostic process and improve diagnostic efficiency in response to the above-mentioned technical problems.

[0005] Firstly, this application provides an image detection method. The method includes:

[0006] Acquire a retinal image, wherein the retinal image is obtained by acquiring an image of the eye region of the target object;

[0007] The retinal image is input into the target prediction model to obtain the prediction result output by the target prediction model;

[0008] The prediction result is used to characterize whether there is a lesion area in the retinal image and the prediction probability of the lesion category corresponding to the lesion area; the target prediction model is obtained by multi-stage training of the initial prediction model, and the sample weights of each sample retinal image in each training stage of the multi-stage training are at least partially different, and the sample weights are determined based on the lesion category corresponding to the sample retinal image, the prediction accuracy corresponding to the sample retinal image, or the default sample weights.

[0009] In one embodiment, the training process of the target prediction model includes:

[0010] Obtain a training sample set, which includes multiple training samples, and each training sample includes a sample retinal image and a lesion category label corresponding to the sample retinal image;

[0011] Based on the training sample set and the default sample weights of each sample retinal image, the initial prediction model is trained in the first stage to obtain the first stage prediction model.

[0012] The weights of each default sample are adjusted according to the lesion category corresponding to each sample retinal image and / or the prediction accuracy corresponding to each sample retinal image to obtain the adjusted sample weights of each sample retinal image. The first-stage prediction model is then trained according to the adjusted sample weights and the training sample set to obtain the target prediction model.

[0013] In one embodiment, adjusting the default sample weights based on the lesion category corresponding to each sample retinal image and / or the prediction accuracy corresponding to each sample retinal image to obtain the adjusted sample weights for each sample retinal image includes:

[0014] Determine the lesion category corresponding to the retinal images of each sample;

[0015] The default sample weights are adjusted according to each of the lesion categories to obtain the first adjusted sample weights for each of the sample retinal images;

[0016] The step of training the first-stage prediction model based on the adjusted sample weights and the training sample set to obtain the target prediction model includes:

[0017] Based on the training sample set and the weights of each of the first adjusted samples, the first-stage prediction model is trained in the second stage to obtain the second-stage prediction model, and the target prediction model is obtained based on the second-stage prediction model.

[0018] In one embodiment, adjusting the default sample weights according to each of the lesion categories to obtain the first adjusted sample weights for each of the sample retinal images includes:

[0019] The number of lesion categories corresponding to each sample retinal image, the number of first sample retinal images containing sample lesion regions in each sample retinal image, the first training round corresponding to the first stage of training, and the second training round corresponding to the second stage of training are obtained.

[0020] Based on the number of each category, the number of the first sample retinal images, the first training round, and the second training round, the sample weight of each first sample retinal image is obtained, and the default sample weight is determined as the sample weight of the second sample retinal image in which the lesion region of the sample does not exist.

[0021] In one embodiment, the method further includes:

[0022] Based on the second-stage prediction model, the prediction accuracy corresponding to each sample retinal image is obtained;

[0023] The default sample weights are adjusted according to the prediction accuracy of each sample to obtain the second adjusted sample weights for each sample retinal image;

[0024] Based on the training sample set and the weights of each of the second adjusted samples, the second-stage prediction model is trained in the third stage to obtain the target prediction model.

[0025] In one embodiment, adjusting the default sample weights based on the prediction accuracy to obtain the second adjusted sample weights for each sample retinal image includes:

[0026] Obtain the third training round corresponding to the third stage of training;

[0027] Based on the number of each category, the prediction accuracy of each category, and the third training round, the sample weights of the third sample retinal images in each sample retinal image with a prediction accuracy less than a preset accuracy threshold are obtained, and the default sample weights are determined as the sample weights of the fourth sample retinal images in each sample retinal image with a prediction accuracy greater than or equal to the preset accuracy threshold.

[0028] In one embodiment, the target prediction model includes an encoder and a classifier, and the step of inputting the retinal image into the target prediction model to obtain the prediction result output by the target prediction model includes:

[0029] The retinal image is input into the encoder to obtain the semantic features of the retinal image;

[0030] The semantic features are input into the classifier to obtain the prediction result.

[0031] Secondly, this application provides a method for training a target prediction model. The method includes:

[0032] Obtain a training sample set, which includes multiple training samples, and each training sample includes a sample retinal image and a lesion category label corresponding to the sample retinal image;

[0033] Based on the training sample set and the default sample weights of each sample retinal image, the initial prediction model is trained in the first stage to obtain the first stage prediction model.

[0034] The weights of each default sample are adjusted according to the lesion category and / or prediction accuracy corresponding to each sample retinal image to obtain the adjusted sample weights of each sample retinal image. The first-stage prediction model is then trained based on the adjusted sample weights and the training sample set to obtain the target prediction model.

[0035] In one embodiment, the initial prediction model includes an initial encoder, and the method further includes:

[0036] The retinal images of each sample are masked to obtain the mask images corresponding to the retinal images of each sample.

[0037] Each of the mask images is converted into a feature vector, and each of the feature vectors is input into the encoder included in the image reconstruction model to obtain the sample semantic feature matrix;

[0038] A virtual mask marker is added to the sample semantic feature matrix to obtain a new sample semantic feature matrix;

[0039] The new sample semantic feature matrix is ​​input into the decoder included in the image reconstruction model to obtain the sample reconstruction image corresponding to each sample retinal image;

[0040] The image reconstruction model is optimized based on the sample reconstructed image to obtain an optimized image reconstruction model, and the encoder parameters in the optimized image reconstruction model are used as the initial parameters of the initial encoder.

[0041] Thirdly, this application also provides an image detection apparatus. The apparatus includes:

[0042] The first acquisition module is used to acquire a retinal image, which is obtained by image acquisition of the eye region of the target object;

[0043] The prediction module is used to input the retinal image into the target prediction model and obtain the prediction result output by the target prediction model;

[0044] The prediction result is used to characterize whether there is a lesion area in the retinal image and the prediction probability of the lesion category corresponding to the lesion area; the target prediction model is obtained by multi-stage training of the initial prediction model, and the sample weights of each sample retinal image in each training stage of the multi-stage training are at least partially different, and the sample weights are determined based on the lesion category corresponding to the sample retinal image, the prediction accuracy corresponding to the sample retinal image, or the default sample weights.

[0045] Fourthly, this application also provides a training apparatus for a target prediction model. The apparatus includes:

[0046] The fifth acquisition module is used to acquire a training sample set, which includes multiple training samples, and the training samples include sample retinal images and lesion category labels corresponding to the sample retinal images;

[0047] The sixth acquisition module is used to perform the first stage training of the initial prediction model based on the training sample set and the default sample weights of each sample retinal image to obtain the first stage prediction model.

[0048] The seventh acquisition module is used to adjust the weights of each default sample according to the lesion category and / or prediction accuracy corresponding to each sample retinal image, to obtain the adjusted sample weights of each sample retinal image, and to train the first-stage prediction model according to the adjusted sample weights and the training sample set to obtain the target prediction model.

[0049] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the methods described in the first and second aspects above.

[0050] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the methods described in the first and second aspects above.

[0051] In a seventh aspect, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the methods described in the first and second aspects above.

[0052] The aforementioned image detection method, target prediction model training method, apparatus, and equipment involve the server first acquiring a retinal image, which is obtained by capturing images of the eye region of the target object. The retinal image is then input into the target prediction model to obtain the prediction result output by the model. Since the prediction result characterizes the presence of lesion areas in the retinal image and the predicted probability of the lesion category corresponding to the lesion area, inputting the retinal image into the target prediction model allows for rapid prediction of the presence of lesion areas in the retinal image, improving the prediction efficiency of the retinal image. This enables rapid diagnosis of lesions based on the prediction results, simplifying the diagnostic process and improving diagnostic efficiency. Furthermore, since the target prediction model is obtained through multi-stage training of the initial prediction model, the sample weights of each sample retinal image in each training stage are at least partially different. These sample weights are determined based on the lesion category corresponding to the sample retinal image, the prediction accuracy corresponding to the sample retinal image, or the default sample weights. During the training process, the imbalance of lesion categories corresponding to the sample retinal images and the differences in prediction results can be comprehensively considered, thereby improving the model accuracy of the trained target prediction model and ultimately improving the accuracy of the prediction results output by the target prediction model. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is an application environment diagram of the image detection method in one embodiment;

[0055] Figure 2 This is a flowchart illustrating an image detection method in one embodiment;

[0056] Figure 3 This is a flowchart illustrating the training process of a target prediction model in one embodiment;

[0057] Figure 4 This is a flowchart illustrating step 303 in one embodiment;

[0058] Figure 5 This is a flowchart illustrating step 202 in one embodiment;

[0059] Figure 6 This is a flowchart illustrating the image detection method in another embodiment;

[0060] Figure 7 This is a flowchart illustrating step 602 in one embodiment;

[0061] Figure 8 This is a flowchart illustrating step 202 in another embodiment;

[0062] Figure 9 This is a flowchart illustrating the training method of a target prediction model in one embodiment;

[0063] Figure 10 This is a flowchart illustrating the training method of the target prediction model in one embodiment;

[0064] Figure 11 This is a structural block diagram of an image detection device in one embodiment;

[0065] Figure 12 This is a structural block diagram of a training device for a target prediction model in one embodiment;

[0066] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0068] In the field of systemic lupus erythematosus (SLE) detection, current clinical diagnosis still relies primarily on complex molecular marker testing, immunological examination, and a comprehensive assessment of clinical symptoms. However, public awareness of this method is low, often leading to diagnostic delays for patients. This is especially true in the asymptomatic or mildly symptomatic stages, where a widely applicable, standardized, non-invasive, and cost-effective screening method is lacking.

[0069] For complications of SLE, such as lupus retinopathy (LR) and lupus nephritis (LN), early detection and treatment can enhance personalized care and improve the long-term prognosis of SLE patients. For LR, because retinal changes associated with LR can be subtle, advanced imaging techniques are needed for screening, such as optical coherence tomography (OCT) or fundus fluorescein angiography (FFA). However, these imaging methods are costly and their application is limited, making them difficult to promote in primary healthcare institutions or community screening settings. For LN, current screening methods mainly rely on collecting urine and blood samples, as well as further urine analysis and serum creatinine measurement. However, in primary healthcare and resource-scarce areas, the accessibility of conducting and analyzing these tests is limited, restricting their application in a broad population. In addition, kidney biopsy is the gold standard for diagnosing lupus nephritis, but it is an invasive procedure and is not routinely performed unless there is a strong suspicion of serious kidney damage.

[0070] Because retinal changes are common in patients with systemic lupus erythematosus (SLE), including cotton wool spots, retinal hemorrhage, vascular tortuosity, hard exudates, small artery stenosis, and arteriovenous crossings, and because these changes often occur concurrently with active SLE episodes, they can provide valuable information about disease activity and severity. Therefore, the retina holds promise as a non-invasive, point-of-care diagnostic, and cost-effective biomarker for detecting SLE and its related complications.

[0071] In view of this, this application proposes an image detection method, which acquires retinal images by capturing images of the eye region of the target object, and then inputs the retinal images into a target prediction model to obtain the prediction results output by the target prediction model. The prediction results can be used to characterize the presence of lesion areas in the retinal images and the prediction probability of the lesion category corresponding to the lesion areas, thereby improving the efficiency of image detection of retinal images, and thus improving the screening efficiency of SLE and its complications, especially in large-scale population screening, improving the early identification rate and hierarchical management level of SLE and its complications.

[0072] The image detection method provided in this application embodiment can be applied to, for example... Figure 1 The implementation environment shown includes servers, which can be implemented using standalone servers or server clusters composed of multiple servers. A data storage system stores the data that the servers need to process. The data storage system can be integrated onto the server or hosted in the cloud or on other network servers.

[0073] In other possible implementations, the image detection method provided in this application embodiment can also be applied to a terminal. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc.

[0074] In one exemplary embodiment, such as Figure 2 As shown, an image detection method is provided, which is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0075] Step 201: Acquire a retinal image. The retinal image is obtained by acquiring an image of the eye region of the target object.

[0076] The retinal image is visual information obtained by imaging the retina of the eyeball using specialized equipment. It can clearly show the anatomical structure and pathological changes of the retina. For example, retinal images can be obtained by taking pictures of the eye area of ​​the target subject using image acquisition equipment such as a fundus camera. It is understood that the retinal images obtained in this embodiment can be acquired in real time by image acquisition equipment of the eye area of ​​the target subject, or they can be acquired from images of the eye area of ​​the target subject over a historical period. The target subject can be a potential or confirmed case of SLE and its complications.

[0077] In this embodiment, the server can obtain a real-time captured retinal image from the image acquisition device, or the server can obtain a retinal image from an image database.

[0078] Step 202: Input the retinal image into the target prediction model to obtain the prediction result output by the target prediction model.

[0079] The prediction results are used to characterize the presence of lesion areas in retinal images and the predicted probability of the lesion category corresponding to the lesion area. The target prediction model is obtained by multi-stage training of the initial prediction model. In the multi-stage training, the sample weights of each sample retinal image in each training stage are at least partially different. The sample weights are determined based on the lesion category corresponding to the sample retinal image, the prediction accuracy corresponding to the sample retinal image, or the default sample weights.

[0080] The target prediction model is a neural network model used to identify and classify input retinal images to determine whether lesion regions exist in the retinal images. In this embodiment, taking SLE as an example, it determines whether lesion regions of SLE and its complications, such as LR and LN, exist in the retinal images, and predicts the probability of the lesion category corresponding to the lesion region when such lesion regions are present. A lesion region refers to a specific area in a body tissue or organ where the structure and function have changed due to disease, injury, or abnormal physiological processes. In this embodiment, the lesion region can be a lesion region related to SLE and its complications. Optionally, if the target object has SLE and its complications, a lesion region will exist in the retinal image; if the target object does not have SLE and its complications, no lesion region will exist in the retinal image. When a lesion region exists in the retinal image, the probability of the lesion category corresponding to the lesion region may be relatively high or relatively low. Therefore, the predicted probability in the prediction result can be used to characterize the likelihood of the target object having a certain lesion category.

[0081] The retinal image may or may not include lesion areas. Therefore, the prediction result can be either no lesion area or a lesion area present. Optionally, if no lesion area is present, the prediction probability is 0; if a lesion area is present, the prediction probability is related to the lesion category corresponding to the lesion area. Optionally, the lesion category corresponding to the lesion area can include multiple categories, for example, the lesion category can be SLE, LR, or LN.

[0082] The initial prediction model is a neural network model built based on initialization parameters. It can be iteratively trained using sample retinal images to obtain the target prediction model. The lesion categories corresponding to the sample retinal images can include multiple lesion categories, such as SLE, LR, and LN. The prediction accuracy corresponding to the sample retinal images refers to the accuracy of the prediction results in each optimization process during multi-stage training. The default sample weights refer to the initial weights automatically assigned to the sample retinal images by the initial prediction model, i.e., the unadjusted weights. For example, the default sample weights can be set to 1.

[0083] Understandably, due to the presence of lesion areas in the sample retinal images and the uneven distribution of lesion categories corresponding to these lesions, a multi-stage training approach can be used to train the initial prediction model to improve its accuracy. Furthermore, the sample weights for each retinal image differ in each training stage. For example, taking a multi-stage training approach with three training stages: in the first stage, all sample retinal images are trained with default sample weights; in the second stage, a larger sample weight is assigned to minority class retinal images, while the default sample weights are assigned to other sample retinal images; in the third stage, the weights of each lesion category are dynamically adjusted based on the prediction accuracy of each training result. For example, the sample weights of retinal images with higher prediction difficulty are increased, while the sample weights of retinal images with lower prediction difficulty are decreased.

[0084] In this embodiment, the server can input a retinal image into the target prediction model, and the target prediction model can identify and classify the retinal image to obtain the prediction result output by the target prediction model.

[0085] As one possible implementation, after obtaining the prediction results output by the target prediction model, in order to improve the interpretability of the prediction results, interpretability analysis can also be performed by generating attention heatmaps, quantitative analysis of retinal vascular morphology features, and comparison of vascular parameter differences between SLE and non-SLE groups.

[0086] For example, the RELPROP (Relevance Propagation) algorithm is introduced to interpret and analyze the Transformer architecture in the target prediction model. This algorithm uses a layer-wise Relevance Propagation (LRP) mechanism to score the relevance of the model's attention heads, and combines gradient information and the operations between the attention maps of each processing layer to generate heatmaps of regions in the input retinal image that are strongly correlated with the classification task. This clarifies the feature regions that the target prediction model focuses on when predicting systemic lupus erythematosus (SLE), lupus nephritis (LN), and lupus retinopathy (LR).

[0087] Furthermore, the results of salient region detection can be combined to perform quantitative analysis of vascular features in color fundus photography (CFP) images. First, binary segmentation of vessels, arteriovenous segmentation, and optic disc segmentation are performed on the complete CFP image. Second, binary segmentation of vessels and arteriovenous segmentation are performed in the standard observation regions Zone B (0.5–1 times the optic disc diameter ring) and Zone C (0.5–2 times the optic disc diameter ring), respectively. Next, based on the segmented image, vascular parameters are calculated, including fractal dimension (a measure of vascular complexity), distance tortuosity and squared curvature torsion (measures of vascular torsion), and central retinal artery equivalence (CRAE) and central retinal vein equivalence (CRVE) to measure vascular diameter. The aforementioned characteristics were used in a comparative analysis between SLE patients and non-SLE individuals. Statistical methods (such as the Mann-Whitney U test) were employed to determine the significant association between vascular morphology indicators and SLE, thereby validating the consistency between the target prediction model's focus area and the disease's pathophysiological characteristics, and improving the model's interpretability. Furthermore, the comparative analysis between SLE patients and non-SLE individuals also helps to identify potential biomarkers.

[0088] Understandably, the aforementioned image detection method can help improve the challenges of current SLE screening methods in primary healthcare and resource-scarce environments. It provides a non-invasive, low-cost, and low-manual-reliance screening method that can achieve early screening of SLE and its complications in primary healthcare settings without laboratory testing facilities or with limited rheumatology resources, thereby improving clinical applicability. Moreover, it helps to achieve "opportunistic screening" of SLE and its complications, which can overcome the technical limitations of high-cost equipment (such as ultra-wide-angle fundus photography and optical coherence tomography) based on retinal fundus images. It can be integrated with existing ophthalmological infrastructure and is particularly suitable for routine fundus screening procedures such as diabetic retinopathy, thereby achieving "opportunistic screening" and improving the early identification rate and intervention timeliness of SLE at the population level.

[0089] In the aforementioned image detection method, the server first acquires a retinal image, which is obtained by capturing images of the eye region of the target object. Then, the retinal image is input into the target prediction model to obtain the prediction result output by the model. Since the prediction result characterizes the presence of lesion areas in the retinal image and the predicted probability of the lesion category corresponding to the lesion area, by inputting the retinal image into the target prediction model, the presence of lesion areas in the retinal image can be quickly predicted, improving the prediction efficiency of the retinal image. This allows for rapid diagnosis of lesions based on the prediction results, simplifying the diagnostic process and improving diagnostic efficiency. Furthermore, since the target prediction model is obtained through multi-stage training of the initial prediction model, the sample weights of each sample retinal image in each training stage are at least partially different. These sample weights are determined based on the lesion category corresponding to the sample retinal image, the prediction accuracy of the sample retinal image, or the default sample weights. During the training process, the imbalance of lesion categories corresponding to the sample retinal images and the differences in prediction results can be comprehensively considered, thereby improving the model accuracy of the trained target prediction model and ultimately improving the accuracy of the prediction results output by the target prediction model.

[0090] In one exemplary embodiment, such as Figure 3 As shown, this embodiment relates to the process of how the server trains the target prediction model. The training process of the target prediction model includes:

[0091] Step 301: Obtain the training sample set.

[0092] The training sample set includes multiple training samples, which include sample retinal images and corresponding lesion category labels.

[0093] The training sample set refers to a collection of multiple training samples. In this embodiment, the training process can be a supervised training process. Therefore, each training sample can include a sample retinal image and a corresponding lesion category label. The lesion category label refers to the lesion category corresponding to the sample retinal image. For example, if the sample retinal image does not have a lesion area, the lesion category label can be an empty label; if the sample retinal image has a lesion area, the lesion category label can be an SLE label, an LN label, or an LR label.

[0094] In this embodiment, the server can obtain retinal images of multiple target objects from the training database as sample retinal images, and obtain the lesion category label corresponding to each sample retinal image from the training database, and determine each sample retinal image and its corresponding lesion category label as a training sample, thereby determining multiple training samples as a training sample set.

[0095] Step 302: Based on the training sample set and the default sample weights of each sample retinal image, perform the first stage training on the initial prediction model to obtain the first stage prediction model.

[0096] The first stage of training refers to the process of training the training sample set with default sample weights for a preset number of rounds. For example, the number of training rounds in the first stage can be determined in advance based on the total number of training rounds and the training strategy. In this embodiment, the model obtained after the first stage of training is defined as the first-stage prediction model. For example, the number of training rounds in the first stage can be represented as: 0 <e< Where e represents each training round in the first phase of training. -1 represents the end round of the first phase of training, for example, It can be 10 or 20; this embodiment is for... There are no restrictions on the specific values ​​that can be obtained.

[0097] In this embodiment, the server can input the training sample set and the default sample weights of each sample retinal image into the initial prediction model, and set the training rounds according to the total number of training rounds and the training strategy. Then, after reaching the maximum number of training rounds, the initial prediction model corresponding to the maximum number of training rounds is determined as the first-stage prediction model.

[0098] As one possible implementation, the training sample set can be divided into a target training set and an optimization set according to a preset ratio. The target training set and the default sample weights of each sample retinal image are input into the initial prediction model. The number of training epochs is set according to the total number of training epochs and the training strategy. Then, after reaching the maximum number of training epochs, the initial prediction model corresponding to the maximum number of training epochs is determined as the first-stage prediction model. For example, the preset ratio can be 8:2, that is, 80% of the sample retinal images and their corresponding lesion category labels in the training sample set are determined as the target training set, and 20% of the sample retinal images and their corresponding lesion category labels in the training sample set are determined as the optimization set.

[0099] Step 303: Adjust the weights of each default sample according to the lesion category and / or the prediction accuracy of each sample retinal image to obtain the adjusted sample weights of each sample retinal image, and train the first-stage prediction model according to the adjusted sample weights and the training sample set to obtain the target prediction model.

[0100] It should be noted that when adjusting the default sample weights, the sample weight adjustment conditions can be set according to the lesion category corresponding to each sample retinal image and / or the prediction accuracy corresponding to each sample retinal image. For example, if a sample retinal image belongs to a minority disease category, it can be determined that the sample retinal image meets the sample weight adjustment conditions, and the default sample weight of the sample retinal image is adjusted to obtain a new sample weight; and / or, if the prediction accuracy corresponding to the sample retinal image is greater than the prediction accuracy threshold, it can be determined that the sample retinal image meets the sample weight adjustment conditions, and the default sample weight of the sample retinal image is adjusted to obtain a new sample weight.

[0101] The adjusted sample weights for each retinal image can include both the new, adjusted sample weights and the default sample weights.

[0102] In this embodiment, the server can first determine the weight coefficients based on the lesion category and / or the prediction accuracy of each sample retinal image. Then, the server uses the weight coefficients to adjust the default sample weights of the sample retinal images that meet the sample weight adjustment conditions to obtain new sample weights. Then, the new sample weights and the default sample weights of some sample retinal images that do not meet the sample weight adjustment conditions are determined as the adjusted sample weights of each sample retinal image. Then, the adjusted sample weights and the training sample set are input into the first-stage prediction model for training. After reaching the preset training rounds, the model corresponding to the preset training rounds is determined as the target prediction model.

[0103] In this embodiment, the server first obtains a training sample set, which includes multiple training samples. The training samples include sample retinal images and corresponding lesion category labels. Then, based on the training sample set and the default sample weights of each sample retinal image, the initial prediction model is trained in the first stage to obtain the first-stage prediction model. Next, the default sample weights are adjusted according to the lesion category and / or the prediction accuracy of each sample retinal image to obtain the adjusted sample weights of each sample retinal image. Then, the first-stage prediction model is trained based on the adjusted sample weights and the training sample set to obtain the target prediction model. Since the training process uses different sample weights to train the training sample set in stages, it can consider the differences in sample class imbalance and prediction difficulty in each training stage, thereby improving the training accuracy of the obtained target prediction model.

[0104] In one exemplary embodiment, such as Figure 4 As shown, this embodiment relates to how the server performs the second-stage training process. Step 303 above includes:

[0105] Step 401: Determine the lesion category corresponding to each sample retinal image.

[0106] Here, lesion category refers to the type of lesion corresponding to the lesion area when a lesion area exists in the sample retinal image. For example, lesion categories can include SLE, LR, and LN. It can be understood that when obtaining the training sample set, the lesion category corresponding to each sample retinal image can be labeled.

[0107] In this embodiment, the server can obtain the lesion category corresponding to each sample retinal image from the lesion category labels included in the training sample set.

[0108] Step 402: Adjust the weights of each default sample according to each lesion category to obtain the first adjusted sample weights of each sample retinal image.

[0109] The first adjusted sample weight refers to the sample weight corresponding to all sample retinal images after adjusting the default sample weights according to each lesion category.

[0110] It is understandable that during the process of adjusting the default sample weights according to each lesion category, the default sample weights corresponding to some retinal images need to be adjusted, while the default sample weights corresponding to other retinal images do not need to be adjusted. Therefore, the first adjustment of sample weights includes a portion of the adjusted sample weights and another portion of the default sample weights.

[0111] In this embodiment, the server can classify each sample retinal image according to each lesion category, determine the sample retinal images belonging to the minority category from the classification results, and adjust the default sample weights corresponding to the sample retinal images of the minority category to obtain the sample weights of the sample retinal images of the minority category. Then, the sample weights of the sample retinal images of the minority category and the default sample weights corresponding to the other sample retinal images are determined as the first adjusted sample weights.

[0112] The above-mentioned "training the first-stage prediction model according to the weights of each adjusted sample and the training sample set to obtain the target prediction model" includes: training the first-stage prediction model in the second stage according to the training sample set and the weights of each first adjusted sample to obtain the second-stage prediction model, and the target prediction model is obtained based on the second-stage prediction model.

[0113] The second-stage prediction model is obtained by training the first-stage prediction model using the training sample set and the weights of each first-stage adjusted sample. It is understood that after obtaining the second prediction model, further training stages can be performed to ultimately obtain the target prediction model. This embodiment does not limit the number of training stages included in the multi-stage training.

[0114] For example, if the multi-stage training process includes two training stages, the second-stage prediction model can be determined as the target prediction model; if the multi-stage training process includes three training stages, the second-stage prediction model can be trained in the third stage, and the prediction model obtained after the third stage training can be determined as the target prediction model.

[0115] In this implementation, the server can input the first adjusted sample weights and the training sample set into the first-stage prediction model for training. After reaching a preset training round, the model corresponding to the preset training round is determined as the second-stage prediction model. Then, the second-stage prediction model is trained again, and after reaching a preset training round, the prediction model corresponding to the preset training round is determined as the target prediction model.

[0116] In this embodiment, the server determines the lesion category corresponding to each sample retinal image and adjusts the weights of each default sample according to each lesion category to obtain the first adjusted sample weights of each sample retinal image. Since the first adjusted sample weights are obtained by adjusting the weights of each default sample according to the lesion category, the first adjusted sample weights can take into account the problem of uneven distribution between different sample retinal images, thereby improving the training accuracy of the second stage training of the first stage prediction model, and thus improving the model accuracy of the obtained second stage prediction model.

[0117] In one exemplary embodiment, such as Figure 5 As shown, this embodiment involves the process by which the server adjusts the weights of each default sample according to each lesion category to obtain the first adjusted sample weights for each sample retinal image. Step 402 includes:

[0118] Step 501: Obtain the number of lesion categories corresponding to each sample retinal image, the number of first sample retinal images containing sample lesion areas in each sample retinal image, the first training round corresponding to the first stage of training, and the second training round corresponding to the second stage of training.

[0119] Here, the number of categories refers to the number of different lesion categories corresponding to each sample retinal image. For example, the number of categories can be 2 or 3. This embodiment does not limit this, and the number of categories can also be different for different lesions. In this embodiment, the sample retinal image containing the sample lesion area is determined as the first sample retinal image. The first training round refers to the number of times the initial prediction model is trained in the first stage according to the training sample set and the default sample weights of each sample retinal image. The second training round refers to the number of times the first-stage prediction model is trained in the second stage.

[0120] In this embodiment, the server can classify each sample retinal image, then obtain the number of lesion categories based on the classification results, then determine the first sample retinal image with a lesion area based on the lesion category label, and count the number of the first sample retinal images. Then, the server calculates the first training round and the second training round based on the total number of training rounds and the training strategy.

[0121] Step 502: Based on the number of each category, the number of first sample retinal images, the first training round, and the second training round, obtain the sample weight of each first sample retinal image, and determine the default sample weight as the sample weight of the second sample retinal image in each retinal image where there is no sample lesion area.

[0122] In this embodiment, the server can substitute the number of each category, the number of first sample retinal images, the first training round, and the second training round into the weight adjustment formula to obtain the sample weight of each first sample retinal image, and then determine the default sample weight as the sample weight of the second sample retinal image in each retinal image where there is no sample lesion area.

[0123] The process of obtaining the sample weights of each first sample retinal image can be expressed as:

[0124]

[0125] in, Here, C represents the sample weights of each first-sample retinal image, C is the number of categories, and c represents the lesion category. The number of retinal images in the first sample. This is the starting round of the second training phase. -1 represents the end round of the second training phase. , where e represents each training round in the second training phase. This is used to indicate the importance of a minority of lesion categories, specifically the importance of the first retinal image, and the fewer the number of first retinal images, the greater the importance. The larger the value, the better.

[0126] Step 503: The sample weights of the first sample retinal image and the second sample retinal image are determined as the first adjusted sample weights.

[0127] In this embodiment, the server can determine the sample weights of the first sample retinal image and the second sample retinal image as the first adjusted sample weights corresponding to each sample retinal image.

[0128] In this embodiment, the server obtains the number of lesion categories corresponding to each sample retinal image, the number of first sample retinal images containing sample lesion areas in each sample retinal image, the first training round corresponding to the first stage of training, and the second training round corresponding to the second stage of training. Then, based on the number of each category, the number of first sample retinal images, the first training round, and the second training round, it obtains the sample weight of each first sample retinal image and determines the default sample weight as the sample weight of the second sample retinal image in each retinal image that does not contain sample lesion areas. This allows the server to quickly obtain the first adjusted sample weight of each sample retinal image during the second stage of training, thereby improving the training efficiency of the second stage of training.

[0129] In one exemplary embodiment, such as Figure 6 As shown, this embodiment relates to how the server performs the third-stage training process. The above method also includes:

[0130] Step 601: Based on the second-stage prediction model, obtain the prediction accuracy corresponding to each sample retinal image.

[0131] The prediction accuracy is determined based on the prediction results corresponding to the target training set and the prediction results corresponding to the optimization set. The target training set can include 80% of the sample retinal images and corresponding lesion category labels in the training sample set, and the optimization set can include 20% of the sample retinal images and corresponding lesion category labels in the training sample set.

[0132] In this embodiment, after the second stage of training is completed, the server can divide the training sample set into a target training set and an optimization set, input the target training set into the second stage prediction model to obtain the prediction result of the target training set, and then input the optimization set into the second stage prediction model to obtain the prediction result of the optimization set. Then, based on the difference between the prediction result of the target training set and the prediction result of the optimization set and the preset loss function, the prediction accuracy is determined.

[0133] The loss function can be expressed as:

[0134]

[0135] in, The value of the loss function. For batch size, The prediction results for the target training set. The prediction results for the optimization set, For the first The weights of each sample.

[0136] Step 602: Adjust the weights of each default sample according to the accuracy of each prediction to obtain the second adjusted sample weights of each sample retinal image.

[0137] The second adjustment of sample weights refers to the sample weights corresponding to all sample retinal images after adjusting the default sample weights according to the prediction accuracy.

[0138] It is understandable that prediction accuracy can characterize the prediction difficulty of each sample retinal image. For example, if the prediction accuracy is higher, the prediction difficulty of the sample retinal image is lower, and if the prediction accuracy is lower, the prediction difficulty of the sample retinal image is higher.

[0139] In this embodiment, the server can reduce the default sample weight of each sample retinal image with high prediction accuracy and increase the default sample weight of each sample retinal image with low prediction accuracy. Then, the adjusted sample weights are determined as the second adjusted sample weights.

[0140] Step 603: Based on the training sample set and the weights of each second adjusted sample, perform third-stage training on the second-stage prediction model to obtain the target prediction model.

[0141] The third stage of training involves performing another stage of training on the second-stage prediction model. It is understandable that, in a multi-stage training process comprising three training stages, the prediction model resulting from the third stage of training on the second-stage prediction model can be designated as the target prediction model.

[0142] In this embodiment, the server can input the second adjusted sample weights and the training sample set into the second-stage prediction model for training, and after reaching a preset training round, determine the prediction model corresponding to the preset training round as the target prediction model.

[0143] In this embodiment, the server first obtains the prediction accuracy corresponding to each sample retinal image based on the second-stage prediction model. Then, it adjusts the weights of each default sample according to the prediction accuracy to obtain the second adjusted sample weights for each sample retinal image. Subsequently, the server performs a third-stage training on the second-stage prediction model based on the training sample set and the second adjusted sample weights to obtain the target prediction model. Since the second adjusted sample weights are obtained by adjusting the weights of each default sample based on the prediction accuracy, the unevenness of the diagnostic difficulty of each sample retinal image is taken into account, thereby improving the training accuracy of the third-stage training and thus improving the prediction accuracy of the obtained target prediction model.

[0144] In one exemplary embodiment, such as Figure 7 As shown, this embodiment relates to the process by which the server adjusts the weights of each default sample according to the prediction accuracy to obtain the second adjusted sample weights for each sample retinal image. Step 602 includes:

[0145] Step 701: Obtain the third training round corresponding to the third stage of training.

[0146] The second training round refers to the number of times the second-stage prediction model is trained in the third stage.

[0147] In this embodiment, the server can calculate the third training round based on the total number of training rounds and the training strategy.

[0148] Step 702: Based on the number of each category, the prediction accuracy, and the third training round, obtain the sample weight of the third sample retinal image in each sample retinal image whose prediction accuracy is less than the preset accuracy threshold, and determine the default sample weight as the sample weight of the fourth sample retinal image in each sample retinal image whose prediction accuracy is greater than or equal to the preset accuracy threshold.

[0149] Understandably, the higher the prediction accuracy in a sample retinal image, the easier it is to diagnose that image; conversely, the lower the prediction accuracy, the more difficult it is to diagnose. The preset accuracy threshold is an indicator used to determine the diagnostic difficulty of a sample retinal image. Therefore, the sample weights of third-sample retinal images with prediction accuracy lower than the preset accuracy threshold can be adjusted.

[0150] In this embodiment, the server can substitute the number of each category and the prediction accuracy into the weight adjustment formula to obtain the sample weight of each third sample retinal image, and then determine the default sample weight as the sample weight of the fourth sample retinal image in each sample retinal image whose prediction accuracy is greater than or equal to the preset accuracy threshold.

[0151] The process of obtaining the sample weights of each third sample retinal image can be expressed as:

[0152]

[0153] in, Here, C represents the sample weights of each third-sample retinal image, C is the number of categories, and c represents the lesion category. The prediction accuracy for the third sample retinal image. This is the starting round of the third training phase. -1 represents the end of the third training phase. , This indicates the difficulty of recognizing the third sample retinal image.

[0154] Step 703: The sample weights of the third sample retinal image and the sample weights of the fourth sample retinal image are determined as the second adjusted sample weights.

[0155] In this embodiment, the server can determine the sample weights of the third sample retinal image and the fourth sample retinal image as the second adjusted sample weights corresponding to each sample retinal image.

[0156] In this embodiment, the server obtains the third training round corresponding to the third stage of training. Based on the number of each category, the prediction accuracy, and the third training round, it obtains the sample weights of the third sample retinal images whose prediction accuracy is less than a preset accuracy threshold. The default sample weights are then determined as the sample weights of the fourth sample retinal images whose prediction accuracy is greater than or equal to the preset accuracy threshold. This allows for the rapid acquisition of the second adjusted sample weights of each sample retinal image during the third stage of training, thereby improving the training efficiency of the third stage of training.

[0157] In one exemplary embodiment, such as Figure 8 As shown, this embodiment relates to the process by which a server inputs a retinal image into a target prediction model and obtains the prediction result output by the target prediction model. Step 202 includes:

[0158] Step 801: Input the retinal image into the encoder to obtain the semantic features of the retinal image.

[0159] The encoder is a processing module in the target prediction model used to extract semantic features from retinal images. Semantic features of retinal images refer to high-level abstract information with biological or medical significance extracted from retinal images. These semantic features reflect the structure, pathological state, or functional relationships of the retina, rather than simply pixel-level visual information.

[0160] In this embodiment, the server can input the retinal image into the encoder of the target prediction model, and the encoder can perform feature extraction processing on the retinal image to obtain the semantic features corresponding to the retinal image.

[0161] As one possible implementation, the retinal image can be data augmented before being input into the encoder. Data augmentation may include geometric transformations (e.g., random rotation, flipping, and scaling), color enhancement (e.g., adjusting brightness and contrast), Gaussian noise injection, and image normalization.

[0162] Step 802: Input the semantic features into the classifier to obtain the prediction result.

[0163] The classifier is a processing module in the target prediction model used to classify retinal images based on semantic features. For example, if a lesion area exists in the retinal image, the prediction result can be the predicted probability of the lesion category corresponding to the lesion area; if no lesion area exists in the retinal image, the prediction result can be that the predicted probability of the lesion category corresponding to the lesion area is 0.

[0164] In this embodiment, the server can input semantic features into a classifier through a target prediction model. The classifier then classifies the semantic features to obtain the lesion category corresponding to the semantic features, and thus obtains the prediction result based on the lesion category.

[0165] In this embodiment, the server obtains the semantic features of the retinal image by inputting the retinal image into the encoder. Since the semantic features can represent the structure, pathological state, or functional relationship of the retinal image, the semantic features are input into the classifier to obtain the prediction result, which can improve the accuracy of the prediction result.

[0166] In one exemplary embodiment, such as Figure 9 As shown, a training method for a target prediction model is provided, which is then applied to... Figure 1 Taking the server in the example, the following steps are included:

[0167] Step 901: Obtain the training sample set.

[0168] The training sample set includes multiple training samples, which include sample retinal images and corresponding lesion category labels.

[0169] In this embodiment, the server can obtain retinal images of multiple target objects from the training database as sample retinal images, and obtain the lesion category label corresponding to each sample retinal image from the training database, and determine each sample retinal image and its corresponding lesion category label as a training sample, thereby determining multiple training samples as a training sample set.

[0170] Step 902: Based on the training sample set and the default sample weights of each sample retinal image, perform the first stage training on the initial prediction model to obtain the first stage prediction model.

[0171] In this embodiment, the server can input the training sample set and the default sample weights of each sample retinal image into the initial prediction model, and set the training rounds according to the total number of training rounds and the training strategy. Then, after reaching the maximum number of training rounds, the initial prediction model corresponding to the maximum number of training rounds is determined as the first-stage prediction model.

[0172] Step 903: Adjust the weights of each default sample according to the lesion category and / or prediction accuracy of each sample retinal image to obtain the adjusted sample weights of each sample retinal image, and train the first-stage prediction model according to the adjusted sample weights and the training sample set to obtain the target prediction model.

[0173] In this embodiment, the server can first determine the weight coefficients based on the lesion category and / or the prediction accuracy of each sample retinal image. Then, the server uses the weight coefficients to adjust the default sample weights of the sample retinal images that meet the sample weight adjustment conditions to obtain new sample weights. Then, the new sample weights and the default sample weights of some sample retinal images that do not meet the sample weight adjustment conditions are determined as the adjusted sample weights of each sample retinal image. Then, the adjusted sample weights and the training sample set are input into the first-stage prediction model for training. After reaching the preset training rounds, the model corresponding to the preset training rounds is determined as the target prediction model.

[0174] In this embodiment, the server first obtains a training sample set, which includes multiple training samples. The training samples include sample retinal images and corresponding lesion category labels. Then, based on the training sample set and the default sample weights of each sample retinal image, the initial prediction model is trained in the first stage to obtain the first-stage prediction model. Next, the default sample weights are adjusted according to the lesion category and / or the prediction accuracy of each sample retinal image to obtain the adjusted sample weights of each sample retinal image. Then, the first-stage prediction model is trained based on the adjusted sample weights and the training sample set to obtain the target prediction model. Since the training process uses different sample weights to train the training sample set in stages, it can consider the differences in sample class imbalance and prediction difficulty in each training stage, thereby improving the training accuracy of the obtained target prediction model.

[0175] In one exemplary embodiment, such as Figure 10 As shown, the initial prediction model includes an initial encoder. This embodiment relates to the process by which the server determines the initial parameters of the initial encoder. The method also includes:

[0176] Step 1001: Perform masking processing on the retinal images of each sample to obtain the mask image corresponding to the retinal image of each sample.

[0177] In this process, the mask image corresponding to each sample retinal image is used to extract the region of interest (ROI) from the sample retinal image and eliminate background interference. Masking refers to creating a "mask" for the retinal image to selectively preserve or mask specific regions in the image, thereby achieving accurate segmentation and extraction of the ROI.

[0178] In this embodiment, the server can first perform grayscale processing on each sample retinal image to obtain grayscale sample retinal images, and then use a mask generation method to process each grayscale sample retinal image to obtain a mask image corresponding to each sample retinal image.

[0179] Step 1002: Convert each mask image into a feature vector, and input each feature vector into the encoder included in the image reconstruction model to obtain the sample semantic feature matrix.

[0180] The encoder can be VisionTransformer Large, which includes 24 Transformer modules, each consisting of alternating multi-head self-attention layers and multilayer perceptron modules.

[0181] In this embodiment, the server can first segment each mask image, cutting each mask image into 16*16 image blocks, and then use a feature transformation algorithm to process each mask image, converting each image block into a feature vector of length 1024. Then, each feature vector is input into the encoder included in the image reconstruction model, and the encoder performs deep feature extraction processing on each mask image to obtain the sample semantic features corresponding to each mask image. Finally, each sample semantic feature is determined as a sample semantic feature matrix.

[0182] Step 1003: Add virtual mask markers to the sample semantic feature matrix to obtain a new sample semantic feature matrix.

[0183] In this embodiment, the server can fill in virtual mask markers into the sample semantic feature matrix to complete the occluded areas in the sample semantic feature matrix, thereby obtaining a new sample semantic feature matrix. For example, the occlusion ratio can be set to 75%.

[0184] Step 1004: Input the new sample semantic feature matrix into the decoder included in the image reconstruction model to obtain the sample reconstruction image corresponding to each sample retinal image.

[0185] The decoder can partially adopt the lightweight ViT-Small structure, which contains 8 Transformer modules to restore the encoded features into a complete image.

[0186] In this embodiment, the server can input the new sample semantic feature matrix into the decoder included in the image reconstruction model, and the decoder can perform reconstruction processing on the new sample semantic feature matrix to generate sample reconstructed images corresponding to each sample retinal image.

[0187] Step 1005: Optimize the image reconstruction model based on the sample reconstructed image to obtain the optimized image reconstruction model, and use the encoder parameters in the optimized image reconstruction model as the initial parameters of the initial encoder.

[0188] In this embodiment, the server can use the sample reconstructed image and loss function to optimize the parameters in the image reconstruction model until the value of the loss function tends to stabilize. The parameters of the encoder in the image reconstruction model are then used as the initial parameters of the initial encoder to train the initial prediction model.

[0189] For example, during the optimization of the image reconstruction model based on the reconstructed images from the samples, four NVIDIA A800 GPUs (80GB VRAM) can be used for parallel training, with a total of 800 training epochs. The batch size for training is 1792 (i.e., 448 images per GPU). Finally, the image reconstruction model after the 800th training epoch is selected as the optimized image reconstruction model.

[0190] In this embodiment, the server performs masking processing on each sample retinal image to obtain a mask image corresponding to each sample retinal image. Then, each mask image is converted into a feature vector, and each feature vector is input into the encoder included in the image reconstruction model to obtain a sample semantic feature matrix. Then, a virtual mask label is added to the sample semantic feature matrix to obtain a new sample semantic feature matrix. The new sample semantic feature matrix is ​​input into the decoder included in the image reconstruction model to obtain the sample reconstructed image corresponding to each sample retinal image. Thus, the image reconstruction model can be optimized based on the sample reconstructed image to obtain an optimized image reconstruction model. The parameters of the encoder in the optimized image reconstruction model are used as the initial parameters of the initial encoder. Since the parameters of the encoder in the image reconstruction model are obtained by pre-training using sample retinal images, the accuracy of the initial encoder in the initial prediction model can be improved, thereby improving the training efficiency of the initial prediction model.

[0191] To facilitate understanding by those skilled in the art, the image detection method provided in this application will be described in detail below. This method may include:

[0192] S1, perform masking processing on each of the sample retinal images to obtain the mask image corresponding to each of the sample retinal images.

[0193] S2, convert each of the mask images into feature vectors, and input each of the feature vectors into the encoder included in the image reconstruction model to obtain the sample semantic feature matrix.

[0194] S3, add virtual mask markers to the sample semantic feature matrix to obtain a new sample semantic feature matrix, and input the new sample semantic feature matrix into the decoder included in the image reconstruction model to obtain the sample reconstruction image corresponding to each sample retinal image.

[0195] S4, optimize the image reconstruction model based on the sample reconstructed image to obtain an optimized image reconstruction model, and use the encoder parameters in the optimized image reconstruction model as the initial parameters of the initial encoder.

[0196] S5, obtain a training sample set, which includes multiple training samples, including sample retinal images and lesion category labels corresponding to the sample retinal images.

[0197] S6. Based on the training sample set and the default sample weights of each sample retinal image, the initial prediction model is trained in the first stage to obtain the first stage prediction model.

[0198] S7, determine the lesion category corresponding to each sample retinal image.

[0199] S8, obtain the number of lesion categories corresponding to each sample retinal image, the number of first sample retinal images containing sample lesion areas in each sample retinal image, the first training round corresponding to the first stage of training, and the second training round corresponding to the second stage of training.

[0200] S9. Based on the number of each category, the number of the first sample retinal images, the first training round, and the second training round, obtain the sample weight of each first sample retinal image, and determine the default sample weight as the sample weight of the second sample retinal image in each of the retinal images in which the sample lesion area does not exist, and determine the sample weight of the first sample retinal image and the sample weight of the second sample retinal image as the first adjusted sample weight.

[0201] S10, based on the training sample set and the weights of each of the first adjusted samples, perform a second-stage training on the first-stage prediction model to obtain a second-stage prediction model.

[0202] S11, Based on the second-stage prediction model, obtain the prediction accuracy level corresponding to each sample retinal image.

[0203] S12, obtain the third training round corresponding to the third stage of training, and according to the number of each category, the prediction accuracy of each category and the third training round, obtain the sample weight of the third sample retinal image in each sample retinal image whose prediction accuracy is less than the preset accuracy threshold, and determine the default sample weight as the sample weight of the fourth sample retinal image in each sample retinal image whose prediction accuracy is greater than or equal to the preset accuracy threshold, and determine the sample weight of the third sample retinal image and the sample weight of the fourth sample retinal image as the second adjusted sample weight.

[0204] S13, based on the training sample set and the weights of each of the second adjusted samples, the second-stage prediction model is trained in the third stage to obtain the target prediction model.

[0205] S14, acquire a retinal image, which is obtained by acquiring an image of the eye region of the target object.

[0206] S15, the retinal image is input into the encoder to obtain the semantic features of the retinal image.

[0207] S16, Input the semantic features into the classifier to obtain the prediction result.

[0208] It should be noted that the descriptions in S1-S16 above can be found in the relevant descriptions in the above embodiments, and their effects are similar, so they will not be repeated here.

[0209] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0210] Based on the same inventive concept, this application also provides an image detection apparatus for implementing the image detection method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image detection apparatus embodiments provided below can be found in the limitations of the image detection method described above, and will not be repeated here.

[0211] In one embodiment, such as Figure 11 As shown, an image detection device is provided, including: a first acquisition module 1101 and a prediction module 1102, wherein:

[0212] The first acquisition module 1101 is used to acquire a retinal image, which is obtained by image acquisition of the eye region of the target object;

[0213] The prediction module 1102 is used to input the retinal image into the target prediction model and obtain the prediction result output by the target prediction model;

[0214] The prediction results are used to characterize the presence of lesion areas in retinal images and the predicted probability of the lesion category corresponding to the lesion area. The target prediction model is obtained by multi-stage training of the initial prediction model. In the multi-stage training, the sample weights of each sample retinal image in each training stage are at least partially different. The sample weights are determined based on the lesion category corresponding to the sample retinal image, the prediction accuracy corresponding to the sample retinal image, or the default sample weights.

[0215] The image detection device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0216] In one embodiment, the above-mentioned apparatus further includes:

[0217] The second acquisition module is used to acquire a training sample set, which includes multiple training samples. The training samples include sample retinal images and lesion category labels corresponding to the sample retinal images.

[0218] The first training module is used to train the initial prediction model in the first stage based on the training sample set and the default sample weights of each sample retinal image, so as to obtain the first stage prediction model.

[0219] The second training module is used to adjust the weights of each default sample according to the lesion category and / or the prediction accuracy of each sample retinal image, to obtain the adjusted sample weights of each sample retinal image, and to train the first-stage prediction model according to the adjusted sample weights and the training sample set to obtain the target prediction model.

[0220] The image detection device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0221] In one embodiment, the second training module described above includes:

[0222] The first determining unit is used to determine the lesion category corresponding to each sample retinal image;

[0223] The adjustment unit is used to adjust the weights of each default sample according to each lesion category to obtain the first adjusted sample weights of each sample retinal image.

[0224] The second training module mentioned above also includes:

[0225] The training unit is used to train the first-stage prediction model in the second stage based on the training sample set and the weights of each first-stage adjusted sample, so as to obtain the second-stage prediction model. The target prediction model is obtained based on the second-stage prediction model.

[0226] The image detection device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0227] In one embodiment, the aforementioned adjustment unit is specifically used for:

[0228] The number of lesion categories corresponding to each sample retinal image, the number of first sample retinal images containing sample lesion areas in each sample retinal image, the first training round corresponding to the first stage of training, and the second training round corresponding to the second stage of training are obtained.

[0229] Based on the number of each category, the number of first sample retinal images, the first training round, and the second training round, the sample weight of each first sample retinal image is obtained, and the default sample weight is determined as the sample weight of the second sample retinal image in which there is no sample lesion area in each retinal image.

[0230] The sample weights of the first sample retinal image and the second sample retinal image are determined as the first adjusted sample weights.

[0231] The image detection device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0232] In one embodiment, the above-mentioned apparatus further includes:

[0233] The third acquisition module is used to acquire the prediction accuracy of each sample retinal image based on the second-stage prediction model.

[0234] The adjustment module is used to adjust the weights of each default sample according to the accuracy of each prediction, so as to obtain the second adjusted sample weights of each sample retinal image.

[0235] The third training module is used to train the second-stage prediction model in the third stage based on the training sample set and the weights of each second adjusted sample, so as to obtain the target prediction model.

[0236] The image detection device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0237] In one embodiment, the adjustment module includes:

[0238] The first acquisition unit is used to acquire the third training round corresponding to the third stage of training.

[0239] The second acquisition unit is used to obtain the sample weights of the third sample retinal images in each sample retinal image whose prediction accuracy is less than a preset accuracy threshold, based on the number of each category, the prediction accuracy, and the third training round, and to determine the default sample weights as the sample weights of the fourth sample retinal images in each sample retinal image whose prediction accuracy is greater than or equal to the preset accuracy threshold.

[0240] The second determining unit is used to determine the sample weights of the third sample retinal image and the sample weights of the fourth sample retinal image as the second adjusted sample weights.

[0241] The image detection device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0242] In one embodiment, the target prediction model includes an encoder and a classifier, and the prediction module includes:

[0243] The third acquisition module is used to input the retinal image into the encoder to obtain the semantic features of the retinal image;

[0244] The fourth acquisition module is used to input semantic features into the classifier to obtain prediction results.

[0245] Each module in the aforementioned image detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0246] In one embodiment, such as Figure 12 As shown, a training device for a target prediction model is provided, comprising: a fifth acquisition module 1201, a sixth acquisition module 1202, and a seventh acquisition module 1203, wherein:

[0247] The fifth acquisition module 1201 is used to acquire a training sample set, which includes multiple training samples. The training samples include sample retinal images and lesion category labels corresponding to the sample retinal images.

[0248] The sixth acquisition module 1202 is used to perform the first stage training of the initial prediction model based on the training sample set and the default sample weights of each sample retinal image to obtain the first stage prediction model.

[0249] The seventh acquisition module 1203 is used to adjust the weights of each default sample according to the lesion category and / or prediction accuracy of each sample retinal image, to obtain the adjusted sample weights of each sample retinal image, and to train the first-stage prediction model according to the adjusted sample weights and the training sample set to obtain the target prediction model.

[0250] The training device for the target prediction model provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0251] In one embodiment, the initial prediction model includes an initial encoder, and the above-described apparatus further includes:

[0252] The first processing module is used to perform masking processing on the retinal images of each sample to obtain the mask image corresponding to the retinal image of each sample.

[0253] The encoding module is used to convert each mask image into a feature vector, and input each feature vector into the encoder included in the image reconstruction model to obtain the sample semantic feature matrix;

[0254] The second processing module is used to add virtual mask markers to the sample semantic feature matrix to obtain a new sample semantic feature matrix.

[0255] The decoding module is used to input the new sample semantic feature matrix into the decoder included in the image reconstruction model to obtain the sample reconstruction image corresponding to each sample retinal image;

[0256] The determination module is used to optimize the image reconstruction model based on the sample reconstructed image to obtain the optimized image reconstruction model, and use the encoder parameters in the optimized image reconstruction model as the initial parameters of the initial encoder.

[0257] The training device for the target prediction model provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0258] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores retinal image data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements an image detection method.

[0259] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0260] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0261] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0262] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0263] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0264] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0265] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image detection method, characterized in that, The method includes: Acquire a retinal image, wherein the retinal image is obtained by acquiring an image of the eye region of the target object; The retinal image is input into the target prediction model to obtain the prediction result output by the target prediction model; The prediction result is used to characterize whether there is a lesion area in the retinal image and the prediction probability of the lesion category corresponding to the lesion area; the target prediction model is obtained by multi-stage training of the initial prediction model, the initial prediction model includes an initial encoder, and the sample weights of each sample retinal image in each training stage of the multi-stage training are at least partially different, the sample weights are determined based on the lesion category corresponding to the sample retinal image, the prediction accuracy corresponding to the sample retinal image, or the default sample weights; The initial parameters of the initial encoder are determined through the following process: The retinal images of each sample are masked to obtain the mask images corresponding to the retinal images of each sample. Each of the mask images is converted into a feature vector, and each of the feature vectors is input into the encoder included in the image reconstruction model to obtain the sample semantic feature matrix; A virtual mask marker is added to the sample semantic feature matrix to obtain a new sample semantic feature matrix; The new sample semantic feature matrix is ​​input into the decoder included in the image reconstruction model to obtain the sample reconstruction image corresponding to each sample retinal image; The image reconstruction model is optimized based on the reconstructed image from the sample to obtain the optimized image reconstruction model. The encoder parameters in the optimized image reconstruction model are then used as the initial parameters of the initial encoder in the initial prediction model.

2. The method according to claim 1, characterized in that, The training process of the target prediction model includes: Obtain a training sample set, which includes multiple training samples, and each training sample includes a sample retinal image and a lesion category label corresponding to the sample retinal image; Based on the training sample set and the default sample weights of each sample retinal image, the initial prediction model is trained in the first stage to obtain the first stage prediction model. The weights of each default sample are adjusted according to the lesion category corresponding to each sample retinal image and / or the prediction accuracy corresponding to each sample retinal image to obtain the adjusted sample weights of each sample retinal image. The first-stage prediction model is then trained according to the adjusted sample weights and the training sample set to obtain the target prediction model.

3. The method according to claim 2, characterized in that, The step of adjusting the weights of each default sample image based on the lesion category corresponding to each sample retinal image and / or the prediction accuracy corresponding to each sample retinal image to obtain the adjusted sample weights of each sample retinal image includes: Determine the lesion category corresponding to the retinal images of each sample; The default sample weights are adjusted according to each of the lesion categories to obtain the first adjusted sample weights for each of the sample retinal images; The step of training the first-stage prediction model based on the adjusted sample weights and the training sample set to obtain the target prediction model includes: Based on the training sample set and the weights of each of the first adjusted samples, the first-stage prediction model is trained in the second stage to obtain the second-stage prediction model, and the target prediction model is obtained based on the second-stage prediction model.

4. The method according to claim 3, characterized in that, The step of adjusting the default sample weights according to each of the lesion categories to obtain the first adjusted sample weights for each of the sample retinal images includes: The number of lesion categories corresponding to each sample retinal image, the number of first sample retinal images containing sample lesion regions in each sample retinal image, the first training round corresponding to the first stage of training, and the second training round corresponding to the second stage of training are obtained. Based on the number of each category, the number of the first sample retinal images, the first training round, and the second training round, the sample weight of each first sample retinal image is obtained, and the default sample weight is determined as the sample weight of the second sample retinal image in which the lesion region of the sample does not exist in each of the retinal images. The sample weights of the first sample retinal image and the sample weights of the second sample retinal image are determined as the first adjusted sample weights.

5. The method according to claim 4, characterized in that, The method further includes: Based on the second-stage prediction model, the prediction accuracy corresponding to each sample retinal image is obtained; The default sample weights are adjusted according to the prediction accuracy of each sample to obtain the second adjusted sample weights for each sample retinal image; Based on the training sample set and the weights of each of the second adjusted samples, the second-stage prediction model is trained in the third stage to obtain the target prediction model.

6. The method according to claim 5, characterized in that, The step of adjusting the default sample weights based on the prediction accuracy of each sample to obtain the second adjusted sample weights for each sample retinal image includes: Obtain the third training round corresponding to the third stage of training; Based on the number of each category, the prediction accuracy of each category, and the third training round, the sample weights of the third sample retinal images in each sample retinal image with a prediction accuracy less than a preset accuracy threshold are obtained, and the default sample weights are determined as the sample weights of the fourth sample retinal images in each sample retinal image with a prediction accuracy greater than or equal to the preset accuracy threshold. The sample weights of the third sample retinal image and the sample weights of the fourth sample retinal image are determined as the second adjusted sample weights.

7. The method according to claim 1, characterized in that, The target prediction model includes an encoder and a classifier. The step of inputting the retinal image into the target prediction model to obtain the prediction result output by the target prediction model includes: The retinal image is input into the encoder to obtain the semantic features of the retinal image; The semantic features are input into the classifier to obtain the prediction result.

8. A method for training a target prediction model, characterized in that, The method includes: Obtain a training sample set, which includes multiple training samples, and each training sample includes a sample retinal image and a lesion category label corresponding to the sample retinal image; Based on the training sample set and the default sample weights of each sample retinal image, the initial prediction model is trained in the first stage to obtain the first stage prediction model. The weights of each default sample are adjusted according to the lesion category and / or prediction accuracy of each sample retinal image to obtain the adjusted sample weights of each sample retinal image. The first-stage prediction model is then trained according to the adjusted sample weights and the training sample set to obtain the target prediction model. The initial prediction model includes an initial encoder, and the process of determining the initial parameters of the initial encoder includes: The retinal images of each sample are masked to obtain the mask images corresponding to the retinal images of each sample. Each of the mask images is converted into a feature vector, and each of the feature vectors is input into the encoder included in the image reconstruction model to obtain the sample semantic feature matrix; A virtual mask marker is added to the sample semantic feature matrix to obtain a new sample semantic feature matrix; The new sample semantic feature matrix is ​​input into the decoder included in the image reconstruction model to obtain the sample reconstruction image corresponding to each sample retinal image; The image reconstruction model is optimized based on the reconstructed image from the sample to obtain the optimized image reconstruction model. The encoder parameters in the optimized image reconstruction model are then used as the initial parameters of the initial encoder in the initial prediction model.

9. An image detection device, characterized in that, The device includes: The first acquisition module is used to acquire a retinal image, which is obtained by image acquisition of the eye region of the target object; The prediction module is used to input the retinal image into the target prediction model and obtain the prediction result output by the target prediction model; The prediction result is used to characterize whether there is a lesion area in the retinal image and the prediction probability of the lesion category corresponding to the lesion area; the target prediction model is obtained by multi-stage training of the initial prediction model, the initial prediction model includes an initial encoder, and the sample weights of each sample retinal image in each training stage of the multi-stage training are at least partially different, the sample weights are determined based on the lesion category corresponding to the sample retinal image, the prediction accuracy corresponding to the sample retinal image, or the default sample weights; The process of determining the initial parameters of the initial encoder includes: The retinal images of each sample are masked to obtain the mask images corresponding to the retinal images of each sample. Each of the mask images is converted into a feature vector, and each of the feature vectors is input into the encoder included in the image reconstruction model to obtain the sample semantic feature matrix; A virtual mask marker is added to the sample semantic feature matrix to obtain a new sample semantic feature matrix; The new sample semantic feature matrix is ​​input into the decoder included in the image reconstruction model to obtain the sample reconstruction image corresponding to each sample retinal image; The image reconstruction model is optimized based on the reconstructed image from the sample to obtain the optimized image reconstruction model. The encoder parameters in the optimized image reconstruction model are then used as the initial parameters of the initial encoder in the initial prediction model.

10. A training device for a target prediction model, characterized in that, The device includes: The fifth acquisition module is used to acquire a training sample set, which includes multiple training samples, and the training samples include sample retinal images and lesion category labels corresponding to the sample retinal images; The sixth acquisition module is used to perform the first stage training of the initial prediction model based on the training sample set and the default sample weights of each sample retinal image to obtain the first stage prediction model. The seventh acquisition module is used to adjust the weights of each default sample according to the lesion category and / or prediction accuracy of each sample retinal image to obtain the adjusted sample weights of each sample retinal image, and to train the first stage prediction model according to the adjusted sample weights and the training sample set to obtain the target prediction model. The initial prediction model includes an initial encoder, the initial parameters of which are determined by the following modules: The first processing module is used to perform masking processing on each of the sample retinal images to obtain a mask image corresponding to each of the sample retinal images; An encoding module is used to convert each of the mask images into feature vectors, and input each of the feature vectors into the encoder included in the image reconstruction model to obtain a sample semantic feature matrix; The second processing module is used to add virtual mask markers to the sample semantic feature matrix to obtain a new sample semantic feature matrix. The decoding module is used to input the new sample semantic feature matrix into the decoder included in the image reconstruction model to obtain the sample reconstruction image corresponding to each sample retinal image; The determination module is used to optimize the image reconstruction model based on the sample reconstructed image to obtain the optimized image reconstruction model, and use the encoder parameters in the optimized image reconstruction model as the initial parameters of the initial encoder in the initial prediction model.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images