A method and apparatus for determining a classification model, an electronic device, and a storage medium

Abstract

This invention discloses a method, apparatus, electronic device, and storage medium for determining a classification model. The method includes: determining at least one set of training sample images; for each set of training sample images, generating a vector to be used corresponding to the training sample image based on the training sample image and corresponding diagnostic and treatment information; and training an initial classification model based on the vectors to be used corresponding to multiple sets of training sample images to obtain a target classification model. This achieves the effect of obtaining a large number of sample images from a small number of pre-diagnosis and post-diagnosis images, generating corresponding vectors to be used based on the diagnostic and treatment information of the sample images, and training the original classification model based on these vectors to obtain a more accurate target classification model.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, electronic device, and storage medium for determining a classification model. Background Technology

[0002] Currently, determining the recovery status of breast cancer areas in patients typically requires pathological examination. However, this process is time-consuming and labor-intensive, and involves invasive procedures. Alternatively, a pre-trained neural network model can be used to detect breast cancer areas; however, this method requires a large number of training samples, which are not easily obtained in practice.

[0003] To address the aforementioned issues, this technical solution proposes a classification model for lesion detection in breast images. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for determining a classification model, in order to solve the problem that when training a classification model for image classification of lesion images, it is difficult to obtain sample images and the data dimension of the model training is too single, resulting in inaccurate image classification results.

[0005] In a first aspect, embodiments of the present invention provide a method for determining a classification model, comprising:

[0006] At least one set of training sample images is determined; wherein, the training sample images include a breast lesion image before diagnosis and treatment, a first breast reference image corresponding to the breast lesion image, a breast lesion image to be compared after diagnosis and treatment corresponding to the breast lesion image, and a second breast reference image corresponding to the breast lesion image to be compared, wherein the first breast reference image is an image that does not contain breast lesions, and the second breast reference image is an image that has not undergone diagnosis and treatment and does not contain the breast lesions;

[0007] For each set of training sample images, a vector to be used corresponding to the training sample image is generated based on the training sample image and the diagnosis and treatment association information corresponding to the training sample image.

[0008] The initial classification model is trained based on the vectors corresponding to the multiple sets of training sample images to obtain the target classification model.

[0009] Secondly, embodiments of the present invention also provide a classification model determination apparatus, comprising:

[0010] A sample image determination module is used to determine at least one set of training sample images; wherein, the training sample images include a breast lesion image before diagnosis and treatment, a first breast reference image corresponding to the breast lesion image, a breast lesion image to be compared after diagnosis and treatment corresponding to the breast lesion image, and a second breast reference image corresponding to the breast lesion image to be compared, wherein the first breast reference image is an image that does not contain breast lesions, and the second breast reference image is an image that has not undergone diagnosis and treatment and does not contain the breast lesions;

[0011] The vector generation module is used to generate a vector to be used corresponding to each group of training sample images, based on the training sample images and the corresponding diagnosis and treatment association information;

[0012] The model determination module is used to train the initial classification model based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model.

[0013] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method for determining the classification model according to any embodiment of the present invention.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute the method for determining a classification model as described in any embodiment of the present invention.

[0018] The technical solution of this invention involves determining at least one set of training sample images, wherein the training sample images include breast lesion images, a first breast reference image, a breast lesion image to be compared, and a second breast reference image. At least one pre-treatment and post-treatment breast image of a user to be trained is acquired. Breast lesion images in the pre-treatment breast image are determined through manual annotation. The first breast reference image is determined based on the symmetrical position of the breast lesion image in the pre-treatment breast image. The breast lesion image to be compared in the post-treatment breast image is determined based on the image position information of the breast lesion image, and a second breast reference image corresponding to the first breast reference image is also determined. For each set of training sample images, a vector to be used is generated based on the training sample image and the corresponding diagnostic and treatment association information. Specifically, a first random diagnostic and treatment information is associated with a breast lesion image to generate a corresponding first vector to be used; a second random diagnostic and treatment information is associated with a first breast reference image to obtain a second vector to be used; a third random diagnostic and treatment information is associated with a second breast reference image to obtain a third vector to be used; and actual diagnostic and treatment information is associated with a breast lesion image to be compared to obtain a fourth vector to be used. The initial classification model is trained based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model. This solves the problem that in image classification models for lesion images, the difficulty in obtaining sample images and the single data dimension of the model training lead to inaccurate image classification results. It achieves the goal of obtaining a large number of sample images based on a small number of pre- and post-treatment images, generating corresponding vectors to be used with the diagnostic and treatment information corresponding to the sample images, and then training the original classification model based on these vectors to obtain a more accurate target classification model.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a method for determining a classification model according to Embodiment 1 of the present invention;

[0022] Figure 2This is a flowchart of a method for determining a classification model according to Embodiment 2 of the present invention;

[0023] Figure 3 This is a flowchart of a method for determining a classification model according to Embodiment 2 of the present invention;

[0024] Figure 4 This is a schematic diagram of the structure of a classification model determination device provided in Embodiment 3 of the present invention;

[0025] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the classification model determination method of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0028] Before elaborating on this technical solution, let's briefly introduce its application scenarios to facilitate a clearer understanding. In the medical field, current treatment methods for certain diseases often involve neoadjuvant therapy, such as neoadjuvant chemotherapy or neoadjuvant radiotherapy, followed by assessment of complete remission of the lesions. Traditional methods often require invasive diagnostic procedures for pathological examination to determine lesion remission. However, this approach is not only time-consuming and laborious for patients but also poses a secondary risk of harm. Therefore, a non-invasive method to confirm complete lesion remission would be far more user-friendly for both patients and those seeking treatment.

[0029] Currently, to confirm whether a patient's lesion area has completely resolved, a deep learning network model can be trained, and the lesion area can be detected based on the trained model. However, training a deep learning network model requires a large number of training samples. Therefore, how to make full use of limited samples to train the deep learning neural network model to make the lesion area identification results more accurate has become an important issue.

[0030] Example 1

[0031] Figure 1 The flowchart of a method for determining a classification model is provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation where a large number of sample images are obtained based on a small number of diagnostic images, and the original classification model is trained based on the sample images and the diagnostic information corresponding to the sample images to obtain the target model. This method can be executed by a device for determining the classification model, which can be implemented in hardware and / or software. The device for determining the classification model can be configured in a computing device that can execute the method for determining the classification model.

[0032] like Figure 1 As shown, the method includes:

[0033] S110. Determine at least one set of training sample images.

[0034] The training sample images can be magnetic resonance imaging (i.e., MRI images) corresponding to patients.

[0035] It should be noted that this technical solution can be applied to diseases in which the lesion area in the patient's body will not spread or metastasize before or after treatment, such as breast cancer.

[0036] Taking breast cancer as an example, to confirm the treatment effect before and after the patient's treatment and to determine whether the lesion area has completely resolved after treatment, corresponding breast images will be taken for comparison before and after the treatment. It is understood that after taking images of the patient's breast area, the resulting breast images include images of both breasts. This technical solution will be explained in detail using a unilateral breast lesion as an example.

[0037] To fully utilize breast images of patients as training samples for the model, this technical solution collects breast images before and after treatment for each patient as training samples. These training samples include a pre-treatment breast lesion image, a first breast reference image corresponding to the pre-treatment lesion image, a post-treatment breast lesion image to be compared, and a second breast reference image corresponding to the post-treatment lesion image. The first breast reference image is an image without breast lesions, and the second breast reference image is an image that has not undergone treatment and does not contain breast lesions.

[0038] It should be noted that the breast lesion image refers to a breast image containing breast lesions before the patient receives treatment. The first breast reference image refers to a breast image without breast lesions before the patient receives treatment, used as a reference image for the breast lesion image to increase training sample data during model training. The breast lesion image to be compared can be understood as the image corresponding to the breast lesion image after the patient receives treatment. It should be noted that if the breast lesion is completely relieved after the patient receives treatment, it will not be included in the breast lesion image to be compared; conversely, if the breast lesion is not completely relieved after treatment, it will still be included in the breast lesion image to be compared. The second breast reference image can be understood as the breast image corresponding to the first breast reference image after the patient receives treatment.

[0039] For example, if a patient's left breast contains a breast lesion, then the image of the left breast before treatment is used as the image of the breast lesion, the image of the right breast before treatment is used as the first breast reference image, the image of the left breast after treatment is used as the image of the breast lesion to be compared, and the image of the right breast after treatment is used as the second breast reference image.

[0040] The advantage of this setup is that traditional methods typically extract only breast lesion images of patients before and after treatment when acquiring training samples. However, this technical solution makes full use of both breast images of patients before and after treatment, and also uses the first and second breast reference images as training sample images to increase the number of training sample images, thus achieving the effect of obtaining more training sample images with the fewest pre- and post-treatment images of patients.

[0041] S120. For each group of training sample images, based on the training sample images and the corresponding diagnostic and treatment association information, generate a vector to be used corresponding to the training sample images.

[0042] Among them, the diagnosis and treatment association information can be understood as the association information of the patient during the diagnosis and treatment process, such as medication information, diagnosis and treatment time information, diagnosis and treatment duration information, diagnosis and treatment suggestions information, and the patient's physical condition information. The vector to be used can be understood as vector information generated based on the training sample image and the corresponding diagnosis and treatment association information.

[0043] Traditional methods for training classification models of breast lesion images typically rely on a large number of training sample images, neglecting the diagnostic and treatment association information associated with these images. In this technical solution, however, the classification model is trained using multi-dimensional data. Specifically, after acquiring at least one set of training sample images corresponding to a patient, each image is associated with relevant diagnostic and treatment information to generate a corresponding vector. This vector is then used to train the classification model, resulting in a more effective classification model.

[0044] Optionally, based on the training sample image and the corresponding diagnostic and treatment association information, a vector to be used corresponding to the training sample image is generated, including: generating a first vector to be used based on a breast lesion image and the first random diagnostic and treatment information corresponding to the breast lesion image; generating a second vector to be used based on a first breast reference image and the second random diagnostic and treatment information corresponding to the first breast reference image; generating a third vector to be used based on a second breast reference image and the third random diagnostic and treatment information corresponding to the second breast reference image; generating a fourth vector to be used based on a breast lesion image to be compared and the actual diagnostic and treatment information corresponding to the breast lesion image to be compared; and using the first vector to be used, the second vector to be used, the third vector to be used, and the fourth vector to be used as the vector to be used.

[0045] In practical applications, when diagnosing and treating patients, it is usually only necessary to treat the breast side containing the lesion, not the breast side without the lesion. Therefore, for each set of training sample images, to assign corresponding diagnostic and treatment association information to each image, the breast lesion image is associated with first random diagnostic and treatment information, the first breast reference image is associated with second random diagnostic and treatment information, and the second breast reference image is associated with third random diagnostic and treatment information. The first, second, and third random diagnostic and treatment information can be pre-set default diagnostic and treatment information, rather than the actual diagnostic and treatment information used when treating the patient. For the breast lesion image to be compared after the patient's visit, since the comparison image corresponds to the breast side containing the lesion, the diagnostic and treatment association information associated with it is the actual diagnostic and treatment information.

[0046] Furthermore, a first vector to be used is generated based on the breast lesion image and the first random diagnosis and treatment information; a second vector to be used is generated based on the first breast reference image and the second random diagnosis and treatment information; a third vector to be used is generated based on the second breast reference image and the third random diagnosis and treatment information; and a fourth vector to be used is generated based on the breast lesion image to be compared and the actual diagnosis and treatment information. Based on this, the classification model is trained using the first, second, third, and fourth vectors to be used for each patient, resulting in a classification model with better classification performance.

[0047] S130. The initial classification model is trained based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model.

[0048] The initial classification model refers to the default classification model that has not been trained, such as a deep learning neural network model. The target classification model refers to the classification model trained on the initial classification model based on the vectors corresponding to multiple sets of training sample images.

[0049] In this technical solution, in order to obtain a target classification model with better classification performance, the target classification model can be obtained by training the initial classification model based on the vectors to be used corresponding to multiple sets of training sample images.

[0050] Optionally, after obtaining the target classification model, the image of the breast lesion to be identified corresponding to the target diagnosis and treatment user is obtained, and a vector to be identified is generated based on the image of the breast lesion to be identified and the corresponding actual diagnosis and treatment information; vector analysis is performed on the vector to be identified based on the target classification model to obtain the image classification result corresponding to the image of the breast lesion to be identified.

[0051] In this context, the target treatment user refers to the user whose lesion results require classification based on a target classification model. For example, if a user has breast cancer, after treatment, by taking breast images of the user before and after treatment, and classifying the lesion areas based on the target classification model, this user can be identified as the target treatment user. The breast lesion image to be identified refers to the breast lesion image of the target treatment user after treatment, to be compared with the lesion image. The vector to be identified is a vector generated based on the target user's breast lesion image and the corresponding actual treatment information. The image classification result includes complete recovery or incomplete recovery.

[0052] In practical applications, the process involves acquiring an image of the breast lesion to be identified corresponding to the target patient, retrieving the corresponding actual medical information, and vectorizing both the image and the information to be identified to obtain the identification vector. Further, a probability analysis is performed on the identification vector based on a target classification model to obtain the probability to be determined. If the probability to be determined is greater than a preset probability threshold, the image classification result corresponding to the breast lesion image is determined to be fully recovered. Conversely, if the probability to be determined is less than the preset probability threshold, the image classification result corresponding to the breast lesion image is determined to be incompletely recovered.

[0053] The technical solution of this invention involves determining at least one set of training sample images, wherein the training sample images include breast lesion images, a first breast reference image, a breast lesion image to be compared, and a second breast reference image. At least one pre-treatment and post-treatment breast image of a user to be trained is acquired. Breast lesion images in the pre-treatment breast image are determined through manual annotation. The first breast reference image is determined based on the symmetrical position of the breast lesion image in the pre-treatment breast image. The breast lesion image to be compared in the post-treatment breast image is determined based on the image position information of the breast lesion image, and a second breast reference image corresponding to the first breast reference image is also determined. For each set of training sample images, a vector to be used is generated based on the training sample image and the corresponding diagnostic and treatment association information. Specifically, a first random diagnostic and treatment information is associated with a breast lesion image to generate a corresponding first vector to be used; a second random diagnostic and treatment information is associated with a first breast reference image to obtain a second vector to be used; a third random diagnostic and treatment information is associated with a second breast reference image to obtain a third vector to be used; and actual diagnostic and treatment information is associated with a breast lesion image to be compared to obtain a fourth vector to be used. The initial classification model is trained based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model. This solves the problem that in image classification models for lesion images, the difficulty in obtaining sample images and the single data dimension of the model training lead to inaccurate image classification results. It achieves the goal of obtaining a large number of sample images based on a small number of pre- and post-treatment images, generating corresponding vectors to be used with the diagnostic and treatment information corresponding to the sample images, and then training the original classification model based on these vectors to obtain a more accurate target classification model.

[0054] Example 2

[0055] Figure 2The flowchart shows a method for determining a classification model according to Embodiment 2 of the present invention. Optionally, the vector to be used corresponding to the training sample image is generated based on the training sample image and the diagnosis and treatment association information corresponding to the training sample image.

[0056] like Figure 2 As shown, the method includes:

[0057] S210. For at least one user to be trained, obtain the pre-treatment breast image and post-treatment breast image corresponding to the current user based on the user identifier of the current user.

[0058] In this context, the users to be trained for diagnosis and treatment can be understood as historical patients whose breast areas contain breast lesions. For example, to train a classification model based on real breast lesion images, it is legal and legitimate to obtain pre-treatment and post-treatment breast images of historical patients from medical institutions. In this case, historical patients can be used as users to be trained for diagnosis and treatment. In this technical solution, taking any user to be trained for diagnosis and treatment as the current user is used as an example. The user identifier can be understood as identifying information corresponding to the current user, such as the user's name or user number.

[0059] Specifically, at least one patient to be trained is identified for training the initial classification model, and pre-treatment and post-treatment breast images corresponding to each patient are retrieved from the data repository according to the user identifier corresponding to each patient. The initial classification model is then trained using the pre-treatment and post-treatment breast images corresponding to each patient.

[0060] S220. Perform image segmentation on the pre-treatment and post-treatment breast images respectively to obtain at least one set of training sample images.

[0061] In practical applications, taking one of the users to be trained as an example, image segmentation is performed on the breast images of the user before and after treatment to obtain a set of corresponding training sample images.

[0062] Optionally, image segmentation is performed on the pre-treatment breast images and the post-treatment breast images to obtain at least one set of training sample images, including: for the pre-treatment breast images, determining the breast lesion images in the pre-treatment breast images and determining the image location information of the breast lesion images in the pre-treatment breast images; determining reference location information corresponding to the image location information from the pre-treatment breast images, and determining a first breast reference image corresponding to the breast lesion images from the pre-treatment breast images based on the reference location information.

[0063] Specifically, for the pre-treatment breast images of the users to be trained, the breast lesion areas in the pre-treatment breast images can be identified through manual annotation, and these areas are used as the breast lesion images. Further, the image position information of the breast lesion image in the pre-treatment breast images is determined based on the image coordinate information. On this basis, a first breast reference image is determined in the pre-treatment breast images based on the reference position information corresponding to the image region symmetrical to the breast lesion image in the pre-treatment breast images.

[0064] For example, using the centerline of the pre-diagnosis breast image as the central axis, if the image region defined by the horizontal coordinates (x1, x2) and vertical coordinates (y1, y2) is determined as the breast lesion region based on manual annotation, then reference location information can be determined based on the image location information of the breast lesion region. The reference location information consists of the horizontal coordinates (-x1, -x2) and the vertical coordinates (-y1, -y2). Based on this, the image region defined by the reference location information is used as the first breast reference image corresponding to the breast lesion image.

[0065] Optionally, image segmentation is performed on the pre-treatment breast image and the post-treatment breast image to obtain at least one set of training sample images, including: for the post-treatment breast image, determining the deformation field to be used between the post-treatment breast image and the pre-treatment breast image; performing registration processing on the pre-treatment breast image and the post-treatment breast image based on the deformation field to be used to obtain a registered image; and performing image segmentation processing on the registered image to obtain at least one training sample image.

[0066] The deformation field to be used can be understood as the registration matrix that registers the pre-treatment breast images and the post-treatment breast images.

[0067] After acquiring the post-treatment breast images corresponding to the users to be trained, the post-treatment breast images can be registered based on the deformation field to make them as similar as possible to the pre-treatment images, resulting in registered images. Further, after registration, the breast lesion image to be compared in the registered images is determined based on the breast lesion image in the pre-treatment breast image, and the second breast reference image is determined based on the first breast reference image in the pre-treatment breast image. Based on this, the breast lesion image to be compared and the second breast reference image obtained from the registered images are used as training sample images.

[0068] Optionally, image segmentation processing is performed on the registered image to obtain at least one training sample image, including: determining a first image region in the registered image corresponding to a breast lesion image, and using the first image region as the breast lesion image to be compared; determining a second image region in the registered image corresponding to a first breast reference image, and using the second image region as a second breast reference image; and using the lesion image to be compared and the second breast reference image as training sample images.

[0069] S230. For each set of training sample images, based on the training sample images and the corresponding diagnostic and treatment association information, generate a vector to be used corresponding to the training sample images.

[0070] S240. The initial classification model is trained based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model.

[0071] In a specific example, such as Figure 3 As shown, breast images of at least one user to be trained were acquired before and after treatment, with the one showing the most significant enhancement being used as the study object. The breast image before treatment was denoted as I. DCE Post-diagnosis breast images are denoted as I. DCE2 Pre-diagnosis breast images I DCE The breast lesion area was manually delineated and marked, and the delineation result was recorded as L. DCE (i.e., breast lesion images). To obtain more accurate breast lesion images, a 3D-UNet neural network can be used to train and iterate the delineation results. Based on the converged model, a large number of pre-treatment breast images of users to be trained can be pre-annotated. The images are then modified based on the patients being trained, resulting in a large number of breast lesion images of users to be trained.

[0072] Furthermore, to compare the image classification results corresponding to breast lesion images after treatment by the training users, the post-treatment MRI images (i.e., post-treatment breast images) were used as templates to classify the pre-treatment MRI images. DCE The image (i.e., the pre-treatment breast image) is registered. That is, a registration matrix (i.e., the deformation field to be used) is learned by machine learning, so that the registered image obtained by applying the registration matrix to the pre-treatment breast image is as similar as possible to the post-treatment breast image. The learned registration matrix is ​​then applied to the delineation matrix in the pre-treatment breast image to obtain the region of interest after treatment.

[0073] Based on this, a dataset for PCR determination is constructed, and image classification is performed on the comparison breast lesion images in the post-treatment breast images. The image classification result includes complete recovery or incomplete recovery. If a breast lesion is present in the comparison breast lesion image, the image classification result is determined to be non-PCR (i.e., incomplete recovery); if no breast lesion is present in the comparison breast lesion image, the corresponding image classification result is PCR (i.e., complete recovery). Therefore, if the region of interest (the registered mask region) on the pre-treatment breast image is labeled according to the actual PCR result and used as the breast lesion image, and the symmetrical region corresponding to the breast lesion image in the pre-treatment breast image is used as the first breast reference image, i.e., PCR=1.

[0074] Furthermore, at least one set of training sample images is obtained based on pre-treatment and post-treatment breast images of at least one user to be trained. Each set of training sample images includes a breast lesion image and a first breast reference image from the pre-treatment breast image, and a breast lesion image to be compared and a second breast reference image from the post-treatment breast image. Based on this, the original classification model is retrieved. The input is divided into two parts: one end is the input of a 3D image, used to input the training sample images of each user; the other end is the diagnosis and treatment association information corresponding to each training sample image. After image convolution and global pooling, the images are input together with the diagnosis and treatment association information into a fully connected layer to obtain the corresponding vector to be used. The original classification model is trained based on the vector to be used for the training sample images to obtain the target classification model. The target classification model is then used to predict the probability of the breast lesion images to be compared for the target user, thus achieving image classification of the breast lesion images to be compared. Specifically, a first vector to be used is obtained based on a breast lesion image and the corresponding first random diagnosis and treatment information; a second vector to be used is obtained based on a first breast reference image and the corresponding second random diagnosis and treatment information; a third vector to be used is obtained based on a second breast reference image and the corresponding third random diagnosis and treatment information; and a fourth vector to be used is obtained based on a breast lesion image to be compared and the corresponding actual diagnosis and treatment information.

[0075] The technical solution of this invention involves determining at least one set of training sample images, wherein the training sample images include breast lesion images, a first breast reference image, a breast lesion image to be compared, and a second breast reference image. At least one pre-treatment and post-treatment breast image of a user to be trained is acquired. Breast lesion images in the pre-treatment breast image are determined through manual annotation. The first breast reference image is determined based on the symmetrical position of the breast lesion image in the pre-treatment breast image. The breast lesion image to be compared in the post-treatment breast image is determined based on the image position information of the breast lesion image, and a second breast reference image corresponding to the first breast reference image is also determined. For each set of training sample images, a vector to be used is generated based on the training sample image and the corresponding diagnostic and treatment association information. Specifically, a first random diagnostic and treatment information is associated with a breast lesion image to generate a corresponding first vector to be used; a second random diagnostic and treatment information is associated with a first breast reference image to obtain a second vector to be used; a third random diagnostic and treatment information is associated with a second breast reference image to obtain a third vector to be used; and actual diagnostic and treatment information is associated with a breast lesion image to be compared to obtain a fourth vector to be used. The initial classification model is trained based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model. This solves the problem that in image classification models for lesion images, the difficulty in obtaining sample images and the single data dimension of the model training lead to inaccurate image classification results. It achieves the goal of obtaining a large number of sample images based on a small number of pre- and post-treatment images, generating corresponding vectors to be used with the diagnostic and treatment information corresponding to the sample images, and then training the original classification model based on these vectors to obtain a more accurate target classification model.

[0076] Example 3

[0077] Figure 4 This is a schematic diagram of a classification model determination device provided in Embodiment 3 of the present invention. Figure 4 As shown, the device includes: a sample image determination module 310, a vector generation module 320, and a model determination module 330.

[0078] The sample image determination module 310 is used to determine at least one set of training sample images. The training sample images include a breast lesion image before diagnosis and treatment, a first breast reference image corresponding to the breast lesion image, a breast lesion image to be compared after diagnosis and treatment, and a second breast reference image corresponding to the breast lesion image to be compared. The first breast reference image is an image that does not contain breast lesions, and the second breast reference image is an image that has not undergone diagnosis and treatment and does not contain breast lesions.

[0079] The vector generation module 320 is used to generate a vector to be used corresponding to the training sample image for each group of training sample images, based on the training sample image and the diagnosis and treatment association information corresponding to the training sample image.

[0080] The model determination module 330 is used to train the initial classification model based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model.

[0081] The technical solution of this invention involves determining at least one set of training sample images, wherein the training sample images include breast lesion images, a first breast reference image, a breast lesion image to be compared, and a second breast reference image. At least one pre-treatment and post-treatment breast image of a user to be trained is acquired. Breast lesion images in the pre-treatment breast image are determined through manual annotation. The first breast reference image is determined based on the symmetrical position of the breast lesion image in the pre-treatment breast image. The breast lesion image to be compared in the post-treatment breast image is determined based on the image position information of the breast lesion image, and a second breast reference image corresponding to the first breast reference image is also determined. For each set of training sample images, a vector to be used is generated based on the training sample image and the corresponding diagnostic and treatment association information. Specifically, a first random diagnostic and treatment information is associated with a breast lesion image to generate a corresponding first vector to be used; a second random diagnostic and treatment information is associated with a first breast reference image to obtain a second vector to be used; a third random diagnostic and treatment information is associated with a second breast reference image to obtain a third vector to be used; and actual diagnostic and treatment information is associated with a breast lesion image to be compared to obtain a fourth vector to be used. The initial classification model is trained based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model. This solves the problem that in image classification models for lesion images, the difficulty in obtaining sample images and the single data dimension of the model training lead to inaccurate image classification results. It achieves the goal of obtaining a large number of sample images based on a small number of pre- and post-treatment images, generating corresponding vectors to be used with the diagnostic and treatment information corresponding to the sample images, and then training the original classification model based on these vectors to obtain a more accurate target classification model.

[0082] Optionally, the sample image determination module includes: an image acquisition submodule, used to acquire, for at least one training user, a pre-treatment breast image and a post-treatment breast image corresponding to the current user based on the user identifier of the current user;

[0083] The sample image determination submodule is used to perform image segmentation on pre-diagnosis and post-diagnosis breast images to obtain at least one set of training sample images.

[0084] Optionally, the sample image determination submodule includes: a location information determination unit, used to determine the breast lesion image in the pre-diagnosis breast image and determine the image location information of the breast lesion image in the pre-diagnosis breast image;

[0085] The first reference image determination unit is used to determine reference position information corresponding to the image position information from the pre-diagnosis breast image, and to determine a first breast reference image corresponding to the breast lesion image from the pre-diagnosis breast image based on the reference position information.

[0086] The first sample image determination unit is used to use the breast lesion image and the first breast reference image as training sample images.

[0087] Optionally, the sample image determination submodule includes: a deformation field determination unit, used to determine the deformation field to be used for post-treatment breast images and pre-treatment breast images;

[0088] The registered image determination unit is used to perform registration processing on the pre-diagnosis breast image and the post-diagnosis breast image based on the deformation field to be used, so as to obtain the registered image;

[0089] The second sample image determination unit is used to perform image segmentation processing on the registered image to obtain at least one training sample image.

[0090] Optionally, the second sample image determination unit includes: a lesion image determination subunit, used to determine a first image region in the registered image that corresponds to the breast lesion image, and to use the first image region as the breast lesion image to be compared;

[0091] The reference image determination subunit is used to determine the second image region in the registered image that corresponds to the first breast reference image, and to use the second image region as the second breast reference image.

[0092] The second sample image determination subunit is used to use the lesion image to be compared and the second breast reference image as training sample images.

[0093] Optionally, the vector generation module includes: a first vector determination submodule, used to generate a first vector to be used based on a breast lesion image and first random diagnostic information corresponding to the breast lesion image;

[0094] The second vector determination submodule is used to generate a second vector to be used based on the first breast reference image and the second random diagnostic information corresponding to the first breast reference image;

[0095] The third vector determination submodule is used to generate a third vector to be used based on the second breast reference image and the third random diagnosis and treatment information corresponding to the second breast reference image.

[0096] The fourth vector determination submodule is used to generate a fourth vector to be used based on the breast lesion image to be compared and the actual diagnosis and treatment information corresponding to the breast lesion image to be compared.

[0097] The submodule for determining the vector to be used is used to select the first vector to be used, the second vector to be used, the third vector to be used, and the fourth vector to be used as the vector to be used.

[0098] Optionally, the model determination module includes: a target vector generation module, used to acquire a target breast lesion image corresponding to the target treatment user, and generate a target vector based on the target breast lesion image and the corresponding actual treatment information; wherein, the target breast lesion image refers to the target breast lesion image to be compared after treatment.

[0099] The classification module is used to perform vector analysis on the vector to be identified based on the target classification model, and obtain the image classification result corresponding to the image of the breast lesion to be identified; the image classification result includes complete recovery or incomplete recovery.

[0100] The classification model determination device provided in the embodiments of the present invention can execute the classification model determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0101] Example 4

[0102] Figure 5 A schematic diagram of the structure of an electronic device 10 according to an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0103] like Figure 5As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0104] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0105] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as methods for determining classification models.

[0106] In some embodiments, the method for determining the classification model may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method for determining the classification model described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the method for determining the classification model by any other suitable means (e.g., by means of firmware).

[0107] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] Computer programs used to implement the classification model determination method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0109] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0110] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0111] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0112] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0113] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0114] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for determining a classification model, characterized in that, include: At least one set of training sample images is determined; wherein, the training sample images include a breast lesion image before diagnosis and treatment, a first breast reference image symmetrical to the breast lesion image, a breast lesion image to be compared after diagnosis and treatment obtained by image registration of the breast lesion image using a deformation field to be used, and a second breast reference image obtained by image registration of the first breast reference image using a deformation field to be used. Based on the reference position information corresponding to the image region symmetrical to the breast lesion image in the breast image before diagnosis and treatment, the first breast reference image in the breast image before diagnosis and treatment is determined. The first breast reference image is an image that does not contain breast lesions, and the second breast reference image is an image that has not undergone diagnosis and treatment and does not contain the breast lesions. For each set of training sample images, a vector to be used corresponding to the training sample image is generated based on the training sample image and the diagnosis and treatment association information corresponding to the training sample image. The initial classification model is trained based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model. The step of generating a vector to be used corresponding to the training sample image based on the training sample image and the corresponding diagnostic and treatment association information includes: generating a first vector to be used based on the breast lesion image and the first random diagnostic and treatment information corresponding to the breast lesion image; generating a second vector to be used based on the first breast reference image and the second random diagnostic and treatment information corresponding to the first breast reference image; generating a third vector to be used based on the second breast reference image and the third random diagnostic and treatment information corresponding to the second breast reference image; generating a fourth vector to be used based on the breast lesion image to be compared and the actual diagnostic and treatment information corresponding to the breast lesion image to be compared; and using the first vector to be used, the second vector to be used, the third vector to be used, and the fourth vector to be used as the vector to be used.

2. The method according to claim 1, characterized in that, Determining at least one set of training sample images includes: For at least one user to be trained for diagnosis and treatment, obtain the pre-diagnosis breast image and post-diagnosis breast image corresponding to the current user based on the user identifier of the current user; Image segmentation is performed on the pre-treatment breast images and the post-treatment breast images to obtain at least one set of training sample images.

3. The method according to claim 2, characterized in that, The process involves image segmentation of the pre-treatment and post-treatment breast images to obtain at least one set of training sample images, including: For the pre-diagnosis breast image, determine the breast lesion image in the pre-diagnosis breast image, and determine the image location information of the breast lesion image in the pre-diagnosis breast image; Determine reference location information corresponding to the image location information from the pre-diagnosis breast image, and determine a first breast reference image corresponding to the breast lesion image from the pre-diagnosis breast image based on the reference location information; The breast lesion image and the first breast reference image are used as the training sample images.

4. The method according to claim 2, characterized in that, The process involves image segmentation of the pre-treatment and post-treatment breast images to obtain at least one set of training sample images, including: For the post-treatment breast image, determine the deformation field to be used between the post-treatment breast image and the pre-treatment breast image; Based on the deformation field to be used, the pre-treatment breast image and the post-treatment breast image are registered to obtain a registered image; The registered image is segmented to obtain at least one training sample image.

5. The method according to claim 4, characterized in that, The step of performing image segmentation processing on the registered image to obtain at least one training sample image includes: A first image region in the registered image that corresponds to the breast lesion image is determined, and the first image region is used as the breast lesion image to be compared. Determine a second image region in the registered image that corresponds to the first breast reference image, and use the second image region as the second breast reference image; The lesion image to be compared and the second breast reference image are used as the training sample images.

6. The method according to claim 1, characterized in that, Also includes: Acquire the breast lesion image to be identified corresponding to the target treatment user, and generate the identification vector based on the breast lesion image to be identified and the corresponding actual treatment information; wherein, the breast lesion image to be identified refers to the breast lesion image to be compared after the target treatment user's treatment; Based on the target classification model, vector analysis is performed on the vector to be identified to obtain the image classification result corresponding to the image of the breast lesion to be identified; wherein, the image classification result includes complete recovery or incomplete recovery.

7. A device for determining a classification model, characterized in that, include: A sample image determination module is used to determine at least one set of training sample images; wherein, the training sample images include a breast lesion image before diagnosis and treatment, a first breast reference image symmetrical to the breast lesion image, a breast lesion image to be compared after diagnosis and treatment obtained by image registration of the breast lesion image using a deformation field to be used, and a second breast reference image obtained by image registration of the first breast reference image using a deformation field to be used. Based on the reference position information corresponding to the image region symmetrical to the breast lesion image in the breast image before diagnosis and treatment, the first breast reference image in the breast image before diagnosis and treatment is determined. The first breast reference image is an image that does not contain breast lesions, and the second breast reference image is an image that has not undergone diagnosis and treatment and does not contain the breast lesions. The vector generation module is used to generate a vector to be used corresponding to each group of training sample images, based on the training sample images and the corresponding diagnosis and treatment association information; The model determination module is used to train the initial classification model based on the vectors to be used corresponding to multiple sets of training sample images to obtain the target classification model. The vector generation module includes: a first vector determination submodule, used to generate a first vector to be used based on the breast lesion image and the first random diagnostic and treatment information corresponding to the breast lesion image; a second vector determination submodule, used to generate a second vector to be used based on the first breast reference image and the second random diagnostic and treatment information corresponding to the first breast reference image; a third vector determination submodule, used to generate a third vector to be used based on the second breast reference image and the third random diagnostic and treatment information corresponding to the second breast reference image; a fourth vector determination submodule, used to generate a fourth vector to be used based on the breast lesion image to be compared and the actual diagnostic and treatment information corresponding to the breast lesion image to be compared; and a vector to be used determination submodule, used to use the first vector to be used, the second vector to be used, the third vector to be used, and the fourth vector to be used as the vector to be used.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method for determining the classification model according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the method for determining the classification model according to any one of claims 1-6.

Images