Method for identifying target in medical image and electronic equipment

Abstract

The invention discloses a method for identifying a target in a medical image and electronic equipment. The method for identifying the target in the medical image comprises the following steps: acquiring the medical image serving as an input image; the prediction result of each trained model for the input image is determined by using at least one trained model, a target prediction result is obtained based on the prediction result of the at least one trained model, the at least one trained model is obtained by training a to-be-trained model, and the target prediction result is obtained based on the prediction result of the at least one trained model. The training set of the to-be-trained model is constructed by a training sample and a region segmentation result corresponding to a labeled region of the training sample, and the region segmentation result is obtained by carrying out under-segmentation on image data in the labeled region. Therefore, the small target can be effectively identified.

Description

[0001] This application is a divisional application of a patent application with an application date of March 2, 2022, application number 2022102054674, and invention name "Model training, method, device and medium for identifying targets in medical images." Technical Field

[0002] The present disclosure relates to the field of image processing based on artificial intelligence, and in particular to a method and electronic device for identifying targets in medical images. Background Art

[0003] In recent years, artificial intelligence technology has made great achievements in the field of computer vision. For example, deep learning technology is increasingly used in semantic segmentation, image classification, and object recognition. Especially in the medical field, it is often necessary to segment, identify, or classify objects in medical images to assist in the analysis of the objects.

[0004] At present, deep learning target recognition technology can achieve high recognition accuracy for large-sized targets, but the recognition performance for small targets (such as thin objects or small objects) is unsatisfactory, which easily leads to missed reports and false alarms, and it is also difficult to distinguish the categories of small targets. For example, in fundus images, small target signs such as punctate hemorrhages and microvascular tumors are not easy to find or distinguish when deep learning is used for target recognition because the targets are small, light in color, and close in color. Therefore, how to effectively recognize small targets remains to be studied. Summary of the invention

[0005] The present disclosure is proposed in view of the above-mentioned state of the prior art, and its purpose is to provide a model training, method, device and medium for identifying targets in medical images that can effectively identify small targets.

[0006] To this end, the first aspect of the present disclosure provides a model training method for identifying targets in medical images, comprising: obtaining the medical image as a training sample and the annotated area corresponding to the target in the training sample; determining the region segmentation result corresponding to the annotated area, and constructing a training set using the training sample and the region segmentation result, wherein the region segmentation result is obtained by under-segmenting the image data in the annotated area; and training a model to be trained based on the training set, and optimizing the model to be trained using a training loss function, wherein in the training loss function, spatial weights are used to reduce the negative impact of pixels in a first area in the training sample on the model to be trained, the first area being an area other than a target area of ​​the target in the annotated area in the training sample, and the target area being determined by the region segmentation result. In this case, by under-segmenting the image data in the annotated area in the training sample to identify pixels of undetermined categories in the annotated area, and combining the spatial weights to train the model to be trained to reduce the negative impact of pixels of undetermined categories in the annotated area on the model to be trained, the accuracy of the prediction results of the input image by the trained model to be trained can be improved. Thus, small targets can be effectively identified.

[0007] In addition, in the model training method involved in the first aspect of the present disclosure, optionally, obtaining the region segmentation result further includes: obtaining the image data to be segmented based on the image data corresponding to the annotated region in the training sample, or obtaining the image data to be segmented based on the image data corresponding to the annotated region in the training sample and the image data corresponding to the annotated region in the segmentation result of interest, wherein the segmentation result of interest is a binary image for identifying the region of interest of the training sample; and performing threshold segmentation on the image data to be segmented using a target segmentation threshold, thereby obtaining the region segmentation result, wherein the region segmentation result is a binary image. In this case, the target region in the image data to be segmented can be identified by threshold segmentation, and when the annotated region includes a region outside the region of interest, the noise outside the region of interest can be eliminated.

[0008] In addition, in the model training method involved in the first aspect of the present disclosure, optionally, the target segmentation threshold is obtained according to the threshold acquisition method of the annotation category to which the target belongs, wherein the threshold acquisition method of each annotation category is determined by the average area and average color of each annotation category, and the threshold acquisition method includes a first method and a second method, wherein the average area of ​​the annotation category corresponding to the first method is greater than the average area of ​​the annotation category corresponding to the second method and the average color of the annotation category corresponding to the first method is lighter than the average color of the annotation category corresponding to the second method; for the first method, a threshold is searched so that the area of ​​pixels with gray values ​​greater than the threshold in the image data to be segmented is less than a preset multiple of the area of ​​the image data to be segmented, and the threshold is used as the target segmentation threshold, wherein the preset multiple is greater than 0 and less than 1; for the second method, if the length of the smallest side of the image data to be segmented is less than the preset length, the average gray value of the pixels in the image data to be segmented is taken as the target segmentation threshold, otherwise the target segmentation threshold is determined based on the gray values ​​of the four corners and the central area of ​​the image data to be segmented. In this case, the target segmentation threshold can be obtained according to the characteristics of the annotation category corresponding to the target. Thereby, the accuracy of threshold segmentation can be improved.

[0009] In addition, in the model training method involved in the first aspect of the present disclosure, optionally, before obtaining the region segmentation result: an erosion operation is further performed on the threshold segmentation result of the image data to be segmented to obtain at least one connected region, and the connected region whose center is closest to the center of the image data to be segmented is selected from the at least one connected region as the region segmentation result. In this way, an accurate target region can be obtained.

[0010] In addition, in the model training method involved in the first aspect of the present disclosure, optionally, in the spatial weight, the pixels of the first area in the training sample are assigned a first weight, wherein the first weight is 0. In this case, samples of undetermined categories can be ignored to reduce the negative impact of samples of undetermined categories on the model to be trained.

[0011] In addition, in the model training method involved in the first aspect of the present disclosure, optionally, the pixels of the first area, the second area, the third area and the fourth area in the training sample are respectively assigned a first weight, a second weight, a third weight and a fourth weight, wherein the second area is the target area, the third area is an area within the region of interest that does not belong to the annotated area, the fourth area is an area outside the region of interest, the first weight is less than the second weight and less than the third weight, and the fourth weight is less than the second weight and less than the third weight. In this case, the negative impact of pixels of undetermined categories and pixels outside the region of interest on the model to be trained can be suppressed, and the positive impact of non-target areas within the target area and the region of interest on the model to be trained can be increased. In this way, the accuracy of the model can be improved.

[0012] In addition, in the model training method involved in the first aspect of the present disclosure, optionally, the model to be trained is a semantic segmentation model, and the prediction result of the model to be trained is the semantic segmentation result of the training sample. Thus, small objects can be identified.

[0013] In addition, in the model training method involved in the first aspect of the present disclosure, optionally, the shape of the labeled area is a rectangle, thereby reducing the difficulty of labeling.

[0014] A second aspect of the present disclosure provides an electronic device, which includes: at least one processing circuit, and the at least one processing circuit is configured to execute the steps of the model training method described in the first aspect of the present disclosure.

[0015] A third aspect of the present disclosure provides a computer-readable storage medium, which stores at least one instruction, and when the at least one instruction is executed by a processor, the steps of the above-mentioned model training method are implemented.

[0016] The fourth aspect of the present disclosure provides a method for identifying targets in a medical image, the method comprising: acquiring the medical image as an input image; and using at least one trained model trained by the model training method described in the first aspect of the present disclosure to determine the prediction results of each trained model for the input image, and obtaining a target prediction result based on the prediction result of the at least one trained model.

[0017] In addition, in the method involved in the fourth aspect of the present disclosure, optionally, the prediction results of each trained model include the probability that each pixel in the input image belongs to the corresponding labeled category, the prediction results of the at least one trained model are integrated according to the labeled category and pixel to obtain the integrated probability that each pixel of the input image belongs to the corresponding labeled category, the connected area is determined based on the integrated probability, and the target prediction results corresponding to each labeled category are obtained based on the connected area, wherein if there is only one trained model, the probability is used as the integrated probability, otherwise the prediction results of multiple trained models are averaged to obtain the average probability that each pixel in the input image belongs to the corresponding labeled category and use it as the integrated probability. In this case, obtaining the target prediction result based on the integrated probability can further improve the accuracy of the target prediction result.

[0018] In addition, in the method involved in the fourth aspect of the present disclosure, optionally, the medical image is a fundus image. In this case, the model obtained after training can recognize small targets in the fundus image.

[0019] In addition, in the method involved in the fourth aspect of the present disclosure, optionally, the target includes microvascular tumors, punctate hemorrhages, sheet hemorrhages and linear hemorrhages. In this case, the model obtained after training can identify small targets in fundus images.

[0020] A fifth aspect of the present disclosure provides an electronic device, comprising: at least one processing circuit, wherein the at least one processing circuit is configured to: obtain the medical image as an input image; and determine the prediction results of each trained model for the input image using at least one trained model trained by the model training method described in the first aspect of the present disclosure, and obtain a target prediction result based on the prediction result of the at least one trained model.

[0021] According to the present disclosure, a model training, method, device and medium for identifying targets in medical images are provided, which can effectively identify small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present disclosure will now be explained in further detail by way of example only with reference to the accompanying drawings, in which:

[0023] Figure 1 is a schematic diagram showing an example of identifying a target environment involved in the examples of the present disclosure.

[0024] Figure 2 is a flowchart showing an example of a model training method involved in the examples of the present disclosure.

[0025] Figure 32 is a schematic diagram showing some example annotation areas involved in the examples of the present disclosure.

[0026] Figure 4 Schematic diagram showing the region segmentation results of some examples involved in the examples of the present disclosure.

[0027] Figure 5 : is a flowchart showing an example of obtaining a region segmentation result according to an example of the present disclosure.

[0028] Figure 6 is an architectural diagram showing an example of a model to be trained using a U-Net architecture involved in the examples of the present disclosure.

[0029] Figure 7 is a schematic diagram showing several areas of some examples involved in the examples of the present disclosure.

[0030] Figure 8 is a flowchart illustrating an example of a method for identifying an object in an image according to an example of the present disclosure. DETAILED DESCRIPTION

[0031] Hereinafter, with reference to the accompanying drawings, the preferred embodiments of the present disclosure are described in detail. In the following description, the same symbols are given to the same parts, and repeated descriptions are omitted. In addition, the accompanying drawings are only schematic diagrams, and the ratio of the sizes of the parts to each other or the shapes of the parts, etc. may be different from the actual ones. It should be noted that the terms "including" and "having" in the present disclosure and any variations thereof, such as the processes, methods, systems, products or devices of a series of steps or units included or possessed are not necessarily limited to those steps or units clearly listed, but may include or have other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. All methods described in the present disclosure may be performed in any suitable order, unless otherwise indicated herein or clearly contradictory to the context.

[0032] The term "circuit" in this document may refer to a hardware circuit and / or a combination of a hardware circuit and software. In the present disclosure, the term "model" is capable of processing inputs and providing corresponding outputs. In this document, the terms "neural network", "deep neural network", "model", "network" and "neural network model" are used interchangeably. In addition, when referring to the rectangular properties (e.g., sides, width, height, width and height) of related objects (e.g., annotated areas, image data to be segmented, and targets), if the object itself is not a rectangle, unless otherwise specified, it can be defaulted to the rectangular properties of the circumscribed rectangle of the object.

[0033] The existing deep learning target recognition scheme uses a labeling method of various shapes of frame marks (that is, a labeling method that does not require accurate boundaries) to identify small targets. However, as briefly mentioned above, the recognition effect of this scheme on small targets in images is not satisfactory, and there is a large risk of missed reports and false alarms. This is because the area of ​​small targets is small, there are few extractable features, and they are easily interfered by noise and other tissues. Therefore, a better solution is to segment small targets by deep learning target segmentation to achieve recognition of small targets. However, this scheme requires accurate labeling of the boundaries of small targets, which makes image labeling difficult. In order to overcome the shortcomings of the above two schemes, the present disclosure obtains regional segmentation results by under-segmenting the labeled area, and uses the results as the gold standard to segment the image, thereby achieving accurate recognition of small targets. In particular, the present disclosure uses a spatial weight method to deal with the negative impact of pixels of undetermined categories on image segmentation caused by under-segmentation in the labeled area. In this case, small targets can be effectively identified.

[0034] Therefore, the examples of the present disclosure propose a training model and a solution for identifying targets in an image to solve one or more of the above problems and / or other potential problems. The solution uses an image segmentation method to perform target recognition (that is, first perform image segmentation on the image data in the labeled area in the training sample to obtain a region segmentation result, and then post-process the region segmentation result to obtain a target recognition result). Specifically, the solution performs under-segmentation on the image data in the labeled area in the training sample to identify pixels of undetermined categories in the labeled area, and trains the neural network model in combination with spatial weights (that is, the setting of weights can be related to the position of pixels) to reduce the negative impact of pixels of undetermined categories in the labeled area on the neural network model, thereby improving the accuracy of the prediction results of the trained model for the input image (for example, a medical image). In addition, the trained model can be a trained neural network model (that is, a trained neural network model. For example, a trained semantic segmentation model. Thus, the performance of the trained model can be optimized and the accuracy of the model in recognizing small targets can be improved. In some examples, the trained model can be the optimal neural network model obtained after training.

[0035] The examples disclosed herein involve training models and schemes for identifying targets in images, which effectively identify small targets. The model training method for identifying targets in images involved in the examples disclosed herein can be referred to as a model training method or a training method. It should be noted that the schemes involved in the examples disclosed herein are also applicable to the identification of large targets.

[0036] The images involved in the examples of the present disclosure may come from a camera, a CT scan, a PET-CT scan, a SPECT scan, an MRI, an ultrasound, an X-ray, an angiogram, a fluorescence image, an image taken by a capsule endoscope, or a combination thereof. In some examples, the image may be a medical image. For example, the medical image may include, but is not limited to, a fundus image, a lung image, a stomach image, a chest image, and a brain image. Thus, small targets in the medical image can be identified. In some examples, the image may be a natural image. A natural image may be an image observed or photographed in a natural scene. Thus, small targets in a natural image can be identified. The following describes an example of the present disclosure using the image as a fundus image in a medical image as an example, and such a description does not limit the scope of the present disclosure. For those skilled in the art, other types of images may be used without limitation.

[0037] The examples of the present disclosure will be described in detail below in conjunction with the accompanying drawings. For ease of understanding, the specific data mentioned in the following description are exemplary and are not intended to limit the scope of protection of the present disclosure. It should be understood that according to the examples of the present disclosure, additional modules not shown may be included, the modules shown may be omitted, additional actions not shown, and / or the actions shown may be omitted, and the scope of the present disclosure is not limited in this respect.

[0038] Figure 1 1 is a schematic diagram showing an example of identifying a target environment 100 according to an example of the present disclosure. Figure 1 As shown, the target environment 100 may include a computing device 110. The computing device 110 may be any device with computing capabilities. For example, the computing device 110 may be a cloud server, a personal computer, a mainframe, a distributed computing system, and the like.

[0039] The computing device 110 can obtain input 120 and generate output 140 corresponding to the input 120 using a neural network model 130 (sometimes also referred to as a model to be trained 130 or model 130). In some examples, the input 120 can be the above-mentioned image, and the output 140 can be a prediction result, a training parameter (e.g., weight), or a performance indicator (e.g., accuracy and error rate). In some examples, the neural network model 130 can include but is not limited to a semantic segmentation model (e.g., U-Net), or other models related to image processing. In addition, the neural network model 130 can be implemented using any suitable network structure. For example, a convolutional neural network (CNN), a recurrent neural network (RNN), and a deep neural network (DNN).

[0040] In some examples, the target recognition environment 100 may also include a model training device and a model application device (not shown). The model training device may be used to implement a training method for training the neural network model 130 to obtain a trained model. The model application device may be used to implement a related method for obtaining a prediction result using the trained model to recognize the target in the image. In addition, in the model training stage, the neural network model 130 may be a model to be trained 130. In the model application stage, the neural network model 130 may be a trained model.

[0041] Figure 2 is a flowchart showing an example of a model training method involved in the examples of the present disclosure. For example, the model training method can be Figure 1 The method may be performed by the computing device 110 shown. In addition, the model training method may train a model for recognizing objects in medical images.

[0042] like Figure 2 As shown, the model training method may include step S102. In step S102, a medical image as a training sample and a marked area corresponding to a target in the training sample may be obtained. That is, in the training stage, a medical image may be obtained as a training sample. Thus, the target in the medical image can be identified. In some examples, the medical image may be a color image. Thus, the accuracy of identifying small targets can be improved.

[0043] In addition, the medical image may contain corresponding targets, and the targets may belong to at least one category of interest (i.e., the category to be identified). In some examples, for the medical image being a fundus image, the targets may include small targets such as microaneurysms, punctate hemorrhages, sheet hemorrhages, and linear hemorrhages. In this case, the model obtained after training can identify small targets in fundus images.

[0044] Figure 3 2 is a schematic diagram showing some example annotation areas involved in the examples of the present disclosure.

[0045] In some examples, the target in the training sample can be annotated to obtain an annotated area. In addition, the shape of the annotated area can be a rectangle, a circle, or a shape that matches the shape of the target in the training sample (for example, the shape of the annotated area can be the outline of the target). Preferably, the shape of the annotated area can be a rectangle. Thus, the difficulty of annotation can be reduced. As an example, Figure 3 The marked area D1 in the fundus image is shown, wherein the shape of the marked area D1 is a rectangle, and the target in the marked area D1 is a sheet-like hemorrhage.

[0046] In addition, the annotated area may have a corresponding annotation label (that is, the annotation category of the target), which can be used to distinguish the category of the target. The annotation category may correspond to the category of the target one by one. For example, for fundus images, the category of the target and the annotation category may include but are not limited to microaneurysm, punctate hemorrhage, sheet hemorrhage, and linear hemorrhage, etc. In some examples, the corresponding annotation category may be represented by a number. Thus, it is convenient for the computing device 110 to perform calculations. In addition, the annotated area and the corresponding annotation label may be referred to as an annotation result.

[0047] like Figure 2 As shown, the model training method may further include step S104. In step S104, the region segmentation result (also referred to as pseudo segmentation result) corresponding to the annotated region in the training sample may be determined, and a training set may be constructed using the training sample and the region segmentation result. It should be noted that in other examples, it is not necessary to determine the region segmentation result corresponding to the annotated region, as long as the target region (described later) in the annotated region can be identified and the pixels in the target region are determined to belong to the target.

[0048] In some examples, according to actual conditions (for example, the quality of training samples does not meet the training requirements or the sizes of training samples are not uniform), before constructing the training set, the training samples may be preprocessed accordingly and then used to construct the training set.

[0049] In some examples, preprocessing the training samples may include unifying the size of the training samples. For example, the size of the training samples may be unified to 1024×1024 or 2048×2048. The present disclosure does not limit the size of the training samples. In some examples, preprocessing the training samples may include cropping the training samples. In some examples, in cropping the training samples, a region of interest in the training samples may be obtained and the training samples may be cropped using the region of interest. Thus, the size of the training samples can be made consistent and include the region of interest. In some examples, the region of interest may be an area where a target may exist (also referred to as a foreground area). For example, for a fundus image, the region of interest may be a fundus area.

[0050] In some examples, the training samples may be segmented to obtain regions of interest. In some examples, the training samples may be threshold segmented to obtain segmentation results of interest, wherein the segmentation results of interest may be used to identify the regions of interest of the training samples. Thus, the regions of interest may be identified. In addition, the segmentation results of interest obtained by threshold segmentation may be binary images (also referred to as binarized images). It is to be understood that, although the segmentation results of interest are obtained by threshold segmentation above, other methods suitable for obtaining segmentation results of interest are also applicable. For example, the segmentation results of interest may be obtained by means of a neural network.

[0051] In some examples, performing threshold segmentation on the training sample may be dividing the training sample into a preset number of parts (e.g., 9 equal parts), determining the segmentation threshold based on the grayscale values ​​of the four corner regions and the central region of the training sample, performing threshold segmentation on the training sample based on the segmentation threshold, and thereby obtaining the segmentation result of interest. In some examples, determining the segmentation threshold based on the grayscale values ​​of the four corner regions and the central region of the training sample may be taking the average of the grayscale mean of the pixels in each region of the four corner regions and the grayscale mean of the pixels in the middle region as the segmentation threshold for performing threshold segmentation, and thereby obtaining the segmentation result of interest.

[0052] In addition, in the threshold segmentation, before obtaining the segmentation result of interest, the threshold segmentation result corresponding to the training sample (that is, the initial segmentation result) can be eroded to obtain the segmentation result of interest. For example, the threshold segmentation result of the training sample can be eroded twice to obtain the segmentation result of interest, wherein the size of the erosion kernel can be 5. In this way, the noise at the edge of the region of interest (such as the fundus region) can be eliminated.

[0053] Return to reference Figure 2 As described above, in step S104, the region segmentation result corresponding to the annotated region can be determined. In addition, the region segmentation result can be used to determine the target region of the target in the annotated region. In this case, the target region in the annotated region can be identified, and then the pixels of undetermined category can be determined based on the target region. Specifically, the pixels outside the target region in the annotated region in the training sample can be pixels of undetermined category.

[0054] In addition, the region segmentation result may be any form of data (e.g., an image) that can identify the target region. In some examples, the region segmentation result may be a binary image. In some examples, for the region segmentation result of a binary image, the region corresponding to the pixel with a value of 1 may be the target region (that is, if the pixel value is 1, it may indicate that the pixel at the corresponding position in the training sample belongs to the target, and if the pixel value is 0, it may indicate that the pixel at the corresponding position in the training sample is a pixel of an undetermined category). In this case, the negative impact of pixels of undetermined category on the training model 130 to be trained can be reduced.

[0055] Figure 4 Schematic diagram showing the region segmentation results of some examples involved in the examples of the present disclosure.

[0056] As an example, Figure 4 Shows Figure 3 The region segmentation result A1 corresponds to the marked region D1 in the figure, where D2 is the target region. In addition, in order to make the region segmentation result A1 more clearly displayed, Figure 4 The region segmentation result A1 is a result of proportional enlargement and does not limit the present disclosure. Figure 4 The region segmentation result A1 can actually be consistent with the size of the marked region D1.

[0057] In some examples, the image data in the annotated area in the training sample can be under-segmented to obtain a regional segmentation result (that is, the target area corresponding to the target in the annotated area can be segmented out by under-segmentation to obtain a regional segmentation result). Thus, the pixels of undetermined category in the annotated area can be identified based on the regional segmentation result obtained by under-segmentation. Generally speaking, the foreground target object is mistakenly segmented as the background but the background is not mistakenly segmented as the foreground target, which can be called under-segmentation. Here, under-segmentation can be that the pixels belonging to the target in the annotated area are mistakenly segmented as non-targets but the pixels that do not belong to the target in the annotated area are not mistakenly segmented as targets. In this case, the pixels in the target area in the regional segmentation result can be determined to belong to the target. In addition, within the annotated area, the pixels outside the target area do not necessarily belong to the target (that is, they can be pixels of undetermined category).

[0058] In some examples, the region segmentation result corresponding to the annotated region can be determined based on the training sample and the image data corresponding to the annotated region in the above-mentioned segmentation result of interest. Specifically, the image data corresponding to the annotated region in the training sample (hereinafter referred to as the first image data) and the image data corresponding to the annotated region in the above-mentioned segmentation result of interest (that is, the segmentation result of interest can be a binary image for identifying the region of interest of the training sample) (hereinafter referred to as the second image data) can be multiplied to obtain the image data to be segmented (that is, the image data in the annotated region), and the image data to be segmented is under-segmented to determine the region segmentation result corresponding to the annotated region. In this case, when the annotated region includes an area outside the region of interest, the noise outside the region of interest can be eliminated.

[0059] Figure 5 1 is a flowchart showing an example of obtaining a region segmentation result involved in the examples of the present disclosure. That is, some examples of the present disclosure are a process of obtaining a region segmentation result.

[0060] like Figure 5 As shown, obtaining the region segmentation result may include step S202. In step S202, the image data to be segmented may be obtained based on the annotated region. As described above, the first image data may be the image data corresponding to the annotated region in the training sample, and the second image data may be the image data corresponding to the annotated region in the segmentation result of interest. That is, the first image data and / or the second image data may be obtained based on the annotated region, and then the image data to be segmented may be obtained based on the first image data, or the first image data and the second image data.

[0061] In some examples, the image data to be segmented can be obtained based on the first image data. In some examples, the image data to be segmented can be obtained based on the color channels (e.g., red channel, green channel, blue channel) of the first image data. Taking the fundus image as an example, the image data to be segmented can be obtained based on the green channel of the first image data. Specifically, the first image data corresponding to the annotated area can be obtained (e.g., cropped) from the training sample, and then the green channel (i.e., G channel) of the first image data is taken, and the image data to be segmented is obtained based on the green channel of the first image data. In some examples, the corresponding color channel (green channel) of the first image data can be used as the image data to be segmented to obtain the image data to be segmented. In addition, the color space and color channel can be selected according to the characteristics of the medical image itself, and the present disclosure does not impose any special restrictions.

[0062] In other examples, the image data to be segmented can be obtained based on the first image data and the second image data. In this case, when the annotated area includes an area outside the area of ​​interest, the noise outside the area of ​​interest can be eliminated. In some examples, the image data to be segmented can be obtained based on the color channel of the first image data and the second image data. Specifically, the color channel of the first image data can be represented as G1, and the second image data can be represented as B1, then the image data to be segmented can be represented as I1=G1⊙B1, where I1 can represent the image data to be segmented, and ⊙ can represent an element (that is, the grayscale value of the pixel) product operation.

[0063] It should be noted that the first image data, the second image data, and the image data to be segmented can represent the image data of the corresponding area (for example, pixel data, data stream, or image). In practice, the pixel values ​​or pixel position marks of the corresponding area can be stored in a corresponding medium (for example, memory or disk) as needed to form image data in a corresponding form, which can be easily processed. In addition, the shapes of the first image data, the second image data, and the image data to be segmented can match the shape of the annotated area, or can be the circumscribed rectangle of the annotated area, which can be selected according to the method of obtaining the area segmentation result.

[0064] In addition, in the process of obtaining the region segmentation result, if the rectangular characteristics of the image data to be segmented (for example, sides, length, width, height, four corners) must be used and the shape of the marked region is not a rectangle, the image data to be segmented can be obtained based on the region corresponding to the circumscribed rectangle of the marked region. That is, after the shape of the marked region is converted into a rectangle, the image data to be segmented can be obtained based on the converted marked region.

[0065] like Figure 5 As shown, obtaining the region segmentation result may further include step S204. In step S204, threshold segmentation may be performed on the image data to be segmented to obtain the region segmentation result. However, the examples disclosed herein are not limited thereto. In other examples, other methods may be used to perform under-segmentation on the image data to be segmented to obtain the region segmentation result.

[0066] In some examples, in step S204, the target segmentation threshold (described later) can be used to perform threshold segmentation on the image data to be segmented, thereby obtaining a region segmentation result. Thus, the target region in the image data to be segmented can be identified by threshold segmentation. In some examples, in threshold segmentation, the value of the pixel whose gray value in the image data to be segmented is not less than the target segmentation threshold can be set to 1, and the value of other pixels can be set to 0, thereby obtaining a region segmentation result.

[0067] In some examples, before obtaining the region segmentation result, the threshold segmentation result (ie, the initial segmentation result) of the image data to be segmented may be subjected to an erosion operation, in which case the probability of pixel isolation in the threshold segmentation result due to noise may be reduced.

[0068] In some examples, in the erosion operation on the threshold segmentation result of the image data to be segmented, the erosion kernel k can satisfy the formula:

[0069]

[0070] Wherein, h may represent the height of the annotated area (that is, the annotated area corresponding to the image data to be segmented), w may represent the width of the annotated area, H may represent the height of the training sample, W may represent the width of the training sample, and p may represent a preset hyperparameter. In this case, an erosion kernel of a suitable size can be obtained according to the size of the training sample, the size of the annotated area, and the preset hyperparameter. Thus, excessive erosion can be suppressed.

[0071] In some examples, the preset hyperparameter may be used to adjust the size of the erosion kernel. In this case, a smaller erosion kernel can be used for a particularly small target. Thus, excessive erosion operations can be avoided, which may cause the target area of ​​a particularly small target to disappear.

[0072] In some examples, the preset hyperparameter may be a fixed value. In some examples, the preset hyperparameter may be determined based on the average size of objects of the same category in the medical image. In some examples, the preset hyperparameter may be determined based on the average width and average height of objects of the same category in the medical image. In some examples, the preset hyperparameter p may satisfy the formula:

[0073]

[0074] in, and can represent the average width and average height of objects of the same category in medical images, σ w and σ h Can represent the width standard deviation and height standard deviation respectively, and Can represent the average width and average height of the medical image respectively. Here, the medical image can be an image in the data source for obtaining the preset hyperparameters. In some examples, the width and height of targets of the same category in multiple training samples and the width and height of the training samples can be counted to obtain the relevant parameters of the preset hyperparameters. That is, the data source can be training data. In some examples, in a medical image with annotated areas (e.g., training samples), when obtaining the preset hyperparameters, the width and height of the target can also be the width and height of the corresponding annotated area. Thereby, the width and height of the target can be easily obtained.

[0075] Generally speaking, there may be multiple connected regions in the threshold segmentation result of the image data to be segmented. In some examples, the threshold segmentation result of the image data to be segmented may be eroded to obtain at least one connected region, and a connected region whose center is closest to the center of the image data to be segmented is selected from the at least one connected region as the region segmentation result. In addition, the connected region closest to the center of the image data to be segmented may represent the identified target region. Thus, an accurate target region can be obtained. In some examples, contours may be searched for the erosion result (i.e., at least one connected region), and a preset number (e.g., 3) of contours with the largest area are taken as candidates, and the connected region corresponding to the contour whose center is closest to the center of the image data to be segmented in the candidate contours is retained as the region segmentation result.

[0076] In addition, in the threshold segmentation of the image data to be segmented, there may be multiple ways to obtain the target segmentation threshold. For example, the target segmentation threshold may be obtained according to the common great law (OTSU) method. In some examples, the method for obtaining the target segmentation threshold may be selected from at least one of the methods described in the examples of the present disclosure.

[0077] In some examples, the target segmentation threshold can be obtained according to the annotation category to which the target belongs. In some examples, the target segmentation threshold can be obtained according to the threshold acquisition method of the annotation category to which the target belongs. In this case, the target segmentation threshold can be obtained according to the characteristics of the annotation category itself corresponding to the target. Thereby, the accuracy of threshold segmentation can be improved. In addition, the threshold acquisition method of the annotation category can include the first method and the second method. In addition, the annotation category to which the target in the training sample belongs can be known. For example, the annotation category to which the target in the training sample belongs can be the annotation label in the annotation result.

[0078] In some examples, the method for obtaining the threshold value of each annotation category can be obtained by the characteristics of each annotation category. In some examples, the method for obtaining the threshold value can be determined according to the average area and average color of each annotation category. However, the examples of the present disclosure are not limited to this. In some other examples, the method for obtaining the threshold value of the annotation category can also be determined based on experience. For example, for fundus images, the first method can be used for sheet hemorrhages in fundus images, and the second method can be used for microaneurysms, punctate hemorrhages, and linear hemorrhages in fundus images.

[0079] In some examples, the average area and average color of each labeled category may be fixed values, which may be obtained by statistically analyzing the sample data. For example, the areas and colors of objects of the same category (for example, for training samples, the same category may refer to the same labeled category) in the sample data (for example, training samples) may be averaged to obtain the average area and average color. In other examples, the fixed value may also be an empirical value.

[0080] In some examples, in the method of obtaining a threshold value for determining a labeling category based on the average area and average color of each labeling category, the average area of ​​the labeling category corresponding to the first method may be greater than the average area of ​​the labeling category corresponding to the second method, and the average color of the labeling category corresponding to the first method may be lighter than the average color of the labeling category corresponding to the second method. For example, the first method may be targeted at targets of such a labeling category that are large in area and light in color (e.g., flake hemorrhages in fundus images). The second method may be targeted at targets of such a labeling category that are small in area and dark in color (e.g., microvascular tumors, punctate hemorrhages, and linear hemorrhages in fundus images).

[0081] In some examples, in the method of determining the threshold value of the annotation category according to the average area and average color of each annotation category, the threshold value method used by the annotation category can be determined by the first preset area and preset color value. Thus, the threshold value method used by the annotation category can be automatically obtained.

[0082] In some examples, if the average area of ​​a labeled category is larger than a first preset area and the average color is smaller than a preset color value (i.e., the area of ​​the target of this labeled category is relatively large and the color is relatively light), then the labeled category can be determined to use the first method; otherwise, if the average area of ​​the labeled category is not larger than the first preset area and the average color is not smaller than the preset color value (i.e., the area of ​​the target of this labeled category is relatively small and the color is relatively dark), then the labeled category can be determined to use the second method.

[0083] In some examples, the first preset area and the preset color value may be adjusted according to the region segmentation result. In some examples, the first preset area and the preset color value may be fixed values, and the fixed values ​​may be obtained by statistically analyzing the sample data. That is, a statistical method may be used to analyze the region segmentation results of a small amount of sample data under different first preset areas and preset color values ​​to determine the best first preset area and preset color value for classification.

[0084] As described above, the target segmentation threshold can be obtained according to the threshold acquisition method of the annotation category to which the target belongs. In some examples, the target segmentation threshold can be obtained according to the threshold acquisition method of the annotation category to which the target belongs and the image data to be segmented corresponding to the training sample.

[0085] In some examples, for the first method (that is, the method for obtaining the threshold value of the annotation category to which the target belongs is the first method), a threshold value can be found so that the area of ​​pixels with grayscale values ​​greater than the threshold value in the image data to be segmented is less than a preset multiple of the area of ​​the image data to be segmented, and the threshold value is used as the target segmentation threshold value, wherein the preset multiple can be greater than 0 and less than 1. Taking the medical image as an 8-bit quantized image as an example, the threshold values ​​from 0 to 255 can be traversed to find a threshold value so that the area of ​​pixels with grayscale values ​​greater than the threshold value in the image data to be segmented is less than a preset multiple of the area of ​​the image data to be segmented, and the threshold value is used as the target segmentation threshold value. In addition, the preset multiple can be any value that prevents the target area from being over-segmented. For example, the preset multiple can be a relatively small value so that the target area is not over-segmented. In some examples, the preset multiple can be determined empirically by the shape of the target.

[0086] In some examples, for the second method (that is, the method for obtaining the threshold of the labeled category to which the target belongs is the second method), the mean of the grayscale values ​​of the pixels in the image data to be segmented can be used as the target segmentation threshold or the target segmentation threshold can be determined based on the grayscale values ​​of the four corner areas and the central area of ​​the image data to be segmented.

[0087] In some examples, for the second method, if the length of the smallest side of the image data to be segmented is less than the preset length, the mean of the grayscale values ​​of the pixels in the image data to be segmented can be taken as the target segmentation threshold. In some examples, the preset length can be any value that does not over-segment the target area. In some examples, the preset length can be a first preset ratio of the smallest side of the training sample. Specifically, the preset length can be expressed as min(rH, rW), where r can represent the first preset ratio, H can represent the height of the training sample, and W can represent the width of the training sample.

[0088] In some examples, the first preset ratio may be a fixed value. In some examples, the first preset ratio may be determined based on the average size of objects of the same category in the medical image. In some examples, the first preset ratio may be determined based on the average width and average height of objects of the same category in the medical image. In some examples, the first preset ratio may satisfy the formula:

[0089]

[0090] in, and can represent the average width and average height of objects of the same category in medical images, σ w and σ h Can represent the width standard deviation and height standard deviation respectively, and The average width and average height of the medical image may be represented respectively. Here, the medical image may be an image in a data source used to obtain the first preset ratio. In some examples, the data source may be training data. In addition, the relevant parameters involved in the first preset ratio may be obtained in a manner similar to the relevant parameters involved in obtaining the preset hyperparameters, which will not be described in detail here.

[0091] In some examples, for the second method, if the length of the smallest side of the image data to be segmented is not less than a preset length, the target segmentation threshold can be determined based on the grayscale values ​​of the four corner areas and the central area of ​​the image data to be segmented. Specifically, the image data to be segmented can be divided into a preset number of parts (e.g., 9 equal parts), and the target segmentation threshold is determined based on the grayscale values ​​of the four corner areas and the central area of ​​the image data to be segmented. For details, see the above description of determining the segmentation threshold based on the grayscale values ​​of the four corner areas and the central area of ​​the training sample.

[0092] Return to reference Figure 2 As described above, in step S104, a training set can be constructed using training samples and region segmentation results. That is, a training set can be constructed based on training samples and at least one region segmentation result corresponding to the training samples. In some examples, the training set can include training samples and a gold standard of the training samples. In some examples, the gold standard of the training samples can be obtained based on the region segmentation results. That is, the target region can be identified based on the region segmentation results, and then the real category to which the pixels in the training sample belong can be determined based on the target region. In this way, the gold standard of the training sample can be obtained.

[0093] In some examples, the true category may include a labeled category of the target (for example, for fundus images, it may include microaneurysm, punctate hemorrhage, sheet hemorrhage, and linear hemorrhage), a no-target category, and at least one of an undetermined category. This is specifically related to the process of optimizing the model to be trained 130.

[0094] In addition, the annotated category of the target in the true category may be the category to which the pixels of the target area (that is, the second area described later) of the target within the annotated area in the training sample belong. In addition, the undetermined category in the true category may be the category to which the pixels of the area outside the target area (that is, the first area described later) of the target within the annotated area in the training sample belong. In addition, the targetless category in the true category may be the category to which the pixels outside the annotated area in the training sample belong. In some examples, the area outside the annotated area in the training sample may include an area within the region of interest and not belonging to the annotated area (that is, the third area described later). For example, for a medical image, the area within the region of interest and not belonging to the annotated area may be an area corresponding to a targetless tissue in the medical image. In some examples, the area outside the annotated area in the training sample may include an area within the region of interest and not belonging to the annotated area, and an area outside the region of interest (that is, the fourth area described later).

[0095] In some examples, the training samples and region segmentation results can also be used to construct validation sets and test sets.

[0096] Return to reference Figure 2 The model training method may further include step S106. In step S106, the model to be trained 130 may be trained based on the training set, and the model to be trained 130 may be optimized using a training loss function.

[0097] In some examples, the model to be trained 130 may include but is not limited to a semantic segmentation model. In addition, the prediction result of the model to be trained 130 may include but is not limited to the semantic segmentation result of the training sample. Thus, small targets can be identified. For example, in the example where the input 120 is image data to be semantically segmented and the model to be trained 130 is a semantic segmentation model, the prediction result may be the semantic segmentation result of the image data. In addition, the input 120 may be color image data.

[0098] In some examples, in the model to be trained 130, high-dimensional feature information can be added. Thereby, the accuracy of recognizing small targets can be improved. In some examples, in the model to be trained 130, feature information of different dimensions in a medical image (e.g., a training sample) can be extracted, and feature information of a preset dimension close to the feature information of the highest dimension can be fused with the feature information of the highest dimension to increase the feature information of the high dimension.

[0099] Figure 6 is an architecture diagram showing an example of a model 130 to be trained using a U-Net architecture involved in the example of the present disclosure.

[0100] As an example, Figure 6 The figure shows a model 130 to be trained using a U-Net architecture, wherein the common network layers in the U-Net architecture are not explained in detail. Figure 6 As shown, the preset dimension can be 2, and the feature information of the two dimensions can include feature information 131a and feature information 132b, wherein feature information 131a can be fused with the feature information of the highest dimension through the upsampling layer 132a, and feature information 131b can be fused with the feature information of the highest dimension through the upsampling layer 132b. In addition, the convolution size of the upsampling layer 132a and the upsampling layer 132b can be any value that makes the feature information (e.g., feature information 131a and feature information 131b) consistent with the size of the feature information of the highest dimension after upsampling.

[0101] In some examples, when training the model to be trained 130, the prediction results corresponding to the training samples can be obtained by the model to be trained 130 based on the training samples of the training set, and then a training loss function can be constructed based on the region segmentation results and the prediction results corresponding to the training samples (that is, the training loss function can be constructed using the gold standard and the prediction results of the training samples obtained based on the region segmentation results). In addition, the training loss function can represent the degree of difference between the gold standard of the training sample and the corresponding prediction result.

[0102] In some examples, the region segmentation result can be directly used as the gold standard of the training sample. In some examples, the region segmentation result can be used as the gold standard of the pixels in the annotated area corresponding to the target in the training sample to obtain the gold standard of the training sample. In addition, the gold standard of the pixels in the area outside the annotated area corresponding to the target in the training sample can be set as needed. For example, it can be fixedly set to a category (for example, it can be the targetless category involved in the example of the present disclosure). For another example, it can be set by manually annotating the training samples or by automatically annotating the training samples through an artificial intelligence algorithm. The example of the present disclosure does not particularly limit the setting method of the gold standard of the pixels in the area outside the annotated area corresponding to the target in the training sample.

[0103] In some examples, in the training loss function, weights may be assigned to the pixels of the undetermined category in the training samples to reduce the negative impact of the pixels of the undetermined category on the model to be trained 130. Thus, the accuracy of the model to be trained 130 can be improved. In some examples, in the training loss function, spatial weights may be used to reduce the negative impact of the pixels of the undetermined category in the training samples on the model to be trained 130.

[0104] In some examples, in the spatial weight, the training samples may be divided into several regions (also referred to as at least one region), and the weights may be used to adjust the influence of each of the several regions on the model 130 to be trained.

[0105] In some examples, the plurality of regions may include a first region. The first region may be a region of pixels of an undetermined category in the training sample (i.e., a region outside the target region within the labeled region in the training sample). In some examples, in the training loss function, spatial weights may be used to reduce the negative impact of pixels in the first region of the training sample on the model to be trained 130. In some examples, in the spatial weights, pixels in the first region of the training sample may be assigned a first weight to reduce the negative impact on the model to be trained 130.

[0106] In addition, the first weight may be any value that reduces the negative impact on the model to be trained 130. In some examples, the first weight may be a fixed value. In some examples, the first weight may be 0. In this case, samples of undetermined categories can be ignored to reduce the negative impact of samples of undetermined categories on the model to be trained 130.

[0107] In some examples, the plurality of regions may include a second region. The second region may be a target region for a training sample. In some examples, in the spatial weight, pixels of the second region may be assigned a second weight. In some examples, the first weight may be less than the second weight. In addition, the second weight may be any value that increases the positive impact of the pixels of the second region on the model to be trained 130. In some examples, the second weight may be a fixed value. In some examples, the second weight may be 1.

[0108] In some examples, the plurality of regions may include a third region. The third region may be a region within the region of interest in the training sample that does not belong to the annotated region. In some examples, in the spatial weight, pixels in the third region may be assigned a third weight. In some examples, the first weight may be less than the third weight. In addition, the setting principle of the third weight may be similar to that of the second weight.

[0109] In some examples, the plurality of regions may include a fourth region. The fourth region may be a region outside the region of interest in the training sample. In some examples, in the spatial weight, pixels in the fourth region may be assigned a fourth weight. In some examples, the fourth weight may be less than the second weight. In addition, the setting principle of the fourth weight may be similar to the first weight.

[0110] In some examples, several regions may include a first region, a second region, a third region, and a fourth region at the same time, and the pixels of the first region, the second region, the third region, and the fourth region may be assigned a first weight, a second weight, a third weight, and a fourth weight, respectively, wherein the first weight may be less than the second weight and less than the third weight, and the fourth weight may be less than the second weight and less than the third weight. In this case, the negative impact of pixels of undetermined categories and pixels outside the region of interest on the model to be trained 130 can be suppressed, and the positive impact of non-target areas within the target region and the region of interest on the model to be trained 130 can be improved. Thus, the accuracy of the model can be improved. Preferably, the first weight may be 0, the second weight may be 1, the third weight may be 1, and the fourth weight may be 0. In this case, the negative impact of pixels of undetermined categories and pixels outside the region of interest on the model to be trained 130 can be avoided, and the positive impact of non-target areas within the target region and the region of interest on the model to be trained 130 can be improved. Thus, the accuracy of the model to be trained 130 can be improved.

[0111] However, the examples of the present disclosure are not limited thereto, and in other examples, the plurality of regions may include any combination of the first region, the second region, the third region, and the fourth region.

[0112] Figure 7 is a schematic diagram showing several areas of some examples involved in the examples of the present disclosure.

[0113] In addition, in order to clearly describe several areas, Figure 7 is a schematic diagram showing each binarized region, and does not limit the present disclosure to being divided into Figure 7 Among them, D3 can represent the first area, D4 can represent the second area, D5 can represent the third area, and D6 can represent the fourth area.

[0114] As described above, in some examples, in spatial weighting, the training samples may be divided into several regions, and weights may be used to adjust the influence of each of the several regions on the model 130 to be trained.

[0115] In some examples, in the training loss function, the loss can be calculated by category. As described above, the true category can include at least one of the labeled category of the target, the no-target category, and the undetermined category. In some examples, in the training loss function, the category can be derived from the above-mentioned true category. That is, the categories in the training loss function can include the labeled category of the target and the no-target category, or the categories in the training loss function can include the labeled category of the target, the no-target category, and the undetermined category. The categories in the specific training loss function are related to the samples selected in the training loss function.

[0116] In some examples, in the training loss function, if samples (i.e., pixels) of each category in the training sample belong to a corresponding region among several regions, the loss of the corresponding sample can be multiplied by the weight of the corresponding region. In this case, the training loss function can be determined based on the spatial weight, thereby adjusting the influence of pixels in different regions on the training model 130.

[0117] In some examples, in the training loss function, the influence of samples of each category on the to-be-trained model 130 can be adjusted based on the weight of each category. Thus, the influence of samples of different categories on the to-be-trained model 130 can be adjusted.

[0118] In some examples, in the training loss function, the influence of the sample on the to-be-trained model 130 can be adjusted based on both the spatial weight and the category weight. Thus, the influence of the sample on the to-be-trained model 130 can be adjusted by region and category.

[0119] In some examples, the training loss function may adopt weighted balanced cross entropy. In this case, the imbalance of positive and negative samples can be suppressed, thereby further improving the accuracy of the model to be trained 130 in recognizing small targets. In some examples, when training the model to be trained 130, the training loss function based on weighted balanced cross entropy can be used, and the spatial weights can be used to control the negative impact of pixels of undetermined categories on the model to be trained 130.

[0120] The following describes the training loss function based on weighted balanced cross entropy, taking the spatial weights in which the first weight of the first area is 0, the second weight of the second area is 1, the third weight of the third area is 1, and the fourth weight of the fourth area is 0 as an example. It should be noted that this does not limit the present disclosure, and those skilled in the art can design a training loss function based on weighted balanced cross entropy by freely combining the weights of each area in a number of areas and the weights of each category according to the circumstances. The training loss function L based on weighted balanced cross entropy can satisfy the formula (that is, equivalent to setting the first weight and the fourth weight to 0 and ignoring the losses of the first area and the fourth area):

[0121]

[0122] Among them, C can represent the number of categories, W i It can represent the weight of the i-th category, M i It can represent the number of samples in the i-th category, y ij It can represent the true value of the jth sample of the i-th category in the gold standard of the above training samples, p ijIt can represent the predicted value of the jth sample of the i-th category in the prediction result (that is, the probability that the jth sample belongs to the i-th category). In addition, the samples of each category can be pixels of the corresponding category in the training sample. In addition, the samples of a category can be determined based on the gold standard of the above training samples. As described above, the weight of the category can adjust the influence of the samples of each category on the training model 130.

[0123] In addition, in formula (1), by setting the first weight and the fourth weight to 0, the samples of the first area and the fourth area are ignored, and the categories in the training loss function can include the labeled category of the target and the non-target category. The labeled category of the target can be the category to which the pixels of the second area in the training sample belong, and the non-target category can be the category to which the pixels of the third area in the training sample belong. Taking the fundus image as an example, the categories in the training loss function of formula (1) can include microvascular tumor, punctate hemorrhage, sheet hemorrhage, linear hemorrhage and non-target category.

[0124] The following combination Figure 8 The present disclosure relates to a method for identifying a target in an image (hereinafter referred to as the identification method). In addition, the identification method can identify a target in a medical image. Figure 8 is a flowchart illustrating an example of a method for identifying an object in an image according to an example of the present disclosure.

[0125] like Figure 8 As shown, the recognition method may include step S302. In step S302, a medical image as an input image may be obtained. In some examples, the input image may be pre-processed like the above-mentioned training sample before being input into the trained model.

[0126] like Figure 8 As shown, the recognition method may further include step S304. In step S304, at least one trained model may be used to determine the prediction results of each trained model for the input image, and a target prediction result may be obtained based on the prediction results of at least one trained model, wherein at least one trained model may be obtained by training according to the above-mentioned model training method. In addition, at least one trained model may be a model based on the same type of network architecture (e.g., U-Net), but with a different network structure and / or different parameters. For example, some branches or network levels may be added or reduced to form at least one trained model. However, the examples disclosed herein are not limited thereto. In other examples, at least one trained model may not be based on the same type of network architecture.

[0127] In addition, the prediction results of each trained model may include the probability that each pixel in the input image belongs to the corresponding labeled category. The labeled category may be the labeled category of the above-mentioned target. In some examples, the prediction results of at least one trained model may be integrated by the labeled category and pixel to obtain the integrated probability that each pixel of the input image belongs to the corresponding labeled category, and the connected area is determined based on the integrated probability, and the target prediction results corresponding to each labeled category are obtained based on the connected area. In this case, obtaining the target prediction result based on the integrated probability can further improve the accuracy of the target prediction result.

[0128] In some examples, in obtaining the integrated probability, if there is only one trained model, the probability that each pixel in the input image belongs to the corresponding labeled category in the prediction result of the trained model can be used as the integrated probability; otherwise, the prediction results of multiple trained models can be averaged to obtain the mean probability that each pixel in the input image belongs to the corresponding labeled category (that is, the probability average can be performed at the pixel level according to the labeled category).

[0129] In some examples, in determining the connected area based on the integrated probability, the connected area can be determined based on the integrated probability and the classification threshold of each labeled category. Specifically, the value of the pixel whose integrated probability is not less than the classification threshold can be set to 1, and the value of other pixels can be set to 0. In some examples, the classification threshold can be determined based on the validation set and using a performance indicator. In addition, if there are connected areas, the number of connected areas can be one or more.

[0130] In some examples, when obtaining target prediction results based on connected areas, the bounding rectangle of each connected area can be obtained. If the area of ​​the bounding rectangle is larger than a second preset area, it can indicate that there is a target at the bounding rectangle; otherwise, it can indicate that there is no target at the bounding rectangle.

[0131] In some examples, the second preset area may be a second preset ratio of the area of ​​the training sample. Specifically, the second preset area may be expressed as sHW, where s may represent the second preset ratio, H may represent the height of the input image, and W may represent the width of the input image.

[0132] In some examples, the second preset ratio may be a fixed value. In some examples, the second preset ratio may be determined based on the median of the area of ​​objects of the same category in the medical image. In some examples, the second preset ratio s may satisfy the formula:

[0133]

[0134] Wherein, m may represent the median of the area of ​​the same category of objects in the medical image, σ may represent the standard deviation of the area of ​​the same category of objects in the medical image, and W and H may represent the average width and average height of the medical image, respectively. Here, the medical image may be an image in a data source for obtaining the second preset ratio. In some examples, the data source may be training data. In addition, the relevant parameters involved in the second preset ratio may be obtained in a manner similar to the relevant parameters involved in obtaining the preset hyperparameters, which will not be described in detail here.

[0135] The present disclosure also relates to a computer-readable storage medium, which may store at least one instruction, and when the at least one instruction is executed by a processor, one or more steps in the above-mentioned model training method or recognition method are implemented.

[0136] The present disclosure also relates to an electronic device, which may include at least one processing circuit. The at least one processing circuit is configured to perform one or more steps in the above-mentioned model training method or recognition method.

[0137] The model training, method, device and medium for identifying targets in medical images of the example disclosed in the present invention, by under-segmenting the image data in the labeled area in the training sample to identify the pixels of undetermined categories in the labeled area, and training the to-be-trained model 130 in combination with the spatial weight to reduce the negative impact of the pixels of undetermined categories in the labeled area on the to-be-trained model 130, can improve the accuracy of the prediction results of the input image by the trained model 130. Thus, small targets can be effectively identified.

[0138] Although the present disclosure is specifically described above in conjunction with the accompanying drawings and examples, it is to be understood that the above description does not limit the present disclosure in any form. Those skilled in the art may modify and change the present disclosure as needed without departing from the essential spirit and scope of the present disclosure, and these modifications and changes all fall within the scope of the present disclosure.

Claims

1. A method for identifying a target in a medical image, characterized in that: include: acquiring the medical image as an input image; And using at least one trained model, determine the prediction results of each trained model for the input image, and obtain the target prediction result based on the prediction result of the at least one trained model, wherein the at least one trained model is obtained by training the model to be trained, and the training set of the model to be trained is constructed by training samples and region segmentation results corresponding to the annotated areas of the training samples, and the region segmentation results are obtained by under-segmenting the image data in the annotated area and are used to determine the target area and the first area, the pixels in the target area are determined to belong to the target, the pixels in the first area are pixels of undetermined category, and the first area is the area outside the target area in the annotated area in the training sample, and in the training loss function of the model to be trained, the pixels of the first area and the pixels of the target area are respectively assigned a first weight and a second weight, and the first weight is less than the second weight to reduce the negative impact of the pixels of the first area on the model to be trained.

2. The method for identifying a target in a medical image according to claim 1, characterized in that: The prediction results of each trained model include the probability that each pixel in the input image belongs to the corresponding labeled category. The prediction results of at least one trained model are integrated according to the labeled category and pixel to obtain the integrated probability that each pixel of the input image belongs to the corresponding labeled category. The connected area is determined based on the integrated probability, and the target prediction results corresponding to each labeled category are obtained based on the connected area. If there is only one trained model, the probability is used as the integrated probability. Otherwise, the prediction results of multiple trained models are averaged to obtain the average probability that each pixel in the input image belongs to the corresponding labeled category and use it as the integrated probability.

3. The method for identifying targets in medical images according to claim 2, characterized in that: In determining the connected area based on the integration probability, the values ​​of pixels whose integration probability is not less than the classification threshold are set to 1, and the values ​​of other pixels are set to 0.

4. The method for identifying a target in a medical image according to any one of claims 1 to 3, characterized in that: The under-segmentation method is threshold segmentation.

5. The method for identifying a target in a medical image according to claim 4, characterized in that: The image data to be segmented is obtained based on the color channel of the image data corresponding to the marked area in the training sample, and the threshold segmentation is performed on the image data to be segmented using the target segmentation threshold, so as to obtain the area segmentation result.

6. The method for identifying a target in a medical image according to claim 4, characterized in that: A method for obtaining a threshold value for each of the labeled categories is obtained by obtaining the characteristics of the labeled category to which each of the targets belongs, or a method for determining the threshold value for the labeled category based on experience. The threshold value obtaining method is a method for obtaining a target segmentation threshold value for the threshold segmentation.

7. The method for identifying a target in a medical image according to any one of claims 1 to 3, characterized in that: In the training loss function, the loss is calculated according to the real category to which the pixels in the training sample belong, and the real category is determined by the region segmentation result and includes at least one of the labeled category of the target, the no-target category and the undetermined category.

8. The method for identifying a target in a medical image according to any one of claims 1 to 3, characterized in that: The first weight is 0.

9. The method for identifying a target in a medical image according to any one of claims 1 to 3, characterized in that: The medical image is a color image.

10. An electronic device, characterized in that: include: At least one processing circuit, the at least one processing circuit being configured to perform the method for identifying an object in a medical image as claimed in any one of claims 1 to 9.

Images