Artificial intelligence model training method and device for image segmentation

Abstract

The invention relates to the technical field of image processing, in particular to an artificial intelligence model training method and device for image segmentation, and the method comprises the steps: marking original image data, and obtaining a marking matrix; segmenting according to the number of pixels occupied by the to-be-identified target, setting a weight for each segment interval, and the more the pixels in the segment interval are, the smaller the weight is; assigning weights to adjacent areas of m circles of pixels around the to-be-identified target, wherein the weight of the circle of pixels closer to the to-be-identified target is larger; and training the artificial intelligence model by using a training set composed of the original image and the labeled result, taking a positive part in the labeled result as a target class, taking a negative part in the labeled result as a background class, taking an absolute value of the labeled result as a weight of a loss function for training, and carrying out loop iteration until the loss value is converged. According to the training method, the sensitivity of the model to the small target developed in the early stage is improved, sensitive monitoring of the specific target is achieved, and practical value is brought to multiple application scenes.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an artificial intelligence model training method and device for image segmentation. Background Art

[0002] Image segmentation, object recognition, and extraction, as key technologies in image processing and computer vision, play a vital role at all levels of society and the economy. In remote sensing, this technology is used to process remote sensing images and identify and monitor changes in mountains, rivers, and lakes. In the medical field, this technology is used in medical image analysis, assisting doctors in accurately segmenting lesions from CT and MRI images. This provides doctors with more intuitive and accurate diagnostic information, helps develop more effective treatment plans, and improves early diagnosis and cure rates, thereby alleviating patient suffering and reducing medical costs. In semiconductor manufacturing, image segmentation can be used to detect defects and flaws on chip surfaces. Segmenting wafer images during the production process can identify potential defective areas, helping to improve product quality, reduce defective rates, and enhance a company's economic efficiency and market competitiveness.

[0003] After years of development, image segmentation, object recognition and extraction technologies have made significant progress, covering traditional methods and new methods based on deep learning. Traditional methods are mainly based on the underlying features of the image, such as pixel intensity, color, texture, threshold, area, edge, etc., which can have certain effects in simple image scenes, but have limitations when facing complex scenes. New methods based on deep learning currently mainly include models based on convolutional neural networks (CNN) (such as U-Net, DeepLabVNN), Transformer architectures (such as Swin-UNet, TransUNet) and emerging state-space models (SSM) (such as CM-UNet and VMKLA-UNet of Mamba architecture). In addition, generative adversarial networks (GAN) and rendering-based methods (such as PointRend and its variant SEMI-PointRend) have also performed well in high-precision segmentation, object recognition and extraction tasks.

[0004] Although image segmentation, object recognition, and extraction technologies have been widely used and have achieved numerous results, they are still being continuously improved. Efficient segmentation and extraction of small targets is one research direction. In remote sensing images, there may be multiple instances of the same type of ground object, such as multiple lakes with vastly different areas. In medical imaging, there may also be multiple instances of the same lesion, with vastly different areas. In manufacturing, photolithography mask images may also contain multiple defects of varying sizes. Small targets refer to instances of a certain type of target to be extracted that are relatively small in area and have a small number of pixels.

[0005] In practical application areas, the segmentation and extraction of small targets is of special value: in the field of remote sensing, accurate segmentation and extraction of small targets can not only provide clues for large-scale regional climate change, but also detect early signs of geological disasters in small areas; in the field of medical imaging, accurate segmentation and extraction of small targets can not only achieve early detection of lesions but also achieve full detection of lesions; in the field of integrated circuit manufacturing, accurate segmentation and extraction of small defects on photolithography masks is crucial to improving the level of semiconductor technology.

[0006] The inconsistent sizes of similar targets make it difficult for existing image segmentation and target recognition and extraction algorithms to achieve a good balance between detail extraction of small targets and overall grasp of large targets. To this end, people have made a lot of efforts and attempts.

[0007] CN118941789A discloses a small target detection method, system, electronic device and computer-readable storage medium. The main process is to build an improved model and train it, gradually obtaining the original feature map, the density feature map, the target feature map and the small target prediction box.

[0008] CN119131369A discloses a colon polyp target detection method and system based on improved YOLOv8. The main process is to improve the YOLO basic model and introduce a new network module to improve the detection accuracy of small targets and colon polyp targets with blurred boundaries.

[0009] CN119444779A discloses a small target medical ultrasound image segmentation method based on coarse and fine granularity attention, the main process of which is to add a multi-channel attention mechanism to the U-Net model.

[0010] CN119152188A discloses a small target defect detection method, equipment, medium and product. The main process is to train a model after coarse-grained and fine-grained defect classification to solve the problem of poor small target defect detection effect of photolithography mask.

[0011] The above solutions all focus on improving the model, but this application adopts a different approach. Although the structures of image segmentation, target recognition, and extraction models vary greatly, the loss function is always an inseparable core module. The loss function calculates the loss value between the predicted result and the true label, that is, the degree of deviation. The training process of the model is essentially a process of constantly adjusting its own parameters in pursuit of a smaller loss value. The calculation of the loss value is the baton of this process, guiding the model to be closer to the annotation. Therefore, the efficiency of segmentation and extraction of specific targets can be improved by combining annotation with the calculation of the loss value. Summary of the Invention

[0012] In order to improve the accuracy of detection, the present invention proposes an artificial intelligence model training method for image segmentation, comprising the following steps:

[0013] The original image is labeled, and the pixels of multiple target areas to be identified in the image are labeled as 1, and the pixels of other areas as background are labeled as -1, to obtain a labeling matrix;

[0014] Segment the target according to the number of pixels it occupies, and set a weight for each segment. The more pixels in a segment, the smaller the weight.

[0015] Assign weights to the neighboring area of ​​m circles of pixels surrounding the target to be identified. The fewer pixels the target to be identified occupies, the larger m is.

[0016] Set a weight for each of the m circles of pixels surrounding the target to be identified, and the closer the circle of pixels is to the target to be identified, the greater the weight;

[0017] Use the set weight value to weight the annotation matrix to obtain the annotation result;

[0018] The artificial intelligence model is trained using a training set consisting of original images and annotation results. The positive pixel positions in the annotation results are used as the class to be identified, and the negative pixel positions in the annotation results are used as the background class. The absolute values ​​of all pixel annotation results are used as the weights of the loss function to calculate the loss value for training, and the cycle is iterated until the loss value converges.

[0019] The present invention also provides an artificial intelligence model training device for image segmentation, which is used in an artificial intelligence model training method for image segmentation, comprising:

[0020] The initial annotation module is used to annotate the original image, annotate the pixels of multiple target areas of the image to be identified as 1, and annotate the pixels of other areas as background as -1, to obtain the annotation matrix;

[0021] The first weighting module divides the target into segments according to the number of pixels occupied by the target in the annotation matrix, sets a weight for each segment interval, wherein the more pixels in the segment interval, the smaller the weight, and uses the set weight to weight each target area to be identified in the annotation matrix;

[0022] The neighboring region selection module selects the size of the neighboring region for the current target based on the current target. The size of the neighboring region is m pixels around the target to be identified. The smaller the number of pixels occupied by the target to be identified, the larger the m;

[0023] The second weighting module is used to weight each circle of pixels in the adjacent area of ​​the annotation matrix, and the weight of a circle of pixels closer to the target to be identified is greater;

[0024] The training module is used to train the artificial intelligence model using a training set consisting of original images and annotation results. The positive pixel positions in the annotation results are used as the class to be identified, and the negative pixel positions in the annotation results are used as the background class. The absolute values ​​of all pixel annotation results are used as the weights of the loss function to calculate the loss value for training, and the cycle is iterated until the loss value converges.

[0025] The present invention provides an artificial intelligence model training method and device for image segmentation to solve the model's bias towards large targets caused by intra-class imbalance in the field of image segmentation, and provides technical means for the accurate and efficient segmentation and extraction of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a structural block diagram of an artificial intelligence model training device for image segmentation according to the present invention;

[0027] Figure 2 This is a flow chart of an artificial intelligence model training method for image segmentation according to the present invention;

[0028] Figure 3 Schematic diagram of setting weights for equally spaced intervals according to the present invention, wherein: Figure (a) shows an embodiment of setting weights for intervals according to exponential changes, and Figure (b) shows an embodiment of setting weights for intervals according to linear changes;

[0029] Figure 4 Schematic diagram of setting weights for unequally spaced segment intervals according to the present invention, wherein: Figure (a) shows an embodiment of setting weights for segment intervals according to exponential changes, and Figure (b) shows an embodiment of setting weights for segment intervals according to linear changes;

[0030] Figure 5 A schematic diagram of the marking of the target to be identified in the prior art;

[0031] Figure 6 This is a schematic diagram of the labeling results of four targets in one embodiment of the present invention;

[0032] Figure 7 is a schematic diagram of an initial marking matrix in an embodiment of the present application;

[0033] Figure 8 It is a schematic diagram of the final marking matrix in one embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0035] The present invention proposes an artificial intelligence model training method for image segmentation, comprising the following steps:

[0036] The original image is labeled, and the pixels of multiple target areas to be identified in the image are labeled as 1, and the pixels of other areas as background are labeled as -1, to obtain a labeling matrix;

[0037] Segment the target according to the number of pixels it occupies, and set a weight for each segment. The more pixels in a segment, the smaller the weight.

[0038] Assign weights to the neighboring area of ​​m circles of pixels surrounding the target to be identified. The fewer pixels the target to be identified occupies, the larger m is.

[0039] Set a weight for each of the m circles of pixels surrounding the target to be identified, and the closer the circle of pixels is to the target to be identified, the greater the weight;

[0040] Use the set weight value to weight the annotation matrix to obtain the annotation result;

[0041] The artificial intelligence model is trained using a training set consisting of original images and annotation results. The positive pixel positions in the annotation results are used as the class to be identified, and the negative pixel positions in the annotation results are used as the background class. The absolute values ​​of all pixel annotation results are used as the weights of the loss function to calculate the loss value for training, and the cycle is iterated until the loss value converges.

[0042] The first step in using an artificial intelligence model to segment and extract targets from images is to train the model. The prerequisite for training the model is to prepare labeled data. Compared with the common binary labeling method of "target is 1, background is 0", this application proposes a new labeling method: label the body of the small target as a positive number with a larger absolute value, label the pixels around the small target as a negative number with a larger absolute value, label the body of the large target as a positive number with a smaller absolute value, label the pixels and background around the large target as negative numbers with a smaller absolute value, and use the absolute value of each pixel label value as a weight to calculate the loss function during model training. The labeling and training scheme proposed in this application enables the model to preferentially fit small targets according to the designer's wishes, making the model more sensitive to small targets.

[0043] The present invention segments the target to be identified based on the number of pixels it occupies, and sets weights for each segment interval in order to assign a larger weight to smaller targets and a smaller weight to larger targets. There are various methods and strategies for achieving the aforementioned objectives, which can be described by different strategy parameters. This embodiment provides a specific implementation method, specifically:

[0044] According to the size segmentation method in the strategy parameters, all targets are placed in a descending queue according to the number of pixels, and targets with the same number of pixels occupy different positions;

[0045] Divide the descending queue into n evenly spaced segments according to the number of segments in the policy parameter;

[0046] According to the segment weight change method in the strategy parameters, assign weight 1 to the first segment target, assign weight 2 to the second segment target, and assign weight 2 to the nth segment target. n-1 ,Right now:

[0047] w i =2 i-1 i∈{1,2,…,n}

[0048] Among them, w i represents the weight of the i-th segment, and n is the number of segments;

[0049] Or, according to the segment weight change method in the strategy parameters, assign a weight of 1 to the first segment target, a weight of 2 to the second segment target, and a weight of n to the nth segment target, that is:

[0050] w i =ii∈{1,2,…,n}

[0051] Among them, w i represents the weight of the i-th segment, and n is the number of segments.

[0052] This embodiment also provides another implementation method of assigning a larger weight to a smaller target and a smaller weight to a larger target, specifically:

[0053] According to the size segmentation method and number of segments in the strategy parameters, count the number of pixels of all targets, find the range of the maximum and minimum pixel numbers, and divide the pixel number range from large to small into n segments of equal proportion;

[0054] Match the number of pixels of the target with the range of the number of pixels of each segment to find the segment to which each target belongs;

[0055] According to the segment weight change method in the strategy parameters, assign weight 1 to the first segment target, assign weight 2 to the second segment target, and assign weight 2 to the nth segment target. n-1 ,Right now:

[0056] w i =2 i-1 ,i∈{1,2,…,n}

[0057] Among them, w i represents the weight of the i-th segment, and n is the number of segments;

[0058] Or, according to the segment weight change method in the strategy parameters, assign a weight of 1 to the first segment target, a weight of 2 to the second segment target, and a weight of n to the nth segment target, that is:

[0059] w i =i,i∈{1,2,…,n}

[0060] Among them, w i represents the weight of the i-th segment, and n is the number of segments.

[0061] When weighting pixels in the neighboring area, the weight of the target neighboring area is set to be less than or equal to the weight corresponding to the target segment interval, and the weight of the neighboring area can be set as a ladder weight, that is, the closer to the target, the greater the weight of the neighboring area, and the farther away from the target, the smaller the weight of the neighboring area. Specifically:

[0062] According to the current target weight w l The size determines the scope of the outer adjacent area affected by it, which is recorded as the m circle of pixels around the target in the annotation matrix, w l The larger , the larger m can be;

[0063] According to the peripheral weight decay method in the strategy parameters, a numerical sequence of length m with step-by-step exponential decay is generated:

[0064] w j =w i *2 1-j j∈{1,2,…,m}

[0065] Among them, w j is the jth value in the sequence, m is the length of the sequence, w l is the weight assigned to the current target l according to its size;

[0066] Or, based on the surrounding weight decay method in the strategy parameters, generate a numerical sequence of length m with linear decay:

[0067] w j =w i -j+1 j∈{1,2,…,m}

[0068] Among them, w j is the jth value in the sequence, m is the length of the sequence, w l is the weight assigned to the current target l according to its size;

[0069] Use the first, second, and mth values ​​in the sequence to multiply the label values ​​of the pixels in the first, second, and mth circles around the target in the label matrix respectively.

[0070] This embodiment further provides an artificial intelligence model training device for image segmentation, which is used to implement an artificial intelligence model training method for image segmentation. The device specifically includes:

[0071] Initial labeling module 1 is used to label the original image, labeling the pixels of multiple target areas of the image as 1 and the pixels of other areas as background as -1, to obtain a labeling matrix;

[0072] The first weighting module 2 segments the target in the annotation matrix according to the number of pixels it occupies, sets a weight for each segmented interval, wherein the more pixels in the segmented interval, the smaller the weight, and uses the set weight to weight each target area to be identified in the annotation matrix;

[0073] The neighboring region selection module 3 selects the size of the neighboring region for the current target according to the current target. The size of the neighboring region is m pixels around the target to be identified. The smaller the number of pixels occupied by the target to be identified, the larger the m;

[0074] A second weighting module 4 is used to weight each circle of pixels in the adjacent area of ​​the labeling matrix, and the weight of a circle of pixels closer to the target to be identified is greater;

[0075] Training module 5 is used to train the artificial intelligence model using a training set consisting of the original image and the annotation results. The positive pixel positions in the annotation results are used as the class to be identified, and the negative pixel positions in the annotation results are used as the background class. The absolute values ​​of all pixel annotation results are used as the weights of the loss function to calculate the loss value for training, and the cycle is iterated until the loss value converges.

[0076] As an optional implementation, this embodiment may further include a policy formulation module for formulating policies, the parameters of which include size segmentation method, number of segments, segment weight change method, and peripheral weight attenuation method.

[0077] As an optional implementation, in this embodiment, the training module 5 uses a plug-in to inject corresponding control points of the neural network model.

[0078] This embodiment takes the classic image segmentation model U-Net as an example. Its loss function is a weighted cross entropy loss function, and the calculation formula is as follows:

[0079]

[0080] where Ω is the set of all pixels in the image, l(x) is the true class label of pixel x (e.g. a certain type of object or background), p(x) is the probability that the model predicts pixel x belongs to class l(x), and w(x) is the weight of pixel x. From this formula, it can be seen that the same prediction probability deviation occurs at pixels with different weights, and the loss value obtained by aggregation is not the same. l(x)

[0081] Image segmentation faces the problem of inter-class imbalance. Inter-class imbalance refers to the fact that the number of pixels in some classes is much larger than that in other classes. The model can quickly reduce the loss value by fitting the features of the class with the most pixels first, which leads to the model being biased towards the majority class. Introducing class weights can solve the problem of inter-class imbalance. When the weights are different, the same number of pixels that are misclassified will result in a larger loss value for high-weight classes and a smaller loss value for low-weight classes. Therefore, under the guidance of the goal of minimizing the loss value, the model will first fit the spectral, textural, and shape features of high-weight classes, thereby preferentially identifying and extracting targets of that class. Therefore, the user can assign a high weight to a certain type of object by using class weights, which can achieve the user's preference of preferentially identifying this type of object.

[0082] However, image segmentation also faces the problem of intra-class imbalance, which refers to the fact that the sizes of multiple target instances in the same class (e.g. multiple tumors in medical images, multiple lakes in remote sensing images, and multiple defects in photomask patterns) differ greatly. Because the number of pixels of large targets is large, the model can quickly obtain a small loss value by correctly identifying large targets during training, which leads to the model being biased towards large targets and small targets being in a less preferred position and being easily misidentified and extracted.

[0083] The technical problem to be solved by the present application is the model bias towards large targets caused by intra-class imbalance in the field of image segmentation. The present application provides an artificial intelligence model training method and device for image segmentation, which provides a technical means for accurate and efficient identification and extraction of small targets.

[0084] To make the application clearer, the annotation and training system of the artificial intelligence model for segmenting and extracting small targets will be described first.

[0085] In an embodiment of the present application, an artificial intelligence model training method for image segmentation is provided, the steps of which are described as follows Figure 2 , including:

[0086] Step S10: An original image is annotated, and the pixels of multiple target regions to be identified of the class to be identified in the image are annotated as 1, and the pixels of other regions are annotated as background class -1 to obtain an annotation matrix.

[0087] ​During AI model training, the amount of training data significantly impacts model efficiency. When the data volume is small, the model may not fully learn the underlying patterns within the data, resulting in underfitting. This can lead to suboptimal performance on both the training and test sets, and poor generalization.

[0088] In one embodiment, the selected image data is sufficiently representative, covering situations under combinations of time, space, and other attributes.

[0089] The present application identifies the target area from the input data by manual or automatic methods, sets the label value of the pixels belonging to the target area to 1, and sets the label value of the pixels belonging to the background area to -1, to obtain a labeling matrix.

[0090] In the field of image segmentation, different numbers are often assigned to pixels in areas where different types of targets are located. For example, in remote sensing images, 1 represents a lake, 2 represents a forest, 3 represents farmland, and 0 represents the background. Artificial intelligence models can simultaneously segment multiple types of targets in an image, but in practical applications, it is more common to extract a single type of target, that is, a binary classification of target and background. The most common annotation method for binary image segmentation is described in patent CN111368843A (A method for extracting glacial lakes based on semantic segmentation): through manual annotation, each pixel is judged to be a glacial lake, and the area judged to be a glacial lake is circled on LabelMe to generate a binary image with a background area value of 0 and a glacial lake area value of 1.

[0091] To better understand common annotation methods, please refer to Figure 5 In the embodiment shown in FIG. 4 , there are four targets in the image field of view, represented by the gray areas 100, 200, 300, and 400. The binary labeling matrix with the background area labeled as 0 and the target area labeled as 1 is shown in FIG. Figure 6 shown. Figure 6 The area with the value of 100, 200, 300, and 400 is marked as 1, and the background area other than that is marked as 0.

[0092] The initial labeling matrix generated in this application labels the target area as 1 and the background area as -1. Compared with the usual labeling method, the background area labeling is changed from 0 to -1. Its purpose is to reflect the impact of the target weight after multiplying it by the target weight. Figure 5 In the embodiment shown, the initial labeling matrix obtained according to the present application is as follows Figure 7 shown. Figure 7 The area with the value of 100, 200, 300, and 400 is marked as 1, and the background area other than that is marked as -1.

[0093] Step S20: Segment the target to be identified according to the number of pixels it occupies, and set a weight for each segmented interval, wherein the more pixels a segmented interval has, the smaller the weight.

[0094] In one embodiment, the segmentation method and number of segments in the strategy parameters are used to define what is a big goal and what is a small goal, and the segment weight change method in the strategy parameters is used to assign a weight to each segment.

[0095] After completing the initial labeling, the overall situation of the target is counted, including the total number of targets, the maximum / minimum pixel number range of the target, and the target number (frequency) distribution under various pixel numbers. The size segmentation method, number of segments, segment weight change method, and peripheral weight attenuation method are determined based on the overall situation of the target.

[0096] In one embodiment, the large object / small object segmentation method is: all objects are added to a descending queue according to the number of pixels, and then the descending queue is divided into multiple uniform or non-uniform segments.

[0097] In one embodiment, the large target / small target segmentation method is: divide the range of the maximum and minimum pixel numbers into multiple segments of equal proportion or other standards, and place each target into each segment according to the number of pixels.

[0098] In one embodiment, the number of segments is determined based on the total number of targets: when the total number of targets is less than 1,000, the number of segments is 2; when the total number of targets is between 1,000 and 10,000, the number of segments is 3; when the total number of targets is greater than 10,000, the number of segments is 4, that is, the number of segment intervals is n=4.

[0099] In one embodiment, the number of segments is determined based on the ratio between the target maximum number of pixels and the target minimum number of pixels: when the ratio is less than or equal to 4, the number of segments is 2; when the ratio is greater than 4 and less than or equal to 8, the number of segments is 3; when the ratio is greater than 8, the number of segments is 4.

[0100] After breaking down the goal into segments, assign weights to each segment, following the principle of giving smaller goals larger weights and larger goals smaller weights. The weights of each segment can vary exponentially, linearly, or in some other way.

[0101] In this application, a larger weight value should be assigned to smaller targets, and a smaller weight value should be assigned to larger targets.

[0102] In one embodiment, all targets are queued in descending order and evenly segmented, and a weight index or linear change strategy is adopted, specifically:

[0103] Put all targets into a queue in descending order according to the number of pixels, and targets with the same number of pixels occupy different positions;

[0104] Divide the descending queue into n even segments;

[0105] Assign a weight of 1 to the first segment goal, a weight of 2 to the second segment goal, and a weight of 2 to the nth segment goal. n-1 ,include:

[0106] w i =2 i-1 i∈{1,2,…,n}

[0107] Among them, w i represents the weight of the i-th segment, and n is the number of segments;

[0108] Or assign a weight of 1 to the first segment goal, a weight of 2 to the second segment goal, and a weight of n to the nth segment goal, including:

[0109] w i =ii∈{1,2,…,n}

[0110] Among them, w i Represents the weight of the nth segment, where n is the number of segments.

[0111] For a clear explanation and understanding of this process, please refer to Figure 3 The embodiment represented by . The X-axis coordinates (1, A, B, C, D) of the two sub-graphs in the figure represent the sequence numbers of the targets in the queue after they are placed in the descending queue according to the number of pixels (the larger target is closer to the head of the queue and the smaller the target is closer to the end of the queue); the Y-axis represents the number of pixels of each target; the curves in the two sub-graphs represent the correspondence between the number of target pixels and the sequence number of the target in the descending queue. In this embodiment, the queue is divided into 4 sections, which are represented by different grayscale blocks (the number of pixels at the dividing points of the blocks are A, B, and C respectively). When the strategy of weight index change is adopted, such as Figure 3 As shown in the sub-figure (a): the targets ranked between 1 and A (including A) are assigned a weight of 1, the targets ranked between A and B (including B) are assigned a weight of 2, the targets ranked between B and C (including C) are assigned a weight of 4, and the targets ranked between C and D (including D) are assigned a weight of 8. When the weight linear change strategy is adopted, such as Figure 3 As shown in subfigure (b): targets ranked between 1-A (including A) are assigned a weight of 1, targets ranked between A and B (including B) are assigned a weight of 2, targets ranked between B and C (including C) are assigned a weight of 3, and targets ranked between C and D (including D) are assigned a weight of 4.

[0112] In this embodiment, the descending queue is divided into four even segments, and the calculation process of the dividing point is:

[0113] Count the total number of targets to get D, divide D by 4, and use it as the segment length N;

[0114] Calculate 1 times the N value to get A;

[0115] Calculate 2 times the value of N to get B;

[0116] Calculate 3 times the value of N to get C.

[0117] In one embodiment, the interval range of the maximum and minimum number of pixels is segmented and a weight exponential change or linear change strategy is adopted, specifically:

[0118] According to the size segmentation method and number of segments in the strategy parameters, count the number of pixels of all targets, find the range of the maximum and minimum pixel numbers, and divide the pixel number range from large to small into n segments of equal proportion;

[0119] According to the number of pixels of each target, find the corresponding segment;

[0120] According to the weight change method in the strategy parameters, assign weight 1 to the first segment target, assign weight 2 to the second segment target, and assign weight 2 to the nth segment target. n-1 ,include:

[0121] w i =2 i-1 i∈{1,2,…,n}

[0122] Among them, w i represents the weight of the i-th segment, and n is the number of segments;

[0123] Or assign a weight of 1 to the first segment goal, a weight of 2 to the second segment goal, and a weight of n to the nth segment goal, including:

[0124] w i =ii∈{1,2,…,n}

[0125] Among them, w i represents the weight of the i-th segment, and n is the number of segments.

[0126] For a clear explanation and understanding of this process, please refer to Figure 4 Representative embodiment. The X-axis coordinates (A, B, C, D, E) of the two sub-graphs in the figure represent the number of target pixels (small targets are on the left, large targets are on the right, in this example, the minimum number of target pixels is A, and the maximum number of target pixels is E); the Y-axis represents the frequency of occurrence of the pixel number; Figure 4 The broken lines of the two subgraphs represent the overall distribution of the number of target pixels and their frequencies. In this embodiment, the target pixel number range is divided into four segments, each represented by a different grayscale block (the block dividing points are B, C, and D). When the weight index change strategy is adopted, as shown in FIG. Figure 4As shown in sub-figure (a): the target with the number of pixels between A and B (including B) is assigned a weight of 8, the target with the number of pixels between B and C (including C) is assigned a weight of 4, the target with the number of pixels between C and D (including D) is assigned a weight of 2, and the target with the number of pixels between D and E (including E) is assigned a weight of 1. When the weight linear change strategy is adopted, as shown in Figure 4 As shown in sub-figure (b): the target with the number of pixels between A and B (including B) is assigned a weight of 4, the target with the number of pixels between B and C (including C) is assigned a weight of 3, the target with the number of pixels between C and D (including D) is assigned a weight of 2, and the target with the number of pixels between D and E (including E) is assigned a weight of 1.

[0127] In this embodiment, the pixel range is divided into four equal-proportion segments, and the specific calculation process is as follows:

[0128] Find the minimum number of pixels A as a benchmark;

[0129] Find the maximum number of pixels E and calculate the ratio of E to A;

[0130] Calculate the fourth root of the ratio of E to A to get the ratio N;

[0131] Calculate the base A multiplied by N to get B;

[0132] Calculate the base A multiplied by the square of N to get C;

[0133] Calculate the base A multiplied by N cubed to get E.

[0134] Step S30: assigning weights to the neighboring area of ​​m circles of pixels surrounding the target to be identified. The fewer pixels occupied by the target to be identified, the larger m is.

[0135] This application multiplies the label values ​​of pixels in the label matrix belonging to the adjacent area outside each target by a number equal to or less than the corresponding weight value of the target, making the model more sensitive to boundaries when segmenting small targets.

[0136] Step S40: setting a weight for each of the m circles of pixels surrounding the target to be identified, and the weight of a circle of pixels closer to the target to be identified is greater.

[0137] This application increases the weight of pixels in the vicinity of small targets, so that the model will suffer greater loss when misjudging pixels around small targets. The specific steps are as follows:

[0138] According to the current target weight w l The size determines the scope of the outer adjacent area affected by it, which is recorded as the m circle of pixels around the target in the annotation matrix, w l The larger , the larger m can be;

[0139] According to the peripheral weight decay method in the strategy parameters, a numerical sequence of length m with step-by-step exponential decay is generated:

[0140] w j =w i ×2 1-j j∈{1,2,…,m}

[0141] Among them, w j is the jth value in the sequence, m is the length of the sequence, w l is the weight assigned to the current target l according to its size;

[0142] Or, based on the surrounding weight decay method in the strategy parameters, generate a numerical sequence of length m with linear decay:

[0143] w j =w i -j+1 j∈{1,2,…,m}

[0144] Among them, w j is the jth value in the sequence, m is the length of the sequence, w l is the weight assigned to the current target l according to its size.

[0145] Step S50: weighting the labeling matrix using the set weight value to obtain the labeling result.

[0146] For a better understanding of step S50, please refer to Figure 8 . Figure 8 Shown with Figure 5 The same image field of view. In one embodiment, according to the execution result of step S20, the 100, 200, 300, and 400 targets shown in the figure are divided into the first, second, third, and fourth segments, respectively, and obtain weights of 1, 2, 4, and 8. Therefore, the weighting of each target area is:

[0147] Multiply the initial annotation value of the pixels belonging to the 100 target area by 1 to obtain Figure 8 The annotation results in the gray area shown as 100;

[0148] Multiply the initial annotation value of the pixels belonging to the 200 target area by 2 to obtain Figure 8 The annotation results in the dark grey area shown in 200;

[0149] Multiply the initial annotation value of the pixels belonging to the 300 target area by 4 to obtain Figure 8 The annotation results in the dark grey area shown in 300;

[0150] Multiply the initial annotation value of the pixels belonging to the 400 target area by 8 to obtain Figure 8 The annotation results in the dark gray area shown in 400.

[0151] The weighting process of the pixels in the outer area of ​​each target can be referred to Figure 8 In one embodiment, the results of steps S30 and S40 are as follows: the influence ranges of targets 100, 200, 300, and 400 are 0, 1, 2, and 3 circles of pixels outside, respectively, and an exponential decay strategy is adopted. The specific process of weighting the outer area is:

[0152] For the 200 targets in the figure, the weight of the first outer circle is 2, and the initial annotation value -1 is multiplied by 2 to obtain the following: Figure 8 The annotation results in the light grey area shown in 201;

[0153] For the target 300 in the figure, the weights of the first and second outer circles are 4 and 2 respectively. The initial marked value -1 is multiplied by 4 or 2 to get the following: Figure 8 The annotation result in the light grey area shown in 301;

[0154] For the target 400 in the figure, the weights of the first, second, and third outer circles are 8, 4, and 2 respectively. The initial marked value -1 is multiplied by 8, 4, or 2 to get the following: Figure 8 The annotation result in the light grey area shown as 401.

[0155] Step S60: Use the training set consisting of the original image and the annotation results to train the artificial intelligence model, take the positive pixel positions in the annotation results as the class to be identified, and the negative pixel positions in the annotation results as the background class, and use the absolute values ​​of all pixel annotation results as the weights of the loss function to calculate the loss value for training, and iterate until the loss value converges.

[0156] exist Figure 8 In one embodiment shown, all pixels labeled with values ​​of 1, 2, 4, and 8 are considered to be the target class, and all pixels labeled with values ​​of -1, -2, -4, and -8 are considered to be the background class. During model training, the model's predicted classification results for each pixel are compared. If the classifications disagree, the pixel is considered misclassified. When calculating the misclassification loss, the absolute value of each pixel's label value is used as a weight.

[0157] exist Figure 8 In the embodiment shown, the loss value caused by the classification error of the same pixel is based on the pixel in the area shown as 100:

[0158] If the pixel is in the area shown as 200, the loss value increases to 2 times;

[0159] If the pixel is in the area shown as 300, the loss value increases to 4 times;

[0160] If the pixel is in the area shown as 400, the loss value increases to 8 times.

[0161] Therefore, when the model misjudges small targets and surrounding pixels, it will suffer a greater loss value due to its higher weight. During the training process where the model adjusts its own parameters to gradually reduce the loss value, the model will give priority to matching small targets.

[0162] The present invention provides a test example for target segmentation and extraction of an image. This embodiment selects an image with a size of 7770×7920 pixels, which includes 271 targets, of which the maximum number of target pixels is 3887 and the minimum number of target pixels (12 in total) is 12. The test used the common annotation training method and the annotation training method proposed in this application, and calculated the U-Net model's segmentation and extraction results for these 12 small targets. The test results show that under the common method, the intersection-over-union (IOU) of these 12 small targets and the true annotations is 20%, while the intersection-over-union (IOU) obtained by the method proposed in this application is 63.8%, and the improvement effect is very obvious. This shows that the method and device of the present application can keenly discover and monitor the early development of related targets.

[0163] The previous discussion of this application uses binary classification as an example, but this does not prevent this idea from being applied to multi-classification situations: different weights are assigned according to the sizes of multiple instances in each target class.

[0164] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for training an artificial intelligence model for image segmentation, characterized in that: The following steps are involved: The original image is labeled, and the pixels of multiple target areas to be identified in the image are labeled as 1, and the pixels of other areas as background are labeled as -1, to obtain a labeling matrix; Segment the target according to the number of pixels it occupies, and set a weight for each segment. The more pixels in a segment, the smaller the weight. Assign weights to the neighboring area of ​​m circles of pixels surrounding the target to be identified. The fewer pixels the target to be identified occupies, the larger m is. Set a weight for each of the m circles of pixels surrounding the target to be identified, and the closer the circle of pixels is to the target to be identified, the greater the weight; Use the set weight value to weight the annotation matrix to obtain the annotation result; The artificial intelligence model is trained using a training set consisting of original images and annotation results. The positive pixel positions in the annotation results are used as the class to be identified, and the negative pixel positions in the annotation results are used as the background class. The absolute values ​​of all pixel annotation results are used as the weights of the loss function to calculate the loss value for training, and the cycle is iterated until the loss value converges.

2. The training method according to claim 1, characterized in that Divide the target to be identified into four segmented intervals according to the number of pixels, determine the minimum number of pixels and the maximum number of pixels of the target to be identified, calculate the fourth root of the ratio of the maximum number of pixels to the minimum number of pixels, which is N, and the number of pixels belongs to (minimum number of pixels × N 3 , the maximum number of pixels] is the first segment interval, and the weight of the target to be identified belonging to this segment interval is set to the lowest weight; the number of pixels belongs to (minimum number of pixels × N 2 , minimum number of pixels × N 3 ] is the second segment interval, and the weight of the target to be identified belonging to this segment interval is set to the second lowest weight; The number of pixels belongs to (minimum number of pixels × N, minimum number of pixels × N 2 ] is the third segment interval, and the weight of the target to be identified belonging to this segment interval is set to the second highest weight; The number of pixels belonging to [minimum number of pixels, minimum number of pixels × N] is the fourth segment interval, and the weight of the target to be identified belonging to this segment interval is set to the highest weight.

3. The training method according to claim 1 or 2, characterized in that: The weight value of the target belonging to the i-th segment interval is: w i =2 i-1 ,i∈{1,2,…,n} Among them, w i represents the weight value of the target belonging to the i-th segment interval, and n is the number of segment intervals.

4. The training method according to claim 1 or 2, characterized in that: The weight value of the target belonging to the i-th segment interval is: w i =i,i∈{1,2,…,n} Among them, w i represents the weight value of the target belonging to the i-th segment interval, and n is the number of segment intervals.

5. The training method according to claim 1, wherein: The maximum weight value of the m-circle pixels in the neighboring area does not exceed the segment interval weight of the corresponding target, and the closer the circle of pixels is to the target, the higher the weight.

6. The training method according to claim 5, characterized in that The weight value of the target j-th circle pixel belonging to the i-th segment interval is: w j =w i ×2 1-j ,j∈{1,2,…,m} Among them, w j represents the weight value of the jth circle of pixels outside the target belonging to the i-th segment interval; w i Represents the weight value of the target belonging to the i-th segment interval.

7. The training method according to claim 5, characterized in that The weight value of the target j-th circle pixel belonging to the i-th segment interval is: w j =w i -j+1,j∈{1,2,…,m} Among them, w j represents the weight value of the jth circle of pixels outside the target belonging to the i-th segment interval; w i Represents the weight value of the target belonging to the i-th segment interval.

8. The training method according to any one of claims 1 to 6, characterized in that: m={0,1,…,n-1}, where n is the number of segment intervals.

9. An artificial intelligence model training device for image segmentation, characterized in that: The method for training an artificial intelligence model for image segmentation according to claim 1 comprises: The initial annotation module is used to annotate the original image, annotate the pixels of multiple target areas of the image to be identified as 1, and annotate the pixels of other areas as background as -1, to obtain the annotation matrix; The first weighting module is used to segment the target to be identified according to the number of pixels occupied by the target to be identified, set a weight for each segment interval, wherein the more pixels in the segment interval, the smaller the weight, and use the set weight to weight each target area to be identified in the annotation matrix; A neighboring region selection module is used to select the size of a neighboring region for the current target to be identified. The size of the neighboring region is m pixels around the current target to be identified. The smaller the number of pixels occupied by the current target to be identified, the larger m is. The second weighting module is used to weight each circle of pixels in the vicinity of the target to be identified in the annotation matrix, and the weight of a circle of pixels closer to the target to be identified is greater; The training module is used to train the artificial intelligence model using a training set consisting of original images and annotation results. The positive pixel positions in the annotation results are used as the class to be identified, and the negative pixel positions in the annotation results are used as the background class. The absolute values ​​of all pixel annotation results are used as the weights of the loss function to calculate the loss value for training, and the cycle is iterated until the loss value converges.

Images