A page layout analysis method and system based on image instance segmentation
By processing images through feature extraction, encoders, and attention mechanisms, and combining random occlusion prediction, this approach addresses the problem of insufficient consideration of positional relationships in traditional instance segmentation algorithms, thereby improving the model's generalization ability and segmentation performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- YUSHAN COUNTY YUTOU EDUCATION INVESTMENT CO LTD
- Filing Date
- 2022-09-26
- Publication Date
- 2026-07-31
AI Technical Summary
Traditional instance segmentation algorithms fail to effectively consider the positional relationships between different categories, resulting in insufficient model generalization ability and long detection and segmentation time.
The image is segmented proportionally by using a feature extraction network. The relationship between feature maps is obtained by using an encoder and attention mechanism. Random occlusion and prediction processing are used to improve the segmentation effect of the model on unknown image data.
It improves the model's generalization ability, enabling it to better handle images with partially blurred or occluded areas, thus improving the accuracy and efficiency of segmentation.
Smart Images
Figure CN115588017B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a layout analysis method and system based on image instance segmentation. Background Technology
[0002] Image segmentation refers to dividing an image into several non-overlapping regions based on features such as grayscale, color, spatial texture, and geometric shape. These features should exhibit consistency or similarity within the same region, while showing significant differences between different regions. Key techniques include object detection, semantic segmentation, and instance segmentation. Instance segmentation combines object detection and semantic segmentation: object detection identifies the target in the image, and then each pixel is labeled (semantic segmentation). However, traditional instance segmentation methods first perform object detection, then add a fully connected network to perform the instance segmentation task. However, traditional methods... Instance segmentation algorithms do not consider the positional relationships between different categories and have long detection and segmentation time. With the development of instance segmentation technology, a new algorithm, SOLO, has been proposed. It divides the image into an S*S grid, extracts features through a fully convolutional network (FCN), and then splits into two branches: a category branch and a mask branch. The category branch predicts the semantic category for the center point of each grid to obtain the semantic category. Then, the image is mapped onto the mask branch to obtain the instance mask. Although the SOLO algorithm considers the positional relationships between different categories, it does not add randomness, which leads to poor generalization ability of the model. Summary of the Invention
[0003] To address the aforementioned technical problems, the present invention aims to provide a layout analysis method and system based on image instance segmentation. This method considers the randomness of occluded areas in images while taking into account the relationship between image position information, thereby increasing the generalization ability of the model and enabling it to achieve better segmentation results for unknown image data.
[0004] The first technical solution adopted in this invention is: a layout analysis method based on image instance segmentation, comprising the following steps:
[0005] The feature extraction network is used to extract features from the image to be detected, resulting in a feature map.
[0006] The feature map is divided into equal parts to obtain the segmented feature map;
[0007] Based on the encoder, the relationship between the segmented feature maps is obtained through the attention mechanism and then applied to the segmented feature maps to obtain the summed feature maps.
[0008] The summed feature maps are then subjected to random occlusion and prediction processing in sequence to obtain the recognition analysis map.
[0009] Furthermore, the step of extracting features from the image to be detected using a feature extraction network to obtain a feature map specifically includes:
[0010] Obtain the image to be detected;
[0011] The image to be detected is input into the feature extraction network model;
[0012] The convolutional layer of the feature extraction network model performs convolution operations on the image to be detected, resulting in shallow and deep feature maps.
[0013] Pooling layers based on feature extraction network models are used to fuse shallow and deep feature maps to obtain a feature map.
[0014] Furthermore, the step of obtaining the relationship between the segmented feature maps based on the encoder through an attention mechanism and assigning it to the segmented feature maps to obtain the summed feature map specifically includes:
[0015] Based on the encoder, the segmented feature map is encoded to obtain the encoded feature map;
[0016] Key node information of the segmented feature map is obtained through an attention mechanism;
[0017] The encoded feature map is summed with the key node information of the feature map to obtain a feature map containing key information.
[0018] The feature map containing key information is processed by forward propagation using an activation function to obtain the activated feature map.
[0019] The activated feature map is summed with the feature map containing key information to obtain the encoder's output.
[0020] The encoder output is summed with the segmented feature map to obtain the summed feature map.
[0021] Furthermore, the step of encoding the segmented feature map based on the encoder to obtain the encoded feature map specifically includes:
[0022] The segmented feature map is input into the encoder;
[0023] The information encoding module based on the encoder performs information encoding processing on the segmented feature map to obtain a feature map with information association.
[0024] The position encoding module based on the encoder performs position encoding processing on the segmented feature map using sine and cosine functions to obtain a feature map with positional association.
[0025] The feature maps with information association and those with location association are summed to obtain the encoded feature map.
[0026] Furthermore, the step of obtaining key node information of the segmented feature map through an attention mechanism specifically includes:
[0027] The query, key, and value matrix is introduced and multiplied with the vector of the segmented feature map to obtain the corresponding query, key, and value matrix after multiplication;
[0028] Multiply the query matrix and the transpose of the key matrix to obtain the fraction matrix;
[0029] The value of the attention matrix is obtained by multiplying the fraction matrix and the value matrix.
[0030] The values of the attention matrix are mapped using a normalized exponential function to obtain the mapped data;
[0031] The mapped data is weighted to obtain the key node information of the segmented feature map.
[0032] Furthermore, the calculation formula for the attention mechanism is as follows:
[0033] Q = W Q *X embedding
[0034] K = W K *X embedding
[0035] V = W V *X embedding
[0036]
[0037] In the above formula, W Q W K W V X represents the matrix corresponding to the key and value of a randomly initialized query in the network. embedding dk represents the encoded feature map, Q, K, and V represent the matrices corresponding to the query, key, and value, respectively, dk represents the network depth, Attention(·) represents the calculation formula of the attention mechanism, and softmax(·) represents the normalized exponential function.
[0038] Furthermore, the step of sequentially performing random occlusion and prediction processing on the summed feature map to obtain the recognition analysis map specifically includes:
[0039] N grid data points from the summed feature map are selected for occlusion processing to obtain the occluded grid data.
[0040] The center point and surrounding nodes are predicted for the occluded grid data to obtain the center point prediction data and surrounding node prediction data.
[0041] Classify the predicted data for the center point and the predicted data for the surrounding nodes;
[0042] If the predicted data of the center point and the predicted data of the surrounding nodes are determined to be of the same category, the predicted data of the center point and the predicted data of the surrounding nodes are connected to obtain the identification analysis map.
[0043] Furthermore, it also includes scaling the recognition and analysis graph;
[0044] Furthermore, the formula for calculating the loss of the occluded grid data is as follows:
[0045]
[0046] In the above formula, Total loss(·) represents the formula for calculating the loss of occluded grid data. This indicates the calculation of classification loss. This represents the calculation of mask loss, where N represents the occluded grid data, and y... i Represents the actual label value. p(y) represents the predicted label value. i ) represents the probability corresponding to the true label value.
[0047] The second technical solution adopted in this invention is: a layout analysis system based on image instance segmentation, comprising:
[0048] The feature extraction module is used to extract features from the image to be detected through a feature extraction network to obtain a feature map.
[0049] The segmentation module is used to perform proportional segmentation on the feature map to obtain the segmented feature map;
[0050] The encoding module, based on the encoder, obtains the relationship between the segmented feature maps through the attention mechanism and assigns it to the segmented feature maps to obtain the summed feature map;
[0051] The output module is used to perform random occlusion and prediction processing on the summed feature map in sequence to obtain the recognition analysis map.
[0052] The beneficial effects of the method and system of this invention are as follows: This invention performs feature extraction processing on the image to be detected and divides it into several grids in an equal proportion. Furthermore, by introducing a position encoding module, it obtains the information correlation between the grids in the image to be detected, better taking into account the positional relationship between each grid. Furthermore, it uses a multi-head attention mechanism to find the most critical node information in the image to be detected, and better learns the relationship between the context, thereby better learning the relationship between each part in the image and obtaining more accurate prediction results. In addition, the idea of random occlusion is introduced, which allows the data of the unoccluded area to have the information of the occluded area, thereby improving the recognition effect of blurred and black dot areas in images with some blurriness and black dots, and increasing the generalization ability of the model. Attached Figure Description
[0053] Figure 1 This is a flowchart of the steps of a layout analysis method based on image instance segmentation according to the present invention;
[0054] Figure 2 This is a structural block diagram of a layout analysis system based on image instance segmentation according to the present invention. Detailed Implementation
[0055] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
[0056] Reference Figure 1 This invention provides a layout analysis method based on image instance segmentation, which includes the following steps:
[0057] S1. The feature extraction network is used to extract features from the image to be detected, resulting in a feature map;
[0058] Specifically, a random image is obtained, and further preprocessing is required so that the computer terminal can recognize the image for subsequent operations. Therefore, the input image is processed by a feature extraction network to extract features. The feature extraction network consists of convolutional layers and pooling layers.
[0059] Convolution is an operation that multiplies corresponding elements and then adds the results. Through convolution, the network can learn specific visual features; at lower layers of the network, these might be spots of certain colors, while at higher layers, they might be regions representing a problem.
[0060] Pooling operations, or pooling layers, mimic the human visual system's dimensionality reduction of data. In building convolutional neural networks, they are often used after convolutional layers to reduce the feature dimension of the convolutional layer's output. This effectively reduces network parameters and prevents overfitting. After the feature map is obtained through the convolutional layer, pooling operations are performed on it. These can be either max pooling or mean pooling. Max pooling divides the input image into several rectangular regions and outputs the maximum value for each sub-region; that is, it takes the point with the largest value in the local receptive field. Similarly, mean pooling takes the average value in the local receptive field, thus obtaining the maximum and average features of a certain region of the feature map and downsampling the data.
[0061] The shallow layers of a network typically extract lower-level features, such as image textures and borders. In layout analysis, this might correspond to a sentence or a word in an image. The deeper layers of the network extract higher-level, more abstract features. Compared to lower-level features, higher-level features are less granular and have lower resolution, but they are richer in semantic information. Often, the information from a single pixel can represent the features of an entire area in the original image. In layout analysis, this might correspond to a title or an inserted image. In other words, after passing through the feature extraction network, the computer knows what the specific content of each location in the image is, thus obtaining a feature map.
[0062] S2. Perform equal-ratio segmentation on the feature map to obtain the segmented feature map;
[0063] Specifically, the feature map is divided into N*N grids in equal proportions: the feature map is divided into N regions according to both height and width, and the N regions divided by height and the N regions divided by width are combined to finally obtain N*N grids on the feature map.
[0064] S3. Based on the encoder, the relationship between the segmented feature maps is obtained through the attention mechanism and assigned to the segmented feature maps to obtain the summed feature maps.
[0065] S31. Based on the encoder, the segmented feature map is encoded to obtain the encoded feature map;
[0066] Specifically, the segmented feature map is first processed by information data encoding, transforming the data in a grid into a vector with the embedding dimension, thus encoding the data input to the encoder. The encoding process involves treating each grid as a value; the closer the values are, the more likely the two grids are to belong to the same label. This encoding step reveals the relationship between the data in each grid. Next, the segmented feature map is processed by positional information encoding. Since the encoder does not have the iterative operation of a recurrent neural network, it must provide positional information for each grid to identify the sequential relationship between them. Specifically, positional encoding is performed on each position using sine and cosine functions. The order in which each grid is input into the network may be random, and the order will be shuffled after passing through the network. This positional encoding step reveals the positional information between each grid, thus maintaining the relationship between the positions of each grid during training.
[0067] The calculation formula for the location information encoding, i.e., the sine and cosine functions, is shown below:
[0068]
[0069]
[0070] In the above formula, pos represents the center point position of each grid, i represents the embedding length, and d model The dimension of the position vector is represented by PE(·), and PE(·) represents the position encoding calculation formula.
[0071] The image information encoding and location data encoding are summed at corresponding positions to ensure that the data maintains both the connection between data within each grid and the connection between positions within each grid. The expression for the summation is as follows:
[0072] X embedding =X embedding +X pos
[0073] In the above formula, the second X embedding Represents the original encoded vector, the first X embedding X represents the result of summing the original encoding vector and the positional encoding vector. pos This represents a feature map with locational associations;
[0074] S32. Obtain key node information of the segmented feature map through an attention mechanism;
[0075] Specifically, the relevance of the input data is calculated by introducing the concepts of query, key, and value. The query, key, and value are obtained through a linear mapping of the input data. First, the key and query are compared to obtain a score. Finally, the score is multiplied by the value to obtain the final result. The calculation process involves obtaining the attention matrix by multiplying the query by the transpose of the key, and then mapping the values of the attention matrix using a normalized exponential function. Ultimately, each value in the attention matrix is mapped to [0, 1], resulting in mapped data. Data closer to 0 is less important, and data closer to 1 is more important. This allows the model to learn the key node information of the segmented feature map through the attention mechanism. Finally, the values are weighted according to the attention matrix. Through multi-head attention, the most useful information in the data can be found.
[0076] The calculation formula for the attention mechanism is as follows:
[0077] Q = W Q *X embedding
[0078] K = W K *X embedding
[0079] V = W V *X embedding
[0080]
[0081] In the above formula, W Q W K W V X represents the matrix corresponding to the key and value of a randomly initialized query in the network. embedding dk represents the encoded feature map, Q, K, and V represent the matrices corresponding to the query, key, and value, respectively, dk represents the network depth, Attention(·) represents the calculation formula of the attention mechanism, and softmax(·) represents the normalized exponential function.
[0082] S33. The encoded feature map is summed with the key node information of the feature map to obtain a feature map with key information.
[0083] Specifically, the encoded feature map is summed with the key node information of the feature map. Then, each element in each row is subtracted from the mean of that row and divided by the standard deviation of that row. This prevents overfitting. The calculation formula for the summation process in step S33 is as follows:
[0084] X hidde =X embedding +Attention(Q, K, V)
[0085] In the above formula, X hidden Feature maps representing key information;
[0086] S34. The feature map containing key information is processed by forward propagation through the activation function to obtain the activated feature map.
[0087] Specifically, the upper-level results are activated through two activation functions. The purpose of this step is to remove negative values from the data while keeping positive values unchanged.
[0088] The calculation formula for the forward propagation is as follows:
[0089] X hidden =ReLU(X) hidden *W1*W2)
[0090] In the above formula, ReLU(·) represents the activation function, and W1 and W2 represent the weights corresponding to the two linear mappings.
[0091] S35. The activated feature map is summed with the feature map containing key information to obtain the encoder output.
[0092] Specifically, the activated feature map is summed with the feature map containing key information. Then, the mean of each row is subtracted from each element in each row, and then divided by the standard deviation of the row. This step is to prevent overfitting and finally obtain the encoder output.
[0093] The summation calculation formula in step S35 is as follows:
[0094] X hidd =X embedding +Attention(Q, K, V)+X hidde
[0095] In the above formula, X hidde This indicates the output of the encoder;
[0096] S36. The encoder output and the segmented feature map are summed to obtain the summed feature map.
[0097] Specifically, the data of the original N*N grid is added to the corresponding positions of the data of the N*N grid output by the encoder;
[0098] X = X + X hidden
[0099] In the above formula, X represents the summed feature map;
[0100] S4. Perform random occlusion and prediction processing on the summed feature map in sequence to obtain the predicted feature map.
[0101] S41. Perform random occlusion processing on the summed feature map;
[0102] Specifically, N grids are randomly selected from the N*N grids of the summed feature map for occlusion. By training the model through random occlusion, the unoccluded areas can better learn the relationships between the occluded areas. Thus, the unoccluded areas not only have the data and positional relationships of their own areas but also the relationships between the occluded areas, thereby increasing the model's generalization ability. For example, when an image with partially blurred areas or black dots is input, the model can also learn the relationships between the blurred areas or black dot areas very well.
[0103] The expression for the random occlusion is as follows:
[0104]
[0105] In the above formula, N represents the occluded grid data, and X represents the feature map with grid segmentation;
[0106] S42. Predict the feature map after random occlusion;
[0107] Specifically, during the model training process, each training process only predicts the data after random occlusion. The specific operation is as follows: first, predict the category of the center point of each occluded grid, and search for local maxima for each occluded grid, suppress non-maximum values, and only retain the output of the maximum value. Then, determine whether the category of the predicted occluded area grid is the same as the category of its surrounding grids. If they are the same, connect the areas with the same category to obtain a mask. If they are different, the occluded area grid is a separate mask. If the predicted category of the occluded area is background or there is no category, then the occluded area has no mask.
[0108] Mask data is a category. A single mask data is a grid whose category is different from all the grids around it, so it is a single mask data. Non-mask data is the background. In terms of layout analysis, it may correspond to an answer sheet, which is divided into N*N grids. For example, if grid 1 and grid 2 to its right are both of the category of "question", then they are merged as mask data. If grid 3 is of the category of "title" and none of its surrounding grids have this category, then grid 3 is a single mask. If grid 4 corresponds to a blank space on the answer sheet and has no category, then it corresponds to non-mask.
[0109] The expression for predicting the feature map after random occlusion is as follows:
[0110]
[0111] In the above formula, X predictThis represents the predicted value of X;
[0112] The loss function for the further random occlusion feature maps is shown below:
[0113]
[0114]
[0115]
[0116] In the above formula, Total loss(·) represents the formula for calculating the loss of occluded grid data. This indicates the calculation of classification loss. This represents the calculation of mask loss, where N represents the occluded grid data, and y... i Represents the actual label value. p(y) represents the predicted label value. i ) represents the probability corresponding to the true label value.
[0117] S5. Scale the predicted feature map to obtain the final recognition and analysis map.
[0118] Specifically, because the height and width of the predicted feature map are very different from those of the input image, people cannot directly obtain the data content information of the image. Therefore, the feature map needs to be scaled proportionally to the size of the input image to facilitate human viewing. The width and height of the predicted feature map are scaled according to the ratio of the width and height of the input image to the width and height of the feature map, respectively, to obtain the final output image.
[0119] Furthermore, the expression for scaling an image proportionally is shown below:
[0120]
[0121]
[0122] In the above formula, (w * h * () represents the coordinates of a point in the output image, (w, h) represents the coordinates of a point on the predicted feature map, and X represents the coordinates of the point. w X h This represents the width and height of the predicted feature map. This indicates the width and height of the input image.
[0123] This invention can be applied in the field of education, but is not limited to this field. When grading papers, it is often necessary to upload the entire answer sheet online so that teachers can identify the location of different questions. By analyzing the layout of the network, the questions can be automatically separated from other areas, and only the questions can be uploaded online for teachers to review.
[0124] Reference Figure 2 A layout analysis system based on image instance segmentation, comprising:
[0125] The feature extraction module is used to extract features from the image to be detected through a feature extraction network to obtain a feature map.
[0126] The segmentation module is used to perform proportional segmentation on the feature map to obtain the segmented feature map;
[0127] The encoding module, based on the encoder, obtains the relationship between the segmented feature maps through the attention mechanism and assigns it to the segmented feature maps to obtain the summed feature map;
[0128] The output module is used to perform random occlusion and prediction processing on the summed feature map in sequence to obtain the recognition analysis map.
[0129] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.
[0130] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. A layout analysis method based on image instance segmentation, characterized in that, Includes the following steps: S1, The feature extraction network is used to extract features from the image to be detected to obtain a feature map; S2, perform equal-ratio segmentation on the feature map to obtain the segmented feature map; S3, based on the encoder, obtains the relationships between the segmented feature maps through an attention mechanism and assigns them to the segmented feature maps to obtain a summed feature map; the summed feature map is then subjected to random occlusion and prediction processing in sequence to obtain a recognition analysis map, and the specific steps are as follows: S31. Based on the encoder, the segmented feature map is encoded to obtain the encoded feature map; Specifically, the segmented feature map is first processed by information data encoding, which transforms the data in a grid into a vector with the embedding dimension, thereby encoding the data input to the encoder. Through this encoding step, the relationship between the data in each grid can be obtained. Then, the segmented feature map is processed by position information encoding. Specifically, each position is encoded using sine and cosine functions. The order in which each grid is input into the network may be random, and the position order will be shuffled after passing through the network. Through this position encoding step, the position information between each grid can be obtained, thereby maintaining the relationship between the positions of each grid during training. The calculation formula for the location information encoding, i.e., the sine and cosine functions, is shown below: ， In the above formula, pos represents the center point position of each grid, i represents the embedding length, dmodel represents the dimension of the position vector, and PE(·) represents the position encoding calculation formula; The image information encoding and location data encoding are summed at corresponding positions to ensure that the data maintains both the connection between data within each grid and the connection between positions within each grid. The expression for the summation is as follows: ， In the above formula, the second X embedding Represents the original encoded vector, the first X embedding X represents the result of summing the original encoding vector and the positional encoding vector. pos This represents a feature map with locational associations; S32. Obtain key node information of the segmented feature map through an attention mechanism; Specifically, the relevance of the input data is calculated by introducing the concepts of query, key, and value. The query, key, and value are obtained through a linear mapping of the input data. First, the key and query are compared to obtain a score. Finally, the score is multiplied by the value to obtain the final result. The calculation process involves obtaining the attention matrix by multiplying the query by the transpose of the key, and then mapping the values of the attention matrix using a normalized exponential function. Ultimately, each value in the attention matrix is mapped to [0, 1], resulting in mapped data. Data closer to 0 is less important, and data closer to 1 is more important. This allows the model to learn the key node information of the segmented feature map through the attention mechanism. Finally, the values are weighted according to the attention matrix. Through multi-head attention, the most useful information in the data can be found. The calculation formula for the attention mechanism is as follows: ， In the above formula, WQ, WK, and WV represent the matrices corresponding to the initial random query, key, and value of the network, respectively; Xembedding represents the encoded feature map; Q, K, and V represent the matrices corresponding to the query, key, and value, respectively; dk represents the network depth; Attention(·) represents the calculation formula for the attention mechanism; and softmax(·) represents the normalized exponential function. S33. The encoded feature map is summed with the key node information of the feature map to obtain a feature map with key information. Specifically, the encoded feature map is summed with the key node information of the feature map. Then, each element in each row is subtracted from the mean of that row and divided by the standard deviation of that row. This prevents overfitting. The calculation formula for the summation process in step S33 is as follows: Xhidde=X embedding+Attention(Q,K,V) In the above formula, Xhidden represents a feature map containing key information; S34. The feature map containing key information is processed by forward propagation through the activation function to obtain the activated feature map. Specifically, the upper-level results are activated through two activation functions. The purpose of this step is to remove negative values from the data while keeping positive values unchanged. The calculation formula for the forward propagation is as follows: Xhidden = ReLU(Xhidden * W1 * W2) In the above formula, ReLU(·) represents the activation function, and W1 and W2 represent the weights corresponding to the two linear mappings. S35. The activated feature map is summed with the feature map containing key information to obtain the encoder output. Specifically, the activated feature map is summed with the feature map containing key information. Then, for each element in each row, the mean of that row is subtracted, and the result is divided by the standard deviation of that row. This step aims to prevent overfitting. Obtain the encoder output; The summation calculation formula in step S35 is as follows: Xhidd=X embedding+Attention(Q,K,V)+Xhidde In the above formula, Xhidde represents the output of the encoder; S36. The output of the encoder and the segmented feature map are summed to obtain the summed feature map. Specifically, the data of the original N*N grid is added to the corresponding positions of the data of the N*N grid output by the encoder. ， In the above formula, X represents the summed feature map; S4. Perform random occlusion and prediction processing on the summed feature map in sequence to obtain the predicted feature map. S41. Perform random occlusion processing on the summed feature map; Specifically, N grids are randomly selected from the N*N grids of the summed feature map for occlusion. By training the model through random occlusion, the unoccluded regions can learn the connections between the occluded regions. Thus, the unoccluded regions not only have the data and positional connections of their own regions but also the connections between the occluded regions, thereby increasing the generalization ability of the model. The expression of the random occlusion is shown below: , In the above formula, N represents the occluded grid data, and X represents the feature map with grid segmentation; S42. Predict the feature map after random occlusion; Specifically, during the model training process, each training process only predicts the data after random occlusion. The specific operation is as follows: first, predict the category of the center point of each occluded grid, and search for local maxima for each occluded grid, suppress non-maximum values, and only retain the output of the maximum value. Then, determine whether the category of the predicted occluded area grid is the same as the category of its surrounding grids. If they are the same, connect the areas with the same category to obtain a mask. If they are different, the occluded area grid is a separate mask. If the predicted category of the occluded area is background or there is no category, then the occluded area has no mask. Mask data represents a category; a single mask data point consists of a grid cell whose category differs from its surrounding grid cells, thus constituting a separate mask data point. Non-mask data represents the background. The expression for predicting the feature map after random occlusion is as follows: , In the above formula, Xpredict represents the predicted value of X; The loss function for the randomly occluded feature map is as follows: ， ， In the above formula, Total loss (·) represents the formula for calculating the loss of occluded grid data. denotes the classification loss computation, represents the mask loss computation, N represents the occluded grid data, and yi represents the actual label value. denotes the predicted label value, p(yi) denotes the probability corresponding to the true label value, S5. Scale the predicted feature map to obtain the final recognition and analysis map. The predicted feature map is scaled proportionally to the size of the input image, where the width scaling ratio is the input image width divided by the feature map width, and the height scaling ratio is the input image height divided by the feature map height. The width and height of the predicted feature map are scaled according to these two ratios to obtain the final output image. The expression for the proportional scaling is as follows: ， In the above formula, (w*, h*) represents the coordinate data of a point in the output image, (w, h) represents the coordinate data of a point in the predicted feature map, and Xw and Xh represent the width and height of the predicted feature map. , denotes the width and height of the input picture.