An image semantic segmentation method and system

Abstract

The present invention discloses an image semantic segmentation method and system based on dynamic convolution attention, which relates to the technical field of image processing. It includes collecting the image to be segmented and preprocessing the image to be segmented, inputting the preprocessed image to be segmented into a preset model for segmentation to obtain a preliminary segmentation result; collecting the morphological features of small targets, obtaining the object overlap coefficient through data processing, screening the image of the object overlap area to be enlarged and based on the texture features, analyzing to obtain the object overlap coefficient and the local detail richness coefficient to define the maximum magnification upper limit formula; then combining the correlation coefficient to set the correction and suppression terms, and determining the dynamic adjustment strategy of the magnification according to the threshold comparison. It can not only divide different overlap areas through the threshold to achieve dynamic combination of local details and global overlap suppression, but also avoid the distortion diffusion of high-overlap areas while ensuring detail enhancement, realizing refined magnification.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an image semantic segmentation method and system. Background Art

[0002] When existing image semantic segmentation technologies identify and segment small targets, due to the small size of small targets, traditional segmentation algorithms often have difficulty accurately distinguishing and processing them, resulting in poor segmentation effects; in addition, the overlap problem between small targets also greatly affects the segmentation quality. Especially when performing image segmentation in natural scenes and complex backgrounds, the mutual occlusion and overlap between small targets become more complex, and existing technologies are difficult to effectively solve this problem;

[0003] Based on the above technical problems, existing image segmentation methods have certain limitations when processing overlapping regions; improving the detailed information of local regions can help improve the segmentation effect, but most of these methods rely on a fixed magnification factor and lack personalized adjustment for the precise magnification requirements of specific regions; therefore, when existing image magnification technologies process overlapping regions, they often cannot balance detail richness and target clarity, resulting in poor final segmentation image effects;

[0004] In summary, existing image semantic segmentation technologies have obvious defects in processing small targets and overlapping regions, and there is an urgent need for an image segmentation method that can accurately segment small targets and flexibly process overlapping regions. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems in the background art, and to propose an image semantic segmentation method and system that can define the maximum magnification upper limit formula according to the object overlap coefficient and the local detail richness coefficient; then set correction and suppression terms based on the above coefficients, and determine the dynamic adjustment strategy of the magnification factor according to the threshold comparison.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions. An image semantic segmentation method includes the following steps:

[0007] Collect the image to be segmented, preprocess the image to be segmented to obtain multiple to-be-segmented images of different scales; input the multiple to-be-segmented images of different scales into a preset image semantic segmentation model for image segmentation to obtain an image segmentation result;

[0008] Collect the morphological feature information of the small target image, perform data processing on it to obtain the object overlap coefficient, compare the object overlap coefficient of each small target image with the overlap threshold, and obtain the to-be-magnified object overlap region image according to the comparison result;

[0009] Based on the overlapping region images of the objects to be enlarged corresponding to each small target image and their corresponding texture feature information, analyze and calculate each texture feature information to obtain a local detail enrichment coefficient, and construct an image magnification detection model in combination with the object overlapping coefficient. Obtain the magnification factor corresponding to each overlapping region image of the object to be enlarged through the image magnification detection model;

[0010] Apply the magnification factor obtained according to the image magnification detection model to the overlapping region image of the object to be enlarged for magnification to obtain the magnification result of the overlapping region image of the object, and judge and iterate whether the magnification result of the overlapping region image of the object meets the preset conditions; if it meets, end the iteration and perform image segmentation to obtain the final image segmentation result. If it does not meet, continue the iteration and return to the step of determining the magnification factor.

[0011] In a preferred embodiment, the image segmentation result is a large target image and a small target image segmented from the image to be segmented.

[0012] In a preferred embodiment, the morphological feature information includes the boundary distance and the contour intersection area; the acquisition logic of the boundary distance is to record the minimum boundary distance as , according to the formula

[0013] ; where, represents the contour C of each small target image i and another contour C j The minimum boundary distance between them, a represents the total number of small target images, i represents the index of the contour of the i-th small target image and , represents the contour point set of the i-th small target image, is the contour point set adjacent to the i-th contour, p and q respectively represent the contour C i and another contour C j A point in, represents the square of the coordinate difference of two points p and q on the x-axis , represents the square of the coordinate difference of two points p and q on the y-axis;

[0014] The acquisition logic of the contour intersection area is defined based on the per-pixel logical AND operation and combined with the Hadamard product that is mathematically equivalent to a matrix to obtain the overlapping region of the two entity contours ; where, represents the pixel value of the i-th small target image at the coordinate (x,y), M j (x,y) represents the pixel value of the j-th small target image adjacent to the i-th small target image at the coordinate (x,y);

[0015] The judgment function is obtained through a per-pixel logical AND operation combined with the Hadamard product that is mathematically equivalent to a matrix. ; where 0 indicates that the pixel does not belong to the contour region of the small target image, and 1 indicates that the pixel belongs to the contour region of the small target image.

[0016] Traverse and sum the above contour overlapping regions to obtain the contour intersection area. ; where represents the intersection area of the contour overlapping region between the i-th small target image and the adjacent small target image, W represents the number of pixels of the image in the x direction, and H represents the number of pixels of the image in the y direction.

[0017] Based on the boundary distance and the contour intersection area perform weighting to obtain the object overlap coefficient ; where represents the minimum boundary distance between the contour C i of the i-th small target image and the adjacent contour C j , represents the intersection area of the contour overlapping region between the i-th small target image and the adjacent small target image, and ω1 and ω2 respectively represent the weights corresponding to the boundary distance and the contour intersection area.

[0018] In a preferred embodiment, comparing the object overlap coefficients of each small target image with the overlap threshold, and obtaining the image of the object overlap region to be enlarged according to the comparison result: comparing the object overlap coefficient OC i a corresponding to each small target image with the overlap threshold P;

[0019] When the object overlap coefficient , then there is an overlap region that needs further analysis in this small target image, which is marked as the image of the object overlap region to be enlarged;

[0020] When the object overlap coefficient , mark the corresponding small target image as the normal segmentation result.

[0021] In a preferred embodiment, the texture feature information includes the gradient amplitude corresponding to the image of the object overlap region to be enlarged; the acquisition logic of the local detail richness coefficient is:

[0022] For each object overlap region Re to be enlarged, calculate its local detail richness coefficient LDR e , defined as the weighted average of the gradient amplitudes within the region: ; where is the number of pixels of each object overlap region Re to be enlarged. is the gradient amplitude of each pixel; e is the index of the overlapping area of the object to be magnified and e=1,2,...,u; u is the number of overlapping areas of the object to be magnified.

[0023] In a preferred embodiment, the object overlap coefficient OC corresponding to the overlapping area image of each object to be magnified is i a Recorded as OC e And combined with its corresponding local detail rich coefficient LDR e The process of building an image magnification detection model is as follows:

[0024] S1: Define the upper limit formula of the maximum magnification factor based on the object overlap coefficient and the local detail richness coefficient;

[0025] The maximum magnification M max Constrained by the overall image characteristics, it is defined as a balance function between detail and overlap coefficients: ;in, is the global average detail coefficient; is the global average overlap coefficient; where e is the index of the overlapping area of the object to be magnified and e=1,2,...,u; u is the number of overlapping areas of the object to be magnified, LDR e is the local detail richness coefficient of the e-th area to be magnified, OC e is the object overlap coefficient of the e-th area to be enlarged, δ is the empirical weight, and ϵ is a minimum constant;

[0026] S2: Determine the optimal magnification based on the maximum magnification and local characteristics through dynamic adjustment;

[0027] The local characteristic is dynamically adjusted to a ratio of a local detail richness coefficient to a global detail richness coefficient;

[0028] ; Where γ is the preset adaptive coefficient, λ is the overlap suppression weight, θ is the preset overlap threshold, and A1 represents the object overlap coefficient When the maximum magnification is reached, the dynamic adjustment coefficient corresponding to the maximum magnification is A2, which represents the object overlap coefficient. When , the dynamic adjustment coefficient corresponding to the maximum magnification;

[0029] when When the optimal magnification is determined, the overlapping area of the object to be magnified is selected;

[0030] when When the overlapping area of the object to be magnified is selected Determine the optimal magnification;

[0031] S3: Integrate the dynamic adjustment processes of calculating the upper limit of the optimal magnification factor in step S1 and determining the optimal magnification factor in step S2 to form a global constraint mechanism and a local adaptive mechanism, and construct an image magnification detection model.

[0032] The present invention provides a system for implementing an image semantic segmentation method, including: an image preprocessing and initial segmentation module, an object overlapping area determination module, a magnification factor decision-making module, and an image magnification and iterative judgment module;

[0033] The image preprocessing and initial segmentation module is used to collect the image to be segmented, preprocess the image to be segmented to obtain multiple to-be-segmented images of different scales; input the multiple to-be-segmented images of different scales into a preset image semantic segmentation model for image segmentation to obtain an image segmentation result;

[0034] The object overlapping area determination module is used to collect the morphological feature information of the small target image, perform data processing on it to obtain the object overlapping coefficient, compare the object overlapping coefficient of each small target image with the overlapping threshold, and obtain the to-be-magnified object overlapping area image according to the comparison result;

[0035] The magnification factor decision-making module is used to analyze and calculate the local detail enrichment coefficient based on the to-be-magnified object overlapping area image corresponding to each small target image and its corresponding texture feature information, and construct an image magnification detection model in combination with the object overlapping coefficient, and obtain the magnification factor corresponding to each to-be-magnified object overlapping area image through the image magnification detection model;

[0036] The image magnification and iterative judgment module is used to apply the magnification factor obtained by the image magnification detection model to the to-be-magnified object overlapping area image for magnification to obtain the magnified result of the object overlapping area image, and judge whether the magnified result of the iterative object overlapping area image meets the preset conditions; if it meets, end the iteration, if it does not meet, continue the iteration.

[0037] Compared with the existing technologies, the beneficial effects of the present invention are as follows:

[0038] 1. The present invention collects the image to be segmented and preprocesses the image to be segmented, inputs the preprocessed image to be segmented into a preset model for segmentation to obtain a preliminary segmentation result; collects the morphological features of small targets, performs data processing to obtain the object overlapping coefficient, and screens the to-be-magnified object overlapping area image; through segmenting images of different scales and targeted processing of the to-be-magnified object overlapping area, it can not only better capture image details and targets of different sizes, clarify the to-be-magnified object overlapping area image, but also make subsequent processing focus on the key parts of the image that are prone to segmentation difficulties due to overlapping, avoid processing the entire image without discrimination, and improve the processing pertinence and resource utilization efficiency;

[0039] 2. The present invention defines the maximum magnification upper limit formula for the image and its texture features to be magnified based on the object overlap coefficient and the local detail enrichment coefficient; and based on the maximum magnification, combines the object overlap coefficient and the local detail enrichment coefficient data, sets the local detail correction term and the overlap suppression term, and obtains the magnification dynamic adjustment strategy through the comparison with the threshold. Then, magnify the image of the area to be magnified according to the magnification, and judge whether the result meets the preset conditions. If it meets, the final segmentation result is obtained; if not, return to recalculate the magnification. It can not only divide different overlapping regions through the threshold, realize the dynamic combination of local details and global overlap suppression, but also avoid the distortion diffusion in the high-overlap region while ensuring the enhancement of details, and achieve refined magnification. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of an image semantic segmentation method proposed by the present invention;

[0041] Figure 2 It is an overall structure block diagram of an image semantic segmentation system proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0043] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0044] Refer to Figure 1 - Figure 2 An image semantic segmentation method includes the following steps:

[0045] Step 1: Collect the image to be segmented, preprocess the image to be segmented to obtain multiple images to be segmented with different scales; input the multiple images to be segmented with different scales into a preset image semantic segmentation model for image segmentation to obtain an image segmentation result;

[0046] It should be noted that the preset image semantic segmentation model described in this application is used to segment large target images and small target images from the image to be segmented, so as to achieve target segmentation; wherein, the image to be segmented is a high-resolution remote sensing image, and the large target image and the small target image represent large-scale ground object targets and small-scale ground object targets in the image to be segmented;

[0047] Specifically, the process of obtaining the image segmentation result is as follows:

[0048] S1: Input and feature extraction;

[0049] The preprocessing of the image to be segmented is specifically to extract the geographic metadata of the image to be segmented (such as projection parameters and affine transformation matrices in the GeoTIFF format), store this information, and determine the target coordinate system;

[0050] Input the preprocessed images to be segmented at multiple different scales into the preset image semantic segmentation model established based on GCM - LANet;

[0051] In the feature extraction stage of the preset image semantic segmentation model, first, the image features are initially extracted by ResNet50, then the local context information is enhanced by PAM, the high-level semantic information is embedded by AEM, and finally, the multi-scale information is fused by the GCM module and the receptive field is further expanded, so as to extract the features in the image more comprehensively;

[0052] S2: Multi-scale prediction and weighted fusion;

[0053] Multi-scale prediction: Based on the input images at different scales, the model makes predictions respectively to obtain the prediction results at different scales; since the images at different scales contain target information of different sizes, multi-scale prediction can capture richer target features;

[0054] Weighted fusion: Perform weighted averaging on the prediction results at different scales to obtain the final image segmentation result. The determination of the weights can be adjusted according to factors such as experimental experience and the performance of the model at different scales; for example, the weights can be determined according to the accuracy rate or other evaluation indicators of the model on the validation set at each scale, so that the scales with better performance have larger weights;

[0055] S3: Output the result;

[0056] Finally, output the image segmentation result, which includes the large target image and the small target image, corresponding to the large-scale ground object target and the small-scale ground object target in the image to be segmented respectively; after the model outputs the segmentation result, extract the boundary pixel coordinates (x, y) of each target through contour detection (such as findContours in OpenCV);

[0057] It should be noted that the above preset image semantic segmentation model is established based on GCM-LANet, including a basic network LANet (Local Attention Network) based on a preliminary feature extraction network (ResNet50), a patch attention module (PAM), and an attention embedding module (AEM); and a global convolutional module (GCM) and an activation function (FReLU) are added to form a complete image semantic segmentation model;

[0058] Model training process: The preprocessed multi-scale images to be segmented are divided into a training set, a validation set, and a test set according to a certain ratio (such as 6∶2∶2). The multi-scale images to be segmented in the training set are input into the preset image semantic segmentation model established based on GCM-LANet;

[0059] In the forward propagation process of the model, the image is processed by each module (including ResNet50, PAM, AEM, GCM, etc.) to obtain the predicted segmentation result; the loss value is calculated based on the predicted result and the true label, and then the gradient is calculated through the backpropagation algorithm, and the parameters of the model are updated according to the optimization algorithm;

[0060] Through the above training process, the image semantic segmentation model is obtained;

[0061] It should be noted that the establishment process of the above preset image semantic segmentation model is prior art and will not be elaborated in this embodiment; and in this embodiment, the size of the small target image is defined as 512×512, with the upper left corner of the small target image as the origin (0,0), the horizontal direction as the x-axis (column coordinate), and the vertical direction as the y-axis (row coordinate).

[0062] Step 2: Collect the morphological feature information of the small target image, perform data processing on it to obtain the object overlap coefficient, compare the object overlap coefficient of each small target image with the overlap threshold, and obtain the image of the object overlap area to be enlarged according to the comparison result;

[0063] Among them, the morphological feature information includes the boundary distance and the contour intersection area;

[0064] The acquisition logic of the boundary distance is to calculate the minimum distance between any two different contours in the contours of all small target images, and take the global minimum value as the boundary distance , according to the formula:

[0065] ; where represents the contour C of the i-th small target image i and the adjacent contour C jThe minimum boundary distance between, a represents the total number of small target images, i represents the index of the contour of the i-th small target image and ,

[0066] represents the set of contour points of the i-th small target image,

[0067] is the set of contour points adjacent to the j-th contour, p and q respectively represent a point on the contour C i and the adjacent contour C j in, represents the square of the coordinate difference between two points p and q on the x-axis, represents the square of the coordinate difference between two points p and q on the y-axis;

[0068] The acquisition logic of the contour intersection area is to define the overlapping area of the two entity contours to obtain ; among them, represents the pixel value of the i-th small target image at the coordinate (x, y), M j (x, y) represents the pixel value of the j-th small target image adjacent to the i-th small target image at the coordinate (x, y);

[0069] The judgment function is obtained through pixel-by-pixel logical AND operation and combined with the Hadamard product that is mathematically equivalent to a matrix ; in the formula, 0 means that the pixel does not belong to the contour area of the small target image, and 1 means that the pixel belongs to the contour area of the small target image;

[0070] It should be noted that the above formula means that only when the pixel belongs to the contour areas of both objects at the same time, the value of the overlapping area mask at this coordinate is 1, otherwise it is 0; in this way, the pixel distribution of the overlapping area in the image is logically determined;

[0071] Traverse and sum the above contour overlapping area to obtain the contour intersection area ; among them, represents the intersection area of the contour overlapping area between the i-th small target image and the adjacent small target image, W represents the number of pixels in the x direction of the image, that is, the image width, H represents the number of pixels in the y direction of the image, that is, the image height, and W and H determine the range of traversing and summing the image;

[0072] Based on the boundary distance and the contour intersection area are weighted to obtain the object overlap coefficient Among them, represents the contour C of the i-th small target image i and the adjacent contour Cj The minimum boundary distance between represents the cross-sectional area of the contour overlap region between the i-th small target image and the adjacent small target image. ω1 and ω2 respectively represent the weights corresponding to the boundary distance and the contour cross-sectional area, and their specific values are set by the researchers according to actual needs;

[0073] The object overlap coefficient OC corresponding to each small target image i a is compared with the overlap threshold P; the specific value of the overlap threshold P is set by the researchers according to actual needs;

[0074] When the object overlap coefficient , then there is an overlap region in the small target image that needs further analysis, and it is marked as an image of the object overlap region to be magnified;

[0075] When the object overlap coefficient , then the overlap situation is within the acceptable range, and no further processing is required. The corresponding small target image is directly marked as the normal segmentation result.

[0076] Step 3: Collect the images of the object overlap regions to be magnified corresponding to each small target image and their corresponding texture feature information, analyze and calculate each texture feature information to obtain the local detail enrichment coefficient, and construct an image magnification detection model in combination with the object overlap coefficient. The magnification factor corresponding to each image of the object overlap region to be magnified is obtained through the image magnification detection model;

[0077] Among them, the texture feature information includes the gradient amplitude corresponding to the image of the object overlap region to be magnified;

[0078] The acquisition logic of the above local detail enrichment coefficient is as follows:

[0079] For each object overlap region Re to be magnified, calculate its local detail enrichment coefficient LDR e , which is defined as the weighted average of the gradient amplitudes within the region: ; where, is the number of pixels in each object overlap region Re to be magnified, is the gradient amplitude of the pixel point (x, y), and e is the number of object overlap regions to be magnified;

[0080] It should be noted that the specific value of the above gradient amplitude can be directly obtained, and the specific process is the prior art and will not be elaborated too much;

[0081] The object overlap coefficient OC corresponding to each image of the object overlap region to be magnified i a is denoted as OC e and combined with its corresponding local detail enrichment coefficient LDRe The process of building an image magnification detection model is as follows:

[0082] S1: Define the upper limit formula of the maximum magnification factor based on the object overlap coefficient and the local detail richness coefficient;

[0083] The maximum magnification M max Constrained by the overall image characteristics, it is defined as a balance function between detail and overlap coefficients: ;in, is the global average detail coefficient; is the global average overlap coefficient; where e is the index of the overlapping area of the object to be magnified and e=1,2,...,u; u is the number of overlapping areas of the object to be magnified, LDR e is the local detail richness coefficient of the e-th area to be magnified, OC e is the object overlap coefficient of the e-th area to be enlarged, δ is the empirical weight, and ϵ is a minimum constant;

[0084] S2: Determine the optimal magnification based on the maximum magnification and local characteristics through dynamic adjustment;

[0085] The local characteristic is dynamically adjusted to a ratio of a local detail richness coefficient to a global detail richness coefficient;

[0086] ; Where γ is the preset adaptive coefficient, λ is the overlap suppression weight, θ is the preset overlap threshold, and A1 represents the object overlap coefficient When the maximum magnification is reached, the dynamic adjustment coefficient corresponding to the maximum magnification is A2, which represents the object overlap coefficient. When , the dynamic adjustment coefficient corresponding to the maximum magnification;

[0087] when When the overlapping area of the object to be magnified is selected Determine the optimal magnification;

[0088] when When the overlapping area of the object to be magnified is selected Determine the optimal magnification; specifically, when The overlapping area of the object to be magnified is characterized as a high overlapping area. The overlapping destructiveness of this area is strong and its influence needs to be suppressed. The OC is limited by a fixed threshold θ. e The role of;

[0089] S3: Integrate the processes of calculating the upper limit of the maximum magnification and the pixel-level optimal magnification to form a global constraint mechanism and a local adaptive mechanism, and construct an image magnification detection model. Specifically, input the object overlap coefficient and local detail richness coefficient data of the small target image, and obtain the upper limit constraint of the magnification of the entire small target image through the formula for calculating the upper limit of the maximum magnification. Based on this, combined with the local characteristics of each pixel, determine the magnification of each pixel in the image through the optimal magnification calculation process, thereby constructing a complete image magnification detection model. This model can adaptively determine the reasonable magnification of each pixel according to the local characteristics of the image.

[0090] In this embodiment, define the formula for the upper limit of the maximum magnification based on the object overlap coefficient and the local detail richness coefficient; and based on the maximum magnification, combined with the object overlap coefficient and local detail richness coefficient data, set the local detail correction term and the overlap suppression term and obtain the magnification dynamic adjustment strategy through the comparison of thresholds. It can not only divide different overlap regions through thresholds to achieve dynamic combination of local details and global overlap suppression, but also avoid the distortion diffusion in high-overlap regions while ensuring detail enhancement, realizing refined magnification.

[0091] Step 4: Apply the magnification obtained from the image magnification detection model to the image of the object overlap region to be magnified for magnification, obtain the magnification result of the object overlap region image, and judge and iterate whether the magnification result of the object overlap region image meets the preset conditions; if it meets, end the iteration, if not, continue the iteration; including:

[0092] According to the magnification corresponding to each image of the object overlap region to be magnified calculated in Step 3, perform image magnification processing on these regions using nearest neighbor interpolation or bilinear interpolation.

[0093] The judgment of whether the iteration-completed segmentation slice meets the preset conditions after each iteration processing includes: calculating the structural similarity between the image of the object overlap region to be magnified before magnification and the image of the object overlap region to be magnified after magnification based on SSIM (Structural Similarity Index Measure).

[0094] And compare it with the preset threshold; if the calculation result of the structural similarity is less than or equal to the preset threshold, the iteration ends, and the corresponding magnified object overlap region image is transmitted to Step 1 for image segmentation to obtain the final image segmentation result.

[0095] If the calculation result of the structural similarity is greater than the preset threshold, continue the iteration, that is, continue to perform the determination and adjustment operations of the magnification of the object overlap region image.

[0096] It should be noted that in this embodiment, the structural similarity between the image of the overlapping area of the object to be magnified before magnification and the image of the overlapping area of the object to be magnified after magnification is calculated, and the structural similarity is used as a marker to end the iteration. The advantage of this is that it can more accurately verify and optimize the magnification operation of the image of the overlapping area of the object to be magnified;

[0097] In this embodiment, by collecting the image to be segmented and preprocessing the image to be segmented, the preprocessed image to be segmented is input into a preset model for segmentation to obtain a preliminary segmentation result; collecting the morphological features of small targets, obtaining the object overlap coefficient through data processing, and screening the image of the overlapping area of the object to be magnified; according to the image of the area to be magnified and its texture features, defining the upper limit formula of the maximum magnification multiple based on the object overlap coefficient and the local detail enrichment coefficient; and based on the maximum magnification multiple, combining the data of the object overlap coefficient and the local detail enrichment coefficient, setting the local detail correction term and the overlap suppression term, and obtaining the dynamic adjustment strategy of the magnification multiple through the comparison with the threshold, and magnifying the image of the area to be magnified according to the magnification multiple, and judging whether the result meets the preset conditions. If it meets, the final segmentation result is obtained; if it does not meet, the magnification multiple is recalculated and returned; it can not only divide different overlapping areas through the threshold, realize the dynamic combination of local details and global overlap suppression, but also avoid the distortion diffusion of high-overlap areas while ensuring the enhancement of details, and realize refined magnification.

[0098] The present invention also includes an image semantic segmentation system, which includes the following modules:

[0099] The image preprocessing and initial segmentation module is used to collect the image to be segmented, preprocess the image to be segmented to obtain multiple images to be segmented with different scales; input the multiple images to be segmented with different scales into a preset image semantic segmentation model for image segmentation to obtain an image segmentation result;

[0100] The object overlapping area determination module is used to collect the morphological feature information of the small target image, perform data processing on it to obtain the object overlap coefficient, compare the object overlap coefficients of each small target image with the overlap threshold, and obtain the image of the overlapping area of the object to be magnified according to the comparison result;

[0101] The magnification multiple decision module is used to analyze and calculate the local detail enrichment coefficient based on the image of the overlapping area of the object to be magnified corresponding to each small target image and its corresponding texture feature information, and construct an image magnification detection model in combination with the object overlap coefficient, and obtain the magnification multiple corresponding to each image of the overlapping area of the object to be magnified through the image magnification detection model;

[0102] The image magnification and iterative judgment module is used to apply the magnification factor obtained from the image magnification detection model to the image of the overlapping area of the object to be magnified for magnification, obtain the magnified result of the image of the object overlapping area, and judge whether the magnified result of the iterative object overlapping area image meets the preset conditions; if it meets, the iteration ends, and if it does not meet, the iteration continues.

[0103] As described above, only the preferred specific implementation manner of the present invention is given, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and all should be covered by the protection scope of the present invention.

Claims

1. An image semantic segmentation method, characterized in that, Including: Collect the image to be segmented, preprocess the image to be segmented, and obtain multiple images to be segmented with different scales; Input the multiple images to be segmented with different scales into a preset image semantic segmentation model for image segmentation to obtain an image segmentation result; Collect the morphological feature information of the small target image, perform data processing on it to obtain an object overlap coefficient, compare the object overlap coefficient of each small target image with an overlap threshold, and obtain an image of the object overlap area to be enlarged according to the comparison result; Based on the image of the object overlap area to be enlarged corresponding to each small target image and its corresponding texture feature information, analyze and calculate each texture feature information to obtain a local detail richness coefficient, and construct an image magnification detection model in combination with the object overlap coefficient. Obtain the magnification factor corresponding to each image of the object overlap area to be enlarged through the image magnification detection model; Apply the magnification factor obtained by the image magnification detection model to the image of the object overlap area to be enlarged for magnification to obtain the enlarged result of the image of the object overlap area, and judge whether the enlarged result of the iterative object overlap area image meets the preset conditions; if it meets, end the iteration to perform image segmentation to obtain the final image segmentation result, if it does not meet, continue the iteration and return to the step of determining the magnification factor; The image segmentation result is a large target image and a small target image segmented from the image to be segmented; The object overlap coefficient is obtained by weighting the boundary distance and the contour intersection area in the morphological feature information; The local detail richness coefficient is defined as: for each object overlap area to be enlarged, calculate the weighted average of the gradient amplitude within the object overlap area to be enlarged and define it as the local detail richness coefficient; Image magnification detection model construction process: the object overlap coefficient and its corresponding local detail rich coefficient are input into the balance function ;in, is the global average detail coefficient; is the global average overlap coefficient; where e is the index of the overlapping area of the object to be magnified and e=1,2,...,u; u is the number of overlapping areas of the object to be magnified, LDR e is the local detail richness coefficient of the e-th area to be magnified, OC e is the object overlap coefficient of the e-th area to be magnified, δ is the empirical weight, and ϵ is the minimum constant; to determine the upper limit of the optimal magnification; and to construct a global constraint mechanism and a local adaptive mechanism based on the ratio of the local detail rich coefficient to the global detail rich coefficient to form an image magnification detection model; Whether it meets the preset conditions includes: calculating the structural similarity based on the structural similarity between the image of the object overlap area to be enlarged before magnification and the image of the object overlap area to be enlarged after magnification; Compare the structural similarity with a preset threshold; if the structural similarity is less than or equal to the preset threshold, the iteration ends; if the calculation result of the structural similarity is greater than the preset threshold, continue the iteration.

2. The image semantic segmentation method according to claim 1, wherein Including: The morphological feature information includes the boundary distance and the contour intersection area; the acquisition logic of the boundary distance is to calculate the minimum distance between any two different contours in the contours of all small target images, and take the global minimum value as the boundary distance , according to the formula: ; wherein, represents the contour C of the i-th small target image i and the minimum boundary distance between adjacent contours C j , a represents the total number of small target images, i represents the index of the contour of the i-th small target image and , represents the set of contour points of the i-th small target image, is the set of contour points adjacent to the j-th contour, p and q respectively represent a point on the contour C i and the adjacent contour C j ; represents the square of the difference in the x-axis coordinates of two points p and q, represents the square of the difference in the y-axis coordinates of two points p and q; The acquisition logic of the contour intersection area is obtained by defining the overlapping area of the two entity contours ; among them, represents the pixel value of the i-th small target image at the coordinate (x, y), and M j (x, y) represents the pixel value of the j-th small target image adjacent to the i-th small target image at the coordinate (x, y); The judgment function is obtained through a per-pixel logical AND operation combined with a Hadamard product that is mathematically equivalent to a matrix ; where 0 indicates that the pixel does not belong to the contour region of the small target image, and 1 indicates that the pixel belongs to the contour region of the small target image Traverse and sum the above contour overlapping regions to obtain the contour intersection area ; where represents the intersection area of the contour overlapping region between the i-th small target image and the adjacent small target image, W represents the number of pixels of the image in the x direction, and H represents the number of pixels of the image in the y direction.

3. A method for image semantic segmentation according to claim 1, characterized in that; Comparing the object overlap coefficient of each small target image with the overlap threshold, and obtaining the image of the object overlap region to be magnified according to the comparison result: the object overlap coefficient OC corresponding to each small target image i a is compared with the overlap threshold P; When the object overlapping coefficient is such that there is an overlapping area in the small target image that requires further analysis, it is marked as an image of the overlapping area of the object to be magnified; When the object overlap coefficient is, mark the corresponding small target image as the normal segmentation result.

4. A method for image semantic segmentation according to claim 1, characterized in that Including: The texture feature information includes the gradient amplitude corresponding to the image of the object overlap area to be enlarged.

5. A method for image semantic segmentation according to claim 1, characterized in that, The specific calculation process of the local detail richness coefficient includes: for each overlapping region Re of the object to be magnified, calculate its local detail richness coefficient LDR e , which is defined as the weighted average of the gradient magnitudes within the region: ; where is the number of pixels in each overlapping region Re of the object to be magnified, is the gradient magnitude of each pixel; e is the index of the overlapping region of the object to be magnified and e = 1, 2,..., u; u is the number of overlapping regions of the object to be magnified.

6. The image semantic segmentation method according to claim 1, wherein Including: Denote the object overlap coefficient OC corresponding to the image of the overlapping region of each object to be magnified i a as OC e and combine it with its corresponding local detail enrichment coefficient LDR e The process of constructing the image magnification detection model is as follows: S1: Define a maximum magnification upper limit formula based on the object overlap coefficient and the local detail richness coefficient; The maximum magnification factor M max is constrained by the overall image characteristics and is defined as a balance function of the detail and the overlap factor; S2: Dynamically adjust based on the maximum magnification and local characteristics to determine the optimal magnification; The dynamic adjustment of the local characteristics is the ratio of the local detail richness coefficient to the global detail richness coefficient; where γ is a preset adaptive coefficient, λ is an overlap suppression weight, θ is a preset overlap threshold, and A1 represents the object overlap coefficient When it is, the dynamic adjustment coefficient corresponding to the maximum magnification, and A2 represents the object overlap coefficient When it is, the dynamic adjustment coefficient corresponding to the maximum magnification; When it is time to determine the best magnification factor by selecting the overlapping area of the object to be magnified When the overlapping area of the object to be magnified is selected to determine the optimal magnification factor; S3: Integrate the dynamic adjustment processes of calculating the optimal magnification upper limit in step S1 and determining the optimal magnification in step S2 to form a global constraint mechanism and a local adaptive mechanism, and construct an image magnification detection model.

7. A system for implementing the image semantic segmentation method according to any one of claims 1-6, characterized in that, Including: The image preprocessing and initial segmentation module is used to collect the image to be segmented, preprocess the image to be segmented, and obtain multiple images to be segmented with different scales; Input the multiple images to be segmented with different scales into a preset image semantic segmentation model for image segmentation to obtain an image segmentation result; The object overlapping region determination module is used to collect the morphological feature information of the small target image, perform data processing on it to obtain the object overlapping coefficient, compare the object overlapping coefficients of each small target image with the overlapping threshold, and obtain the image of the object overlapping region to be magnified according to the comparison result; The magnification decision module is used to analyze and calculate the local detail enrichment coefficient based on the image of the object overlapping region to be magnified corresponding to each small target image and its corresponding texture feature information, construct an image magnification detection model in combination with the object overlapping coefficient, and obtain the magnification corresponding to each image of the object overlapping region to be magnified through the image magnification detection model; The image magnification and iteration judgment module is used to apply the magnification obtained by the image magnification detection model to the image of the object overlapping region to be magnified for magnification, obtain the magnification result of the object overlapping region image, and judge whether the magnification result of the iterative object overlapping region image meets the preset conditions; if it meets, the iteration ends, and if it does not meet, the iteration continues.

Images