Image instance recognition method, system, device and medium based on semantic segmentation

Abstract

The application provides a semantic segmentation-based image instance recognition method, system and computer device, and relates to the technical field of image recognition. The method comprises: performing structural preprocessing on a to-be-processed image to obtain an image block feature sequence; inputting the image block feature sequence into an encoder to perform adaptive mask analysis on the image block feature sequence according to a visual change relationship between the image block features to generate mask information corresponding to different feature changes, and fusing each mask information and the image block feature sequence to generate adaptive coding features; inputting the adaptive coding features into a decoder to perform channel mixing and dilated convolution on the adaptive coding features according to a pixel perception mechanism to obtain semantic segmentation features; and performing instance recognition on the to-be-processed image according to the semantic segmentation features to obtain an instance recognition result. The method can reflect targeted structural improvement and efficient semantic expression ability in both the feature extraction and feature restoration stages.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to an image instance recognition method and system based on semantic segmentation and a computer device. BACKGROUND

[0002] Image instance recognition is one of the core tasks in the field of computer vision, which not only needs to identify the semantic category of each pixel in the image (semantic segmentation), but also needs to distinguish different individuals under the same semantic category (instance segmentation). This technology has wide application value in many fields such as automatic driving, medical image analysis, robot vision, and remote sensing image interpretation. In recent years, with the rapid development of deep learning, semantic segmentation methods based on convolutional neural networks (CNN) and visual Transformers (ViT) have become the mainstream technology route to solve this problem.

[0003] Early work mainly focuses on fully convolutional networks (FCN) and its variants, which realize end-to-end pixel-level prediction through an encoder-decoder structure. For example, U-Net fuses shallow details and deep semantic information through a skip connection and achieves remarkable results in medical image segmentation. The DeepLab series of networks introduces dilated convolution (Dilated Convolution) to expand the receptive field and uses an atrous spatial pyramid pooling (ASPP) module to capture multi-scale context information. However, when dealing with instance-level recognition, these methods usually need to rely on additional post-processing steps (such as conditional random fields CRF, clustering algorithms, etc.) or add complex instance-aware modules (such as Mask R-CNN) to the network head, resulting in a long process, limited computational efficiency, and insufficient refinement of instance boundaries.

[0004] To achieve instance awareness more directly, subsequent research attempts to incorporate stronger structural priors or relationship modeling capabilities into the feature encoding stage. For example, attention mechanisms are used to enable the network to adaptively focus on different regions. Visual Transformers (ViT) and their adapted models in the segmentation field (such as SegFormer, SETR) model long-range dependencies through global self-attention, showing advantages in capturing global context. However, this type of method usually treats images as a sequence of independent image patches (Patch) for processing, and does not adequately model the local visual correlation inherent to the image, guided by object boundaries and structural changes, during the encoding process. The encoding structure is often fixed or homogeneous, lacking adaptive analysis and differentiated processing capabilities for feature variation intensity in different regions of the image (such as smooth regions, complex texture regions, and object edge regions), resulting in a blurred response of the generated feature map at key boundaries and insufficient clarity of the internal consistency and external distinction of semantic regions.

[0005] On the other hand, in the feature decoding and spatial information recovery stage, the existing methods mostly adopt symmetric upsampling, deconvolution or feature pyramid fusion strategy. Although these methods can gradually recover the spatial resolution, they usually adopt equal or fixed strategies (such as point-by-point addition or channel splicing) in the integration of channel dimension information, without fully considering the semantic importance difference of different channel features at different spatial positions. This leads to the key boundary information being smoothed and the small instances being lost or adhered to adjacent instances in the upsampling process when restoring details. Although some improvement works try to introduce edge enhancement branches or loss functions to optimize the boundaries, these are external auxiliary means and cannot achieve accurate semantic perception and reconstruction at the pixel level from the fundamental mechanism of feature reorganization and fusion.

[0006] In summary, in the field of image recognition technology, a method for identifying different semantic instances in an image based on semantic segmentation is implemented. In related image instance recognition methods, fixed structure encoding and decoding networks are usually used for feature extraction and restoration. However, such structures lack adaptive modeling capability for feature changes during feature analysis and lack fine-grained perception of pixel-level semantics during spatial restoration, resulting in rigid feature expression and insufficient semantic association, making it difficult to achieve efficient semantic expression capability. SUMMARY

[0007] To at least solve one of the problems existing in the prior art, the present application provides a semantic segmentation-based image instance recognition method, system, computer device and computer readable storage medium.

[0008] In a first aspect, the present application provides a semantic segmentation-based image instance recognition method, comprising:

[0009] Obtaining a to-be-processed image, performing structured preprocessing on the to-be-processed image to obtain an image block feature sequence;

[0010] Inputting the image block feature sequence into a preset encoder, performing adaptive mask analysis on the image block feature sequence in the encoder according to the visual change relationship between the image block features to generate mask information corresponding to different feature changes, and fusing each mask information and the image block feature sequence to generate adaptive encoding features;

[0011] Inputting the adaptive encoding features into a preset decoder, performing channel shuffling and dilated convolution on the adaptive encoding features according to a preset pixel perception mechanism to obtain semantic segmentation features;

[0012] Performing instance recognition on the to-be-processed image according to the semantic segmentation features to obtain an instance recognition result corresponding to the to-be-processed image.

[0013] In a second aspect, the present application further provides an image instance recognition system based on semantic segmentation, comprising:

[0014] An acquisition module is configured to acquire a to-be-processed image, perform structural preprocessing on the to-be-processed image, and obtain an image block feature sequence.

[0015] An encoding module is configured to input the image block feature sequence into a preset encoder, perform adaptive mask analysis on the image block feature sequence according to a visual change relationship between image block features in the encoder to generate mask information corresponding to different feature changes, and fuse each mask information and the image block feature sequence to generate adaptive encoding features.

[0016] A decoding module is configured to input the adaptive encoding features into a preset decoder, perform channel shuffling and dilated convolution on the adaptive encoding features according to a preset pixel perception mechanism in the decoder to obtain semantic segmentation features.

[0017] A recognition module is configured to perform instance recognition on the to-be-processed image according to the semantic segmentation features to obtain an instance recognition result corresponding to the to-be-processed image.

[0018] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above steps when executing the computer program.

[0019] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above steps.

[0020] The image instance recognition method, system, computer device and computer readable storage medium based on semantic segmentation described above, firstly, the image to be processed is structurally preprocessed to obtain an image block feature sequence, so that the local features of the original image can be structured in sequence form, providing an ordered input basis for subsequent feature correlation analysis; then, the encoder performs mask analysis and fusion processing on the visual change relationship between the image block features in the image block feature sequence to obtain adaptive coding features, thereby extracting and strengthening the feature correlation of the key semantic region in the global range; further, combined with the pixel perception mechanism, the decoder performs channel mixing and dilated convolution processing on the adaptive coding features to obtain semantic segmentation features, thereby maintaining the integrity of the features while restoring the pixel-level spatial structure and generating semantic segmentation features with higher discrimination; finally, the image to be processed is recognized according to the semantic segmentation features, thereby accurately distinguishing different semantic instances and efficiently outputting the corresponding instance recognition results; based on this, in the entire technical solution, by introducing adaptive mask analysis operation in the encoder and introducing pixel perception analysis operation in the decoder, fine instance recognition driven by semantic segmentation is realized, thereby reflecting the targeted structural improvement and efficient semantic expression ability in both the feature extraction and feature restoration stages. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flowchart of an embodiment of the image instance recognition method based on semantic segmentation.

[0022] Figure 2 A data processing schematic diagram of an embodiment of the image instance recognition method based on semantic segmentation.

[0023] Figure 3 A data processing schematic diagram of the encoding stage in an embodiment of the image instance recognition method based on semantic segmentation.

[0024] Figure 4 A data processing schematic diagram of the decoding stage in an embodiment of the image instance recognition method based on semantic segmentation.

[0025] Figure 5 A structural block diagram of an embodiment of the image instance recognition system based on semantic segmentation. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0027] In an exemplary embodiment, the image instance recognition method based on semantic segmentation comprises the following steps: Figure 1 and Figure 2As shown, a semantic segmentation-based image instance recognition method is provided, and in this embodiment, the method is applied to a server for illustration. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps S101 to S104.

[0028] In step S101, an image to be processed is obtained, and structured preprocessing is performed on the image to be processed to obtain an image block feature sequence.

[0029] The image block feature sequence represents a set of ordered feature vectors formed by the structured preprocessing of the image to be processed, and is used to express the texture, brightness, edge and spatial distribution of each local region in the entire image in the feature space.

[0030] The input image to be processed is preliminarily structured and preprocessed to ensure the effectiveness of subsequent semantic feature extraction. Exemplarily, the structured preprocessing includes image block division and vectorization processing. Specifically, first, the entire image to be processed is divided into a plurality of image blocks of the same size, so that the original continuous image is discretized into local region units. Thus, the complex overall visual scene is converted into a basic unit that can be analyzed in zones, so as to extract the texture, edge, color and brightness features contained in each local region. Subsequently, each image block is subjected to vectorization processing, i.e., the pixel matrix data is converted into a feature vector form through feature mapping to form a unified feature representation. Thus, the vectorization process not only reduces the data dimension, but also enables each image block to have a clear mathematical representation in the feature space.

[0031] Based on this, the image block feature sequence obtained in this way is a high-dimensional vector sequence organized in a spatial arrangement order, and is used to reflect the relative position and visual difference relationship between local regions in the entire image. This image block feature sequence not only retains the spatial distribution structure, but also provides a standardized input basis for subsequent feature learning, thereby systematically analyzing and modeling the visual correlation between different image blocks at the feature level.

[0032] In step S102, the image block feature sequence is input to a preset encoder, and in the encoder, the image block feature sequence is adaptively analyzed by mask according to the visual change relationship between the image block features to generate mask information corresponding to different feature changes, and each mask information and the image block feature sequence are fused to generate adaptive coding features.

[0033] In an embodiment, the encoder is configured to implement a feature extraction structure based on a Vision Transformer model, to model semantic association of image block feature sequences and to compress feature expression.

[0034] It can be understood that, although the encoder of the present application is based on the Vision Transformer model, it is not limited to this model structure. In other embodiments, other models can also be used, as long as they can establish semantic association between image blocks at the feature level and support regionalized feature reorganization. For example, a multi-layer self-attention network or a hybrid convolution-attention structure can also achieve the functions and effects of the present application.

[0035] The mask information represents the weight identification generated in the encoder according to the visual change relationship between the image block features, and is used to indicate the importance and association of each image block feature in the semantic association modeling.

[0036] The adaptive coding feature is a high-dimensional feature formed by fusing the mask information and the original image feature, and is used to comprehensively reflect the visual differences and structural semantics between different regions in the entire image.

[0037] Exemplarily, the image block feature sequence generated by the foregoing steps is input into a preset encoder structure to establish semantic association of image local changes in the feature space. Specifically, first, the encoder processes the input image block feature sequence layer by layer through multi-layer linear or nonlinear transformation to extract the visual change rule therein. Then, the encoder generates corresponding mask information according to the differences between different image block features in the analysis process, to identify the relevance and difference between the image block features. That is, the mask analysis process is to determine which features are representative in semantic expression through the interaction between the features, so as to moderately suppress irrelevant or redundant features. Subsequently, each mask information and the image block feature sequence are fused to generate adaptive coding features, so that each local feature forms a unified representation structure under the action of the mask, to obtain the adaptive coding features.

[0038] Based on this, the adaptive coding feature is a high-dimensional expression of the visual changes of the entire image after mask fusion based on the original image feature, which contains both local feature details and global structural semantics. Through this process, the input image block feature sequence is converted into coding features that can reflect multi-scale structural differences, providing an accurate semantic basis for subsequent segmentation and recognition.

[0039] Step S103, input the adaptive coding feature into a preset decoder, and perform channel shuffling and dilated convolution on the adaptive coding feature according to a preset pixel perception mechanism in the decoder to obtain a semantic segmentation feature.

[0040] The decoder represents a feature reconstruction structure constructed based on the channel shuffling and dilated convolution branch, and is used for spatial restoration and semantic segmentation feature extraction on the coded high-dimensional feature.

[0041] The pixel perception mechanism represents a mapping rule used to establish a correspondence between the feature space and the image space in the decoder, so as to restore the spatial layout and pixel-level semantic correspondence of the image in the feature restoration process. On one hand, the mapping rule is used to determine the correspondence between the feature and the pixel position in the spatial mapping process, and on the other hand, the mapping rule is used to guide the alignment of the spatial distribution and the semantic structure in the feature restoration, so that the decoding result has semantic consistency and complete expression of local details in the spatial level.

[0042] The semantic segmentation feature represents a spatial semantic representation obtained after the channel shuffling and dilated convolution processing of the decoder, and is used to distinguish different semantic categories of regions in the image at the pixel level.

[0043] Exemplarily, the adaptive coding feature generated in the foregoing steps is input into a preset decoder structure to map and restore the high-dimensional feature compressed in the coding stage to a semantic representation with spatial resolution. Specifically, first, the decoder performs pixel-level mapping on the input adaptive coding feature according to the preset pixel perception mechanism, so that each spatial position reacquires local information corresponding to the original image. Subsequently, the decoder performs feature reorganization in the channel dimension, rearranges the feature order between different channels through channel shuffling operation, so that the information across channels is redistributed in the spatial position, so as to enhance the boundary separation effect between different semantic regions. On this basis, the decoder further performs dilated convolution processing on the redistributed feature to expand the receptive field while maintaining the resolution, so as to capture larger scale semantic structures.

[0044] Based on this, through the continuous mapping, reorganization and convolution operation, the final semantic segmentation feature not only has the representation ability for the spatial difference between pixels, but also maintains the global semantic consistency, providing a structured input for the final instance recognition.

[0045] Step S104, performing instance recognition on the to-be-processed image according to the semantic segmentation feature to obtain an instance recognition result corresponding to the to-be-processed image.

[0046] The instance recognition result represents a structured output result obtained by pixel clustering and region distinguishing of the to-be-processed image based on the semantic segmentation feature, and is used to identify the category, position and contour information of each independent semantic instance in the image.

[0047] Exemplarily, the semantic instance recognition operation is performed on the input to-be-processed image according to the semantic segmentation feature generated in the foregoing steps. Specifically, first, the preliminary classification of different semantic regions is realized by establishing a mapping relationship between each pixel point in the mapping space and its corresponding semantic label of the semantic segmentation feature. In this process, the feature response value of each pixel under different semantic categories is calculated, and the matching degree between the pixel and the average feature vector of each semantic category is measured by means of cosine similarity or Euclidean distance, so as to determine the semantic category to which the pixel most likely belongs, and then the complete semantic segmentation feature is divided into a plurality of semantic regions with independent semantics. Subsequently, the boundary refinement and instance distinction are performed on the semantic regions, the attribution range of pixels of the same category is determined by judging the feature similarity within the semantic region and the feature difference between the regions, so as to determine the spatial boundary of each independent semantic instance. In this process, the context relationship between the features is comprehensively calculated, for example, the average feature gradient change or the feature correlation distribution of adjacent regions are used to adjust and optimize the smoothness and connectivity of the region boundary, so that the segmented region is not only based on the pixel similarity, but also considers the coherence of the semantic structure.

[0048] Based on this, through this coherent recognition process, the identification result corresponding to each semantic instance is finally output, and the spatial mapping relationship between the to-be-processed image and each semantic instance is established, so as to obtain the instance recognition result of the to-be-processed image. The data form of the instance recognition result is usually a complete semantic instance set containing the position, category and region contour, which provides a directly usable structured output for subsequent image understanding or target detection.

[0049] In the image instance recognition method based on semantic segmentation, in step S101, the image to be processed is structurally preprocessed to obtain an image block feature sequence, so that the local features of the original image can be expressed in a sequence form, thereby providing an ordered input basis for subsequent feature correlation analysis; in step S102, the visual change relationship between the image block features in the image block feature sequence is analyzed and fused by the encoder to obtain adaptive coding features, thereby extracting and strengthening the feature correlation of the key semantic region in a global range; in step S103, the adaptive coding features are processed by the decoder to obtain semantic segmentation features by combining the pixel perception mechanism and channel mixing and expansion convolution, thereby maintaining the integrity of the features while restoring the pixel-level spatial structure and generating semantic segmentation features with higher discrimination; in step S104, the image to be processed is recognized based on the semantic segmentation features, thereby accurately distinguishing different semantic instances and efficiently outputting the corresponding instance recognition results; based on this, in the entire technical solution, by introducing adaptive mask analysis operation in the encoder and pixel perception analysis operation in the decoder, fine instance recognition driven by semantic segmentation is realized, thereby reflecting the targeted structural improvement and efficient semantic expression ability in both the feature extraction and feature restoration stages.

[0050] In an exemplary embodiment, the method comprises the following steps: Figure 3 In the encoder, the image block feature sequence is adaptively analyzed according to the visual change relationship between the image block features to generate mask information corresponding to different feature changes, specifically comprising steps S201 to S204.

[0051] Step S201, in the encoder, the image block features in the image block feature sequence are calculated according to the visual change relationship between the image block features to obtain a change response map representing the change intensity.

[0052] The change response map represents the numerical distribution result calculated according to the visual change relationship between the image block features, and is used to represent the change intensity of the image block features in the spatial and feature dimensions; when the image block features differ greatly, the response value difference at the corresponding position is more obvious, thereby forming a region with significant change in the change response map.

[0053] For example, the encoder performs response value calculation and difference measurement operations on the input image patch feature sequence based on the visual change relationship between the features of each image patch, in order to obtain a change response map that can quantify the intensity of feature changes. Specifically, this process unfolds on two levels: on the one hand, response value calculation and difference analysis are performed on the features between adjacent image patches to extract the visual differences between local regions, so that the brightness, texture or structural changes between adjacent regions can be expressed in numerical form; on the other hand, response value calculation and difference analysis are performed on the features within the same image patch to reflect the consistency and internal change rules of the image patch in the feature dimension, thereby establishing a quantitative measure of feature stability within a single patch.

[0054] Furthermore, when performing response value calculation, the input image patch feature sequence is first arranged in a structured manner according to the visual change relationship, so that the features of each image patch maintain a comparable mapping relationship with the features of adjacent image patches in space, while preserving its internal feature distribution in the feature dimension, so as to carry out response analysis between and within patches simultaneously. Subsequently, after obtaining the response values ​​at each location, difference analysis is performed on these response values. By measuring the response differences between different image patches and within the same image patch, the intensity of feature change in the spatial and semantic dimensions is extracted. In this process, the numerical differences between response values ​​are mapped to the corresponding spatial locations, thereby forming a quantitative distribution basis that can reflect the degree of feature change.

[0055] Finally, all response value differences are reorganized according to the feature positions of image patch features in the original image to form a two-dimensional structured change response map, so that each position corresponds to a quantitative result reflecting the intensity of visual feature changes, thereby providing a clear basis for subsequent boundary region division.

[0056] Optionally, Equation (1) embodies the comprehensive measurement process of feature changes by the change response map at both intra-block and inter-block levels, where:

[0057] (1)

[0058] In equation (1), The change in response value represents the change in the image patch. Feature location At this point, through calculation of all The quantified feature change intensity is obtained from the inter-block feature differences and intra-block feature differences in each channel dimension. Indicates the first The image patch in the first Each feature dimension, feature location eigenvalues ​​at that location Indicates the relationship with the first Image blocks spatially adjacent to each other in the first image block space feature dimension, feature position feature value at the feature position, reflects the feature position difference between the feature of the adjacent image block in the first feature dimension, used to measure the visual change between regions, such as structural transition or boundary difference; representing the average value of all feature positions in the first feature dimension within the first image block, reflects the feature position difference between the average feature within the same image block in the first feature dimension, used to measure the stability or internal change of the feature distribution within the block; coefficient for controlling the weight of the intra-block difference in the overall change measure, to balance the inter-block difference and intra-block consistency.

[0059] Based on this, first, the weighted results of the inter-block difference and the intra-block difference of the feature position in the first feature dimension are calculated; secondly, the weighted results of the feature position in all feature dimensions are averaged to eliminate single channel bias, and the final is a normalized change response value, representing the change intensity of the feature position in the visual feature dimension; finally, the change response values of all positions together constitute a change response map , used to depict the change intensity distribution of different positions in the feature space, thereby reflecting the semantic fluctuation and structural difference of each region in the overall feature mapping.

[0060] Step S202, in the change response map, the region with a change intensity greater than or equal to a preset change threshold is marked as a boundary region, and the region with a change intensity less than the preset change threshold is marked as a non-boundary region.

[0061] Wherein, the change threshold represents a preset numerical limit for distinguishing the boundary region and the non-boundary region, used to determine in the change response map which regions belong to significant change and which belong to smooth change.

[0062] Wherein, the boundary region represents a feature section with a change intensity greater than or equal to the preset change threshold in the change response map, used to identify the edge or structural transition position with obvious visual difference in the image, such as the object edge or texture mutation position, which is usually defined as the boundary region.

[0063] The non-boundary region represents a feature section with a change intensity less than a preset change threshold in the change response map, and is used to identify a position of a visual feature in the image that is relatively continuous and smooth, for example, a region with uniform color or consistent texture is usually defined as a non-boundary region.

[0064] By way of example, the change response map generated by the foregoing steps is regionally divided to identify the boundary region and the non-boundary region on the feature level. Specifically, the process performs interval determination on the change response map according to the feature change intensity by using a preset change threshold. On one hand, when the feature change intensity at a position is greater than or equal to the change threshold, the position is marked as a boundary region to represent the presence of a significant change transition part. On the other hand, when the feature change intensity at a position is less than the change threshold, the position is marked as a non-boundary region to represent a stable part of the feature distribution. Based on this, through the thresholding marking process, the original continuous change response is converted into a discrete region label distribution, thereby forming a regionally divided result that is operable on the feature level.

[0065] Optionally, the setting of the change threshold is based on the statistical characteristics of the overall change response to ensure the balance and reasonableness of the division. For example, in the change response map, all feature change intensities will present a certain statistical distribution, such as a high change region, a low change region, and an intermediate transition region. Thus, by performing statistical analysis on these feature change intensities, the mean, variance, or percentile interval thereof is calculated to determine a threshold range that can distinguish the boundary region from the non-boundary region. For example, when the overall mean of the feature change intensity is 0.45 and the standard deviation is 0.15, a region higher than the sum of the mean and the standard deviation (0.6) can be defined as a boundary region, and a region lower than the difference between the mean and the standard deviation (0.3) can be defined as a non-boundary region, so as to ensure that the boundary division is neither too sparse nor overly dense, thereby maintaining the overall balance and reasonableness.

[0066] In step S203, a boundary mask corresponding to each adjacent boundary region and non-boundary region is determined according to the positional relationship between the adjacent boundary region and the non-boundary region.

[0067] The boundary mask represents reference information for limiting a corresponding interface transition position and assigning a feature weight to the interface transition position according to the positional relationship between the adjacent boundary region and the non-boundary region; for example, a boundary mask covers the interface transition position between the boundary region and the non-boundary region, and the internal weight value gradually decays from the edge to the smooth region to guide the weighted processing of the feature.

[0068] Exemplarily, according to the spatial adjacency relationship between the boundary region and the non-boundary region, the corresponding boundary masks of the two are calculated and determined. Specifically, by scanning the label distribution of the adjacent regions in the change response map, the contact positions of the boundary region and the non-boundary region are detected, and the corresponding boundary masks are generated according to the relative positions of the two in space; wherein the construction of the boundary mask takes the spatial adjacency relationship as the core, defines the junction part of the adjacent regions as the core section of the mask, and dynamically adjusts the coverage range according to the continuity of the feature change (for example, when the boundary part feature change is gentle, the mask coverage range is expanded to include a wider transition zone; when the change is violent, the coverage range is contracted to focus on the mutation region; thus, the continuity of the feature change corresponds to the width and shape of the mask boundary, so that it can adapt to the actual feature transition situation). In addition, in order to ensure the smooth transition of the boundary, position interpolation calculation is introduced in the mask generation process, so that the numerical distribution of the mask presents continuous and gradual change, so as to avoid the feature fragmentation caused by mutation.

[0069] Based on this, each boundary mask is composed of feature position index and its weight distribution, forming a mask structure with clear coverage range in space. In this way, a specific corresponding relationship is established between each pair of adjacent boundary region and non-boundary region, so that the corresponding boundary mask can not only represent the local edge information, but also maintain the continuity of the overall feature distribution, thereby realizing the smooth transition and structural expression between regions at the feature level.

[0070] Step S204, according to the coverage range and weight distribution defined by each boundary mask, the image block feature sequence is adaptively masked and analyzed to generate mask information corresponding to different feature changes.

[0071] Exemplarily, first, according to the coverage range defined by each boundary mask, the features at the corresponding positions in the image block feature sequence are extracted, and a weighted calculation is performed according to the weight distribution in the boundary mask, so that each feature obtains a weight value in the overall calculation that matches its change intensity, to reflect the local significance of the features at different positions. In addition, in order to ensure the integrity of the calculation, the feature channel and the spatial position are traversed at the same time in the weighting process, so that the mask effect can take effect in the full-dimensional range.

[0072] Based on this, after the iterative fusion of multiple groups of boundary masks, the mask information corresponding to different feature changes is generated in turn. Each mask information is composed of coverage range, position index and weight distribution, used to describe the change attribute and spatial distribution relationship of the corresponding feature region, and is a feature distribution structure with hierarchical expression ability. Thus, through this processing process, the originally unified image block feature sequence is transformed into a change expression with regional distinction, so that the feature modeling process can be described hierarchically according to different change structures.

[0073] Further, the respective mask information and the image block feature sequence are fused to generate adaptive coding features. Specifically, the respective mask information corresponds to the feature changes of different boundary transition positions. Through the fusion operation, the local features are aligned and integrated in a unified space and feature dimension, so that the originally dispersed mask analysis results form a consistent expression structure. After fusion, the boundary features, change intensity and spatial correlation of different boundary transition positions are uniformly mapped, and the adaptive coding features generated thereby can reflect the comprehensive relationship of image features at a global level, providing a consistent input representation for subsequent decoding and semantic segmentation.

[0074] In the embodiment, in step S201, the response value calculation is performed on the image block feature sequence to form a change response map, thereby reflecting the change intensity of the features between and within the image blocks in a numerical manner, providing a quantitative basis for region division; in step S202, the threshold division processing of the change intensity in the change response map is performed, thereby distinguishing the significant change region from the stable region, and realizing the structural identification of the boundary region and the non-boundary region; in step S203, the boundary mask is constructed according to the positional relationship between the boundary region and the non-boundary region, thereby establishing a continuous and smooth region transition expression at the feature level; in step S204, the adaptive mask analysis is performed on the image block feature sequence according to the boundary mask, thereby generating different mask information capable of reflecting the local change features. Based on this, in the entire technical solution, the hierarchical change modeling from response analysis to mask generation is realized, so that the features have distinguishability and continuity in the spatial and semantic dimensions.

[0075] In one exemplary embodiment, according to the positional relationship between the adjacent boundary region and the non-boundary region, the boundary mask corresponding to each adjacent boundary region and non-boundary region is determined, including steps S301 to S303.

[0076] Step S301, according to the relative distribution position between the adjacent boundary region and the non-boundary region, the coverage range defined by the corresponding boundary mask is determined.

[0077] Exemplarily, firstly, the spatial positions of the adjacent boundary region and the non-boundary region are obtained from the change response map, and the contact relationship between the two is analyzed, and the extension range and the morphological change of the contact boundary are calculated to reflect the spatial correlation characteristics of the two types of regions in the transition position. Subsequently, the contact range is comprehensively evaluated in combination with the relative distribution direction of the two types of regions, that is, by analyzing the position distribution and direction consistency of the boundary region and the non-boundary region on the contact line, for example, when the edge direction of the two types of regions remains continuous and the direction connection is smooth in the local space, the transition position between the boundary region and the non-boundary region is determined accordingly. That is, in the analysis process, not only the contact position of the two is concerned, but also the continuity of the distribution direction and the boundary shape is considered, so that the coverage range of the boundary mask can be more in line with the actual region contour and the spatial connection relationship.

[0078] By statistically and screening the position information of multiple groups of adjacent regions, the spatial interval covered by each boundary mask is gradually determined, so that the boundary mask can accurately correspond to the transition position between the boundary and the non-boundary; finally, the coverage range of the boundary mask is defined in the form of a spatial section corresponding to the coordinates, which includes the starting point, the end point and the corresponding region identifier. Based on this, through the execution of this step, the accurate positioning of the boundary mask is realized in the spatial level, so that the coverage range of the boundary mask can not only reflect the region distribution relationship, but also have clear spatial constraints and interpretability.

[0079] Optionally, for the adjacent boundary region and the non-boundary region, the contact boundary in space is extracted, and the coverage range of the corresponding boundary mask is determined according to the continuity of the boundary shape. For example, if the contact boundary of the two regions is relatively straight and changes gently, the coverage range of the boundary mask can be limited to a relatively narrow strip region; if the contact boundary has a curve or irregular change in shape, the coverage range will be expanded to ensure that the boundary mask can completely cover the boundary extension part. Thus, in this way, the coverage range defined by the boundary mask can accurately reflect the actual distribution shape of the boundary transition position.

[0080] Optionally, the boundary mask does not need to and should not completely cover the adjacent boundary region and the non-boundary region, and the role of the boundary mask is to locally model the transition position between the two. If the coverage range of the boundary mask is too large, it will weaken the difference between the boundary features and the non-boundary features, resulting in the smoothing of the boundary information during feature fusion; if the coverage range is too small, it will make it difficult for the boundary mask to accurately capture the boundary extension features, thereby affecting the continuous expression of the transition position. Therefore, the coverage of the boundary mask should be limited to the local strip region at the contact of the two types of regions, so that it can not only completely reflect the change characteristics of the boundary, but also maintain the differentiation degree between the boundary and the non-boundary in numerical terms, thereby providing accurate spatial constraints for the subsequent determination process of the weight distribution.

[0081] Step S302, determining the weight distribution of the corresponding boundary mask according to the position influence degree between the adjacent boundary region and the non-boundary region.

[0082] Exemplarily, within the determined mask coverage range, the weight distribution of the boundary mask is calculated and determined according to the relative position influence degree between the adjacent boundary region and the non-boundary region, that is, the influence relationship of different interface transition positions is quantitatively analyzed to determine the weight proportion of each position in the boundary mask. Specifically, first, representative position points are selected in the coverage range, and the distance difference of each position point to the center of the boundary region and the center of the non-boundary region is calculated to reflect the relative degree of influence of the two types of regions on the position. Subsequently, the influence relationship is converted into a weight value, so that the points close to the boundary region obtain a higher weight in the calculation, and the points close to the non-boundary region obtain a lower weight, thereby forming a continuous weight distribution gradually changing along the space. In addition, in order to avoid the calculation instability caused by the too large weight difference, the overall weight is uniformly adjusted after being determined, so that it is distributed within a reasonable range and keeps continuous transition.

[0083] Finally, the obtained weight distribution can be used to describe the influence transition between the adjacent boundary region and the non-boundary region, so that the corresponding boundary mask not only has the coverage significance at the numerical level, but also has the weight characteristics that can reflect the spatial change trend.

[0084] Optionally, since the boundary mask only covers the local strip region at the contact position of the two types of regions, the weight distribution of the boundary mask presents the feature of gradually decreasing along the boundary center to both sides. The region close to the boundary center has the most significant feature change and the highest weight value; when extending to the non-boundary direction, the weight value gradually decreases with the increase of the spatial distance, forming a continuous gradient distribution. This distribution mode enables the boundary mask to strengthen the feature response at the boundary and realize smooth connection, thereby maintaining the stability and natural transition of the feature fusion process while maintaining the sensitivity of boundary recognition.

[0085] Step S303, determining the boundary mask corresponding to each adjacent boundary region and non-boundary region according to the coverage range and weight distribution defined by each boundary mask.

[0086] Exemplarily, the coverage range and weight distribution of the boundary mask obtained by the foregoing steps are combined to determine the complete boundary mask corresponding to each pair of adjacent boundary region and non-boundary region. Specifically, first, the coverage range is mapped to the feature index space in the spatial dimension to ensure that the mask action position and the feature data position are one-to-one corresponding; then, the weight distribution is superimposed on the coverage range, so that the boundary mask simultaneously embodies the spatial action information and the weight gradient information in the numerical value.

[0087] After the final mask structure is formed, a unique identifier is assigned to each boundary mask, and its corresponding region index, coverage area and weight distribution are recorded to form a standardized mask definition format. This format enables each boundary mask to be directly called and reused in subsequent calculations, thereby achieving an accurate expression of local differences in the feature analysis stage.

[0088] Furthermore, in the construction of boundary masks, besides the case where a single boundary mask corresponds to a pair of adjacent boundary regions and non-boundary regions, there are indeed situations where multiple pairs of regions correspond to the same mask. When a boundary region is adjacent to multiple adjacent non-boundary regions, to maintain boundary continuity, a single boundary mask can be used to uniformly model the transition position of this composite boundary. Similarly, when multiple boundary regions are in contact with the same non-boundary region and their boundaries are connected, they can also be jointly covered by a single boundary mask. The simultaneous correspondence between multiple boundary regions and multiple non-boundary regions only holds true when the boundary overlaps or contact relationships are complex and spatially continuous, ensuring that the boundary mask still maintains structural consistency and computational controllability.

[0089] Optionally, a boundary mask The data can be represented in the following form:

[0090] (2)

[0091] In equation (2), Represents boundary mask Unique identifier, Represents boundary mask The coverage area Represents boundary mask The weight distribution.

[0092] Suppose that if in Location ,Record For this position The distance to the center line (or boundary line) of the boundary area, and the record. For this position The distance to the center of the non-boundary area, thus, the location The mask weights can be defined as:

[0093] (3)

[0094] In equation (3), if the position If it is located on the boundary line, then Therefore, the weight is 1; if the position The closer to the interior of the non-boundary region, the more... Increase and decrease, thus the weight corresponding to the small strain, forming a step-by-step transition of the numerical distribution. Finally, the function of this weight distribution is superimposed on the corresponding coverage to form a boundary mask with both clear spatial boundaries and continuous numerical gradients . .

[0095] Based on the foregoing, in step S301, the coverage of the boundary mask is determined according to the analysis of the relative distribution positions of the adjacent boundary region and the non-boundary region, so that the mask action area has clear definition and structural correspondence in space; in step S302, the weight distribution is determined according to the analysis of the position influence degree of the adjacent boundary region and the non-boundary region, so that the boundary mask has the quantization ability to reflect the strength of the boundary features; in step S303, the boundary mask is generated according to the comprehensive definition of the coverage and the weight distribution, so that the unity of spatial transition and numerical expression is realized at the feature level; based on this, in the entire technical solution, through the collaborative design of spatial positioning and weight allocation, the accurate modeling of the feature transition relationship between the boundary region and the non-boundary region is realized, so that the boundary expression is more coherent, smooth and computable.

[0096] In an exemplary embodiment, according to the coverage and weight distribution defined by each boundary mask, the image block feature sequence is adaptively masked and analyzed to generate mask information corresponding to different feature changes, including steps S401 to S402.

[0097] Step S401, according to the coverage defined by each boundary mask, the image block features in the image block feature sequence are grouped to divide the image block features within the coverage of the same boundary mask into corresponding feature groups.

[0098] Exemplarily, according to the coverage defined by each boundary mask, the image block features in the image block feature sequence are grouped and processed to realize the corresponding matching between the features and the mask regions. Specifically, first, the coverage of each boundary mask is read in space, which defines the spatial region where the boundary mask acts in the image block feature sequence. Then, the image block features corresponding to the coverage are retrieved one by one in the image block feature sequence, and these features are divided into corresponding groups according to their spatial coordinates and the overlap relationship with the coverage of the boundary mask. Thus, each feature group corresponds to a boundary mask, and contains all the feature information within the coverage of the boundary mask.

[0099] In this process, to ensure the integrity of the division, the case of partial overlap between the boundary regions also needs to be handled. When the image block feature is in the overlapping position of two boundary masks, the feature is assigned to the boundary mask with higher coverage ratio or preset priority rule, so as to maintain the uniqueness of the grouping division and the continuity of the spatial logic.

[0100] The priority rule is that, on the one hand, when the image block feature is in the overlapping position of two boundary masks, the image block feature is assigned to the boundary mask with higher coverage ratio, to represent that the image block feature has more significant influence on the region corresponding to the boundary mask; on the other hand, the image block feature can also be assigned to the boundary mask with greater change intensity, higher boundary continuity or closer distance to the spatial center of the region, according to the priority set by each boundary mask based on the change intensity of the region, the boundary continuity or the distance to the spatial center of the region when the boundary mask is generated, to ensure the rationality of the assignment result.

[0101] In this way, the image block feature sequence is reorganized into a plurality of feature groups corresponding to the mask structure one by one, and the features of each feature group represent a local feature set of the transition position between the boundary region and the non-boundary region, and the spatial attribution relationship is logically synchronized with the mask definition, so as to realize regionalization division at the feature level.

[0102] Optionally, if there is a case that the boundary region has complex morphology or branch structure, for example, when a boundary region is spatially split into a plurality of slender branches and contacts with non-boundary regions in different directions, the coverage range of the plurality of boundary masks corresponding to the boundary region will have multiple local overlaps. At this time, according to the correspondence between the image block feature and the main direction of the boundary, the feature is preferentially assigned to the boundary mask with the same direction, to ensure the consistency of the boundary extension. If a feature is close to multiple branch boundaries at the same time, the attribution is performed in the direction with more significant feature change by comparing the change intensity of the adjacent regions, so as to maintain the spatial coherence and calculation stability of the boundary mask.

[0103] In step S402, each feature group is weighted and fused according to the weight distribution defined by each boundary mask to obtain each feature fusion result, and each feature fusion result is taken as the mask information corresponding to different feature changes.

[0104] For example, based on the weight distribution defined by each boundary mask, weighted fusion processing is performed on each feature group obtained in the preceding steps to generate mask information reflecting the changing attributes of different features; that is, through the differentiated effect of the weight distribution, each feature reflects its computational contribution corresponding to its spatial position and intensity of change in the fusion. Specifically, firstly, the features and mask weights within each feature group are matched accordingly, that is, by establishing a positional mapping relationship in the feature space, the mask weights at corresponding positions in the boundary mask are paired with the features at the same positions in the feature group, so that each feature obtains a weight value consistent with its spatial position; subsequently, weighted accumulation is performed within each feature group according to the weight distribution, so that the feature values ​​form a smooth fusion effect in space.

[0105] Specifically, features closer to the boundary center have larger weights and a more significant impact on the final feature fusion result; features farther from the boundary or located in non-boundary regions have relatively smaller weights and a limited impact on the final feature fusion result. Through this weighted calculation method based on spatial gradients, the fusion result of each feature group can reflect the intensity of change and structural distribution within the region.

[0106] After fusion, the feature fusion result of each feature group is defined as an independent mask information. Each mask information contains weighted feature values ​​and their corresponding spatial range, thus numerically characterizing the significance of feature changes and the region affiliation. Finally, all mask information is aggregated to form a complete mask information set, which represents the feature fusion state within different boundary transition locations.

[0107] Optionally, if there are uneven distributions or inconsistent directions of change in boundary features—for example, when one side of the boundary region exhibits drastic feature changes while the other side changes more gradually—the weight distribution is adaptively adjusted during the weighted fusion process. This ensures that features on the side with higher changes occupy a higher proportion in the feature fusion result, thus more accurately reflecting local differences. Simultaneously, for regions with curved or converging boundaries, the weight distribution will shift directionally along the boundary shape, ensuring that the feature fusion result numerically aligns with the actual boundary trend, achieving refined representation and continuous feature expression of complex boundary structures.

[0108] Alternatively, the data representation of the mask information can refer to the following formula:

[0109] (4)

[0110] In equation (4), Indicates the first The mask information formed by a boundary mask; Indicates the first a feature position set covered by the boundary mask, i.e., a corresponding feature group; a weight distribution of the boundary mask; a feature fusion result obtained by fusing the feature groups corresponding to the boundary mask according to the corresponding weight distribution, used to depict the final generated feature expression of the boundary transition position.

[0111] Based on this, the adaptive coding feature is to integrate all such local feature fusion results with the original image block feature sequence into a feature map, and the overall expression obtained therefrom can be referred to as follows:

[0112] (5)

[0113] In formula (5), denotes the original mapping of the image block feature sequence in the feature space, denotes the sum of the feature fusion results corresponding to all boundary masks, and the fusion of the two according to the corresponding feature positions, so as to superimpose all the feature fusion results at the boundaries on the original features. The generated thereby not only retains the overall structure of the original image block features, but also injects refined features fused according to the mask weights at each boundary transition position, and the data form is still a multi-channel feature mapping with the same size as the feature map, so as to be directly sent to the subsequent decoding stage.

[0114] In this embodiment, in step S401, the image block feature sequence is grouped according to the coverage range defined by each boundary mask to obtain each feature group, so that the feature data is one-to-one corresponding to the mask action area in space, realizing the regionalized feature attribution division; in step S402, the corresponding feature groups are weighted and fused according to the weight distribution of each boundary mask, so as to form a response distribution gradually decaying from the boundary to the non-boundary in feature calculation, so that the feature fusion result can reflect the structural hierarchy in the boundary transition position; based on this, in the whole technical solution, through the collaborative design of feature grouping and weighted fusion, the ordered mapping of feature data from spatial division to hierarchical expression is realized, and the mask analysis result is more structured and accurate.

[0115] In an exemplary embodiment, in combination Figure 4 , in the decoder, the adaptive coding feature is subjected to channel shuffling and dilated convolution according to a preset pixel perception mechanism, to obtain a semantic segmentation feature, including steps S501 to S503.

[0116] Step S501, in the decoder, pixel perception weight calculation is performed on each pixel position in the spatial mapping of the adaptively encoded feature according to a pixel perception mechanism, to obtain a pixel perception result expressed at the pixel perception weight level.

[0117] The pixel perception result represents reference information for characterizing the importance of different pixels in the feature space. For example, when a certain pixel is in a region with obvious texture changes or significant semantic boundaries, the corresponding pixel perception weight is higher, so that greater calculation weight is given to this region in the subsequent feature processing.

[0118] By way of example, the decoder performs pixel perception weight calculation on each pixel position in the spatial mapping of the adaptively encoded feature according to a preset pixel perception mechanism, to obtain a pixel perception result reflecting the importance of pixels at the pixel perception level. Specifically, first, the adaptively encoded feature is restored to a spatial distribution form, for example, by deconvolution or upsampling, the mapped feature compressed in the encoding stage is restored to the original spatial resolution, so that each pixel position corresponds to a set of multi-dimensional feature values again; then, according to the feature difference, relative position and local structure relationship between the pixel and the surrounding region, and in combination with the mask structure feature involved in the pixel position, pixel perception weight calculation is performed on the multi-dimensional feature values of the pixel position, to determine the semantic association degree and local importance of the pixel position in the overall feature. The calculation process of the pixel perception weight comprehensively considers the response relationship between the pixel position and the neighborhood feature and the mask structure feature, so that the pixel position is affected by both its own feature and the mask structure feature involved.

[0119] Finally, the pixel perception weights corresponding to all pixel positions are integrated into a pixel perception result, which reflects the response difference of different pixel positions in the image in numerical form, so that the adaptively encoded feature regains the ability of semantic correspondence and local information recovery at the spatial level, thereby providing a structured spatial perception basis for subsequent channel mixing and dilated convolution processing.

[0120] Alternatively, for example, in a complex boundary region of an image, a certain pixel is located at the boundary transition position covered by the mask, and there is a significant difference in texture direction and brightness between the surrounding pixels, at this time, when calculating the pixel perception weight, the variation amplitude and direction consistency of the pixel and the neighborhood feature are considered, and in combination with the mask weight distribution involved, the perception weight of the pixel is improved, to highlight its importance in the expression of semantic boundaries. On the contrary, if the pixel is inside the mask and the surrounding feature difference is small, the pixel perception weight is low, only for maintaining the smoothness and structural continuity of the local region.

[0121] Further, the boundary mask weighting in the aforementioned embodiments and the pixel-aware weighting in the present embodiment do not conflict, but are in a hierarchical relationship. The former acts in the encoding stage and is a region-level weighting, which is used to determine the contribution of different spatial regions in feature fusion and construct the structural distribution of boundaries and non-boundaries; the latter acts in the decoding stage and is a pixel-level weighting, which is used to refine the local differences between pixels in the feature map after mask fusion and highlight semantic boundaries and detail information. The two complement each other, the former determines the overall morphology of the region structure, and the latter strengthens the fine expression within the region, thereby achieving coordination and unity between spatial consistency and semantic accuracy.

[0122] Optionally, the data representation form of the pixel-aware weight value can refer to the following formula:

[0123] (6)

[0124] In formula (6), denotes the spatial position of the adaptive coding feature in the feature space, denotes the pixel-aware weight value of the position , which is in the range of 0 to 1.

[0125] Based on this, the calculation method of the pixel-aware weight value can refer to the following formula:

[0126] (7)

[0127] In formula (7), denotes the basic feature of the position , denotes the average value of the neighborhood feature of the position , denotes the difference value between the two, which is used to measure the change intensity of the position and the surrounding region in the local structure or texture level; denotes the structure factor related to the spatial position of the pixel, which is used to reflect the position effect of the pixel in the image geometric layout, such as the spatial sensitivity of positions such as boundaries, corner points or centers, so that the perception calculation can reflect the spatial hierarchical features; denotes the mask semantic mapping function of the adaptive coding feature, which converts the adaptive coding feature after mask fusion into a scalar response value, so that the weight value calculation can consider the influence of the boundary mask, thereby reflecting the semantic difference of different mask regions at the pixel level; denotes a normalization function to compress or map the numerical range to a stable interval.

[0128] Finally, the coefficients , , The local difference factor, the spatial structure factor and the mask semantic factor are weighted respectively to obtain a weighted result, and a normalization function is used The weighted result is mapped to the interval [0, 1] to obtain the final pixel perception weight Based on this, the process integrates the information of the local difference factor, the spatial structure factor and the mask semantic factor, so that the final pixel perception weight not only pays attention to the local difference, but also considers the global structure and semantic distribution, realizing multi-level pixel perception modeling.

[0129] In step S502, according to the pixel perception result, the adaptive coding features are mixed in the channel, and the channel features that are spatially adjacent and similar in pixel perception weight are recombined to obtain the channel feature sequence corresponding to each channel.

[0130] The channel feature sequence represents an ordered feature set formed by reordering the channel features that are spatially adjacent and similar in pixel perception weight after the channel mixing operation, and is used to describe the relevance and semantic consistency between different channels at the feature level.

[0131] For example, according to the pixel perception result obtained in the foregoing steps, the adaptive coding features are mixed in the channel to realize structure reorganization based on pixel perception in the feature dimension. Since the adaptive coding features have integrated multiple mask information in the previous steps, the feature response distribution contained in different channels is unbalanced, so the channel distribution structure needs to be adjusted according to the pixel perception result to make the channel arrangement of feature data consistent with the spatial perception distribution.

[0132] Specifically, first, the pixel perception weight corresponding to each pixel position is read, and the correspondence between the pixel position and the channel feature is established in the spatial mapping; then, according to the high and low of the pixel perception weight, the channel features with high or low response intensity are identified in each channel, and the arrangement order of the channel features is adjusted based on this; then, the channel features that are spatially adjacent and similar in pixel perception weight are aggregated and recombined into continuous channel feature sequences, so that the channel features with similar pixel perception characteristics in space form an ordered mapping relationship in the channel dimension. At this time, the channel mixing process is no longer a simple channel rearrangement, but a feature structure reconstruction process guided by the pixel perception result, aiming to make the channel feature distribution consistent with the spatial semantic feature.

[0133] In this way, the relevant features originally dispersed in different channels are reorganized, so that the semantic correlation areas are gathered in the channel space, and the redundancy and separation between channels are reduced. After the reorganization is completed, the channel feature sequence of each output channel is output as a structured feature expression generated by pixel perception guidance, which can maintain the continuity of regional semantics in space and reflect the hierarchical distribution in feature dimension.

[0134] In step S503, the channel feature sequence of each channel is dilated convolution, and pixel perception weight is introduced in the dilated convolution process to weight the corresponding convolution area, to obtain semantic segmentation features that fuse local perception and global semantics.

[0135] For example, according to the multi-branch dilated convolution structure in the decoder, the channel feature sequence of each channel obtained in the foregoing step is dilated convolution to realize joint expression of different spatial scale features in the same level; wherein, based on the unified input feature, the same channel feature is calculated by multiple parallel dilated convolution branches, and the convolution kernel inside each branch is set to have different dilation rates in space, so that the convolution area forms a multi-scale coverage relationship from local to global on the feature map.

[0136] Specifically, the convolution branch with a smaller dilation rate is used to capture local features at the detail level, and the branch with a larger dilation rate expands the sampling interval of the convolution kernel to obtain structural correlation between distant areas. In order to ensure that the convolution area has response difference in calculation, pixel perception weight is introduced in the convolution operation of the corresponding convolution branch to weight the channel features in the convolution area, so that the contribution of different channel features in convolution calculation matches the perception weight of the pixels. In one embodiment, the smaller dilation rate corresponds to the convolution kernel sampling interval close to the standard convolution, such as a dilation rate of 1 or 2; and the larger dilation rate corresponds to a significantly increased convolution kernel sampling interval, such as a dilation rate of 4 to 8 or higher.

[0137] Furthermore, the design of the multi-branch structure makes the convolution area hierarchically complementary in the spatial receptive field, and the convolution results output by each convolution branch are fused and aggregated in the channel dimension to form a unified feature representation. In addition, during the fusion process, the outputs of each convolution branch are combined according to the preset weight, so that the local convolution response and the global convolution response are balanced in the feature expression.

[0138] Finally, the unified output feature representation is defined as semantic segmentation features, which integrates the weighted response results of different convolution areas, presents continuous semantic distribution in the spatial level, and reflects hierarchical expression from local details to global structure in the feature level.

[0139] In this embodiment, in step S501, pixel perception results are obtained by calculating pixel perception weights of pixel positions of adaptive coding features, so that the features obtain semantic correspondence and local information recovery ability in space, and provide response basis at the pixel level for subsequent feature reorganization; in step S502, channel mixing processing is performed on the adaptive coding features according to the pixel perception results, so that structure reorganization based on pixel perception weights is realized in the channel dimension, and similar channel features form an ordered association between channels; in step S503, pixel perception weights are introduced in the dilated convolution structure to weight the convolution region, so that joint expression of local details and global semantics is realized within the same feature layer; based on this, in the entire technical solution, through layer-by-layer coupling of pixel perception guidance, channel reorganization and dilated convolution, continuous modeling from pixel response to semantic features is realized, so that the semantic segmentation features have hierarchical expression ability and spatial consistency.

[0140] In one exemplary embodiment, according to the pixel perception results, the adaptive coding features are mixed and washed, and the channel features that are spatially adjacent and similar in pixel perception weight are recombined to obtain a channel feature sequence corresponding to each channel, including steps S601 to S602.

[0141] In step S601, according to the pixel perception results, the correlation degree of each channel feature in the adaptive coding features is analyzed to obtain a feature index relationship reflecting spatial proximity and similarity of pixel perception weights.

[0142] Exemplarily, first, based on the preset multi-channel structure in the adaptive coding features, the feature mapping corresponding to each channel is extracted to obtain the original channel feature set; each channel feature represents a feature distribution formed in response to a specific semantic, structure or region in the coding stage. Subsequently, the pixel perception weights corresponding to each pixel position in the feature space are read, and these pixel perception weights are matched with the channel features through a spatial mapping mechanism, so that each channel feature in each channel is in one-to-one correspondence with the perception weight of the corresponding pixel.

[0143] Then, the feature response relationship is counted inside the channel and between the channels, that is, the perceptual weight distribution features of each pixel in the channel are extracted, and the correlation degree of the channel features inside the channel and between the channels is calculated by comparing the response overlap degree of different channels in the spatial position and the consistency of the pixel perception distribution; for example, on the one hand, inside the channel, the correlation degree between the channel features in the same channel is obtained by calculating the difference between the pixel perceptual weights of different spatial positions in the same channel; on the other hand, between the channels, the correlation degree between the channel features of different channels is obtained by calculating the difference between the pixel perceptual weights of the same or different spatial positions in different channels. Thus, the obtained correlation degree reflects the comprehensive response relationship between the channel features in different channels in terms of spatial proximity and pixel perceptual weight similarity, for example: when two channels have similar pixel perceptual weight distribution in multiple positions and the spatial positions are close, the correlation degree of the corresponding channel features is high; otherwise, it is low. Thus, in this way, a quantitative structural description is formed in the channel dimension, so that the feature response relationship inside the channel and between the channels is mapped into ordered correlation data.

[0144] Finally, the correlation degrees of all channel features are integrated into a feature index relationship table, the channel index relationship table takes the channel features as index items, and is sorted according to the comprehensive response relationship of the spatial proximity and the pixel perceptual weight similarity inside the channel and between the channels, so that different channel features maintain the internal consistency of their respective channels in the overall sorting, and also present semantic coherence across channels, thereby forming a unified and ordered feature organization structure in the channel dimension.

[0145] Step S602, according to the feature index relationship, the adaptive coding features are mixed and shuffled to recombine the channel features according to the spatial proximity and the pixel perceptual weight similarity, and the channel feature sequence corresponding to each channel is obtained.

[0146] Exemplarily, first, according to the comprehensive response relationship of the spatial proximity and the pixel perceptual weight distribution similarity reflected by the feature index relationship table obtained in the foregoing steps, the original channel features are rearranged in the feature dimension, that is, according to the correlation degree between each channel feature in the feature index relationship table, the channel features with spatial proximity and similar pixel perceptual weight are arranged in adjacent positions, thereby forming a continuous distribution of semantic association relationship in the channel structure. In this process, for the channel features with high pixel perceptual weight, the dominant position in the sorting is preferentially retained to strengthen the expression of the significant area in the channel structure; and for the channel features with low pixel perceptual weight, they are merged to the secondary position in the arrangement to supplement the feature information of the edge or non-significant area.

[0147] In addition, not only the reordering of the channel features within a single channel is performed, but also the exchange and aggregation of the channel features between different channels are performed according to the similarity of the perceptual weights, so that similar feature patterns are kept locally continuous and globally ordered in the overall channel dimension.

[0148] Finally, according to the correlation degree between the channel features in the feature index relationship table and the numerical distribution of the pixel perceptual weights of the channel features in any local spatial range, a plurality of channel features that are spatially adjacent and similar in the distribution of the pixel perceptual weights are aggregated into a channel feature group, or only the channel features that are similar in the distribution of the pixel perceptual weights are aggregated into a channel feature group; thus, any aggregated channel feature group presents a locally continuous and globally ordered organization form in the channel dimension, which not only retains the difference between the channel feature groups, but also reflects consistent correlation at the semantic level.

[0149] In this way, the originally dispersed channel features are reorganized in structure, and the channel feature sequences of the output channels reflect the channel structure distribution guided by the pixel perceptual weights, so that the channel features realize the corresponding relationship in the spatial and semantic levels, and provide a more consistent and structured input basis for subsequent convolution processing.

[0150] Optionally, a channel can include a plurality of channel feature groups; each channel feature group is composed of local features that are spatially adjacent and similar in the distribution of the pixel perceptual weights, and reflects the perceptual response distribution of the channel in different spatial regions. The plurality of channel feature groups in the same channel have a mutually independent and locally continuous relationship, which is used to express the feature difference of the channel in different spatial regions. The channel feature groups between different channels establish a cross-channel semantic correspondence through the channel index relationship, so that the feature structure forms a hierarchical correlation from local to global in the channel dimension. Thus, the regionalized feature distribution is realized within a single channel, and the semantic complementarity is realized between different channels, so that the overall feature structure maintains consistency and mapping in the spatial and semantic levels.

[0151] Optionally, the adaptive coding feature is wherein, represents the original channel feature of the i-th channel, represents the original channel feature of the i-th channel, represents the number of channels.

[0152] Then, according to the pixel perceptual result, the correlation degree of each channel feature in the i-th channel is analyzed, and a position rearrangement mapping relationship reflecting the spatial proximity and the similarity of the pixel perceptual weights is obtained, i.e. ; wherein the position rearrangement mapping relationship is used to reorganize the feature positions within the channel or between the channels, so that a certain position is rearranged to a new position rearranged to another position so that the channel feature distribution in the first channel is locally adjusted in space according to the corresponding mapping relationship, realizing the structured rearrangement of the channel feature in the first channel; accordingly, the position rearrangement mapping relationship of the first channel is represented.

[0153] On this basis, in order to further maintain the semantic consistency across channels, the adaptive coding feature is subjected to channel shuffle operation, the process of which can refer to the following formula:

[0154] (8)

[0155] In formula (8), denotes the channel feature sequence of the first channel after shuffle; denotes in the first channel, according to the position rearrangement mapping relationship of the first channel , the feature value of the corresponding spatial position is taken, so that the spatial rearrangement result of the first channel is taken as the spatial reference for cross-channel aggregation, realizing the unification of channel position rearrangement and inter-channel feature fusion.

[0156] Further, denotes the contribution weight of the first channel to the output of the first channel, which is calculated according to the pixel perception weight similarity between channels; when the pixel perception distribution of two channels is similar, the contribution weight is larger, and the fusion effect is more significant. And denotes the channel set similar to the first channel in pixel perception weight or semantic distribution, which is used to limit the range of cross-channel feature participation; in addition, if = , it is equivalent to that the channel feature in the first channel is rearranged according to the position rearrangement mapping relationship of the channel feature in the first channel, to independently realize the feature rearrangement in the channel.

[0157] In the embodiment, in step S601, the correlation degree of each channel feature in the adaptive coding feature is analyzed according to the pixel perception result, thereby establishing a feature index relationship of the channel feature in the spatial proximity and the pixel perception weight similarity, so that the channel structure has a quantifiable semantic association basis; in step S602, the original channel feature is sorted and aggregated according to the feature index relationship, thereby forming a continuous distribution of the channel features with similar spatial proximity and pixel perception weight similarity in the channel dimension, realizing the structured channel reorganization; based on this, in the whole technical scheme, the continuity mapping from the pixel perception to the channel structure is realized through the correlation degree analysis of the channel feature and the hierarchical connection of the feature reorganization, so that the feature expression is consistent and semantically coherent in the spatial and channel dimensions.

[0158] In an exemplary embodiment, the channel feature sequence of each channel is dilated convolution, and the pixel perception weight is introduced in the dilated convolution process to weight the corresponding convolution region, to obtain semantic segmentation features that fuse local perception and global semantics, including steps S701 to S703.

[0159] Step S701, according to the preset first dilated convolution branch, the channel feature sequence of each channel is dilated convolution, and the initial branch result is obtained, and the convolution region corresponding to the initial branch result is weighted according to the corresponding pixel perception weight, to obtain the first branch result based on local perception.

[0160] Among them, the first dilated convolution branch represents a feature extraction path preset in the dilated convolution structure to perform convolution operation with a smaller dilation rate, which is used to capture the detail changes and structural features in the local neighborhood.

[0161] Among them, the first branch result based on local perception represents the convolution response result obtained by weighting the first dilated convolution branch combined with the pixel perception weight, such as the brightness mutation at the boundary or the subtle changes of local structure.

[0162] Exemplarily, the first dilated convolution branch dilates the channel feature sequence of each channel with a smaller dilation rate, that is, the convolution kernel moves on the spatial mapping of the channel feature according to the predetermined dilation step, so that each convolution calculation covers a wider spatial neighborhood under the premise of keeping the resolution unchanged, thereby obtaining the initial branch result containing local structural relationship.

[0163] Subsequently, the convolution regions in the initial branch result are weighted according to the corresponding pixel perception weight. Specifically, first, the perception weight of each pixel in the convolution region is extracted and matched with the convolution response value in the initial branch result; then, the numerical contribution of the convolution response value is adjusted according to the pixel perception weight, so that the pixel with a higher pixel perception weight has a greater weighting coefficient in the convolution result, and the pixel with a lower pixel perception weight has a weakened influence on the convolution output.

[0164] Through this weight weighting mechanism, the convolution response in the convolution region is redistributed, so that the convolution response result can more accurately reflect the difference relationship between pixels at the perception level. Based on this, the final first branch result based on local perception not only contains the structural feature distribution of the local region, but also embodies the differentiated feature expression introduced by the pixel perception weight, so that the feature has an interpretable response level in numerical value.

[0165] Further, in the local feature modeling stage of the first dilated convolution branch, in addition to pixel perception weighting, pixel perception weight can also be combined for dynamic modulation of the convolution kernel. Specifically, the weight parameters of the convolution kernel can be adaptively adjusted according to the distribution characteristics of the pixel perception weight, for example, the proportion coefficients of each weight parameter in the convolution kernel are dynamically adjusted according to the mean and variance of the pixel perception weight, so that the convolution weight of the region with a higher perception weight is increased, and the convolution weight of the region with a lower perception weight is decreased, thereby making the convolution kernel form a biased response in the perception concentrated region. In this way, the sampling direction, receptive strength or response center of the convolution kernel in the local region is shifted with the importance of the pixel, so as to realize dynamic focusing of local information without changing the overall convolution structure. Based on this, this way directly integrates pixel perception into the calculation logic of the convolution kernel, so that the convolution operation has the local self-adjusting feature; unlike simple numerical weighting, this way makes the convolution feature extraction process and the pixel perception mechanism form close cooperation, which not only enhances the response sensitivity to key pixels, but also maintains the integrity of the convolution in structure.

[0166] Step S702, according to the preset second dilated convolution branch, the channel feature sequence of each channel is dilated convolution, and the second branch result based on global semantics is obtained.

[0167] Among them, the second dilated convolution branch represents a feature extraction path with a larger dilated rate in the dilated convolution structure, which is used to capture the global feature relationship and semantic association relationship across regions.

[0168] Among them, the second branch result based on global semantics represents the convolution response result obtained by the second dilated convolution branch, such as the semantic connectivity or overall morphological profile feature between target regions.

[0169] Exemplarily, the second dilated convolution branch performs dilated convolution on the channel feature sequence of each channel according to a larger dilated rate, so that the convolution kernel covers the input feature in a more sparse sampling manner during calculation, thereby expanding the receptive field range of the convolution. At the same time, the second dilated convolution branch realizes the aggregation of global information and the promotion of structural levels while keeping the feature dimension unchanged, so that the convolution response result can provide overall expression reference at the semantic level for the subsequent feature fusion stage.

[0170] Based on this, the final global semantic-based second branch result forms a cross-regional feature correlation distribution in spatial mapping, provides a structured input for the unified modeling of local and global features, enables the subsequent fusion operation to have a global context alignment basis, and thus guarantees the completeness and consistency of the feature expression at the global level.

[0171] Further, the pixel perception is introduced in the first dilated convolution branch with a smaller dilated rate because the convolution receptive field area of this branch is mainly concentrated in a local range, which can more accurately reflect the pixel-level differences and detailed structures. By combining the pixel perception weight, the response intensity of the local area can be finely adjusted during the convolution process, so that subtle changes are strengthened. The second dilated convolution branch with a larger dilated rate focuses on the global semantic correlation across regions, and its convolution coverage is wider. If pixel-level weights are introduced, the overall consistency of the global features will be weakened, so pixel perception is more suitable for the local feature modeling stage of the first dilated convolution branch.

[0172] Step S703, fuse the first branch result and the second branch result to obtain semantic segmentation features that fuse local perception and global semantics.

[0173] Exemplarily, the first branch result and the second branch result are fused to simultaneously retain the detailed expression of the local perception feature and the overall structure of the global semantic feature. Among them, the fusion process is based on the spatial correspondence relationship, aligns the two types of branch results in the feature dimension, and generates unified semantic segmentation features through feature superposition or weighted integration, that is, forms semantic segmentation features that have detailed expression ability at the spatial level and overall correlation at the semantic level.

[0174] Optionally, the data representation of the semantic segmentation features that fuse local perception and global semantics may refer to the following formula:

[0175] (9)

[0176] In formula (9), represents the output of the channel feature sequence of the i-th channel after convolution in the first dilated convolution branch (local perception branch); ​Indicates the first The output of the channel feature sequence of each channel after convolution under the second dilated convolution branch (global semantic branch).

[0177] Furthermore, , These represent the first and second dilated convolution branches, respectively. The channel weights of each channel reflect the response contribution of different channel feature sequences during the fusion process; This represents the balance coefficient between local and global features, used to control the proportion of the two types of features in the overall semantic expression.

[0178] Based on this, the output of the first dilated convolution branch Based on all Perform a full-channel weighted summation on each channel, and then apply the output of the second dilated convolution branch. Based on all Each channel is weighted and summed, and then the two summation results are combined according to the balance coefficient. By controlling the overall proportions, a unified semantic segmentation feature can be obtained. .

[0179] In this embodiment, in step S701, the channel feature sequence is subjected to local convolution calculation according to the first dilated convolution branch, and the convolution region is weighted by pixel-aware weights to highlight the detailed differences and regional saliency of local features; in step S702, the channel feature sequence is subjected to large-scale convolution calculation according to the second dilated convolution branch to extract global semantic relationships and overall structural information across regions; in step S703, the results of the two branches are fused to unify and integrate local perception and global semantics, thereby efficiently obtaining semantic segmentation features that fuse local perception and global semantics; based on this, in the entire technical solution, by introducing pixel-aware multi-scale dilated convolution operation, the unified modeling of local features and global features in the semantic segmentation process is realized, improving the structural integrity and semantic continuity of feature expression.

[0180] It should be understood that although the steps in the flowcharts involved in the embodiments described above are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0181] Based on the same inventive concept, the embodiments of the present application also provide a semantic segmentation-based image instance recognition system for implementing the semantic segmentation-based image instance recognition method described above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme described in the above method, so the specific limitations in one or more semantic segmentation-based image instance recognition system embodiments provided below can refer to the limitations of the semantic segmentation-based image instance recognition method described above, which will not be repeated here.

[0182] In an exemplary embodiment, as shown in Figure 5 a semantic segmentation-based image instance recognition system is provided, comprising: an acquisition module 101, an encoding module 102, a decoding module 103 and an identification module 104, wherein:

[0183] The acquisition module 101 is configured to acquire a to-be-processed image, perform structured preprocessing on the to-be-processed image, and obtain an image block feature sequence.

[0184] The encoding module 102 is configured to input the image block feature sequence into a preset encoder, perform adaptive mask analysis on the image block feature sequence in the encoder according to the visual change relationship between the image block features to generate mask information corresponding to different feature changes, and fuse each mask information and the image block feature sequence to generate adaptive encoding features.

[0185] The decoding module 103 is configured to input the adaptive encoding features into a preset decoder, perform channel shuffling and dilated convolution on the adaptive encoding features in the decoder according to a preset pixel perception mechanism, and obtain semantic segmentation features.

[0186] The identification module 104 is configured to perform instance recognition on the to-be-processed image according to the semantic segmentation features, and obtain an instance recognition result corresponding to the to-be-processed image.

[0187] In an example embodiment, the encoding module 102 is further configured to: in the encoder, perform response value calculation on the image block features in the image block feature sequence according to the visual change relationship between the image block features, to obtain a change response map representing change intensity; in the change response map, mark a region with change intensity greater than or equal to a preset change threshold as a boundary region, and mark a region with change intensity less than the preset change threshold as a non-boundary region; determine a boundary mask corresponding to each adjacent boundary region and non-boundary region according to the positional relationship between the adjacent boundary region and the non-boundary region; and perform adaptive mask analysis on the image block feature sequence according to the coverage range and the weight distribution defined by each boundary mask, to generate mask information corresponding to different feature changes.

[0188] In an example embodiment, the encoding module 102 is further configured to: determine the coverage range defined by the corresponding boundary mask according to the relative distribution position between the adjacent boundary region and the non-boundary region; determine the weight distribution defined by the corresponding boundary mask according to the position influence degree between the adjacent boundary region and the non-boundary region; and determine the boundary mask corresponding to each adjacent boundary region and non-boundary region according to the coverage range and the weight distribution defined by each boundary mask.

[0189] In an example embodiment, the encoding module 102 is further configured to: group the image block features in the image block feature sequence according to the coverage range defined by each boundary mask, to divide the image block features within the coverage range of the same boundary mask into corresponding feature groups; and perform weighted fusion processing on each feature group respectively according to the weight distribution defined by each boundary mask to obtain each feature fusion result, and take each feature fusion result as the mask information corresponding to different feature changes.

[0190] In an example embodiment, the decoding module 103 is further configured to: in the decoder, perform pixel perception weight calculation on each pixel position of the adaptive encoding feature in the spatial mapping according to a pixel perception mechanism, to obtain a pixel perception result represented at the pixel perception weight level; perform channel shuffle on the adaptive encoding feature according to the pixel perception result, and recombine channel features that are spatially adjacent and similar in pixel perception weight, to obtain a channel feature sequence corresponding to each channel; perform dilated convolution on the channel feature sequence of each channel, and introduce pixel perception weights to weight the corresponding convolution regions in the dilated convolution process, to obtain semantic segmentation features that fuse local perception and global semantics.

[0191] In an example embodiment, the decoding module 103 is further configured to: according to the pixel perception result, perform correlation analysis on the channel features in the adaptive coding features to obtain a feature index relationship reflecting spatial proximity and pixel perception weight similarity; and perform channel mixing on the adaptive coding features according to the feature index relationship to recombine the channel features according to spatial proximity and pixel perception weight similarity to obtain a channel feature sequence corresponding to each channel.

[0192] In an example embodiment, the decoding module 103 is further configured to: perform dilated convolution on the channel feature sequence of each channel according to a preset first dilated convolution branch to obtain an initial branch result, perform weighted processing on a convolution region corresponding to the initial branch result according to a corresponding pixel perception weight to obtain a first branch result based on local perception; perform dilated convolution on the channel feature sequence of each channel according to a preset second dilated convolution branch to obtain a second branch result based on global semantics; and fuse the first branch result and the second branch result to obtain semantic segmentation features fusing local perception and global semantics.

[0193] The above-described modules in the image instance recognition system based on semantic segmentation can be implemented by software, hardware, or a combination thereof. The above-described modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform operations corresponding to the above-described modules.

[0194] In an example embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps in any of the above-described embodiments when executing the computer program.

[0195] In an example embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program is executed by a processor to implement the steps in any of the above-described embodiments.

[0196] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, database or other medium used in each embodiment of the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in each embodiment of the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without limitation. The processor involved in each embodiment of the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without limitation.

[0197] In the embodiments of the present application, for the overall goal of "image instance recognition based on semantic segmentation", the encoding stage and the decoding stage of the existing network are respectively improved in structure: on the one hand, the encoder introduces an adaptive coding idea based on change response and mask division when generating features, and on the other hand, the decoder introduces a decoding idea based on pixel perception, channel mixing and dilated convolution when restoring features, thereby forming a set of instance recognition scheme from feature division, feature coding to semantic recovery.

[0198] (I) Structural improvement for the encoding stage:

[0199] 1.1 Differences between the encoder and related technologies: Traditional encoding stages mostly use unified convolution or attention mechanism to extract the entire feature map, such as ResNet encoder, Swin Transformer encoder, and other traditional encoding networks, which do not differentiate the region change intensity or boundary transition position, resulting in a lack of spatial hierarchy in the encoding result. The encoding stage of the present application introduces an adaptive encoding idea based on change response and mask division: first detect local feature changes during feature generation, and then generate mask information based on the change region, so that the encoded features can reflect regional differences during generation, thereby establishing an encoding expression corresponding to the image structure.

[0200] The encoder of the embodiment of the present application combines change response and mask division, changes the encoding process from "global uniform extraction" to "region adaptive generation"; that is, by detecting the strength of feature difference through change response, and then performing feature fusion in mask units, the dynamic balance of local features and global distribution is realized; thus, the design structure directly completes the explicit expression of region division, change intensity and spatial attribution in the encoding stage, thereby reducing the dependence on boundary recovery in the subsequent stage, and improving the structural clarity and interpretability of the features as a whole.

[0201] The encoder of the embodiment of the present application can adaptively reflect regional change differences; can realize early modeling of boundary features, and reduce the decoding burden; the encoding output has hierarchical and structural constraints; and the region expression is more in line with the semantic distribution law.

[0202] (II) Structural improvement for the decoding stage:

[0203] 2.1 Differences between the decoder and related technologies: Traditional decoding stages mostly use fixed convolution or upsampling operations to restore spatial resolution, such as UNet decoder, SegNet decoder, and other traditional decoding networks, in which the arrangement between channels has no corresponding relationship with pixel importance, resulting in that the restored features cannot maintain semantic consistency; the decoding stage of the present application introduces a decoding idea based on pixel-aware channel shuffling and dilated convolution: first, quantify the importance of each position based on pixel features, and then guide the channel shuffling based on this, so that the channel feature distribution is aligned with the spatial semantics, and then utilize convolution branches with different dilation rates to realize collaborative modeling of local and global semantics, forming a hierarchical restoration structure.

[0204] The decoder of the embodiment of the application combines channel shuffling and dilated convolution based on pixel perception, maps the importance of the spatial layer to the channel layer, completes semantic aggregation before decoding input, and realizes adaptive restoration in cooperation with a multi-dilated convolution path, wherein local details are captured through a small dilated rate branch, global semantics are extracted through a large dilated rate branch, and finally, cross-scale consistent semantic reconstruction is realized in the fusion stage; in this way, the decoding stage has a dynamic attention mechanism, and the convolution calculation is no longer evenly distributed, but is developed around the semantic significant area, and the restoration effect is more accurate and stable.

[0205] The decoder of the application realizes pixel perception guided channel structure rearrangement, complementary local and global convolution branches, coherent output feature semantics, and clear details, and forms a structural closed loop with adaptive coding.

[0206] (Three) example evaluation:

[0207] 3.1 Example evaluation of the encoding stage: when performing encoding processing under the same data set (for example, Cityscapes), the feature map of the traditional encoding network using uniform convolution or attention mechanism described above often appears blurred transition at the complex boundary, the boundary and non-boundary area response overlap is serious, and the region details are difficult to retain; compared with the ResNet encoder, Swin Transformer encoder and other traditional encoding networks, the encoder of the embodiment of the application can make the generated encoding features have an increase of about 5% to 10% in feature separation degree at the edge and structure mutation area, effectively reduce the boundary drift phenomenon, and make the encoding result more spatially interpretable.

[0208] 3.2 Example evaluation of the decoding stage: when performing decoding processing under the same feature input, the semantic map output by the traditional decoding network using fixed convolution or upsampling operation has information smoothing and small target missing problems in the local detail recovery stage; compared with the UNet decoder, SegNet decoder and other traditional decoding networks, the decoder of the embodiment of the application based on pixel perception channel shuffling and double dilated convolution structure can strengthen the edge and high-frequency details, and the overall detail retention rate is improved by about 6% to 9%, and the recognition in small targets and complex scenes is more stable.

[0209] 3.3 Example evaluation of the overall network: when performing end-to-end training on the same data set, the network of the application shows stronger semantic distinction and structure stability in the instance recognition task; compared with the traditional network combining the traditional encoding network and the traditional decoding network described above, the application improves the average intersection over union (mIoU) by about 2% to 4%, and the boundary accuracy index (BF Score) by about 3% to 5%, and the overall output result is obviously improved in semantic boundary clarity and target structure integrity.

[0210] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered to be within the scope of the present disclosure.

[0211] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An image instance recognition method based on semantic segmentation, characterized in that, Includes the following steps: The image to be processed is acquired, and the image to be processed is subjected to structured preprocessing to obtain an image block feature sequence; The image patch feature sequence is input into a preset encoder. In the encoder, adaptive mask analysis is performed on the image patch feature sequence based on the visual change relationship between the image patch features to generate mask information corresponding to different feature changes. The mask information and the image patch feature sequence are then fused to generate adaptive encoded features. In the encoder, response values ​​are calculated for the image patch features in the image patch feature sequence based on the visual change relationship between the image patch features, resulting in a change response map representing the change intensity. In the change response map, regions with change intensity greater than or equal to a preset change threshold are marked as boundary regions, and regions with change intensity less than the preset change threshold are marked as non-boundary regions. Based on the positional relationship between adjacent boundary regions and non-boundary regions, a boundary mask corresponding to each adjacent boundary region and non-boundary region is determined. Based on the coverage area and weight distribution defined by each boundary mask, adaptive mask analysis is performed on the image patch feature sequence to generate mask information corresponding to different feature changes. The adaptive coding features are input into a preset decoder. In the decoder, according to a preset pixel-aware mechanism, the adaptive coding features are subjected to channel shuffling and dilated convolution to obtain semantic segmentation features. Specifically, this includes: calculating pixel-aware weights for each pixel position of the adaptive coding features in the spatial mapping according to the pixel-aware mechanism, obtaining pixel-aware results represented at the pixel-aware weight level; performing channel shuffling on the adaptive coding features based on the pixel-aware results, and recombining spatially adjacent channel features with similar pixel-aware weights to obtain channel feature sequences corresponding to each channel; the pixel-aware mechanism represents the mapping rules used in the decoder to establish the correspondence between the feature space and the image space, so as to restore the spatial layout and pixel-level semantic correspondence of the image during feature restoration; performing dilated convolution on the channel feature sequences of each channel, and introducing pixel-aware weights to weight the corresponding convolution regions during the dilated convolution process, to obtain semantic segmentation features that fuse local perception and global semantics. The image to be processed is identified based on the semantic segmentation features to obtain the instance identification result corresponding to the image to be processed.

2. The method according to claim 1, characterized in that, The step of determining the boundary mask corresponding to each adjacent boundary region and non-boundary region based on the positional relationship between adjacent boundary regions and non-boundary regions includes: The coverage area defined by the corresponding boundary mask is determined based on the relative distribution of adjacent boundary regions and non-boundary regions. The weight distribution defined by the corresponding boundary mask is determined based on the degree of positional influence between adjacent boundary regions and non-boundary regions. Based on the coverage area and weight distribution defined by each boundary mask, determine the boundary mask corresponding to each adjacent boundary region and non-boundary region.

3. The method according to claim 1, characterized in that, The step of performing adaptive mask analysis on the image patch feature sequence based on the coverage area and weight distribution defined by each boundary mask to generate mask information corresponding to different feature changes includes: Based on the coverage area defined by each boundary mask, the image block features in the image block feature sequence are grouped so that the image block features within the coverage area of ​​the same boundary mask are divided into corresponding feature groups; Based on the weight distribution defined by each boundary mask, each feature group is subjected to weighted fusion processing to obtain the feature fusion result, and the feature fusion result is used as the mask information corresponding to different feature changes.

4. The method according to claim 1, characterized in that, The step of performing channel shuffling on the adaptive coding features based on the pixel perception results, and recombining spatially adjacent channel features with similar pixel perception weights to obtain the channel feature sequence corresponding to each channel includes: Based on the pixel perception results, correlation analysis is performed on the features of each channel in the adaptive coding features to obtain the feature index relationship that reflects spatial proximity and pixel perception weight similarity; Based on the feature index relationship, the adaptive coding features are shuffled into channels to recombine the channel features according to spatial proximity and pixel perception weight similarity, thereby obtaining the channel feature sequence corresponding to each channel.

5. The method according to claim 1, characterized in that, The process involves performing dilated convolution on the channel feature sequences of each channel, and introducing pixel-aware weights to weight the corresponding convolution regions during the dilated convolution process, resulting in semantic segmentation features that fuse local perception and global semantics. This includes: The channel feature sequences of each channel are dilated and convolved by the first dilated convolution branch to obtain the initial branch result. The convolution region corresponding to the initial branch result is weighted according to the corresponding pixel perception weight to obtain the first branch result based on local perception. The second branch of dilated convolution is used to dilate the channel feature sequences of each channel to obtain the second branch result based on global semantics. The results of the first branch and the second branch are fused to obtain semantic segmentation features that integrate local perception and global semantics.

6. An image instance recognition system based on semantic segmentation, characterized in that, The system for implementing the method according to any one of claims 1-5 comprises: The acquisition module is used to acquire the image to be processed, perform structured preprocessing on the image to be processed, and obtain an image block feature sequence; The encoding module is used to input the image block feature sequence into a preset encoder. In the encoder, based on the visual change relationship between the image block features, adaptive mask analysis is performed on the image block feature sequence to generate mask information corresponding to different feature changes. The mask information and the image block feature sequence are then fused to generate adaptive encoded features. The decoding module is used to input the adaptive coding features into a preset decoder, in which the adaptive coding features are subjected to channel shuffling and dilated convolution according to a preset pixel awareness mechanism to obtain semantic segmentation features. The recognition module is used to perform instance recognition on the image to be processed based on the semantic segmentation features, and obtain the instance recognition result corresponding to the image to be processed.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images