Instance segmentation method in low-contrast environment and readable storage medium

Abstract

The invention discloses an instance segmentation method in a low-contrast environment and a readable storage medium, and belongs to the technical field of image processing, the instance segmentation method in the low-contrast environment introduces an adaptive state space module, performs long-range dependence modeling based on a down-sampled input image to obtain a state space feature, and obtains a state space feature; and performing multi-scale edge feature extraction based on the state space features by adopting a multi-scale adaptive edge enhancement module to obtain multi-scale edge enhancement features, screening a final prediction frame of the target object through an instance segmentation network based on the multi-scale edge enhancement features, generating a final mask of the target object, and obtaining the final mask of the target object. And the target object is subjected to instance segmentation according to the final mask, so that sequence modeling of linear complexity is realized, excellent long-range dependence modeling capability is obtained while efficient calculation is kept, and the edge features of the object to be subjected to instance segmentation can be well adaptively enhanced in a low-contrast situation.

Description

Technical Field

[0001] This invention relates to an instance segmentation method and a readable storage medium in a low-contrast environment, belonging to the field of image processing technology. Background Technology

[0002] Instance segmentation, a core task in computer vision, requires the simultaneous completion of object detection and pixel-level semantic segmentation, and has wide applications in fields such as medical image analysis, autonomous driving, and financial credentials. Taking bank cards as an example, automated instance segmentation is of great significance for ensuring smooth financial transactions, improving user experience, and promoting the construction of smart banks. With the increasing complexity of digital financial scenarios, bank card images often exhibit low contrast due to factors such as insufficient ambient light, material reflections, or interference from similar backgrounds, resulting in blurred object edges and loss of texture details. Traditional image segmentation methods, such as edge detection-based and threshold-based segmentation, suffer a sharp decline in recognition and segmentation accuracy when faced with such low-contrast, complex-textured, and blurred-boundary images, making it difficult to meet the high availability requirements of financial services.

[0003] Current mainstream instance segmentation methods primarily rely on deep learning models based on Convolutional Neural Networks (CNNs) or Transformers. CNNs possess strong inductive bias properties, but their receptive field is limited; while Transformers can effectively model long-range dependencies through a global attention mechanism. The combination of the two can simultaneously address local feature extraction and global context awareness. However, the attention computation of Transformers suffers from a quadratic complexity of O(n²). Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an instance segmentation method and readable storage medium in low-contrast environments. By constructing a lightweight adaptive instance segmentation model, it achieves linear complexity sequence modeling, obtains excellent long-range dependency modeling capability while maintaining high computational efficiency, enhances the edge features of objects to be segmented in low-contrast environments, and improves the overall segmentation accuracy.

[0005] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution: This invention provides an instance segmentation method for low-contrast environments, comprising: Image data acquired in a low-contrast environment is input into a pre-trained adaptive instance segmentation model to segment the target object in the image. The adaptive instance segmentation model includes a backbone network and an instance segmentation network; The backbone network includes an adaptive state space module and a multi-scale adaptive edge enhancement module; The adaptive state space module is used to perform long-range dependency modeling based on the downsampled input image to obtain state space features; The multi-scale adaptive edge enhancement module is used to extract multi-scale edge features based on state space features to obtain multi-scale edge enhancement features. The instance segmentation network is used to filter the final predicted bounding box of the target object based on multi-scale edge enhancement features, generate the final mask of the target object, and perform instance segmentation of the target object based on the final mask.

[0006] Furthermore, the adaptive state space module is used to perform long-range dependency modeling based on the downsampled input image to obtain state space features, including: Perform on the downsampled input image Convolution calculation yields the first feature; The sampling point positions are adjusted using deformable convolution on the first feature to obtain the second feature, which is represented as: ; in, Indicates the second characteristic, Represents deformable convolution. This represents the first feature of the input image; Represents the total number of aggregate groups, for The first in Aggregation group: This indicates the number of pixels in each aggregation group. Indicates the first element in the aggregate group Each sampled pixel point Indicates the first The value learned by each sampled pixel in relation to the location of the aggregate group; express Convolution is used to sample the position of pixels in the receptive field. , This indicates that the K×K convolution kernel is in the receptive field of the th... The position of each sampling point relative to the center point; Indicates the first The feature vector after adaptive deformable sampling in the aggregation group; This represents the input pixel at the current position in the feature map. Represents points on the feature map The position offset performed; Indicates an aggregation group Group 1 Weighting coefficients at each sampled pixel; The second feature is input into the structured state space submodule, and the zero-order preservation technique of parameter discretization is used to scan all pixels in the second feature and capture global context information to obtain the third feature; The third feature and the residual of the first feature are concatenated, and deformable convolution is used to obtain the fourth feature; The residuals of the fourth and third features are concatenated to obtain the state-space features; The second feature is input into the structured state space submodule, and the zero-order preservation technique of parameter discretization is used to scan all pixels in the second feature and capture global context information to obtain the third feature; The third feature and the residual of the first feature are concatenated, and deformable convolution is used to obtain the fourth feature; The state-space features are obtained by concatenating the residuals of the fourth and third features.

[0007] Furthermore, the second feature is input into the structured state space submodule, and the zero-order preservation technique of parameter discretization is used to scan all pixels in the second feature and capture global context information to obtain the third feature, including: The second feature is expanded into a sequence. The zero-order preservation technique of parameter discretization is used to analyze the sequence. Full-pixel scanning using structured convolution is represented as: ; ; ; ; in, Indicates the third characteristic, Represents a structured convolution kernel. , Represents a sequence Length; This represents a transition matrix that allows control over the changes in the hidden state over time. , Represents the set of real numbers. express Dimensions express Discretized representation; This represents the weight matrix of an input matrix relative to the hidden states. , express Dimensions express Discretized representation; This represents an observation matrix that maps hidden intermediate states to output. , express The dimension; Indicates the time scale parameter. Represents the identity matrix; Represented by real numbers e An exponential function with base 0.

[0008] Furthermore, the multi-scale adaptive edge enhancement module is used to extract multi-scale edge features based on state space features to obtain multi-scale edge enhancement features, including: The state space features are adopted The number of channels adjusted during convolution is expressed as: ; in, This represents the adjusted state-space characteristics. Represents the state space features of the input. express convolution; Features of the state space Deformable convolution is used to adaptively analyze the edge feature information of the target object, resulting in adaptive features, represented as: ; Represents adaptive features. This indicates the use of batch normalization and the Sigmoid function. convolution; right Use three times Pooling is used to extract multi-scale edge information and enhance the edge information, as shown below: ; ; in, The proportion is Feature map, The proportion is Feature map, express Pooling, This indicates max pooling. Indicates average pooling. Let the given input feature map be... One of them, This represents the number of channels in the input feature map. Represents the first... aisle; Introducing edge enhancement methods To enhance multi-scale feature maps , and The detailed information on the surface is used to obtain the feature maps after edge enhancement. , and ; By channel cascading , , and By concatenating these features, we obtain multi-scale edge enhancement features that contain fine-grained feature information and multi-scale edge information, represented as: ; in, This represents multi-scale edge enhancement features. express convolution.

[0009] Furthermore, the proposed edge enhancement method To enhance multi-scale feature maps , and The detailed information on the surface is used to obtain the feature maps after edge enhancement. , and ,include: ; ; in, This represents the feature map after edge enhancement. Indicates the first Branching Convolution and Features of pooling output express , and One of them, according to , and Choose the branch where it is located.

[0010] Furthermore, the backbone network also includes a downsampling module, which is used to downsample the instance segmentation dataset input to the pre-trained lightweight adaptive instance segmentation model to obtain the downsampled input image.

[0011] Furthermore, the backbone network also includes a cross-pixel mixing module, which is used to perform sparse dynamic receptive field downsampling and mixing based on multi-scale edge enhancement features to obtain downsampled cross-pixel mixed features.

[0012] Furthermore, the instance segmentation network is used to filter the final predicted bounding box of the target object based on multi-scale edge enhancement features, generate the final mask of the target object, and perform instance segmentation of the target object based on the final mask, including: An instance segmentation head is set up, consisting of a classification branch, a bounding box branch, and a segmentation branch. Each branch includes two 3×3 convolutions and one 1×1 convolution. The classification branch is used to determine the category of the target object and generate the target category probability. The bounding box branch is used to provide a predicted bounding box and a corresponding confidence score for the region of the suspected target object. The segmentation branch is used to generate a basic mask based on the predicted bounding box. Based on the confidence scores of the predicted boxes given by the bounding box branch, the final target box is selected from all the predicted boxes using the non-maximum suppression method; Based on the target category probability and the final target bounding box, the base mask is adjusted and refined to obtain the final target mask; Based on the final target mask, the target object is segmented into instances.

[0013] Furthermore, during the training of the adaptive instance segmentation model, the bounding box branch employs the AInnerCIoU loss function and the DFL loss function; The AInnerCIoU loss function is used to adjust the position of the predicted bounding box to ensure that only the true target is contained within the box, as shown below: ; ; ; ; ; ; in, Represents the AInnerCIoU loss function; Represents the true bounding box. This represents the width of the actual bounding box. Indicates the height of the actual bounding box. Represents the center point of the true bounding box ; Indicates the prediction box. Indicates the width of the prediction box. Indicates the height of the predicted bounding box. Indicates the center point of the prediction box ; Represents pi (π). The diagonal of the smallest bounding box between the ground truth box and the predicted box; 、 、 、 、 、 、 、 This represents the scaling factor that allows for adaptive learning during the network training phase, with each adaptive scaling factor initially set to 1.0; This represents the function that takes the minimum value. This represents the function that takes the maximum value. Represents the consistency parameter. This represents the weighting coefficients for the orthogonal shapes of the ground truth bounding box and the predicted bounding box. This represents the area of ​​the adaptive intersection of the ground truth bounding box and the predicted bounding box. This indicates that the left boundary of the help box adapts to the actual bounding box. This indicates the right boundary of the guide box that adapts to the actual bounding box. This indicates the upper boundary of the actual bounding box that adapts to the auxiliary bounding box. This represents the lower boundary of the actual bounding box that adapts to the auxiliary bounding box. This represents the left boundary of the adaptive auxiliary box for the predicted bounding box. This represents the right boundary of the adaptive auxiliary box of the predicted box. This represents the upper boundary of the adaptive auxiliary box of the predicted box. This indicates the lower boundary of the adaptive auxiliary box for the predicted bounding box; The DFL loss function is expressed as: ; Represents the DFL loss function; and Indicates that it contains real tags The boundary values ​​of two adjacent discrete intervals; This indicates that the coordinates predicted by the model fall within The probability value corresponding to the interval. This indicates that the coordinates predicted by the model fall within The probability value for the corresponding interval.

[0014] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the instance segmentation method in a low-contrast environment as described above.

[0015] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: This invention introduces an adaptive state space module, which performs long-range dependency modeling based on the downsampled input image to obtain state space features. A multi-scale adaptive edge enhancement module is then used to extract multi-scale edge features based on the state space features, resulting in multi-scale edge enhancement features. Based on these multi-scale edge enhancement features, an instance segmentation network is used to filter the final predicted bounding box of the target object, generating the final mask of the target object. Instance segmentation of the target object is then performed based on the final mask, achieving linear complexity sequence modeling. While maintaining high computational efficiency, it achieves excellent long-range dependency modeling capabilities and can effectively adaptively enhance the edge features of the object to be segmented in low-contrast scenarios. This solves the problem that current instance segmentation models suffer from a sharp drop in recognition accuracy and computational complexity in low-contrast environments. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of the adaptive instance segmentation model provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the downsampling module provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the adaptive state space module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the multi-scale adaptive edge enhancement module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the cross-pixel mixing module provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of the instance segmentation head provided in an embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the segmentation effect of an instance segmentation method in a low-contrast environment provided by an embodiment of the present invention. Detailed Implementation

[0017] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention. Example 1

[0018] An instance segmentation method for low-contrast environments includes: Image data acquired in a low-contrast environment is input into a pre-trained adaptive instance segmentation model to segment the target object in the image. The training methods for the adaptive instance segmentation model include: Establish an image instance segmentation dataset, specifically: First, real bank card image data under low contrast environment is collected by ordinary camera, and the bank card data is expanded by image enhancement method. The images are labeled by segmentation labeling tool and divided into training set, validation set and test set according to proportion to form a bank card image instance segmentation dataset. It should be noted that the image enhancement method in this embodiment includes: geometric transformation, color transformation, brightness transformation, simulated distortion, and noise addition. Among them, geometric transformation includes vertical flip, horizontal flip, random rotation from -10 to 10 degrees, and scaling ±15%; color transformation includes HSV (Hue Saturation Value) color model transformation; brightness transformation includes Gamma transformation; simulated distortion includes simulated camera distortion; and noise addition includes Gaussian noise.

[0019] like Figure 1 As shown, an adaptive instance segmentation model is constructed by combining a CNN network and a Mamba network. The adaptive instance segmentation model includes a backbone network and an instance segmentation network. The backbone network includes an adaptive state space module and a multi-scale adaptive edge enhancement module; The adaptive state-space module is used to perform long-range dependency modeling based on the downsampled input image to obtain state-space features; The multi-scale adaptive edge enhancement module is used to extract multi-scale edge features based on state space features, and obtain multi-scale edge enhancement features. The instance segmentation network is used to filter the final predicted bounding box of the target object based on multi-scale edge enhancement features, generate the final mask of the target object, and perform instance segmentation of the target object based on the final mask; In this embodiment, the backbone network includes a downsampling module, seven first adaptive state space modules, a first multi-scale adaptive edge enhancement module, a first cross-pixel mixing module, three second adaptive state space modules, a second multi-scale adaptive edge enhancement module, a second cross-pixel mixing module, three third adaptive state space modules, and a third multi-scale adaptive edge enhancement module connected in sequence; the instance segmentation network includes a first instance segmentation head, a second instance segmentation head, and a third instance segmentation head, wherein the first instance segmentation head is connected to the first multi-scale adaptive edge enhancement module, the second instance segmentation head is connected to the second multi-scale adaptive edge enhancement module, and the third instance segmentation head is connected to the third multi-scale adaptive edge enhancement module.

[0020] An adaptive instance segmentation model is trained using an image instance segmentation dataset. Specifically: like Figure 2 As shown, the training set from the instance segmentation dataset is input into the adaptive instance segmentation model. After processing by the downsampling module, the downsampled input image is obtained. The downsampling module includes one... The convolution and a depthwise separable convolution with residual connections, the depthwise separable convolution including: a first Convolution, layer-by-layer convolution, point-by-point convolution, and the second convolution; like Figure 3 As shown, the downsampled input image is processed... Convolution calculation yields the first feature; The sampling point positions are adjusted using deformable convolution on the first feature to obtain the second feature, which is represented as: ; in, Indicates the second characteristic, Represents deformable convolution. This represents the first feature of the input image; Represents the total number of aggregate groups, for The first in Aggregation group: This indicates the number of pixels in each aggregation group. Indicates the first element in the aggregate group Each sampled pixel point Indicates the first The value learned by each sampled pixel in relation to the location of the aggregate group; express Convolution is used to sample the position of pixels in the receptive field. , This indicates that the K×K convolution kernel is in the receptive field of the th... The position of each sampling point relative to the center point, for example: if the center point of a 3×3 convolution kernel is (1,1), then the distribution of the sampling points in the receptive field is represented as follows: ; Indicates the first The feature vector after adaptive deformable sampling in the aggregation group; This represents the input pixel at the current position in the feature map. Represents points on the feature map The position offset performed; Indicates an aggregation group Group 1 Weighting coefficients at each sampled pixel; The second feature is input into the Structured State Space Submodule (SSM2D). The zero-order preservation technique of parameter discretization is used to scan all pixels in the second feature and capture global context information, resulting in the third feature, which includes: The second feature is expanded into a sequence. The zero-order preservation technique of parameter discretization is used to analyze the sequence. Full-pixel scanning using structured convolution is represented as: ; ; ; ; in, Indicates the third characteristic, Represents a structured convolution kernel. , Represents a sequence Length; This represents a transition matrix that allows control over the changes in the hidden state over time. , Represents the set of real numbers. express Dimensions express Discretized representation; This represents the weight matrix of an input matrix relative to the hidden states. , express Dimensions express Discretized representation; This represents an observation matrix that maps hidden intermediate states to output. , express The dimension; Indicates the time scale parameter. Represents the identity matrix; Represented by real numbers e An exponential function with base 0; The third feature and the residual of the first feature are concatenated, and deformable convolution is used to obtain the fourth feature; The state-space features are obtained by concatenating the residuals of the fourth and third features.

[0021] like Figure 4 As shown, the state space features are applied using... The number of channels adjusted during convolution is expressed as: ; in, This represents the adjusted state-space characteristics. Represents the state space features of the input. express convolution; Features of the state space Deformable convolution is used to adaptively analyze the edge feature information of the target object, resulting in adaptive features, represented as: ; Represents adaptive features. This indicates the use of batch normalization and the Sigmoid function. convolution; right Use three times Pooling is used to extract multi-scale edge information and enhance the edge information, as shown below: ; ; in, The feature map representing the scale is . The proportion is Feature map, express Pooling, This indicates max pooling. Indicates average pooling. Let the given input feature map be... One of them, This represents the number of channels in the input feature map. Represents the first... aisle; Introducing edge enhancement methods To enhance multi-scale feature maps , and The detailed information on the surface is used to obtain the feature maps after edge enhancement. , and ,include: ; ; in, This represents the feature map after edge enhancement. Indicates the first Branching Convolution and Features of pooling output express , and One of them, according to , and Choose the branch where it is located; By channel cascading , , and By concatenating these features, we obtain multi-scale edge enhancement features that contain fine-grained feature information and multi-scale edge information, represented as: ; in, This represents multi-scale edge enhancement features. express convolution.

[0022] like Figure 5 As shown, the cross-pixel blending module is used to perform sparse dynamic receptive field downsampling and blending based on multi-scale edge enhancement features to obtain downsampled cross-pixel blending features, including: The input multi-scale edge enhancement features are flattened, and the elements of the flattened features are cross-blended to obtain the initial blended features; The initial mixture features are sequentially input into two... The convolution is performed in a deformable convolution with residual connection to obtain downsampled cross-pixel blending features; In this embodiment, the input of the first cross-pixel blending module is the multi-scale edge enhancement feature output by the first multi-scale adaptive edge enhancement module, and the output is the first cross-pixel blending feature. The first cross-pixel blending feature is used as the input of the second adaptive state space module. The input to the second cross-pixel blending module is the multi-scale edge enhancement feature output by the second multi-scale adaptive edge enhancement module, and the output is the second cross-pixel blending feature. The second cross-pixel blending feature is used as the input to the third adaptive state space module.

[0023] like Figure 6 As shown, during the training process of the adaptive instance segmentation model, each instance segmentation head has three branches: a classification branch, a bounding box branch, and a segmentation branch. Each branch includes two 3×3 convolutions and one 1×1 convolution. The classification branch is used to determine the category of the target object and generate the target category probability. The bounding box branch is used to provide a predicted bounding box and a corresponding confidence score for the region of the suspected target object. The segmentation branch is used to generate a basic mask based on the predicted bounding box. Based on the confidence scores of the predicted boxes given by the bounding box branch, the final target box is selected from all the predicted boxes using the non-maximum suppression method; Based on the target category probability and the final target bounding box, the base mask is adjusted and refined to obtain the final target mask; Based on the final target mask, the target object is segmented into instances; The three branches use bounding box regression loss, classification loss, and mask segmentation loss respectively, where the bounding box regression loss is the AInnerCIoU loss function and the DFL loss function; The AInnerCIoU loss function is used to adjust the position of the predicted bounding box to ensure that only the true target is contained within the box, as follows: ; ; ; ; ; ; in, Represents the AInnerCIoU loss function; Represents the true bounding box. This represents the width of the actual bounding box. Indicates the height of the actual bounding box. Represents the center point of the true bounding box ; Indicates the prediction box. Indicates the width of the prediction box. Indicates the height of the predicted bounding box. Indicates the center point of the prediction box ; Represents pi (π). The diagonal of the smallest bounding box between the ground truth box and the predicted box; 、 、 、 、 、 、 、 This represents the scaling factor that allows for adaptive learning during the network training phase, with each adaptive scaling factor initially set to 1.0; This represents the function that takes the minimum value. This represents the function that takes the maximum value. This represents a consistency parameter, used to measure the consistency of the aspect ratio between the ground truth bounding box and the predicted bounding box. This represents the weighting coefficients for the orthogonal shapes of the ground truth bounding box and the predicted bounding box. This represents the area of ​​the adaptive intersection of the ground truth bounding box and the predicted bounding box. This indicates that the left boundary of the help box adapts to the actual bounding box. This indicates the right boundary of the guide box that adapts to the actual bounding box. This indicates the upper boundary of the actual bounding box that adapts to the auxiliary bounding box. This represents the lower boundary of the actual bounding box that adapts to the auxiliary bounding box. This represents the left boundary of the adaptive auxiliary box for the predicted bounding box. This represents the right boundary of the adaptive auxiliary box of the predicted box. This represents the upper boundary of the adaptive auxiliary box of the predicted box. This indicates the lower boundary of the adaptive auxiliary box for the predicted bounding box; The DFL loss function optimizes the correlation with the true label using a cross-entropy approach. Two adjacent discrete positions and The predicted probability allows the network to focus its attention on the output distribution in the vicinity of the target, thus making the prediction results more accurate, as expressed as: ; Represents the DFL loss function; and Indicates that it contains real tags The boundary values ​​of two adjacent discrete intervals; This indicates that the coordinates predicted by the model fall within The probability value corresponding to the interval. This indicates that the coordinates predicted by the model fall within The probability value for the corresponding interval; Both classification loss and mask segmentation loss use the BCE loss function, expressed as: ; in, Represents the BCE loss function. Indicates the number of samples. Indicates the weight value. Indicates the tag value. Indicates the model's predicted value; The model is iteratively trained using bounding box regression loss, classification loss, and mask segmentation loss. During the iterative training process, the model training results are verified using the validation set in the dataset after a certain number of training iterations. The model parameters are set as follows: anchor box width ratio 0.21, anchor box height ratio 0.15, no pre-trained weights, Lion optimizer, initial learning rate 0.015, dual momentum coefficients (0.9, 0.99), mask downsampling ratio 4, number of images per batch 4, and training cycle 50 epochs. Finally, the training result with the minimum verification loss after training and verification is used as the prediction weight of the adaptive instance segmentation model.

[0024] The trained adaptive instance segmentation model is deployed to an embedded edge intelligent device. Instance segmentation of target objects is achieved by collecting a validation set using an external camera, obtaining the target object mask, category, bounding box, and confidence score. The bank card instance segmentation effect in low-contrast environments is shown below. Figure 7 As shown, a balance between detection speed and accuracy has been achieved.

[0025] Example 2 Based on the same inventive concept, this embodiment 2 provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements an instance segmentation method in a low-contrast environment as described in the above embodiment.

[0026] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0027] This application is described with reference to flowchart illustrations of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each step in the flowchart can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 One or more processes or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0028] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 The function specified in one or more processes.

[0029] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 Steps of a specified function in one or more processes.

[0030] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for instance segmentation in low-contrast environments, characterized in that, include: Image data acquired in a low-contrast environment is input into a pre-trained adaptive instance segmentation model to segment the target object in the image. The adaptive instance segmentation model includes a backbone network and an instance segmentation network; The backbone network includes an adaptive state space module and a multi-scale adaptive edge enhancement module; The adaptive state space module is used to perform long-range dependency modeling based on the downsampled input image to obtain state space features; The multi-scale adaptive edge enhancement module is used to extract multi-scale edge features based on state space features to obtain multi-scale edge enhancement features. The instance segmentation network is used to filter the final predicted bounding box of the target object based on multi-scale edge enhancement features, generate the final mask of the target object, and perform instance segmentation of the target object based on the final mask.

2. The instance segmentation method in a low-contrast environment according to claim 1, characterized in that, The adaptive state space module is used to perform long-range dependency modeling based on the downsampled input image to obtain state space features, including: Perform on the downsampled input image Convolution calculation yields the first feature; The sampling point positions are adjusted using deformable convolution on the first feature to obtain the second feature, which is represented as: ； in, Indicates the second characteristic, Represents deformable convolution. This represents the first feature of the input image; Represents the total number of aggregate groups, for The first in Aggregation group: This indicates the number of pixels in each aggregation group. Indicates the first element in the aggregate group Each sampled pixel point Indicates the first The value learned by each sampled pixel in relation to the location of the aggregate group; express Convolution is used to sample the position of pixels in the receptive field. , This indicates that the K×K convolution kernel is in the receptive field of the th... The position of each sampling point relative to the center point; Indicates the first The feature vector after adaptive deformable sampling in the aggregation group; This represents the input pixel at the current position in the feature map. Represents points on the feature map The position offset performed; Indicates an aggregation group Group 1 Weighting coefficients at each sampled pixel; The second feature is input into the structured state space submodule, and the zero-order preservation technique of parameter discretization is used to scan all pixels in the second feature and capture global context information to obtain the third feature; The third feature and the residual of the first feature are concatenated, and deformable convolution is used to obtain the fourth feature; The state-space features are obtained by concatenating the residuals of the fourth and third features.

3. The instance segmentation method in a low-contrast environment according to claim 2, characterized in that, The second feature is input into the structured state space submodule, and the zero-order preservation technique of parameter discretization is used to scan all pixels in the second feature and capture global context information to obtain the third feature, including: The second feature is expanded into a sequence. The zero-order preservation technique of parameter discretization is used to analyze the sequence. Full-pixel scanning using structured convolution is represented as: ； ； ； ； in, Indicates the third characteristic, Represents a structured convolution kernel. , Represents a sequence Length; This represents a transition matrix that allows control over the changes in the hidden state over time. , Represents the set of real numbers. express Dimensions express Discretized representation; This represents the weight matrix of an input matrix relative to the hidden states. , express Dimensions express Discretized representation; This represents an observation matrix that maps hidden intermediate states to output. , express The dimension; Indicates the time scale parameter. Represents the identity matrix; Represented by real numbers e An exponential function with base 0.

4. The instance segmentation method in a low-contrast environment according to claim 2, characterized in that, The multi-scale adaptive edge enhancement module is used to extract multi-scale edge features based on state space features, resulting in multi-scale edge enhancement features, including: The state space features are adopted The number of channels adjusted during convolution is expressed as: ； in, This represents the adjusted state-space characteristics. Represents the state space features of the input. express convolution; Features of the state space Deformable convolution is used to adaptively analyze the edge feature information of the target object, resulting in adaptive features, represented as: ； Represents adaptive features. This indicates the use of batch normalization and the Sigmoid function. convolution; right Use three times Pooling is used to extract multi-scale edge information and enhance the edge information, as shown below: ； ； in, The proportion is Feature map, The proportion is Feature map, express Pooling, This indicates max pooling. Indicates average pooling. Let the given input feature map be... One of them, This represents the number of channels in the input feature map. Represents the first... aisle; Introducing edge enhancement methods To enhance multi-scale feature maps , and The detailed information on the surface is used to obtain the feature maps after edge enhancement. , and ; By channel cascading , , and By concatenating these features, we obtain multi-scale edge enhancement features that contain fine-grained feature information and multi-scale edge information, represented as: ； in, This represents multi-scale edge enhancement features. express convolution.

5. The instance segmentation method in a low-contrast environment according to claim 4, characterized in that, The introduced edge enhancement method To enhance multi-scale feature maps , and The detailed information on the surface is used to obtain the feature maps after edge enhancement. , and ,include: ； ； in, This represents the feature map after edge enhancement. Indicates the first Branching Convolution and Features of pooling output express , and One of them, according to , and Choose the branch where it is located.

6. The instance segmentation method in a low-contrast environment according to claim 1, characterized in that, The backbone network also includes a downsampling module, which is used to downsample the instance segmentation dataset input to the pre-trained lightweight adaptive instance segmentation model to obtain the downsampled input image.

7. The instance segmentation method in a low-contrast environment according to claim 6, characterized in that, The backbone network also includes a cross-pixel mixing module, which is used to perform sparse dynamic receptive field downsampling and mixing based on multi-scale edge enhancement features to obtain downsampled cross-pixel mixing features.

8. The instance segmentation method in a low-contrast environment according to claim 1, characterized in that, The instance segmentation network is used to filter the final predicted bounding boxes of target objects based on multi-scale edge enhancement features, generate the final mask of the target objects, and perform instance segmentation of the target objects based on the final mask, including: An instance segmentation head is set up, consisting of a classification branch, a bounding box branch, and a segmentation branch. Each branch includes two 3×3 convolutions and one 1×1 convolution. The classification branch is used to determine the category of the target object and generate the target category probability. The bounding box branch is used to provide a predicted bounding box and a corresponding confidence score for the region of the suspected target object. The segmentation branch is used to generate a basic mask based on the predicted bounding box. Based on the confidence scores of the predicted boxes given by the bounding box branch, the final target box is selected from all the predicted boxes using the non-maximum suppression method; Based on the target category probability and the final target bounding box, the base mask is adjusted and refined to obtain the final target mask; Based on the final target mask, the target object is segmented into instances.

9. The instance segmentation method in a low-contrast environment according to claim 8, characterized in that, During the training of the adaptive instance segmentation model, the bounding box branch adopts the AInnerCIoU loss function and the DFL loss function; The AInnerCIoU loss function is used to adjust the position of the predicted bounding box to ensure that only the true target is contained within the box, as shown below: ； ； ； ； ； ； in, Represents the AInnerCIoU loss function; Represents the true bounding box. This represents the width of the actual bounding box. Indicates the height of the actual bounding box. Represents the center point of the true bounding box ; Indicates the prediction box. Indicates the width of the prediction box. Indicates the height of the predicted bounding box. Indicates the center point of the prediction box ; Represents pi (π). The diagonal of the smallest bounding box between the ground truth box and the predicted box; 、 、 、 、 、 、 、 This represents the scaling factor that allows for adaptive learning during the network training phase, with each adaptive scaling factor initially set to 1.0; This represents the function that takes the minimum value. This represents the function that takes the maximum value. Represents the consistency parameter. This represents the weighting coefficients for the orthogonal shapes of the ground truth bounding box and the predicted bounding box. This represents the area of ​​the adaptive intersection of the ground truth bounding box and the predicted bounding box. This indicates that the left boundary of the help box adapts to the actual bounding box. This indicates the right boundary of the guide box that adapts to the actual bounding box. This indicates the upper boundary of the actual bounding box that adapts to the auxiliary bounding box. This represents the lower boundary of the actual bounding box that adapts to the auxiliary bounding box. This represents the left boundary of the adaptive auxiliary box for the predicted bounding box. This represents the right boundary of the adaptive auxiliary box of the predicted box. This represents the upper boundary of the adaptive auxiliary box of the predicted box. This indicates the lower boundary of the adaptive auxiliary box for the predicted bounding box; The DFL loss function is expressed as: ； Represents the DFL loss function; and Indicates that it contains real tags The boundary values ​​of two adjacent discrete intervals; This indicates that the coordinates predicted by the model fall within The probability value corresponding to the interval. This indicates that the coordinates predicted by the model fall within The probability value for the corresponding interval.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the instance segmentation method in a low-contrast environment as described in any one of claims 1 to 9.

Images