16results about How to "Improve Semantic Segmentation" patented technology

The invention discloses a multi-source remote sensing image semantic segmentation method based on Transform, Mama and a diffusion model, belongs to the technical field of electric digital data processing, graphic data reading and image data processing, and is used for extracting image information of a remote sensing image. Comprising the following steps: acquiring remote sensing images, constructing an image information extraction network based on the remote sensing images, and respectively inputting the remote sensing images in a training set, a verification set and a test set into the constructed remote sensing image information extraction network based on Transform, Mama and diffusion models. According to the method, the RGB features and the depth features are subjected to Transform feature extraction, and a cross Mama method is provided to perform cross enhancement on the RGB features and the depth features; according to the method, a row and column pooling Transform module is used for replacing a traditional Transform module, and detail texture information and large-scale structure information of a feature map in the vertical direction and the horizontal direction are extracted at the same time; according to the invention, a cross Mama attention feature fusion module is used to replace a traditional Mama module, and the modeling capability for complex scenes and diversified targets is enhanced.

The invention discloses a remote sensing image semantic segmentation-oriented spatial spectrum context feature enhancement method and device. The method comprises the following steps of: constructing a neural network model based on an encoder-decoder architecture; inputting the preprocessed remote sensing image set into a feature coding block, and extracting hierarchical features; according to the model, spectrum attention and position self-attention mechanisms are integrated through a collaborative attention module, and the sensitivity of network spectrum and spatial context information is remarkably enhanced; the network coding module efficiently compresses and codes the features before entering a decoding stage; a cross fusion module is applied to intelligently fuse features between a coding block and a decoding block to ensure that information is not lost in a decoding process; the feature map is mapped into a pixel-level category label through a Softmax layer, precise semantic segmentation is achieved, and a high-quality classification result is obtained. According to the method, through collaborative modeling of the spectrum and the spatial context, high-frequency details are effectively reserved, and the precision of semantic segmentation is improved.

The invention discloses a complex model clinical image semantic segmentation method based on frequency phase guidance. The method comprises the following steps: preprocessing an original clinical image; the data are input into a Gabor wavelet transformation module to be converted into a complex value domain image; inputting the complex-value-domain image into a coding network, and selectively adjusting perception of a phase component and preference for different frequency components through a complex-value mixed attention module to obtain a complex-value-domain feature; inputting the complex-value-domain features into a frequency adaptive separation module, decomposing the complex-value-domain features into high-frequency features and low-frequency features, combining the low-frequency features and the high-frequency features, and screening out important features through a frequency channel attention module; and inputting the important features into a next-level encoder, and combining the high-frequency features with the features of each level of the decoder to obtain a segmentation result of the clinical image. According to the method, the phase information and the frequency information are effectively utilized, the target area and the similar background in the clinical image can be better distinguished, and the segmentation effect is improved.

The invention relates to a 3D semantic scene completion method. The method comprises the following steps: step 1, acquiring an RGB image of a 3D scene by using a binocular camera; 2, constructing a voxel occupancy prediction network, and inputting the collected image into the voxel occupancy prediction network to obtain a voxel grid; 3, constructing an attention enhancement feature fusion module, and performing fine adjustment on the voxel grid by using the attention enhancement feature fusion module to obtain a three-dimensional voxel query set; 4, extracting two-dimensional features from the collected images, associating the three-dimensional voxel query set with the two-dimensional features through a variable cross attention mechanism, and updating voxel data; 5, introducing mask marks to supplement voxel data, and extracting voxel features from all voxel data by using a self-attention mechanism; and 6, inputting the voxel features into an AUHead semantic segmentation head for semantic segmentation, and projecting a result to an output space to obtain a 3D scene graph. According to the invention, the space occupation integrity and semantic segmentation accuracy of 3D semantic scene completion are improved.

The invention relates to the technical field of computer machine vision, and discloses a plant growth condition monitoring method, system and device and a storage medium, and the method comprises the steps: obtaining a target plant image, inputting the preprocessed target plant image into an improved YOLOv9 model (comprising a backbone network, a feature fusion network and a prediction classification network), and obtaining a target plant image; obtaining a target plant detection image; performing feature extraction on the preprocessed target plant image through a backbone network to obtain plant multi-scale features; performing feature conversion, branch processing and feature fusion on the plant multi-scale features through a feature fusion network to obtain plant fusion features; performing classification processing on the plant fusion features through a prediction classification network to obtain a target plant detection image; and inputting the target plant detection image into an image semantic segmentation model (including an image super-resolution network and an image segmentation network) to obtain a plant binary image. The method effectively improves the accuracy of plant growth condition monitoring.

The invention provides a semantic segmentation method and device for an SAR image flood area, a storage medium and a terminal, and the method comprises the steps: carrying out the semantic extraction of an SAR image through a preset encoder, so as to obtain a multi-stage semantic feature map; performing preset processing on each feature map in the shallow feature map area, and then performing noise filtering to obtain a noise-filtered feature map of a corresponding stage; performing multi-scale convolution processing on the last stage feature map to obtain a processed deep feature map; pre-processing the feature map in the middle feature map area to obtain a corresponding pre-processed middle feature map, and splicing the pre-processed middle feature map, the feature map after noise filtering and the processed deep feature map to obtain an optimized feature map; and inputting the optimized feature map into a preset classifier to obtain a semantic segmentation result of the SAR image. According to the method, the semantic segmentation effect of the SAR image flood area can be effectively improved.

The invention relates to the field of neural networks, and discloses a method and device for determining a target semantic segmentation network model and a processor. The method comprises the steps that semantic segmentation results are input into a plurality of discriminators to obtain a plurality of classification results, the classification results belong to any one of a 2D source domain, a 3D source domain, a 2D target domain and a 3D target domain, and the discriminators comprise at least one of a first discriminator, a second discriminator, a third discriminator and a fourth discriminator; the first discriminator is used for judging whether the semantic segmentation result belongs to a 2D source domain or a 2D target domain, the second discriminator is used for judging whether the semantic segmentation result belongs to a 3D source domain or a 3D target domain, the third discriminator is used for judging whether the semantic segmentation result belongs to the 2D source domain or the 3D target domain, and the fourth discriminator is used for judging whether the semantic segmentation result belongs to the 3D source domain or the 2D target domain; obtaining a plurality of semantic segmentation network models to be tested after adversarial training; and determining a target semantic segmentation network model according to the test segmentation result. And the semantic segmentation accuracy of the target domain is improved.

This invention relates to the field of image processing technology and discloses a semi-supervised semantic segmentation method, apparatus, device, and medium for remote sensing images. The method includes: acquiring several remote sensing images and a panchromatic image corresponding to each remote sensing image; employing various panchromatic sharpening techniques to fuse the spectral information in each remote sensing image with the high spatial resolution information in the corresponding panchromatic image to obtain a high-resolution multispectral image; fusing the RGB band information in each high-resolution multispectral image with the remaining band information to obtain a band-fused image; during semantic segmentation model training, the supervised learning part uses weak perturbation techniques to perturb the labeled image, while the unsupervised learning part uses the high-resolution multispectral image and the band-fused image as multimodal fusion perturbations, combining weak perturbation techniques, strong perturbation techniques, and multimodal fusion perturbations to perturb the unlabeled image, so as to perform semantic segmentation through the trained semantic segmentation model.

The present invention is applicable to the fields of deep learning and autonomous driving technology, and provides a semantic segmentation method, system, electronic device and medium for autonomous driving, the method comprising: acquiring three-dimensional point cloud data, mapping it into a two-dimensional depth map including multiple channel data, and forming a sample data set according to the multiple channel data; constructing an initial model of a semantic segmentation network including a first model and a second model, using the sample data set to train the initial model of the semantic segmentation network to obtain a target model, wherein the second model comprises an encoder and a decoder; acquiring target three-dimensional point cloud data, mapping it into a target two-dimensional depth map, and inputting the target two-dimensional depth map into the target model to obtain a target semantic segmentation result; the method is used to solve the problem of low semantic segmentation performance of three-dimensional point clouds.

The application discloses a point supervision semantic segmentation model construction and segmentation method based on SAM and CNN feature fusion, comprising the following steps: step one, acquiring remote sensing image data as a training set, and performing point labeling on target ground objects in the remote sensing image data to obtain a point label set, wherein the point label set is the category and position of the target ground objects; a point supervision semantic segmentation model is constructed; object structure information generated by using a convolutional neural network branch and a SAM driven fusion branch is used to realize collaborative modeling of image semantic information and structure information in combination with a cross-level fusion module CFM; meanwhile, under the condition of point supervision, the model can enhance the integrity control of the target region, improve the boundary positioning accuracy, and reduce the category confusion phenomenon, thereby significantly improving the effect of remote sensing image semantic segmentation. The technical problem that the existing point supervision semantic segmentation method is difficult to fully utilize image structure information and semantic information under the condition of sparse supervision information is solved.

The embodiment of the invention discloses an image semantic segmentation method and device and a computer storage medium, and belongs to the technical field of computer vision. The method comprises the following steps: determining a plurality of features of an image; determining initial category embedding based on the first feature; determining learned category embedding based on the at least one second feature and the initial category embedding; and determining a semantic segmentation result of the image based on the learned category embedding and the plurality of features. In the embodiment of the invention, category embedding is introduced in the semantic segmentation process, so that the semantic segmentation effect of the image is improved through interaction between the category embedding and each feature in the image. Moreover, the initial category embedding is determined based on one feature of the image instead of random initialization, so that the initial category embedding is fused with the spatial prior information of the image, and compared with the mode that the initial category embedding is randomly initialized to be 0, the semantic segmentation effect of the image can be further improved.

The present invention discloses a semi-supervised training method, device and medium of a video semantic segmentation network, the method comprising: obtaining a first data set including a plurality of first video frames carrying labels and a second data set including a plurality of second video frames not carrying labels; obtaining a video semantic segmentation network, which comprises a first segmentation network and a second segmentation network; sampling a first video frame from the first data set and then inputting it into the first segmentation network for processing, and determining a first classification loss in combination with the carried labels; sampling a plurality of second video frames from the second data set and then inputting them into the first segmentation network for processing to obtain a plurality of pseudo labels; performing data enhancement on the plurality of second video frames and then inputting them into the second segmentation network for processing, and using a plurality of pseudo labels for supervision to determine a second classification loss; and training the video semantic segmentation network according to the first classification loss and the second classification loss. The present invention can make full use of the time information in the video to improve the semantic segmentation performance.

The invention provides a method and a system for improving semantic segmentation performance of a multi-source remote sensing image based on complementary prompt. According to the multi-modal remote sensing image semantic segmentation method, the high-level semantic features of the remote sensing images are extracted by using the basic model, and the middle features are mutually used as visual prompts during feature extraction, so that the complementary synergistic effect between the multi-modal remote sensing images is fully utilized, the multi-modal feature representation is perfected, and the multi-modal remote sensing image semantic segmentation performance is improved. According to the method, the cooperation and prompt effect between the multi-modal remote sensing images can be fully utilized, and the problem that the existing method excessively pursues common feature expression between modals is better solved. According to the method provided by the invention, middle features can be mutually utilized as visual prompts during extraction of other modalities in the modal feature extraction process, so that the model is more focused on the missing feature part of the current modal, more comprehensive multi-modal feature representation is learned, and the method is applied to subsequent downstream tasks.

Embodiments of the invention provide a semantic segmentation model training method and apparatus, a device and a storage medium. The method comprises the steps of obtaining training data; the training data comprises a night image set and a daytime image set, the night image set comprises night image information and night event information, and the daytime image set comprises daytime image information with corresponding semantic categories; extracting daytime event information corresponding to the daytime image information, daytime content information corresponding to the daytime image information and night content information corresponding to the night image information; and training a to-be-trained semantic segmentation model by using the target training data to obtain a target semantic segmentation model, the target semantic segmentation model being used for performing semantic segmentation on a to-be-segmented image shot at night. According to the embodiment of the invention, the target semantic segmentation model obtained through training can better understand the association between the image information and the event information, so that the performance of semantic segmentation of the image shot at night is remarkably improved.

The application belongs to the field of computer vision, and discloses a weakly supervised semantic segmentation method, system, device and medium based on random combination, which comprises the following steps: training classification networks N1, N2 and N3 respectively by using a training data set, a slice training data set and a slice training data set randomly combined, so that each network can extract different active regions in the picture, and the learning results of the other two networks are learned by using the mutual supervision training mode; finally, the prediction results of the three networks are combined to obtain the final semantic segmentation result, which is used as a semantic segmentation training data set to train a semantic segmentation model to predict the final semantic segmentation result. The application effectively utilizes the different perception areas of the network for the randomly combined slice pictures, and utilizes the different classification networks to perceive the categories of the same picture, thereby improving the semantic segmentation ability and prediction accuracy of the semantic segmentation model through the semantic segmentation data set obtained by combining the results of the three classification networks.

The invention discloses an image semantic segmentation method for separating attention based on a space channel. The method comprises the following steps: preprocessing an input image and extracting multi-scale features by an encoder; constructing a visual potential source item and solving a Poisson equation to obtain a potential field and a potential gradient; generating a space weight graph by utilizing the potential field, and forming a boundary enhancement weight graph by combining the potential gradient to realize space gating; channel characteristics are regarded as parallel capacitor layers to generate channel coupling weights for channel gating; fusing space and channel enhancement features; and a semantic segmentation result is output through step-by-step decoding. According to the method, by introducing a visual potential propagation and flux conservation mechanism, energy balance and accurate boundary control are realized in space and channel modeling, and the structural integrity and stability of a semantic segmentation result are improved.