Training method of image segmentation model, and road ponding identification method and device

Abstract

The invention relates to the technical field of artificial intelligence, and discloses an image segmentation model training method and a road ponding recognition method and device, and the method comprises the steps: obtaining a plurality of sample images, and constructing a dictionary database; performing feature transformation on the dictionary database by adopting an image segmentation model to obtain a dictionary feature vector set, and extracting segmentation features from the dictionary feature vector set to obtain a sample feature matrix; performing similarity calculation based on the dictionary feature vector set and the sample feature matrix to obtain an initial segmentation mask; based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask, performing multiple times of completion segmentation to obtain a segmentation mask of each time; and based on the initial segmentation mask and the segmentation mask of each time, determining training loss, and performing training. According to the method, the training cost is reduced by using a small number of sample images, the over-fitting problem of the model obtained through final training is avoided through multiple times of complementation segmentation, the segmentation precision and stability are improved, and meanwhile, the method is suitable for multiple application scenes.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a training method for an image segmentation model, and a method and device for identifying road waterlogging. Background Art

[0002] Image segmentation technology plays an important role in many fields, especially in medical image analysis, autonomous driving, remote sensing image analysis, industrial inspection, video surveillance, virtual reality and augmented reality, agriculture, security, smart cities and entertainment. For example, in rainy seasons, some low-lying urban roads, tunnels, underground passages and low-lying shopping malls are prone to waterlogging. Road waterlogging can easily have a great impact on urban traffic safety and urban infrastructure construction, so it is necessary to train an image segmentation model to identify waterlogged areas so that they can be handled in a timely manner.

[0003] Currently, commonly used image segmentation models use strongly supervised semantic segmentation technology and require a large number of labeled training samples to participate in training, resulting in high training costs and easy overfitting on small sample data sets, resulting in unstable model performance and low segmentation accuracy. Summary of the invention

[0004] In view of this, the present invention provides a training method for an image segmentation model, a method and device for identifying road waterlogging, so as to solve the problems of high training cost, unstable model performance and low image segmentation accuracy.

[0005] In a first aspect, the present invention provides a method for training an image segmentation model, the method comprising:

[0006] Acquire multiple sample images, and build a dictionary database based on the multiple sample images, where the number of the sample images is less than an image number threshold;

[0007] The image segmentation model is used to perform feature transformation on the dictionary database to obtain a dictionary feature vector set, and segmentation features are extracted from the dictionary feature vector set to obtain a sample feature matrix;

[0008] Using the image segmentation model, similarity calculation is performed based on the dictionary feature vector set and the sample feature matrix to obtain the initial segmentation mask;

[0009] Using the image segmentation model, based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask, a preset number of completion segmentations are performed to obtain a segmentation mask obtained for each completion segmentation;

[0010] Based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, a training loss is determined, and the image segmentation model is trained based on the training loss.

[0011] The training method of the image segmentation model provided by the embodiment of the present invention uses a small amount of sample images to build a dictionary database, thereby reducing the need for a large amount of labeled data and effectively reducing the training cost. The image segmentation model is used to perform feature transformation on the dictionary database to obtain a dictionary feature vector set, and then the segmentation features are extracted to obtain a sample feature matrix, effectively capturing the important features in the sample image. Then, an initial segmentation mask is generated by similarity calculation, and then the segmentation results are gradually optimized through multiple completion segmentations to improve the robustness and generalization ability of the model. Each completion segmentation can further refine and improve the initial segmentation mask, and can more accurately identify and segment the target area. Finally, the training loss is determined by comprehensively considering the initial segmentation mask and the results of multiple completion segmentations, thereby optimizing the model, ensuring that the model can be continuously improved during the training process, so that the image segmentation model finally trained can avoid the overfitting problem, improve the segmentation accuracy and the stability of the model, and can effectively process complex and changeable image content, and is suitable for a variety of application scenarios.

[0012] In an optional implementation, an image segmentation model is used to perform feature transformation on a dictionary database to obtain a dictionary feature vector set, and segmentation features are extracted from the dictionary feature vector set to obtain a sample feature matrix, including:

[0013] The feature extraction network in the image segmentation model is used to perform feature transformation on the dictionary database to obtain a set of sample feature maps;

[0014] Perform average pooling on the sample feature map set according to the channel dimension to obtain a dictionary feature vector set;

[0015] The skeleton network in the image segmentation model is used to extract segmentation features from the dictionary feature vector set to obtain the sample feature matrix.

[0016] The training method of the image segmentation model provided by the embodiment of the present invention performs feature transformation on the sample image through the feature extraction network of the image segmentation model to generate a sample feature map set, which can effectively capture high-level abstract features in the image, and then performs an average pooling operation on the sample feature map set to compress the multi-channel feature map into a one-dimensional feature vector, thereby reducing the dimension of the feature and retaining the main feature information, simplifying the subsequent processing process, and improving the calculation efficiency. Then, the skeleton network is used to extract features related to the segmentation task from the dictionary feature vector set to generate a sample feature matrix so that it can better reflect the target area in the image and improve the accuracy of segmentation.

[0017] In an optional implementation, an image segmentation model is used to perform a preset number of complementary segmentations based on a dictionary feature vector set, a sample feature matrix, and an initial segmentation mask to obtain a segmentation mask obtained for each complementary segmentation, including:

[0018] Based on the initial segmentation mask and the sample feature matrix, aggregate the target feature prototype and the background feature prototype;

[0019] Based on the dictionary feature vector set, the target feature prototype and the background feature prototype are spliced ​​and fused to obtain the target feature vector;

[0020] The similarity is calculated based on the target feature vector and the sample feature matrix to obtain the segmentation mask obtained by the first completion segmentation;

[0021] The segmentation mask obtained from the first completion segmentation is used as the initial segmentation mask for the next completion segmentation, and the process returns to the step of aggregating the target feature prototype and the background feature prototype based on the initial segmentation mask until the preset number of completion segmentations are completed to obtain the segmentation mask obtained for each completion segmentation.

[0022] The training method of the image segmentation model provided by the embodiment of the present invention helps the model to better understand and distinguish the target area and the background area by splicing and fusing the dictionary feature vector set, the target feature prototype and the background feature prototype, thereby improving the accuracy of segmentation. By gradually optimizing the segmentation results, a more accurate segmentation mask can be generated, thereby improving the accuracy of segmentation.

[0023] In an optional implementation, aggregating target feature prototypes based on the initial segmentation mask and the sample feature matrix includes:

[0024] Determining a target mask region greater than a first preset threshold from the initial segmentation mask;

[0025] Extracting a first feature vector set corresponding to a target mask region from a sample feature matrix;

[0026] Average pooling is performed on the first feature vector set to obtain the target feature prototype.

[0027] The training method of the image segmentation model provided by the embodiment of the present invention can effectively extract and aggregate the feature prototypes of the target mask area by only selecting high-confidence areas for feature extraction, generate a high-quality target feature vector representing the target area, effectively reduce the interference of noise and background areas, and compress multiple feature vectors into a single feature vector through average pooling operations, thereby simplifying the feature representation, reducing the consumption of computing resources, and making the feature vector easier to be processed and utilized by subsequent models.

[0028] In an optional implementation, aggregating background feature prototypes based on the initial segmentation mask and the sample feature matrix includes:

[0029] Determine a background mask area greater than a first preset threshold value from a difference between the initial segmentation mask and the set value;

[0030] Extracting a second feature vector set corresponding to the background mask area from the sample feature matrix;

[0031] For any feature vector in the second feature vector set, a deformable attention mechanism is used to obtain the relationship between each dimension of the feature vector to obtain a new feature vector;

[0032] The new feature vectors corresponding to the second feature vector set are average pooled to obtain the background feature prototype.

[0033] The training method of the image segmentation model provided by the embodiment of the present invention can better reflect the characteristics of the background area by only selecting the background area with high confidence for feature extraction, generate high-quality background feature vectors representing non-waterlogged areas, effectively reduce the interference of noise and target areas, and improve the purity and reliability of features. By using a deformable attention mechanism, the weight of the feature vector can be dynamically adjusted to capture the complex relationship between the various dimensions in the feature vector. This enhances the flexibility of feature representation, allowing the model to better adapt to different background features. Through the average pooling operation, multiple feature vectors are compressed into a single feature vector, which simplifies the feature representation, reduces the consumption of computing resources, and makes the feature vector easier to be processed and used by subsequent models.

[0034] In an optional implementation, any sample image includes a labeled mask, and based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, determining the training loss includes:

[0035] For any segmentation mask in the initial segmentation mask and the segmentation mask obtained by each completed segmentation, based on the segmentation mask and the annotation mask corresponding to the segmentation mask, determine the cross entropy loss and the similarity metric loss of the segmentation mask;

[0036] The sum of the cross entropy loss and the similarity metric loss for each segmentation mask is determined as the training loss.

[0037] The training method of the image segmentation model provided by the embodiment of the present invention can more comprehensively evaluate the accuracy of the segmentation results of the image segmentation model by determining the cross entropy loss and the similarity metric loss based on the segmentation mask obtained by the image segmentation model and the actual annotated mask, and then train the model based on the two losses, so that the trained model can better handle various complex images and scenes, improve its robustness and generalization ability, and reduce the risk of overfitting, thereby improving the accuracy and robustness of the segmentation and being suitable for a variety of image segmentation tasks.

[0038] In an optional implementation, before acquiring the plurality of sample images, the method further includes:

[0039] Obtain multiple initial images;

[0040] Preprocessing the multiple initial images to obtain multiple preprocessed images;

[0041] The structural similarity extraction method is used to extract the structural information of multiple pre-processed images respectively, and the color channel decomposition technology is used to extract the color components and brightness components of multiple pre-processed images respectively;

[0042] The mean clustering technique is used to cluster multiple pre-processed images after fusing structural information, color components and brightness components to obtain multiple clustering results.

[0043] The center point image of each clustering result in the multiple clustering results is used as multiple sample images.

[0044] The training method of the image segmentation model provided by the embodiment of the present invention can effectively reduce random noise in the image and improve the quality of the image through preprocessing with the median filtering technology, thereby providing a better basis for subsequent processing. The structural information in the image, such as edges, textures, etc., can be retained through the structural similarity extraction method, which is helpful for subsequent feature extraction and image segmentation. The color channel decomposition technology is then used to extract the color component and the brightness component, which can comprehensively capture the multi-dimensional features of the image and improve the richness and accuracy of the feature representation. Finally, the mean clustering technology is used to classify similar image features into one category to obtain multiple clustering results. The center point image of each clustering result is selected as the sample image, which is helpful to screen out the most representative sample image from a large number of initial images and improve the diversity and representativeness of the samples.

[0045] In a second aspect, the present invention provides a method for identifying road waterlogging, the method comprising:

[0046] Acquire road water accumulation images;

[0047] Preprocessing the road waterlogging image to obtain a preprocessed road waterlogging image;

[0048] An image segmentation model is used to perform image segmentation based on a preprocessed road waterlogging image, and a segmentation result greater than a second preset threshold is used as a target segmentation mask. The image segmentation model is trained based on the training method of the image segmentation model of the first aspect or any corresponding embodiment thereof.

[0049] The road waterlogging identification method provided by the embodiment of the present invention applies the trained image segmentation model to the field of smart cities, uses the image segmentation model to perform image segmentation on the preprocessed road waterlogging image, and uses the segmentation results greater than the second preset threshold in the segmentation results as the target segmentation mask. It can accurately segment the waterlogging area and non-waterlogging area in the image, which is helpful for the construction of smart cities.

[0050] In a third aspect, the present invention provides a training device for an image segmentation model, the device comprising:

[0051] A first acquisition module is used to acquire a plurality of sample images and construct a dictionary database based on the plurality of sample images, wherein the number of the sample images is less than an image number threshold;

[0052] The first extraction module is used to perform feature transformation on the dictionary database using an image segmentation model to obtain a dictionary feature vector set, and extract segmentation features from the dictionary feature vector set to obtain a sample feature matrix;

[0053] A first determination module is used to use an image segmentation model to perform similarity calculation based on a dictionary feature vector set and a sample feature matrix to obtain an initial segmentation mask;

[0054] The segmentation module is used to use the image segmentation model to perform a preset number of complementary segmentations based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask to obtain a segmentation mask obtained for each complementary segmentation;

[0055] The training module is used to determine the training loss based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, and train the image segmentation model based on the training loss.

[0056] In a fourth aspect, the present invention provides a device for identifying water accumulation on a road, the device comprising:

[0057] The second acquisition module is used to acquire the road water accumulation image;

[0058] A preprocessing module, used for preprocessing the road waterlogging image to obtain a preprocessed road waterlogging image;

[0059] The second determination module is used to use an image segmentation model to perform image segmentation based on a preprocessed road waterlogging image, and use the segmentation result greater than a second preset threshold as a target segmentation mask. The image segmentation model is trained based on the training method of the image segmentation model of the first aspect or any corresponding embodiment thereof.

[0060] In a fifth aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the training method of the image segmentation model of the above-mentioned first aspect or any corresponding embodiment thereof, or executes the road water identification method of the above-mentioned second aspect by executing the computer instructions.

[0061] In a sixth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the image segmentation model training method of the first aspect or any corresponding embodiment thereof, or to execute the road water identification method of the second aspect.

[0062] In a seventh aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the image segmentation model training method of the first aspect or any corresponding embodiment thereof, or to execute the road water identification method of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 is a flowchart of a method for training an image segmentation model according to an embodiment of the present invention;

[0065] Figure 2 is a flowchart of constructing a dictionary database according to an embodiment of the present invention;

[0066] Figure 3 is a flowchart of a training method for another image segmentation model according to an embodiment of the present invention;

[0067] Figure 4 is a flow chart of a method for identifying road waterlogging according to an embodiment of the present invention;

[0068] Figure 5 is a structural block diagram of a training device for an image segmentation model according to an embodiment of the present invention;

[0069] Figure 6 is a structural block diagram of a device for identifying road water accumulation according to an embodiment of the present invention;

[0070] Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0072] Image segmentation technology plays an important role in many fields. For example, in rainy seasons, some low-lying urban roads, tunnels, underground passages and low-lying shopping malls are prone to waterlogging. Road waterlogging can easily have a great impact on urban traffic safety and urban infrastructure construction, so it is necessary to train an image segmentation model to identify the waterlogged area so as to deal with it in time. At present, the commonly used image segmentation model adopts a strongly supervised semantic segmentation technology, and requires a large number of labeled training samples to participate in the training, resulting in high training costs, and easy overfitting on small sample data sets, resulting in unstable model performance and low segmentation accuracy. The training method of the image segmentation model provided by the embodiment of the present invention effectively reduces the training cost by using a small number of sample images to build a dictionary database, and gradually optimizes the segmentation results through multiple completion segmentations, so that the target area can be more accurately identified and segmented. Finally, by comprehensively considering the initial segmentation mask and the results of multiple completion segmentations, the model is optimized to ensure that the model can be continuously improved during the training process, so that the image segmentation model finally trained can avoid the problem of overfitting, improve the segmentation accuracy and the stability of the model, and can effectively handle complex and changeable image content, and is suitable for a variety of application scenarios.

[0073] According to an embodiment of the present invention, an embodiment of a training method for an image segmentation model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0074] In this embodiment, a method for training an image segmentation model is provided, which can be used in electronic devices. Figure 1 is a flowchart of a method for training an image segmentation model according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0075] Step S101, obtain multiple sample images, and build a dictionary database based on the multiple sample images, and the number of sample images is less than the image number threshold. Specifically, the image segmentation model can be applied to any scenario that requires image segmentation. The application scenario is to use the image segmentation model to perform image segmentation on road waterlogging images to determine the waterlogging area as an example. First, obtain a sample image whose image content is road waterlogging. The number of sample images is less than the image number threshold, and the value of the image number threshold should not be too large to effectively reduce the training cost and avoid the overfitting problem. The embodiment of the present invention is described using 10 sample images as an example. These 10 sample images are used as a dictionary database to provide a basis for subsequent feature extraction.

[0076] Step S102, using the image segmentation model to perform feature transformation on the dictionary database, obtain a dictionary feature vector set, and extract segmentation features from the dictionary feature vector set to obtain a sample feature matrix. Specifically, the image segmentation model is used to perform feature transformation on each sample image in the dictionary database to obtain 10 dictionary feature vectors, thereby forming a dictionary feature vector set. These feature vectors can represent key information in the sample image, such as color, texture, shape, etc. Then, features related to segmentation, such as edge information, texture information, etc., are extracted from the dictionary feature vector set to construct a sample feature matrix. The sample feature matrix contains the segmentation features of the 10 sample images. By feature transformation and extraction of segmentation features, the multi-dimensional features of the sample image can be fully captured, the richness and accuracy of the feature representation can be improved, and the sample feature matrix can be used as the input of the model to better guide the learning process of the model and improve the performance of the model.

[0077] Step S103, using the image segmentation model, based on the dictionary feature vector set and the sample feature matrix, similarity calculation is performed to obtain an initial segmentation mask. Specifically, using the image segmentation model, based on the dictionary feature vector set and the sample feature matrix, cosine point multiplication is performed to obtain a similarity matrix, and the matrix is ​​subjected to sigmoid activation transformation to obtain an initial segmentation mask. The initial segmentation mask is a preliminary segmentation result, which indicates a possible waterlogging area in the sample image and can be represented by a binary image, in which the value of each pixel in the binary image indicates that the pixel belongs to the target or the background. Usually, the value of the target pixel is 1 and the value of the background pixel is 0. In the scene of road waterlogging, a pixel value of 1 indicates a target pixel, that is, it belongs to the waterlogging area; a pixel value of 0 indicates a background pixel, that is, it belongs to the non-waterlogging area. Since the dictionary feature vector set includes 10 dictionary feature vectors, and the sample feature matrix includes features corresponding to the 10 dictionary feature vectors, the initial segmentation mask includes segmentation masks corresponding to the 10 sample images, that is, the initial segmentation mask is used as a set of preliminary segmentation results of the 10 sample images, which provides a basis for subsequent complementary segmentation. More specifically, during the calculation, all dictionary feature vectors in the dictionary feature vector set and the features corresponding to each dictionary feature vector are required to be sequentially applied through the following formula (1) to calculate the initial segmentation mask corresponding to each sample image.

[0078] F 1 =softmax(sigmoid(cosine(P ds ,M O ))) (1)

[0079] Among them, F 1 represents the initial segmentation mask; P ds Represents the dictionary feature vector set; M O Represents the sample feature matrix.

[0080] Step S104, using the image segmentation model, based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask, a preset number of complementary segmentations are performed to obtain a segmentation mask obtained by each complementary segmentation. Specifically, using the image segmentation model, based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask, multiple complementary segmentations are performed. Each complementary segmentation is optimized according to the previous segmentation result to generate a new segmentation mask, so that the segmentation result can be gradually optimized and the accuracy of the final segmentation can be improved.

[0081] Step S105, based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, the training loss is determined, and the image segmentation model is trained based on the training loss. Specifically, based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, the training loss is calculated, and the image segmentation model is trained based on the training loss. During the training process, the model will continuously adjust parameters to minimize the training loss and improve the accuracy of segmentation.

[0082] The training method of the image segmentation model provided by the embodiment of the present invention uses a small amount of sample images to build a dictionary database, thereby reducing the need for a large amount of labeled data and effectively reducing the training cost. The image segmentation model is used to perform feature transformation on the dictionary database to obtain a dictionary feature vector set, and then the segmentation features are extracted to obtain a sample feature matrix, effectively capturing the important features in the sample image. Then, an initial segmentation mask is generated by similarity calculation, and then the segmentation results are gradually optimized through multiple completion segmentations to improve the robustness and generalization ability of the model. Each completion segmentation can further refine and improve the initial segmentation mask, and can more accurately identify and segment the target area. Finally, the training loss is determined by comprehensively considering the initial segmentation mask and the results of multiple completion segmentations, thereby optimizing the model, ensuring that the model can be continuously improved during the training process, so that the image segmentation model finally trained can avoid the overfitting problem, improve the segmentation accuracy and the stability of the model, and can effectively process complex and changeable image content, and is suitable for a variety of application scenarios.

[0083] In this embodiment, a training method for an image segmentation model is provided, which can be used for the above-mentioned electronic device, etc. The method specifically includes the following steps:

[0084] Step S201, obtaining multiple initial images. Specifically, multiple initial images of road waterlogging are obtained through a camera or other image acquisition device. These initial images can be multiple images taken at different angles in the same time period, or images taken in different time periods, or can be obtained from an image database. The above image acquisition method is only an example and is not intended to be limiting.

[0085] Step S202, preprocessing the multiple initial images to obtain multiple preprocessed images. Specifically, the multiple initial images are denoised using a median filtering technique, and then any existing deblurring technique is used to deblur the denoised images to improve image quality.

[0086] Step S203, adopting the structural similarity extraction method to extract the structural information of multiple preprocessed images respectively, and adopting the color channel decomposition technology to extract the color components and brightness components of multiple preprocessed images respectively. Specifically, for any preprocessed image, the structural similarity extraction method (Structural Similarity Index, SSIM) is adopted to extract the structural information of the preprocessed image by calculating the local structural similarity of the preprocessed image. Then, the color channel decomposition technology (YUV color space, YUV) is adopted to decompose the preprocessed image into different color channels, such as RGB (Red GreenBlue, red, green and blue) channels, and extract the color components of each color channel. At the same time, the brightness component of the preprocessed image is extracted. By extracting the structural information, color components and brightness components, the characteristics of the image can be fully described, and the accuracy of subsequent processing can be improved.

[0087] Step S204, using mean clustering technology, clustering is performed after fusing structural information, color components and brightness components of multiple pre-processed images to obtain multiple clustering results. Specifically, the structural information, color components and brightness components of the pre-processed images are fused together to form a comprehensive feature vector. The comprehensive feature vector is clustered using a mean clustering (K-means) algorithm. Mean clustering is an unsupervised learning method that divides data points into multiple clusters through iteration, and each cluster is represented by a center point. Multiple clustering results are obtained through mean clustering, and each clustering result contains a group of similar images. Optionally, the number of clustering results is consistent with the number of sample images. Images with similar features can be found through clustering, thereby improving the efficiency and accuracy of image processing.

[0088] Step S205, the center point image of each clustering result in the multiple clustering results is used as a plurality of sample images. Specifically, the center point image of each clustering result is the most representative image in the clustering result. Therefore, the center point image of each clustering result is used as a plurality of sample images for subsequent training. Preprocessing by median filtering technology can effectively reduce random noise in the image, improve the quality of the image, and provide a better basis for subsequent processing. The structural similarity extraction method can retain the structural information in the image, such as edge, texture, etc., which is helpful for subsequent feature extraction and image segmentation. The color channel decomposition technology is then used to extract the color component and the brightness component, which can fully capture the multi-dimensional features of the image and improve the richness and accuracy of the feature representation. Finally, the mean clustering technology is used to classify similar image features into one category to obtain multiple clustering results. The center point image of each clustering result is selected as the sample image, which is helpful to screen out the most representative sample image from a large number of initial images and improve the diversity and representativeness of the sample.

[0089] Step S206, obtaining multiple sample images, and constructing a dictionary database based on the multiple sample images, wherein the number of sample images is less than the image number threshold. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0090] In some optional embodiments, Figure 2 is a flowchart of constructing a dictionary database according to an embodiment of the present invention. Figure 2 As shown, for any initial image, the initial image is preprocessed to obtain a preprocessed image. The structural similarity extraction method is used to extract structural information from the preprocessed image, and the color channel decomposition technology is used to extract the color component and brightness component from the preprocessed image. Then, the mean clustering technology is used to fuse the structural information, color component and brightness component of the preprocessed image, and clustering is performed to obtain the clustering result. Then, the image located at the center point of each clustering result is used as a sample image. Finally, these sample images are used as a dictionary database.

[0091] Step S207: perform feature transformation on the dictionary database using the image segmentation model to obtain a dictionary feature vector set, and extract segmentation features from the dictionary feature vector set to obtain a sample feature matrix.

[0092] Specifically, the above step S207 includes:

[0093] Step S2071, using the feature extraction network in the image segmentation model, perform feature transformation on the dictionary database to obtain a set of sample feature maps. Specifically, assuming that 10 sample images are obtained, that is, the dictionary database includes 10 sample images. Using the Swin-transformer feature extraction network to perform feature transformation on the dictionary database to obtain a set of sample feature maps p = {p 1 ,p 2 ,…,p 10}, whose dimension is 10×C d ×H d ×W d Among them, 10 represents the number of sample feature maps in the sample feature map set; C d ×H d ×W d Indicates the dimension of each sample feature map; C d represents the channel dimension; H d Indicates height; W d Indicates width.

[0094] Step S2072: average pooling is performed on the sample feature map set according to the channel dimension to obtain a dictionary feature vector set. Specifically, for p = {p 1 ,p 2 ,…,p 10Each sample feature map in} is averagely pooled according to the channel dimension to obtain a dictionary feature vector, thereby obtaining a dictionary feature vector set consisting of 10 dictionary feature vectors with a dimension of 10×C d ×1×1.

[0095] Step S2073, using the skeleton network in the image segmentation model, extracting segmentation features from the dictionary feature vector set to obtain a sample feature matrix. Specifically, using the Swin-transformer skeleton network to extract features for segmentation, a sample feature matrix is ​​obtained, whose dimension is C O ×H O ×W O . The feature extraction network is used to transform the features of the sample image and generate a set of sample feature maps, which can effectively capture the high-level abstract features in the image. The sample feature map set is then averaged and pooled to compress the multi-channel feature map into a one-dimensional feature vector, which reduces the dimension of the feature and retains the main feature information, simplifies the subsequent processing and improves the computational efficiency. The skeleton network is then used to extract features related to the segmentation task from the dictionary feature vector set to generate a sample feature matrix, which can better reflect the target area in the image and improve the accuracy of segmentation.

[0096] Step S208: Using the image segmentation model, similarity calculation is performed based on the dictionary feature vector set and the sample feature matrix to obtain an initial segmentation mask. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0097] Step S209 , using the image segmentation model, based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask, performs a preset number of complementary segmentations to obtain a segmentation mask obtained by each complementary segmentation.

[0098] Specifically, the above step S209 includes:

[0099] Step S2091, based on the initial segmentation mask and the sample feature matrix, aggregate the target feature prototype and the background feature prototype.

[0100] In some optional implementations, the above step S2091 includes:

[0101] Step a1, determine the target mask area greater than the first preset threshold from the initial segmentation mask. Specifically, for the road waterlogging image, the target mask area is the area to be segmented, that is, the waterlogging area. The first preset threshold is used to filter out the target mask area, and the first preset threshold is 0.6 as an example. By comparing the pixel value in the initial segmentation mask with the first preset threshold, the area greater than the first preset threshold is filtered out, and the area is the target mask area where the waterlogging area is located.

[0102] Step a2, extracting the first feature vector set corresponding to the position of the target mask area from the sample feature matrix. Specifically, each element in the sample feature matrix is ​​a feature vector, which represents the feature of a certain position in the sample image. According to the pixel position of the target mask area, the corresponding feature vector is extracted from the sample feature matrix, and the set of these feature vectors is the first feature vector set.

[0103] Step a3, average pooling is performed on the first feature vector set to obtain the target feature prototype. Specifically, all vectors in the first feature vector set are averaged to obtain a single feature vector, that is, the target feature prototype, which represents the comprehensive characteristics of the waterlogged area. It should be noted that since the initial segmentation mask and the sample feature matrix are both sets, that is, they include the initial segmentation masks and features corresponding to 10 sample images, the above steps a1 and a2 must be performed for each sample image, and the set of results obtained by the above two steps for the 10 sample images is used as the first feature vector set. By only selecting high-confidence areas for feature extraction, the feature prototypes of the target mask area can be effectively extracted and aggregated, and a high-quality target feature vector representing the waterlogged area can be generated, effectively reducing the interference of noise and background areas. Through the average pooling operation, multiple feature vectors are compressed into a single feature vector, which simplifies the feature representation, reduces the consumption of computing resources, and makes the feature vector easier to be processed and used by subsequent models.

[0104] Step a4, determining the background mask area greater than the first preset threshold from the difference between the initial segmentation mask and the set value. Specifically, the set value is 1. Since the target area where the waterlogged area is located is determined in the above step a1, each pixel value in the initial segmentation mask is subtracted from 1, and the area formed by the pixel values ​​greater than the first preset threshold in the difference result is used as the background mask area of ​​the non-waterlogged area.

[0105] Step a5, extracting a second feature vector set corresponding to the background mask area from the sample feature matrix. Specifically, extracting corresponding feature vectors from the sample feature matrix according to the pixel positions of the background mask area, the set of these feature vectors is the second feature vector set.

[0106] Step a6: For any feature vector in the second feature vector set, a deformable attention mechanism is used to obtain the relationship between each dimension of the feature vector to obtain a new feature vector. Specifically, the deformable attention mechanism is used to allow the attention weight to be dynamically adjusted at different positions, so as to better capture the correlation between each dimension of the feature vector, thereby obtaining a new feature vector.

[0107] Step a7, average pooling is performed on the new feature vectors corresponding to the second feature vector set to obtain the background feature prototype. Specifically, all vectors in the second feature vector set are averaged to obtain a single feature vector, that is, the background feature prototype, which represents the comprehensive features of the non-waterlogged area. It should be noted that, as mentioned above, for each sample image, the above steps a4 to a6 must be performed, and the set of results obtained by the above three steps of the 10 sample images is used as the second feature vector set. By only selecting high-confidence background areas for feature extraction, the characteristics of the background area can be better reflected, and high-quality background feature vectors representing non-waterlogged areas can be generated, effectively reducing the interference of noise and target areas, and improving the purity and reliability of the features. By using a deformable attention mechanism, the weights of the feature vectors can be dynamically adjusted to capture the complex relationship between the dimensions in the feature vector. This enhances the flexibility of feature representation, enables the model to better adapt to different background features, and compresses multiple feature vectors into a single feature vector through the average pooling operation, simplifies the feature representation, reduces the consumption of computing resources, and makes the feature vector easier to be processed and used by subsequent models.

[0108] Step S2092, based on the dictionary feature vector set, the target feature prototype and the background feature prototype, the target feature vector is obtained by splicing and fusing. Specifically, the dictionary feature vector set, the target feature prototype and the background feature prototype are spliced ​​according to the channel dimension, and then the spliced ​​feature vectors are fused using a 3x3 convolution operation to obtain the target feature vector.

[0109] Step S2093, perform similarity calculation based on the target feature vector and the sample feature matrix to obtain the segmentation mask obtained by the first completion segmentation. Specifically, referring to the above formula (1), the target feature vector replaces the dictionary feature vector set, and the segmentation mask obtained by the first completion segmentation is obtained by similarity calculation.

[0110] Step S2094, the segmentation mask obtained by the first complementary segmentation is used as the initial segmentation mask for the next complementary segmentation, and the process returns to the step of aggregating the target feature prototype and the background feature prototype based on the initial segmentation mask until the preset number of complementary segmentations is completed to obtain the segmentation mask obtained by each complementary segmentation. Specifically, steps S2091 to S2093 are a process of one complementary segmentation, and the segmentation mask obtained by this complementary segmentation is used as the initial segmentation mask for the next complementary segmentation, and the process returns to step S2091, and the subsequent operations are continued until the number of complementary segmentations reaches the preset number, and the segmentation mask obtained by each complementary segmentation is obtained. By splicing and fusing the dictionary feature vector set, the target feature prototype and the background feature prototype, the model can better understand and distinguish between the waterlogged area and the non-waterlogged area, and improve the accuracy of the segmentation. By gradually optimizing the segmentation results, a more accurate segmentation mask can be generated to improve the segmentation accuracy.

[0111] Step S210, determining a training loss based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, and training the image segmentation model based on the training loss, wherein any sample image includes a labeled mask.

[0112] Specifically, the above step S210 determines the training loss based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, including:

[0113] Step S2101, for any segmentation mask in the initial segmentation mask and the segmentation mask obtained by each complementary segmentation, based on the segmentation mask and the annotation mask corresponding to the segmentation mask, determine the cross entropy loss and similarity metric loss of the segmentation mask. Specifically, since the initial segmentation mask and the segmentation mask obtained by each complementary segmentation are actually a set of segmentation masks of 10 sample images, for the segmentation mask corresponding to any sample image in any segmentation mask set, based on the segmentation mask of the sample image and the actual annotation mask, the cross entropy loss is determined by the following formula (2), and the similarity metric loss is determined by the following formula (3). Among them, the cross entropy loss function is used to calculate the difference between the segmentation mask and the annotation mask, and the similarity metric loss is used to measure the similarity between the segmentation mask and the annotation mask.

[0114] BCE(gt,F n )=-(F n log(p(F n )))+(1-F n )log(1-p(F n )) (2)

[0115] Among them, gt represents the annotation mask corresponding to the sample image; F n represents the segmentation mask corresponding to the nth sample image; BCE(gt,F n) represents the cross entropy loss between the segmentation mask and the annotation mask corresponding to the nth sample image; p(F n ) represents the predicted probability of each pixel value of the segmentation mask corresponding to the nth sample image by the image segmentation model, and its value range is [0,1].

[0116]

[0117] Among them, gt represents the annotation mask corresponding to the sample image; F n represents the segmentation mask corresponding to the nth sample image; DICE(gt,F n ) represents the similarity metric loss between the segmentation mask and the annotation mask corresponding to the nth sample image.

[0118] Step S2102, the sum of the cross entropy loss and the similarity metric loss of each segmentation mask is determined as the training loss. Specifically, assuming that two complementary segmentations are performed, an initial segmentation mask, a segmentation mask of the first complementary segmentation, and a segmentation mask of the second complementary segmentation are obtained. Considering that each segmentation mask is a set of segmentation masks of 10 sample images, based on the segmentation mask and annotation code corresponding to each sample image, the cross entropy loss and the similarity metric loss are determined by the above formulas (2) and (3). Through the following formula (4), the sum of the cross entropy loss and the similarity metric loss of all sample images in all segmentation masks, that is, the sum of 10 cross entropy losses and 10 similarity metric losses corresponding to the initial segmentation mask, 10 cross entropy losses and 10 similarity metric losses corresponding to the segmentation mask obtained by the first completion segmentation, and 10 cross entropy losses and 10 similarity metric losses corresponding to the segmentation mask obtained by the second completion segmentation, a total of 60 losses, is determined as the training loss of the image segmentation model, so that the image segmentation model can be trained based on the training loss, and the minimization of the training loss is taken as the optimization goal, and the optimization is continuously performed, so that the trained image segmentation model has a higher segmentation accuracy, while avoiding the overfitting problem and having universal applicability.

[0119] L=BCE(gt,F 1 )+DICE(gt,F 1 )+BCE(gt,F 2 )+DICE(gt,F 2 )+(gt,F 3 )+DICE(gt,F 3 ) (4)

[0120] Among them, L represents the training loss; gt represents the annotation mask corresponding to the sample image; F 1 represents the initial segmentation mask; BCE(gt,F 1) represents the sum of the cross entropy losses corresponding to the initial segmentation masks of all sample images; DICE(gt,F 1 ) represents the sum of the similarity metric losses corresponding to the initial segmentation masks of all sample images; F 2 represents the segmentation mask obtained by the first completion segmentation; BCE(gt,F 2 ) represents the sum of the cross entropy losses corresponding to the segmentation masks obtained by the first completion segmentation of all sample images; DICE(gt,F 2 ) represents the sum of the similarity metric losses corresponding to the segmentation masks obtained by the first completion segmentation of all sample images; F 3 represents the segmentation mask obtained by the second segmentation completion; BCE(gt,F 3 ) represents the sum of the cross entropy losses corresponding to the segmentation masks obtained by the second completion segmentation of all sample images; DICE(gt,F 3 ) represents the sum of the similarity metric losses corresponding to the segmentation masks obtained by the second completion segmentation of all sample images.

[0121] In some optional implementations, Figure 3 is a flowchart of another method for training an image segmentation model according to an embodiment of the present invention. Figure 3 As shown, first, all sample images are input into the image segmentation model as a dictionary database. Then, feature extraction is performed through the feature extraction network in the image segmentation model to obtain a set of sample feature maps. Then, the sample feature map set is average pooled to obtain a set of dictionary feature vectors. Then, segmentation features are extracted from the dictionary feature vector set through the skeleton network to obtain a sample feature matrix. Then, the initial segmentation mask is obtained by calculating the similarity between the sample feature matrix and the dictionary feature vector set. Then, based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask, a segmentation mask for each complementary segmentation is obtained. Based on the segmentation mask of each complementary segmentation, the initial segmentation mask and the annotation mask of the sample image, the training loss is calculated, and the image segmentation model is trained based on the training loss.

[0122] The training method of the image segmentation model provided by the embodiment of the present invention uses a small amount of sample images to build a dictionary database, thereby reducing the need for a large amount of labeled data and effectively reducing the training cost. The image segmentation model is used to perform feature transformation on the dictionary database to obtain a dictionary feature vector set, and then the segmentation features are extracted to obtain a sample feature matrix, effectively capturing the important features in the sample image. Then, an initial segmentation mask is generated by similarity calculation, and then the segmentation results are gradually optimized through multiple completion segmentations to improve the robustness and generalization ability of the model. Each completion segmentation can further refine and improve the initial segmentation mask, and can more accurately identify and segment the target area. Finally, the training loss is determined by comprehensively considering the initial segmentation mask and the results of multiple completion segmentations, thereby optimizing the model, ensuring that the model can be continuously improved during the training process, so that the image segmentation model finally trained can avoid the overfitting problem, improve the segmentation accuracy and the stability of the model, and can effectively process complex and changeable image content, and is suitable for a variety of application scenarios.

[0123] In this embodiment, a method for identifying road water accumulation is provided, which can be used in the above-mentioned electronic equipment, etc. Figure 4 is a flow chart of a method for identifying road waterlogging according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0124] Step S401, obtaining a road waterlogging image. Specifically, assuming that the image segmentation model obtained through the above training is used for road waterlogging recognition, firstly, a road waterlogging image can be obtained through a camera, a drone or other shooting device. The number of the road waterlogging images is one or more.

[0125] Step S402, preprocessing the road water accumulation image to obtain a preprocessed road water accumulation image. Specifically, the road water accumulation image is subjected to existing denoising and deblurring processing, referring to step S202, to obtain a preprocessed road water accumulation image to improve image quality.

[0126] Step S403, using an image segmentation model, image segmentation is performed based on the preprocessed road waterlogging image, and the segmentation results greater than the second preset threshold are used as the target segmentation mask. The image segmentation model is trained based on the training method of the image segmentation model provided in the above embodiment. Specifically, the road waterlogging image is input into the image segmentation model, and the image segmentation model generates a segmentation result for each pixel. The segmentation result is a probability map, which indicates the probability that each pixel belongs to the waterlogging area. A second preset threshold is set, and the pixels in the segmentation result that are greater than the second preset threshold are marked as waterlogging areas, thereby obtaining a target segmentation mask that can accurately segment the waterlogging area and the non-waterlogging area in the image. The target segmentation mask is a binary image, in which the pixel value of the waterlogging area is 1, and the pixel value of the non-waterlogging area is 0.

[0127] The road waterlogging identification method provided by the embodiment of the present invention applies the trained image segmentation model to the field of smart cities, uses the image segmentation model to perform image segmentation on the road waterlogging image, and uses the segmentation results greater than the second preset threshold in the segmentation results as the target segmentation mask. It can accurately segment the waterlogging area and non-waterlogging area in the image, which is helpful for the construction of smart cities.

[0128] In this embodiment, a training device for an image segmentation model is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0129] This embodiment provides a training device for an image segmentation model. Figure 5 As shown, including:

[0130] The first acquisition module 501 is used to acquire multiple sample images and construct a dictionary database based on the multiple sample images, where the number of the sample images is less than an image number threshold.

[0131] The first extraction module 502 is used to perform feature transformation on the dictionary database using an image segmentation model to obtain a dictionary feature vector set, and extract segmentation features from the dictionary feature vector set to obtain a sample feature matrix.

[0132] The first determination module 503 is used to use the image segmentation model to perform similarity calculation based on the dictionary feature vector set and the sample feature matrix to obtain an initial segmentation mask.

[0133] The segmentation module 504 is used to use the image segmentation model to perform a preset number of complementary segmentations based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask to obtain a segmentation mask obtained by each complementary segmentation.

[0134] The training module 505 is used to determine the training loss based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, and train the image segmentation model based on the training loss.

[0135] In some optional implementations, the first extraction module 502 includes:

[0136] The first extraction unit is used to use the feature extraction network in the image segmentation model to perform feature transformation on the dictionary database to obtain a set of sample feature maps.

[0137] The pooling unit is used to perform average pooling on the sample feature map set according to the channel dimension to obtain a dictionary feature vector set.

[0138] The second extraction unit is used to extract segmentation features from the dictionary feature vector set using the skeleton network in the image segmentation model to obtain a sample feature matrix.

[0139] In some optional implementations, the segmentation module 504 includes:

[0140] The aggregation unit is used to aggregate the target feature prototype and the background feature prototype based on the initial segmentation mask and the sample feature matrix.

[0141] The first determining unit is used to obtain a target feature vector by performing splicing and fusing based on a dictionary feature vector set, a target feature prototype and a background feature prototype.

[0142] The second determining unit is used to perform similarity calculation based on the target feature vector and the sample feature matrix to obtain a segmentation mask obtained by the first completion segmentation.

[0143] The segmentation unit is used to use the segmentation mask obtained by the first completion segmentation as the initial segmentation mask for the next completion segmentation, and return to the step of aggregating the target feature prototype and the background feature prototype based on the initial segmentation mask until a preset number of completion segmentations are completed to obtain the segmentation mask obtained for each completion segmentation.

[0144] In some optional embodiments, the polymeric unit includes:

[0145] The first determining subunit is used to determine a target mask area greater than a first preset threshold from the initial segmentation mask.

[0146] The first extraction subunit is used to extract a first feature vector set at a position corresponding to the target mask area from the sample feature matrix.

[0147] The first pooling subunit is used to perform average pooling on the first feature vector set to obtain a target feature prototype.

[0148] In some optional embodiments, the polymeric unit includes:

[0149] The second determining subunit is used to determine the background mask area greater than the first preset threshold from the difference between the initial segmentation mask and the set value.

[0150] The second extraction subunit is used to extract a second feature vector set at a position corresponding to the background mask area from the sample feature matrix.

[0151] The third determination subunit is used to use a deformable attention mechanism to obtain the relationship between each dimension of any feature vector in the second feature vector set to obtain a new feature vector.

[0152] The second pooling subunit is used to perform average pooling on the new feature vectors corresponding to the second feature vector set to obtain a background feature prototype.

[0153] In some optional implementations, any sample image includes a label mask, and the training module 505 includes:

[0154] The third determining unit is used to determine the cross entropy loss and the similarity metric loss of the segmentation mask for any segmentation mask in the initial segmentation mask and the segmentation mask obtained by each completed segmentation based on the segmentation mask and the annotation mask corresponding to the segmentation mask.

[0155] The fourth determining unit is used to determine the sum of the cross entropy loss and the similarity metric loss of each segmentation mask as the training loss.

[0156] In some optional implementations, before the first acquisition module 501, the device further includes:

[0157] The third acquisition module is used to acquire multiple initial images.

[0158] The image processing module is used to preprocess the multiple initial images to obtain multiple preprocessed images.

[0159] The second extraction module is used to respectively extract the structural information of the plurality of pre-processed images by using a structural similarity extraction method, and respectively extract the color components and brightness components of the plurality of pre-processed images by using a color channel decomposition technology.

[0160] The clustering module is used to adopt the mean clustering technology to cluster multiple pre-processed images after fusing structural information, color components and brightness components to obtain multiple clustering results.

[0161] The third determination module is used to use the center point image of each clustering result in the multiple clustering results as multiple sample images.

[0162] In this embodiment, a road water accumulation identification device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0163] This embodiment provides a road waterlogging identification device. Figure 6 As shown, including:

[0164] The second acquisition module 601 is used to acquire a road waterlogging image.

[0165] The preprocessing module 602 is used to preprocess the road waterlogging image to obtain a preprocessed road waterlogging image.

[0166] The second determination module 603 is used to use an image segmentation model to perform image segmentation based on the preprocessed road waterlogging image, and use the segmentation result greater than the second preset threshold as the target segmentation mask. The image segmentation model is trained based on the image segmentation model training method provided in the above embodiment.

[0167] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0168] The image segmentation model training device and road water identification device in this embodiment are presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0169] The embodiment of the present invention also provides a computer device having the above Figure 5 The training device of the image segmentation model shown and Figure 6 The road water accumulation identification device shown.

[0170] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0171] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0172] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0173] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0174] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0175] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0176] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0177] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0178] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0179] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A training method for an image segmentation model, characterized in that: The method comprises: Acquire multiple sample images and use the multiple sample images as a dictionary database, wherein the number of the sample images is less than an image number threshold; Using an image segmentation model to perform feature transformation on the dictionary database to obtain a dictionary feature vector set, and extracting segmentation features from the dictionary feature vector set to obtain a sample feature matrix; Using the image segmentation model, similarity calculation is performed based on the dictionary feature vector set and the sample feature matrix to obtain an initial segmentation mask; Using the image segmentation model, based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask, a preset number of completion segmentations are performed to obtain a segmentation mask obtained for each completion segmentation; Based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, a training loss is determined, and the image segmentation model is trained based on the training loss.

2. The method according to claim 1, characterized in that The image segmentation model is used to perform feature transformation on the dictionary database to obtain a dictionary feature vector set, and segmentation features are extracted from the dictionary feature vector set to obtain a sample feature matrix, including: Using the feature extraction network in the image segmentation model, the dictionary database is subjected to feature transformation to obtain a set of sample feature maps; Performing average pooling on the sample feature map set according to the channel dimension to obtain the dictionary feature vector set; The skeleton network in the image segmentation model is used to extract segmentation features from the dictionary feature vector set to obtain the sample feature matrix.

3. The method according to claim 1, characterized in that The image segmentation model is used to perform a preset number of complementary segmentations based on the dictionary feature vector set, the sample feature matrix, and the initial segmentation mask to obtain a segmentation mask obtained for each complementary segmentation, including: Aggregating target feature prototypes and background feature prototypes based on the initial segmentation mask and the sample feature matrix; Based on the dictionary feature vector set, the target feature prototype and the background feature prototype, they are spliced ​​and fused to obtain a target feature vector; Perform similarity calculation based on the target feature vector and the sample feature matrix to obtain a segmentation mask obtained by the first completion segmentation; The segmentation mask obtained from the first completion segmentation is used as the initial segmentation mask for the next completion segmentation, and the process returns to the step of aggregating the target feature prototype and the background feature prototype based on the initial segmentation mask until a preset number of completion segmentations are completed to obtain a segmentation mask obtained for each completion segmentation.

4. The method according to claim 3, characterized in that The aggregating target feature prototypes based on the initial segmentation mask and the sample feature matrix includes: Determining a target mask area greater than a first preset threshold from the initial segmentation mask; Extracting a first feature vector set corresponding to the target mask area from the sample feature matrix; Average pooling is performed on the first feature vector set to obtain the target feature prototype.

5. The method according to claim 4, characterized in that The aggregating background feature prototypes based on the initial segmentation mask and the sample feature matrix includes: Determine a background mask area greater than the first preset threshold from the difference between the initial segmentation mask and the set value; Extracting a second feature vector set at a position corresponding to the background mask area from the sample feature matrix; For any feature vector in the second feature vector set, using a deformable attention mechanism to obtain the relationship between each dimension of the feature vector to obtain a new feature vector; Average pooling is performed on the new feature vectors corresponding to the second feature vector set to obtain the background feature prototype.

6. The method according to claim 1, characterized in that Any sample image includes a labeled mask, and determining the training loss based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation includes: For any segmentation mask among the initial segmentation mask and the segmentation mask obtained by each completed segmentation, determine a cross entropy loss and a similarity metric loss of the segmentation mask based on the segmentation mask and a label mask corresponding to the segmentation mask; The sum of the cross entropy loss and the similarity metric loss of each segmentation mask is determined as the training loss.

7. The method according to claim 1, characterized in that Before acquiring the plurality of sample images, the method further includes: Obtain multiple initial images; Preprocessing the multiple initial images to obtain multiple preprocessed images; Using a structural similarity extraction method to extract structural information of the plurality of pre-processed images respectively, and using a color channel decomposition technology to extract color components and brightness components of the plurality of pre-processed images respectively; Using mean clustering technology, clustering is performed after the plurality of pre-processed images are fused with structural information, color components and brightness components to obtain a plurality of clustering results; The center point image of each clustering result in the multiple clustering results is used as the multiple sample images.

8. A method for identifying road waterlogging, characterized in that: The method comprises: Acquire road water accumulation images; Preprocessing the road waterlogging image to obtain a preprocessed road waterlogging image; An image segmentation model is used to perform image segmentation based on the preprocessed road waterlogging image, and a segmentation result greater than a second preset threshold is used as a target segmentation mask. The image segmentation model is trained based on the training method of the image segmentation model described in any one of claims 1 to 7.

9. A training device for an image segmentation model, characterized in that: The device comprises: A first acquisition module is used to acquire a plurality of sample images and construct a dictionary database based on the plurality of sample images, wherein the number of the sample images is less than an image number threshold; A first extraction module is used to perform feature transformation on the dictionary database using an image segmentation model to obtain a dictionary feature vector set, and extract segmentation features from the dictionary feature vector set to obtain a sample feature matrix; A first determination module is used to use the image segmentation model to perform similarity calculation based on the dictionary feature vector set and the sample feature matrix to obtain an initial segmentation mask; A segmentation module, configured to use the image segmentation model to perform a preset number of complementary segmentations based on the dictionary feature vector set, the sample feature matrix and the initial segmentation mask, and obtain a segmentation mask obtained for each complementary segmentation; A training module is used to determine a training loss based on the initial segmentation mask and the segmentation mask obtained by each completed segmentation, and train the image segmentation model based on the training loss.

10. A road water accumulation identification device, characterized in that: The device comprises: The second acquisition module is used to acquire the road water accumulation image; A preprocessing module, used for preprocessing the road waterlogging image to obtain a preprocessed road waterlogging image; The second determination module is used to use an image segmentation model to perform image segmentation based on the preprocessed road waterlogging image, and use the segmentation result greater than a second preset threshold as a target segmentation mask. The image segmentation model is trained based on the training method of the image segmentation model described in any one of claims 1 to 7.

Images