Semantic segmentation network model establishment method for road surface detection and electronic device

Abstract

The application provides a semantic segmentation network model establishment method and electronic equipment for road surface detection. The method comprises the following steps: establishing an initial cross-regularization segmentation comprehensive model, including an initial semantic segmentation network model, an initial edge prediction network model and a cross-regularization module; establishing a training data set; training the initial semantic segmentation network model and the initial edge prediction network model respectively; comparing the edge information in the semantic segmentation prediction result with the edge ground truth image by using the cross-regularization module, comparing the edge information in the segmentation ground truth image with the edge prediction result, and comparing the semantic segmentation prediction result with the edge prediction result; and obtaining the semantic segmentation network model according to the comparison result, the error between the semantic segmentation prediction result and the segmentation ground truth image, and the error between the edge prediction result and the edge ground truth image. The semantic segmentation network model established by the method can recognize various types of road surface diseases and has better prediction accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road surface detection, and in particular to a semantic segmentation network model establishment method for road surface detection and an electronic device. BACKGROUND

[0002] The detection and segmentation of road surface diseases of airports, highways and the like and the maintenance and repair thereof are one of the key links for ensuring the safe operation of the public transportation field. In particular, in the aspect of safety detection of airport road surfaces, safety work is the premise of all work, and the safety detection of airport road surfaces is one of the important tasks for ensuring the safety of civil aviation.

[0003] Traditional road surface disease detection is mostly realized by manual work, which requires a large amount of manpower and material resources, and due to the diversity of disease types and sizes, the detection of road surface diseases is greatly affected by human subjectivity, and the recognition efficiency and accuracy are difficult to meet the actual demand.

[0004] With the continuous development of artificial intelligence technology, automatic detection and segmentation technology of diseases is also emerging, for example, a mobile robot platform is used to collect data images through path planning, and then a detection algorithm is used to obtain a segmentation result. Such road surface disease detection method is usually based on image processing technology, and analyzes in the spatial domain (morphological method, region growing method, etc.), in the frequency domain (wavelet transform, etc.) and in the fusion of spatial and frequency domains. With the development of deep learning, road surface detection technology can be mainly divided into methods based on convolutional neural network, full convolutional network and U-Net network.

[0005] However, the above disease detection technology focuses on the identification of a single disease of road surface cracks, and cannot effectively identify other types of diseases. On the other hand, due to the noise of the detection data caused by weather and light and the diversity of different types of disease morphology in road surface detection, the above detection technology has low detection accuracy for road surface detection. SUMMARY

[0006] The present application provides a semantic segmentation network model establishment method for road surface detection and an electronic device to solve the defects of low detection accuracy and difficulty in detecting multiple types of road surface diseases in the prior art, and to realize an establishment method of a semantic segmentation network model which can detect multiple types of road surface diseases and has high detection accuracy.

[0007] The present application provides a semantic segmentation network model establishment method for road surface detection, comprising:

[0008] An initial cross-regularization segmentation comprehensive model is established, the initial cross-regularization segmentation comprehensive model comprising an initial semantic segmentation network model, an initial edge prediction network model and a cross-regularization module;

[0009] establish a training data set, the training data set comprising training images, segmentation ground truth images corresponding to the training images, and edge ground truth images corresponding to the training images;

[0010] train the initial semantic segmentation network model and the initial edge prediction network model based on the training data set respectively, and output semantic segmentation prediction results and edge prediction results;

[0011] compare the edge information in the semantic segmentation prediction results with the edge ground truth images, compare the edge information in the segmentation ground truth images with the edge prediction results, and compare the semantic segmentation prediction results with the edge prediction results by using the cross-regularization module;

[0012] optimize the initial cross-regularization segmentation comprehensive model according to the comparison results, errors between the semantic segmentation prediction results and the segmentation ground truth images, and errors between the edge prediction results and the edge ground truth images, to obtain a cross-regularization segmentation comprehensive model, so as to obtain a semantic segmentation network model.

[0013] According to the semantic segmentation network model establishment method for road surface detection provided by the application, the training data set is established, including:

[0014] collect road surface disease images on a road surface as sampling images;

[0015] perform data expansion on the sampling images to obtain the training images;

[0016] perform artificial labeling on the training images to obtain the segmentation ground truth images and the edge ground truth images;

[0017] establish a training data set according to the training images, the segmentation ground truth images, and the edge ground truth images.

[0018] According to the semantic segmentation network model establishment method for road surface detection provided by the application, the training data set is established, including:

[0019] train the semantic segmentation network model by using the training images and the segmentation ground truth images in the training data set, with the training images as input and the semantic segmentation prediction results as output;

[0020] train the edge prediction network model by using the training images and the edge ground truth images in the training data set, with the training images as input and the edge prediction results as output.

[0021] According to the present invention, a method for establishing a semantic segmentation network model for road surface detection is provided, wherein the cross-regularization module includes a first branch, a second branch, and a third branch;

[0022] The step of comparing the edge information in the semantic segmentation prediction result with the ground truth edge image using the cross-regularization module, comparing the edge information in the ground truth segmentation image with the edge prediction result, and comparing the semantic segmentation prediction result with the edge prediction result includes:

[0023] The first branch is used to compare the edge information in the semantic segmentation prediction result with the edge ground image to obtain the first loss;

[0024] A second loss is obtained by comparing the edge information in the segmented ground image with the edge prediction result using a second branch.

[0025] A third loss is obtained by comparing the semantic segmentation prediction result with the edge prediction result using a third branch.

[0026] According to the present invention, a method for establishing a semantic segmentation network model for road surface detection includes, wherein the first loss is obtained by comparing the edge information in the semantic segmentation prediction result with the ground truth edge image using a first branch.

[0027] The first branch is used to extract edge information from the semantic segmentation prediction result;

[0028] The edge information extracted from the semantic segmentation prediction result is compared with the ground truth edge image to obtain the first loss.

[0029] According to the present invention, a method for establishing a semantic segmentation network model for road surface detection, wherein the step of comparing the edge information in the segmentation ground truth image with the edge prediction result using a second branch to obtain a second loss includes:

[0030] The second branch is used to extract edge information from the segmented ground truth image;

[0031] The edge information extracted from the segmentation ground truth image is compared with the edge ground truth image to obtain a second loss.

[0032] According to the present invention, a method for establishing a semantic segmentation network model for road surface detection includes optimizing an initial cross-regularized segmentation synthesis model based on comparison results, the error between the semantic segmentation prediction result and the ground truth segmentation image, and the error between the edge prediction result and the ground truth edge image, thereby obtaining a semantic segmentation network model. The method comprises:

[0033] A first loss function is established based on the error between the semantic segmentation prediction result and the segmentation ground image;

[0034] A second loss function is established based on the error between the edge prediction result and the ground truth edge image;

[0035] The first loss function and the second loss function are optimized, and the parameters of the initial semantic segmentation network model and the initial edge prediction network model are updated in combination with the comparison results to obtain a cross-regularized segmentation comprehensive model, thereby obtaining a semantic segmentation network model.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the semantic segmentation network model establishment method for road surface detection as described above.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the semantic segmentation network model establishment method for road surface detection as described above.

[0038] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the semantic segmentation network model establishment method for road surface detection as described above.

[0039] The semantic segmentation network model establishment method and electronic device for road surface detection provided by this invention train and optimize an initial cross-regularized segmentation comprehensive model to obtain a semantic segmentation network model. During the training process, the semantic segmentation network model is assisted by the edge prediction network model and constrained by the cross-regularization module, so that the obtained semantic segmentation network model can identify multiple types of road surface defects with high recognition accuracy and better robustness. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is one of the flowcharts illustrating the semantic segmentation network model establishment method for road surface detection provided by the present invention;

[0042] Figure 2This is the second flowchart illustrating the semantic segmentation network model establishment method for road surface detection provided by this invention.

[0043] Figure 3 This is a flowchart illustrating the method for establishing a training dataset provided by the present invention;

[0044] Figure 4 This is a flowchart illustrating the method for training the initial semantic segmentation network model and the initial edge prediction network model provided by the present invention.

[0045] Figure 5 This is one of the flowcharts illustrating the method for comparison using the cross-regularization module provided by the present invention;

[0046] Figure 6 This is a second flowchart illustrating the method for comparison using the cross-regularization module provided by the present invention;

[0047] Figure 7 This is a schematic diagram illustrating the possible values ​​of the number of pixels in the field provided by the present invention;

[0048] Figure 8 This is a flowchart illustrating the method for obtaining the first loss provided by the present invention;

[0049] Figure 9 This is a flowchart illustrating the method for obtaining the second loss provided by the present invention;

[0050] Figure 10 This is a flowchart illustrating the method for obtaining a semantic segmentation network model provided by the present invention;

[0051] Figure 11 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0053] The following is combined Figures 1-10 This invention describes a method for establishing a semantic segmentation network model for road surface detection, the method comprising:

[0054] S101: Establish an initial cross-regularized segmentation synthesis model, which includes an initial semantic segmentation network model, an initial edge prediction network model, and a cross-regularization module.

[0055] Specifically, first, an initial cross-regularized segmentation synthesis model is established, referring to... Figure 2 As shown, the model includes an initial semantic segmentation network model, an initial edge prediction network model, and a cross-regularization module. The initial semantic segmentation model is used to segment the diseased areas in the road surface image, the initial edge prediction network model is used to segment the edges of the diseased areas in the road surface image, and the cross-regularization module is used to apply cross-constraints to the outputs of the initial semantic segmentation network model and the initial edge prediction network model.

[0056] S102: Establish a training dataset, which includes training images, segmentation ground truth images corresponding to the training images, and edge ground truth images corresponding to the training images.

[0057] Specifically, a training dataset is established, which includes training images, ground truth segmentation images corresponding to the training images, and ground truth edge images corresponding to the training images.

[0058] The training images are road surface defects, which can be either color or grayscale. The segmentation ground truth images and edge ground truth images are labeled based on the training images.

[0059] S103: Train the initial semantic segmentation network model and the initial edge prediction network model based on the training dataset, and output the semantic segmentation prediction result and the edge prediction result.

[0060] It is understandable that the initial semantic segmentation network model and the initial edge prediction network model are trained based on the training dataset. The initial semantic segmentation network model outputs the semantic segmentation prediction result, and the initial edge prediction network module outputs the edge prediction result. Both the semantic segmentation prediction result and the edge prediction result can be output in the form of an image.

[0061] S104: The cross-regularization module is used to compare the edge information in the semantic segmentation prediction result with the edge ground truth image, compare the edge information in the segmentation ground truth image with the edge prediction result, and compare the semantic segmentation prediction result with the edge prediction result.

[0062] Specifically, a cross-regularization module is used to compare the semantic segmentation prediction results and the edge prediction results. Based on the semantic segmentation prediction results and the edge prediction results, the cross-regularization module is used to compare the edge information in the semantic segmentation prediction results with the ground truth edge image, and the edge information in the ground truth segmentation image is compared with the edge prediction results. After the comparison, three comparison results are obtained.

[0063] S105: Based on the comparison results, the error between the semantic segmentation prediction result and the segmentation ground truth image, and the error between the edge prediction result and the edge ground truth image, the initial cross-regularized segmentation synthesis model is optimized to obtain the cross-regularized segmentation synthesis model, thereby obtaining the semantic segmentation network model.

[0064] For example, the initial cross-regularized segmentation synthesis model is optimized based on the comparison results, the error between the semantic segmentation prediction result and the ground truth segmentation image, and the error between the edge prediction result and the ground truth edge image to obtain the cross-regularized segmentation synthesis model. Optimizing the initial cross-regularized segmentation synthesis model is equivalent to optimizing the initial semantic segmentation network model and the initial edge prediction network model, thereby obtaining the semantic segmentation network model.

[0065] Existing pavement distress segmentation algorithms often suffer from spatial resolution loss in the predicted image due to operations such as pooling, resulting in a lack of fine-grained details in the segmentation results, affecting segmentation accuracy and leading to classification errors. This method introduces an edge prediction network model, which learns the binarized prediction values ​​of distress edges to compensate for the lack of fine-grained object detail modeling, thereby improving the segmentation and prediction capabilities of the resulting semantic segmentation network model for multi-class pavement distresses.

[0066] Furthermore, this method proposes a cross-regularization approach. For the semantic segmentation network model and the edge prediction network model, the edge prediction results are constrained by the segmentation ground truth image, while the semantic segmentation prediction results are constrained by the edge ground truth image. This is used to train the initial cross-regularized segmentation synthesis model, so that the semantic segmentation network model obtained after training has higher accuracy and robustness.

[0067] The semantic segmentation network model establishment method for road surface detection provided by this invention trains and optimizes an initial cross-regularized segmentation comprehensive model to obtain a semantic segmentation network model. During the training process, the edge prediction network model is used as an aid, and the cross-regularization module provides constraints, so that the obtained semantic segmentation network model can identify multiple types of road surface defects with high recognition accuracy and better robustness.

[0068] In one embodiment, such as Figure 3 As shown, the establishment of the training dataset includes:

[0069] S301: Collect images of road surface defects as sample images.

[0070] Specifically, images of road surface defects are captured by a camera as sample images, which can be either color or grayscale.

[0071] S302: Perform data expansion on the sampled image to obtain the training image.

[0072] Understandably, since the number of available sampled images may be relatively small and insufficient to meet training requirements, data expansion processing can be performed on the sampled images to obtain more road surface damage images as training images.

[0073] S303: Manually annotate the training images to obtain the segmentation ground truth image and the edge ground truth image.

[0074] For example, training images can be manually labeled, specifically using specialized labeling software. This manual labeling yields ground truth segmentation images corresponding to the training images. After obtaining the ground truth segmentation images, edge extraction is performed on them to obtain ground truth edge images.

[0075] S304: Establish a training dataset based on the training image, the segmentation ground truth image, and the edge ground truth image.

[0076] Specifically, the training images, segmentation ground truth images, and edge ground truth images obtained from the above steps are used to build the training dataset.

[0077] In one embodiment, such as Figure 4 As shown, the initial semantic segmentation network model and the initial edge prediction network model are trained based on the training dataset, and the semantic segmentation prediction result and edge prediction result are output, including:

[0078] S401: The semantic segmentation network model is trained using the training images and the ground truth segmentation images in the training dataset. The input is the training images, and the output is the semantic segmentation prediction result.

[0079] Specifically, the semantic segmentation network model is trained using training images and ground truth segmentation images from the training data. During training, the input is the training images, the output is the semantic segmentation prediction results, and the ground truth segmentation images are used as the training ground truth.

[0080] The output semantic segmentation prediction result can be an image, which can use different colors to represent different types of road surface defects. For example, the pixels in the healthy road surface area are black, the pixels in the cracked road surface area are red, the pixels in the broken road surface joint area are green, and the pixels in the road surface patch area are yellow, etc.

[0081] Understandably, when performing manual annotation, the annotation color of the disease type corresponding to the segmentation ground truth image should correspond to the same color of each disease type in the output semantic segmentation prediction result. For example, when manually annotating, the color of the pixels in the road surface crack area is sampled in red, the color of the pixels in the road surface joint breakage area is sampled in green, and the color of the pixels in the road surface patch area is sampled in yellow.

[0082] S402: The edge prediction network model is trained using the training images and ground truth edge images in the training dataset, with the training images as input and the edge prediction results as output.

[0083] Specifically, the edge prediction network model is trained using training images and ground truth edge images from the training data. The input during training is the training image, and the output can be a binarized edge prediction result. The ground truth edge images are used as training ground truth.

[0084] The output binarized edge prediction result can be an image, which uses two different colors to represent the edge of the road surface defect area and the rest of the area. For example, the pixels of the rest of the output image are black, and the pixels of the road surface defect edge are white, thus indicating whether the pixels in the image are edges or not.

[0085] It is understandable that the ground truth image extracted by segmenting the ground truth image can also be a binarized edge ground truth image. During extraction, the color of the edge of the road surface defect corresponding to the ground truth image should correspond to the same color of the edge of the defect in the output edge prediction result. For example, when performing edge extraction, white is used as the color of the pixel point of the edge of the road surface defect, and black is used as the color of the pixel point of the rest of the image.

[0086] In one embodiment, such as Figure 5 As shown, the cross-regularization module includes a first branch, a second branch, and a third branch;

[0087] The step of comparing the edge information in the semantic segmentation prediction result with the ground truth edge image using the cross-regularization module, comparing the edge information in the ground truth segmentation image with the edge prediction result, and comparing the semantic segmentation prediction result with the edge prediction result includes:

[0088] S501: The first branch is used to compare the edge information in the semantic segmentation prediction result with the edge ground truth image to obtain the first loss.

[0089] For details, please refer to Figure 6As shown, the first branch is used to compare the edge information in the semantic segmentation prediction result with the ground truth edge image to obtain the first loss L. re1 .

[0090] S502: The second branch is used to compare the edge information in the segmented ground truth image with the edge prediction result to obtain the second loss.

[0091] For details, please refer to Figure 6 As shown, the second branch is used to compare the edge information in the segmented ground truth image with the edge prediction results to obtain the second loss L. re2 .

[0092] S503: The semantic segmentation prediction result is compared with the edge prediction result using the third branch to obtain the third loss.

[0093] For details, please refer to Figure 6 As shown, the third branch is used to compare the semantic segmentation prediction results with the edge prediction results to obtain the third loss L. re,i :

[0094]

[0095] Where C represents the number of road surface distress types, and N represents the number of neighboring pixels of pixel i, such as... Figure 7 The value of N shown can be 4, 8, 24, etc. s represents all pixels in the neighborhood of pixel i. c,i ,s c,j ε represents the semantic segmentation prediction results for pixel i and pixel j, respectively. i ,ε j These represent the edge prediction results for pixel i and pixel j, respectively.

[0096] It is understandable that, in the above content, the choice of the number of neighboring pixels N of pixel i is related to the type of road surface defects. If the value is too small, the amount of neighboring information will be small, which may affect the judgment of the semantic segmentation prediction result. If the value is too large, there will be too many irrelevant points in the neighboring points, which will also affect the judgment of the semantic segmentation prediction result. Generally speaking, a value of 8 is better.

[0097] In one embodiment, such as Figure 8 As shown, the step of comparing the edge information in the semantic segmentation prediction result with the ground truth edge image using the first branch to obtain the first loss includes:

[0098] S801: Use the first branch to extract edge information from the semantic segmentation prediction result.

[0099] It is understandable that when comparing the edge information in the semantic segmentation prediction result with the ground truth edge image, it is first necessary to extract the edge information in the semantic segmentation prediction result. The edge information is the image that can represent the edge of the road surface defect area in the semantic segmentation prediction result.

[0100] S802: The edge information extracted from the semantic segmentation prediction result is compared with the ground truth edge image to obtain the first loss.

[0101] In one embodiment, such as Figure 9 As shown, the step of comparing the edge information in the segmented ground truth image with the edge prediction result using the second branch to obtain the second loss includes:

[0102] S901: Use the second branch to extract edge information from the segmented ground truth image.

[0103] It is understandable that when comparing the edge information in the ground truth segmentation image with the edge prediction results, it is first necessary to extract the edge information in the ground truth segmentation image. The edge information is an image that can represent the edge of the road surface defect area in the ground truth segmentation image.

[0104] S902: The edge information extracted from the segmented ground truth image is compared with the edge ground truth image to obtain a second loss.

[0105] In one embodiment, such as Figure 10 As shown, the initial cross-regularized segmentation synthesis model is optimized based on the comparison results, the error between the semantic segmentation prediction result and the segmentation ground truth image, and the error between the edge prediction result and the edge ground truth image to obtain the cross-regularized segmentation synthesis model, thereby obtaining the semantic segmentation network model, including:

[0106] S1001: Establish a first loss function based on the error between the semantic segmentation prediction result and the segmentation ground truth image.

[0107] Specifically, a first loss function L is established based on the error between the semantic segmentation prediction result and the ground truth segmentation value. se :

[0108]

[0109] Where C represents the number of pavement distress types, and for each type of distress, y represents the probability that a pixel in the semantic segmentation prediction result is predicted to belong to the current disease category. c To segment the actual disease category of pixels in the ground truth image, y cThe value can be 0 or 1. 1 can represent that the pixel belongs to the current category, and 0 can represent that the pixel does not belong to the current category.

[0110] S1002: Establish a second loss function based on the error between the edge prediction result and the ground truth edge image.

[0111] Specifically, a second loss function L is established based on the error between the edge prediction result and the ground truth edge image. ed :

[0112]

[0113]

[0114] in, Let L represent the probability that a pixel in the edge prediction result is predicted as an edge. α is a scaling parameter, representing the ratio of positive to negative samples. Positive samples are pixels at the edges of pavement defects, and negative samples are pixels at the edges of non-pavement defects. m is the number of positive samples (pixels at the edges of pavement defects), and n is the number of negative samples (pixels at the edges of non-pavement defects). The ground truth value for the probability that a pixel in the ground truth image is an edge is set to 1. Therefore, when the probability that a pixel in the edge prediction result is predicted as an edge approaches 1, the loss function L... ed If the probability of a pixel in the edge prediction result being predicted as an edge approaches 0, then the loss function L... ed The value of should tend towards positive infinity.

[0115] S1003: Optimize the first loss function and the second loss function, and update the parameters of the initial semantic segmentation network model and the initial edge prediction network model in combination with the comparison results to obtain a cross-regularized segmentation comprehensive model, thereby obtaining a semantic segmentation network model.

[0116] Specifically, after obtaining the first loss function and the second loss function, the comparison result L obtained in the aforementioned steps is combined with the optimization of the first and second loss functions. re L re and L re,i The parameters of the initial semantic segmentation network model and the initial edge prediction network model are updated until the initial semantic segmentation network model converges, or until the preset number of iterations is reached to obtain the cross-regularized segmentation synthesis model, thus obtaining the semantic segmentation network model. This semantic segmentation network model can then be directly used for the prediction of road surface defects.

[0117] The semantic segmentation network model establishment method for road surface detection provided by this invention trains and optimizes an initial cross-regularized segmentation comprehensive model to obtain a semantic segmentation network model. During the training process, the edge prediction network model is used as an aid, and the cross-regularization module provides constraints, so that the obtained semantic segmentation network model can identify multiple types of road surface defects with high recognition accuracy and better robustness.

[0118] Figure 11 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 11 As shown, the electronic device may include: a processor 1110, a communications interface 1120, a memory 1130, and a communications bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other through the communications bus 1140. Processor 1110 can call logical instructions in memory 1130 to execute a method for establishing a semantic segmentation network model for road surface detection. This method includes: establishing an initial cross-regularized segmentation synthesis model, which includes an initial semantic segmentation network model, an initial edge prediction network model, and a cross-regularization module; establishing a training dataset, which includes training images, ground truth segmentation images corresponding to the training images, and ground truth edge images corresponding to the training images; training the initial semantic segmentation network model and the initial edge prediction network model based on the training dataset, and outputting semantic segmentation prediction results and edge prediction results; comparing edge information in the semantic segmentation prediction results with the ground truth edge images using the cross-regularization module, comparing edge information in the ground truth segmentation images with the edge prediction results, and comparing the semantic segmentation prediction results with the edge prediction results; and optimizing the initial cross-regularized segmentation synthesis model based on the comparison results, the error between the semantic segmentation prediction results and the ground truth segmentation images, and the error between the edge prediction results and the ground truth edge images to obtain a cross-regularized segmentation synthesis model, thereby obtaining a semantic segmentation network model.

[0119] Furthermore, the logical instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0120] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the semantic segmentation network model establishment method for road detection provided by the above methods. The method includes: establishing an initial cross-regularized segmentation synthesis model, which includes an initial semantic segmentation network model, an initial edge prediction network model, and a cross-regularization module; establishing a training dataset, which includes training images, segmentation ground truth images corresponding to the training images, and edge ground truth images corresponding to the training images; and based on the training dataset... The initial semantic segmentation network model and the initial edge prediction network model are trained separately, outputting semantic segmentation prediction results and edge prediction results. The cross-regularization module is used to compare the edge information in the semantic segmentation prediction results with the ground truth edge image, compare the edge information in the ground truth segmentation image with the edge prediction results, and compare the semantic segmentation prediction results with the edge prediction results. Based on the comparison results, the error between the semantic segmentation prediction results and the ground truth segmentation image, and the error between the edge prediction results and the ground truth edge image, the initial cross-regularized segmentation synthesis model is optimized to obtain the cross-regularized segmentation synthesis model, thereby obtaining the semantic segmentation network model.

[0121] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for establishing a semantic segmentation network model for road surface detection provided by the methods described above. This method includes: establishing an initial cross-regularized segmentation synthesis model, the initial cross-regularized segmentation synthesis model including an initial semantic segmentation network model, an initial edge prediction network model, and a cross-regularization module; establishing a training dataset, the training dataset including training images, segmentation ground truth images corresponding to the training images, and edge ground truth images corresponding to the training images; and applying the training dataset to the initial semantic segmentation network model respectively. The initial edge prediction network model is trained to output semantic segmentation prediction results and edge prediction results. The cross-regularization module is used to compare the edge information in the semantic segmentation prediction results with the ground truth edge image, compare the edge information in the ground truth segmentation image with the edge prediction results, and compare the semantic segmentation prediction results with the edge prediction results. Based on the comparison results, the error between the semantic segmentation prediction results and the ground truth segmentation image, and the error between the edge prediction results and the ground truth edge image, the initial cross-regularized segmentation synthesis model is optimized to obtain a cross-regularized segmentation synthesis model, thereby obtaining the semantic segmentation network model.

[0122] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for establishing a semantic segmentation network model for road surface detection, characterized in that, include: An initial cross-regularized segmentation synthesis model is established, which includes an initial semantic segmentation network model, an initial edge prediction network model, and a cross-regularization module; Establish a training dataset, which includes training images, segmentation ground truth images corresponding to the training images, and edge ground truth images corresponding to the training images; The initial semantic segmentation network model and the initial edge prediction network model are trained based on the training dataset, and the semantic segmentation prediction result and the edge prediction result are output. The cross-regularization module is used to compare the edge information in the semantic segmentation prediction result with the ground truth edge image, compare the edge information in the ground truth segmentation image with the edge prediction result, and compare the semantic segmentation prediction result with the edge prediction result. Based on the comparison results, the error between the semantic segmentation prediction result and the segmentation ground truth image, and the error between the edge prediction result and the edge ground truth image, the initial cross-regularized segmentation synthesis model is optimized to obtain the cross-regularized segmentation synthesis model, thereby obtaining the semantic segmentation network model. The cross-regularization module includes a first branch, a second branch, and a third branch; The step of comparing the edge information in the semantic segmentation prediction result with the ground truth edge image using the cross-regularization module, comparing the edge information in the ground truth segmentation image with the edge prediction result, and comparing the semantic segmentation prediction result with the edge prediction result includes: The first branch is used to compare the edge information in the semantic segmentation prediction result with the edge ground image to obtain the first loss; A second loss is obtained by comparing the edge information in the segmented ground image with the edge prediction result using a second branch. A third loss is obtained by comparing the semantic segmentation prediction result with the edge prediction result using a third branch.

2. The method for establishing a semantic segmentation network model for road surface detection according to claim 1, characterized in that, The establishment of the training dataset includes: Images of road surface defects were collected as sampling images. The sampled image is data-expanded to obtain the training image; The training images are manually annotated to obtain the segmentation ground truth image and the edge ground truth image; A training dataset is established based on the training images, the segmentation ground truth images, and the edge ground truth images.

3. The method for establishing a semantic segmentation network model for road surface detection according to claim 1, characterized in that, The initial semantic segmentation network model and the initial edge prediction network model are trained based on the training dataset, and the semantic segmentation prediction result and the edge prediction result are output, including: The semantic segmentation network model is trained using the training images and the ground truth segmentation images in the training dataset, with the training images as input and the semantic segmentation prediction results as output. The edge prediction network model is trained using the training images and ground truth edge images in the training dataset, with the training images as input and the edge prediction results as output.

4. The method for establishing a semantic segmentation network model for road surface detection according to claim 1, characterized in that, The step of comparing the edge information in the semantic segmentation prediction result with the ground truth edge image using the first branch to obtain the first loss includes: The first branch is used to extract edge information from the semantic segmentation prediction result; The edge information extracted from the semantic segmentation prediction result is compared with the ground truth edge image to obtain the first loss.

5. The method for establishing a semantic segmentation network model for road surface detection according to claim 1, characterized in that, The step of comparing the edge information in the segmented ground truth image with the edge prediction result using the second branch to obtain the second loss includes: The second branch is used to extract edge information from the segmented ground truth image; The edge information extracted from the segmentation ground truth image is compared with the edge ground truth image to obtain a second loss.

6. The method for establishing a semantic segmentation network model for road surface detection according to any one of claims 1 to 5, characterized in that, The initial cross-regularized segmentation synthesis model is optimized based on the comparison results, the error between the semantic segmentation prediction result and the ground truth segmentation image, and the error between the edge prediction result and the ground truth edge image to obtain a cross-regularized segmentation synthesis model, thereby obtaining a semantic segmentation network model, including: A first loss function is established based on the error between the semantic segmentation prediction result and the segmentation ground image; A second loss function is established based on the error between the edge prediction result and the ground truth edge image; The first loss function and the second loss function are optimized, and the parameters of the initial semantic segmentation network model and the initial edge prediction network model are updated in combination with the comparison results to obtain a cross-regularized segmentation comprehensive model, thereby obtaining a semantic segmentation network model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the semantic segmentation network model establishment method for road surface detection as described in any one of claims 1 to 6.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the semantic segmentation network model establishment method for road surface detection as described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the semantic segmentation network model establishment method for road surface detection as described in any one of claims 1 to 6.

Images