Tunnel inspection method, device, electronic equipment and storage medium

Abstract

The embodiment of the present invention provides a tunnel inspection method, which obtains the optical flow component and the brightness component corresponding to the captured image in the target tunnel; combines the optical flow component and the brightness component with the color component of the captured image to obtain the target image; processes the target image through a trained defect recognition model to obtain the defect recognition result of the target tunnel; and determines the inspection result of the target tunnel based on the defect recognition result of the target tunnel. By combining the optical flow component and the brightness component of the captured image with the color component of the captured image, the target image has the dynamic information of the optical flow component and the static information of the brightness component, and uses the dynamic information of the optical flow component and the static information of the brightness component to assist the defect recognition model in identifying road defects, thereby improving the accuracy of the defect recognition result, and further improving the accuracy of the tunnel inspection result.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a tunnel inspection method, device, electronic equipment and storage medium. Background Art

[0002] With the implementation of artificial intelligence in the transportation field, visual technology can be used to replace manual inspections for road defects. A road defect inspection vehicle is a vehicle equipped with an image sensor and an image processor. It uses image sensors to collect images or point cloud data of the road for image recognition and processing, thereby identifying road defects on the road. However, when inspecting in a tunnel, due to the weak light in the tunnel, if image data is used for defect identification, the light of the acquired image will be weak due to the lighting problem in the tunnel. If the acquired image is used directly for defect identification, the accuracy of the defect identification result will be low, thereby reducing the accuracy of the tunnel inspection result. Summary of the invention

[0003] The embodiment of the present invention provides a tunnel inspection method, which aims to solve the problem that when inspecting defects in tunnels, the light in the tunnel is weak, and if the collected images are directly used for defect identification, the accuracy of the defect identification result will be low. By combining the optical flow component and the brightness component of the captured image with the color component of the captured image, the target image has the dynamic information of the optical flow component and the static information of the brightness component, and the dynamic information of the optical flow component and the static information of the brightness component are used to assist the defect identification model in identifying road defects, thereby improving the accuracy of the defect identification result, and further improving the accuracy of the tunnel inspection result.

[0004] In a first aspect, an embodiment of the present invention provides a tunnel inspection method, the method comprising:

[0005] Obtain the optical flow component and brightness component corresponding to the image taken in the target tunnel;

[0006] Combining the optical flow component and the brightness component with the color component of the captured image to obtain a target image;

[0007] Processing the target image by using a trained defect recognition model to obtain a defect recognition result of the target tunnel;

[0008] The inspection result of the target tunnel is determined based on the disease identification result of the target tunnel.

[0009] Optionally, before acquiring the optical flow component and the brightness component of the image captured in the target tunnel, the method further includes:

[0010] Collect road sign information before the target tunnel;

[0011] Determining shooting parameters of the target tunnel according to the road sign information;

[0012] After entering the target tunnel, the target tunnel is photographed using the photographing parameters.

[0013] Optionally, the step of acquiring an optical flow component and a brightness component of an image captured in a target tunnel includes:

[0014] Acquire images shot inside the target tunnel and images shot before;

[0015] Extracting an optical flow component between the previous captured image and the captured image to obtain an optical flow component corresponding to the captured image;

[0016] The captured image is converted into an HSV color mode, and a brightness component corresponding to the captured image is determined based on the HSV color mode.

[0017] Optionally, determining the brightness component corresponding to the captured image based on the HSV color model includes:

[0018] determining a first brightness component of the captured image based on the HSV color model;

[0019] A second brightness component of the captured image is predicted according to the first brightness component, and the second brightness component is determined as a brightness component corresponding to the captured image.

[0020] Optionally, the captured image is in RGB color mode, and combining the optical flow component and the brightness component with the color component of the captured image to obtain a target image includes:

[0021] splicing the optical flow component and the brightness component to the color component of the captured image to obtain a spliced ​​image;

[0022] The spliced ​​images are subjected to convolution fusion processing using a preset convolution kernel to obtain a target image.

[0023] Optionally, before processing the target image by using the trained disease recognition model to obtain the disease recognition result of the target tunnel, the method further includes:

[0024] Acquire a first data set and a disease recognition model to be trained, wherein the first data set includes sample images and disease labels corresponding to the sample images, and the sample images are acquired in the same manner as the target images;

[0025] Performing supervised training on the disease recognition model to be trained using the first data set;

[0026] After the training is completed, the trained disease recognition model is obtained.

[0027] Optionally, the determining the inspection result of the target tunnel based on the disease identification result of the target tunnel includes:

[0028] Determining a comprehensive disease score of the target tunnel based on the disease identification result of the target tunnel;

[0029] The inspection result of the target tunnel is determined according to the comprehensive disease score.

[0030] In a second aspect, an embodiment of the present invention further provides a tunnel inspection device, the tunnel inspection device comprising:

[0031] A first acquisition module is used to acquire the optical flow component and the brightness component corresponding to the image captured in the target tunnel;

[0032] A combining module, used for combining the optical flow component and the brightness component with the color component of the captured image to obtain a target image;

[0033] A first processing module is used to process the target image by using a trained defect recognition model to obtain a defect recognition result of the target tunnel;

[0034] The first determination module is used to determine the inspection result of the target tunnel based on the disease identification result of the target tunnel.

[0035] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the tunnel inspection method provided in the embodiment of the present invention when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the tunnel inspection method provided in the embodiment of the invention are implemented.

[0037] In an embodiment of the present invention, the optical flow component and the brightness component corresponding to the captured image in the target tunnel are obtained; the optical flow component and the brightness component are combined with the color component of the captured image to obtain the target image; the target image is processed by a trained defect recognition model to obtain the defect recognition result of the target tunnel; and the inspection result of the target tunnel is determined based on the defect recognition result of the target tunnel. By combining the optical flow component and the brightness component of the captured image with the color component of the captured image, the target image has the dynamic information of the optical flow component and the static information of the brightness component, and the dynamic information of the optical flow component and the static information of the brightness component are used to assist the defect recognition model in identifying road defects, thereby improving the accuracy of the defect recognition result, and further improving the accuracy of the tunnel inspection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a flow chart of a tunnel inspection method provided by an embodiment of the present invention;

[0040] Figure 2 is a schematic structural diagram of a tunnel inspection device provided in an embodiment of the present invention;

[0041] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] like Figure 1 As shown, Figure 1 1 is a method flow chart of a tunnel inspection method provided by an embodiment of the present invention. The tunnel inspection method comprises the following steps:

[0044] 101. Obtain an optical flow component and a brightness component corresponding to an image captured in a target tunnel.

[0045] In an embodiment of the present invention, the tunnel may be a tunnel for vehicles to pass through, the tunnel inspection method may be mounted on a server or an edge end, the edge end may be an electronic device for image processing and analysis on an inspection vehicle, and the inspection vehicle is provided with an image capture device, which can be used to capture the road surface and arch surface of the tunnel to obtain corresponding captured images. It should be noted that when the inspection vehicle is driving, the image capture device will capture at a certain frame rate to obtain continuous captured images.

[0046] The above-mentioned target tunnel may be the tunnel in which the inspection vehicle is currently located. In this case, the image captured in the target tunnel may be the image captured by the inspection vehicle in real time; or the above-mentioned target tunnel may be a tunnel selected by the user through an interactive interface. In this case, the image captured in the target tunnel may be a historical image of the tunnel selected by the user, and the historical image is a historical image captured by the inspection vehicle in the selected tunnel.

[0047] After capturing the image, the image capture device can transmit the image to a server or edge terminal through a data transmission protocol for image processing and analysis.

[0048] After the server or edge terminal obtains the image shot in the target tunnel, it can extract the optical flow component and brightness component corresponding to the shot image through image processing technology, and use the optical flow component and brightness component corresponding to the shot image to assist in the identification of defects in tunnel inspection.

[0049] In a possible embodiment, after receiving the image captured by the image capture device, the above-mentioned edge end extracts the optical flow component and the brightness component corresponding to the captured image through image processing technology, and uploads the captured image and the corresponding optical flow component and brightness component to the server. The server assists in the identification of defects in tunnel inspection through the optical flow component and the brightness component corresponding to the captured image.

[0050] The optical flow component corresponding to the above-mentioned captured image can represent the changes of each pixel point from the previous captured image to the current captured image. The Lucas-Kanade algorithm, Farneback algorithm, and PyrLK algorithm can be used to extract the optical flow information between the previous captured image and the current captured image as the optical flow component corresponding to the captured image.

[0051] The brightness component corresponding to the captured image may be determined based on a grayscale image of the captured image, or may be obtained by converting the captured image into an HSV color mode.

[0052] It should be noted that in a grayscale image, the grayscale value can be represented by an integer between 0 and 255. Different grayscale values ​​represent different brightness. The larger the grayscale value, the higher the brightness. 0 represents black, which is the lowest brightness, and 255 represents white, which is the highest brightness. Specifically, the captured image can be converted into a grayscale image, and the grayscale image is used as the brightness component corresponding to the captured image.

[0053] In the HSV color mode, the color of each pixel can be represented as a triplet (H, S, V), where H represents hue, S represents saturation, and V represents brightness. In the HSV color mode, the brightness component of the image can be represented by the V component, which can also be called the V channel. Specifically, the captured image is converted into the HSV color mode, and the V component is extracted in the HSV color mode as the brightness component corresponding to the captured image.

[0054] 102. Combine the optical flow component and the brightness component with the color component of the captured image to obtain a target image.

[0055] In an embodiment of the present invention, the captured image includes multiple color components, different color modes correspond to different color components, and one color component can also be called a channel. The color modes of the image include RGB color mode, HSV color mode, CMYK color mode, LAB color mode, etc. Among them, the color components corresponding to the RGB color mode are R component, G component and B component, R component represents the red component, G component represents the green component, and B component represents the blue component. The color components corresponding to the HSV color mode are H component, S component and V component, wherein the H component represents the hue component, the S component represents the saturation component, and the V component represents the brightness component. The color components corresponding to the CMYK color mode are C component, M component, Y component and K component, wherein the C component represents the cyan component, the M component represents the magenta component, the Y component represents the yellow component, and the K component represents the black component. The color components corresponding to the LAB color mode are L component, A component and B component, wherein the L component represents the brightness component, the A component represents the green / red component, and the B component represents the blue / yellow component. In the embodiment of the present invention, the color mode of the captured image is preferably the RGB color mode, because the RGB color mode can provide rich color expression capabilities, so that the captured image has more image details.

[0056] Taking the color mode of the captured image as the RGB color mode for illustration, after obtaining the optical flow component and brightness component corresponding to the captured image, the server or edge end can splice the optical flow component and brightness component with the color component of the captured image to obtain a target image, in which case the target image includes the RGB color component, the optical flow component, and the brightness component. In a possible embodiment, the optical flow component and the brightness component can be superimposed on the color component of the captured image to obtain a target image, in which case the target image includes the superimposed RGB color component.

[0057] 103. The target image is processed by the trained defect recognition model to obtain the defect recognition result of the target tunnel.

[0058] In an embodiment of the present invention, the above-mentioned disease recognition model is used to identify the type, location and degree of disease in the target image. The above-mentioned disease recognition model can be a deep convolutional neural network. Specifically, the above-mentioned disease recognition model can be a model constructed based on deep convolutional neural networks such as ResNet, YOLOV, Faster R-CNN, SSD, RetinaNet, etc.

[0059] The above-mentioned disease recognition model can be trained with a pre-prepared data set, and a trained disease recognition model is obtained after the training is completed. The above-mentioned trained disease recognition model is deployed on a server or an edge end. After the target image is obtained on the server or the edge end, the target image is input into the disease recognition model for processing. The disease recognition model outputs the disease information such as the disease type, disease location and disease degree existing in the target image as the disease recognition result.

[0060] The trained disease recognition model outputs a disease detection frame (x, y, w, h, r, u, v), where (x, y) represents the center position of the disease detection frame, w represents the width of the disease detection frame, h represents the height of the disease detection frame, r represents the confidence of the disease detection frame, u represents the disease type in the disease detection frame, and v represents the degree of the disease in the disease detection frame.

[0061] The above-mentioned disease type can be determined by u in the disease detection frame, and the disease type may include pavement disease and arch surface disease, wherein the above-mentioned pavement disease can be understood as a disease that occurs on the road surface, and the above-mentioned arch surface disease can be understood as a disease that occurs on the inner surface of the arch hole. Specifically, the above-mentioned road surface disease can be road cracks, potholes, wavy road surface, settlement and other diseases that affect the life of the road surface and traffic safety, and the above-mentioned arch surface disease can be surface peeling, cracks, collapse and other diseases that affect the life of the tunnel and tunnel safety.

[0062] The above-mentioned disease position may be the position of the disease in the target image, and the disease position may be represented by the center position of the disease detection frame. In a possible embodiment, each frame of the captured image has a corresponding shooting time and shooting location. After the shooting location of the captured image is determined, the disease position in the target image may be converted to the disease position in the real world through the conversion relationship between the camera coordinate system of the image capturing device and the real world coordinate system.

[0063] The above-mentioned disease degree can be determined by v of the disease detection frame, and the disease degree can be expressed in the form of a score or a level. The higher the score of the disease degree, the more serious the disease, or the higher the disease level, the more serious the disease.

[0064] By performing the defect recognition results on all target images in the target tunnel through the trained defect recognition model, the defect recognition results of the target tunnel can be obtained. The defect recognition results of the target tunnel may include defect information such as the number of defects, defect type, defect location, and defect degree.

[0065] 104. Determine the inspection result of the target tunnel based on the disease identification result of the target tunnel.

[0066] In the embodiment of the present invention, after obtaining the disease identification result of the target tunnel, data analysis can be performed based on the disease identification result of the target tunnel to obtain the inspection result of the target tunnel.

[0067] The inspection result includes at least one of whether it is qualified and the repair level. After obtaining the defect identification result of the target tunnel, the server or edge end can determine whether the inspection result of the target tunnel is qualified according to the defect identification result of the target tunnel. Alternatively, the repair level of the target tunnel can be determined according to the defect identification result of the target tunnel. Alternatively, if it is determined that the inspection result of the target tunnel is unqualified, the repair level of the target tunnel can be further determined.

[0068] Whether the inspection result of the target tunnel is qualified can be determined based on the number of defects, the type of defects, the location of defects, the degree of defects, etc. of the target tunnel. For example, when the number of defects is greater than or equal to the preset number of defects, the inspection result of the target tunnel can be determined to be unqualified, and when the number of defects is less than the preset number of defects, the inspection result of the target tunnel can be determined to be qualified; when the defect type exists in the preset type, the inspection result of the target tunnel can be determined to be unqualified, and when the number of defects does not exist in the preset number of defects, the inspection result of the target tunnel can be determined to be qualified; when the defect location appears in the center of the road surface or the center of the arch surface, the inspection result of the target tunnel can be determined to be unqualified, and when the defect location appears at the edge of the road surface or the edge of the arch surface, the inspection result of the target tunnel can be determined to be qualified; when the degree of defects is greater than or equal to the preset degree of defects, the inspection result of the target tunnel can be determined to be unqualified, and when the degree of defects is less than the preset degree of defects, the inspection result of the target tunnel can be determined to be qualified. The number of defects, the type of defects, the location of defects, and the degree of defects of the target tunnel may also be combined to determine whether the inspection result of the target tunnel is qualified.

[0069] The above-mentioned repair grade can be determined by the degree of damage of the target tunnel. The maximum degree of damage in the target tunnel can be determined as the repair grade of the target tunnel, or the average degree of damage in the target tunnel can be determined as the repair grade of the target tunnel. In a possible embodiment, the repair grade of the target tunnel can also be determined according to the above-mentioned comprehensive damage score. The higher the comprehensive damage score of the target tunnel, the higher the repair grade of the target tunnel, and the lower the comprehensive damage score of the target tunnel, the lower the repair grade of the target tunnel.

[0070] After obtaining the above inspection results, the server or edge terminal can send the inspection results to the management system of the relevant road department or the user terminal, so that the relevant road department or the user can view the inspection results of the target tunnel.

[0071] In the embodiment of the present invention, the optical flow component and the brightness component corresponding to the captured image in the target tunnel are obtained; the optical flow component and the brightness component are combined with the color component of the captured image to obtain the target image; the target image is processed by a trained disease recognition model to obtain the disease recognition result of the target tunnel; and the inspection result of the target tunnel is determined based on the disease recognition result of the target tunnel. By combining the optical flow component and the brightness component of the captured image with the color component of the captured image, the target image has the dynamic information of the optical flow component and the static information of the brightness component, and the dynamic information of the optical flow component and the static information of the brightness component are used to assist the disease recognition model in identifying road diseases, thereby improving the accuracy of the disease recognition result, and further improving the accuracy of the tunnel inspection result.

[0072] It is understandable that in the specific implementation of this application, related data such as images taken inside and outside the tunnel are involved. When the embodiments in this application are applied to specific products or technologies, it is necessary to obtain the permission or consent of the relevant departments, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0073] Optionally, before the step of obtaining the optical flow component and the brightness component of the image captured in the target tunnel, the road sign information in front of the target tunnel may be collected; the shooting parameters of the target tunnel may be determined according to the road sign information; and after entering the target tunnel, the target tunnel may be photographed with the shooting parameters.

[0074] In an embodiment of the present invention, the captured images collected by the inspection vehicle before entering the target tunnel can be obtained, and the captured images collected before entering the target tunnel can be identified to obtain the road sign information before the target tunnel. The above road sign information may include the tunnel name, tunnel speed limit, tunnel length, tunnel height, etc. The above shooting parameters include shooting light compensation, shooting frame rate, shooting resolution, focus position, etc.

[0075] Different road sign information corresponds to different shooting parameters. The association between the road sign information and the shooting parameters can be recorded through a mapping table. After obtaining the road sign information of the target tunnel, the corresponding shooting parameters can be found through the mapping table to obtain the shooting parameters of the target tunnel.

[0076] In a possible embodiment, the shooting parameters can be outputted through a trained convolutional neural network. The road sign information can be inputted into the trained convolutional neural network for encoding, feature extraction and result output to obtain the corresponding shooting parameters. Specifically, the road sign information includes tunnel speed limit, tunnel length, tunnel height, etc. The trained convolutional neural network includes an encoding layer, a convolutional layer and an output layer. The road sign information is encoded through the encoding layer, and the road sign information is encoded from a string type to a numerical type to obtain the encoding features corresponding to the road sign information. The encoding features are convolutionally calculated through the convolutional layer to obtain the feature vector of the road sign information. The feature vector is linearly regressed and classified through the output layer to obtain the corresponding shooting parameters for output. Further, the convolutional neural network collects a second data set for supervised training during training, and the second data set includes sample identification information and shooting parameter labels, wherein the sample identification information includes tunnel speed limit, tunnel length, tunnel height, etc., and the shooting parameter labels include shooting light compensation, shooting frame rate, shooting resolution, focus position, etc. that actually correspond to the sample identification information. The shooting parameter labels can be obtained by manual annotation by relevant professionals. During the training process, the sample identification information is input into the convolutional neural network to be trained. After being processed by the encoding layer, convolutional layer and output layer, the shooting parameter result corresponding to the sample identification information is obtained. The error loss between the shooting parameter result corresponding to the sample identification information and the shooting parameter label corresponding to the sample identification information is calculated. With minimizing the error loss as the optimization goal, the parameters in the convolutional neural network to be trained are adjusted by the error back propagation algorithm. The above error calculation and parameter adjustment process is iterated until the convolutional neural network to be trained converges at the minimum error loss or the number of iterations reaches a preset number. The training is completed to obtain a trained convolutional neural network. After obtaining the trained convolutional neural network, the road sign information of the target tunnel is input into the trained convolutional neural network for processing, and the shooting parameters of the target tunnel are output through the trained convolutional neural network.

[0077] After obtaining the shooting parameters of the target tunnel, the shooting parameters are converted into corresponding control instructions. After the inspection vehicle enters the target tunnel, the corresponding image shooting device is controlled by the control instructions to shoot, so that the image shooting device can shoot the target tunnel with the shooting parameters to obtain the shooting image inside the target tunnel.

[0078] Optionally, in the step of obtaining the optical flow component and the brightness component of the image taken in the target tunnel, the image taken in the target tunnel and the previous image taken can be obtained; the optical flow component between the previous image and the captured image is extracted to obtain the optical flow component corresponding to the captured image; the captured image is converted into the HSV color mode, and the brightness component corresponding to the captured image is determined based on the HSV color mode.

[0079] In an embodiment of the present invention, when the server or edge obtains the image captured in the target tunnel, it also obtains the previous image of the captured image. The previous image of the above-mentioned image captured in the target tunnel can be understood as an image captured at the previous moment, and the captured image and the previous image are two adjacent frames of captured images. The optical flow information between the previous image and the captured image can be extracted as the optical flow component corresponding to the captured image by using the optical flow extraction algorithm such as the Lucas-Kanade algorithm, the Farneback algorithm, and the PyrLK algorithm deployed in the server or edge.

[0080] After the server or edge end obtains the captured image in the target tunnel, the color mode of the captured image can be converted into the HSV color mode through the color mode conversion algorithm, and the corresponding brightness component can be extracted in the HSV color mode as the brightness component corresponding to the captured image. It should be noted that in the HSV color mode, the brightness component of the image can be represented by the V component, and the above-mentioned V component can also be called the V channel. Specifically, the captured image is converted into the HSV color mode, and the V component is extracted in the HSV color mode as the brightness component corresponding to the captured image. Since the resolution size of the image remains unchanged during the color mode conversion process, the extracted brightness component also has the same resolution size as the captured image.

[0081] Optionally, in the step of determining the brightness component corresponding to the captured image based on the HSV color model, the first brightness component of the captured image can also be determined based on the HSV color model; the second brightness component of the captured image is predicted based on the first brightness component, and the second brightness component is determined as the brightness component corresponding to the captured image.

[0082] In an embodiment of the present invention, after the server or edge terminal converts the captured image into the HSV color mode, the V component in the HSV color mode is extracted as the first brightness component of the captured image, the resolution size of the first brightness component is the same as the resolution size of the captured image, and each pixel in the first brightness component corresponds to a pixel with the same coordinates in the captured image, wherein the value of each pixel in the first brightness component represents the brightness value of the corresponding pixel in the captured image.

[0083] After obtaining the first brightness component, the first brightness component may be predicted to obtain a second brightness component of the captured image, wherein the brightness value of each pixel in the second brightness component is greater than the brightness value of each pixel in the first brightness component.

[0084] In one embodiment, in the step of predicting the second brightness component of the captured image based on the first brightness component, the first brightness component can be subjected to singular value decomposition (SVD) or principal component analysis (PCA) to obtain the singular value or principal component of the first brightness component, and then the first brightness component is adjusted by the singular value or principal component to obtain the second brightness component.

[0085] In one embodiment, a generator can be trained to generate a second brightness component based on a first brightness component. Specifically, a generative adversarial network (GAN) can be constructed, and the generative adversarial network includes a generator and a discriminator. It should be noted that the generator attempts to generate samples similar to real data, while the discriminator attempts to distinguish between real data and data generated by the generator. During the training process, the generator and the discriminator compete with each other, and finally the generator can generate realistic samples. A third data set is constructed, and the third data set includes a sample brightness component and a target brightness component. The target brightness component can be understood as the brightness component after the sample brightness component is brightness enhanced, and the target brightness component can also be understood as the V component in the brighter HSV image. The sample brightness component can also be understood as the V component in the darker HSV image. The darker HSV image can be obtained by lowering the brightness of the brighter HSV image, and the brighter HSV image can also be obtained by raising the brightness of the darker HSV image. The sample brightness component has the same resolution size as the target brightness component, and the brightness value of each pixel in the sample brightness component is less than the brightness value of each pixel in the target brightness component. During the training process, the sample brightness component is input into the generator for generation processing to obtain the generated result, and the generated result is input into the discriminator for discrimination processing with the target brightness component. If the discrimination result is false, it indicates that the discriminator determines that the generated result of the generator is not similar to the target brightness component. At this time, the error loss between the generated result and the target brightness component is calculated, and the parameters of the generator are adjusted by taking the minimum error loss as the optimization goal, so that the generator can generate a result more similar to the target brightness component; if the discrimination result is true, it indicates that the discriminator determines that the generated result of the generator is very similar to the target brightness component, then the error loss between the generated result and the target brightness component can be calculated, and the parameters of the discriminator are adjusted by taking the minimum error loss as the optimization goal, so that the generator can improve the discrimination performance, thereby improving the discrimination accuracy of the discriminator for the generated result, iterating the parameter adjustment process of the above generator and discriminator until the number of iterations reaches the preset number, stopping the iteration, and obtaining the trained generator. The first brightness component of the captured image is input into the trained generator, and the corresponding generated result is output by the generator, and the generated result is used as the second brightness component of the captured image.

[0086] After obtaining the second brightness component, the second brightness component can be determined as the brightness component of the captured image. Since the second brightness component has a higher brightness value than the first brightness component, the second brightness component is determined as the brightness component of the captured image. After combining the brightness component with the color component of the captured image, the brightness of the target image can be further improved.

[0087] Optionally, the captured image is in RGB color mode. In the step of combining the optical flow component and the brightness component with the color component of the captured image to obtain the target image, the optical flow component and the brightness component can be spliced ​​with the color component of the captured image to obtain a spliced ​​image; the spliced ​​image is convolutionally fused using a preset convolution kernel to obtain the target image.

[0088] In an embodiment of the present invention, the captured image is preferably in RGB color mode, which can provide rich color expression capabilities, so that the captured image has more image details. After combining the optical flow component and the brightness component, the target image has the dynamic information of the optical flow component and the static information of the brightness component, while retaining rich image details.

[0089] The color components of the captured image include R color component, G color component and B color component. The optical flow component and the brightness component can be spliced ​​before or after the R color component, G color component and B color component, so that the spliced ​​image has 5 components (which can be understood as 5 channels). The R color component, G color component and B color component in the spliced ​​image record the image detail information, the optical flow component records the dynamic information of the brightness, and the brightness component records the static information of the brightness.

[0090] The preset convolution kernel may be a 1*1 convolution kernel, which is used to linearly combine multiple channels to obtain a new channel. The 1*1 convolution kernel may be used to perform convolution operations on the color component, optical flow component, and brightness component in the spliced ​​image, and the multiple components may be fused into a single-channel image, and the single-channel image may be determined as the target image, which incorporates image detail information, dynamic brightness information, and static brightness information.

[0091] Optionally, before the step of processing the target image through the trained disease recognition model to obtain the disease recognition result of the target tunnel, a first data set and a disease recognition model to be trained can also be obtained, the first data set includes sample images and disease labels corresponding to the sample images, and the sample images are obtained in the same way as the target images; the disease recognition model to be trained is supervisedly trained through the first data set; and the trained disease recognition model is obtained after the training is completed.

[0092] In an embodiment of the present invention, the first data set includes a certain number of sample images and disease labels corresponding to the sample images, that is, the sample images are obtained by combining the color component, optical flow component and brightness component of the original image. For the specific combination method, please refer to the combination method of the target image in the above embodiment, which will not be described in detail here. The disease labels corresponding to the sample images can be obtained by annotation by relevant professionals. One sample image can correspond to a group of disease labels, and a group of disease labels includes annotation information such as disease type, disease location, and disease degree.

[0093] During the training process, the sample image is input into the disease recognition model to be trained for processing, and the disease recognition result corresponding to the sample image is output. The error loss between the disease recognition result corresponding to the sample image and the disease label corresponding to the sample image is calculated. The error loss is minimized as the optimization goal. The parameters of the disease recognition model to be trained are adjusted by the error back propagation algorithm. The above error calculation and parameter adjustment process is iterated until the disease recognition model to be trained converges at the minimum error loss or the number of iterations reaches a preset number. The training is completed to obtain a trained disease recognition model. After obtaining the trained disease recognition model, the target image can be input into the trained disease recognition model for processing, and the disease recognition result corresponding to the target image is output. The disease recognition result corresponding to the target image is the disease recognition result of the captured image. Furthermore, the disease recognition results of all captured images in the tunnel can be recorded to obtain the disease recognition result of the target tunnel.

[0094] Optionally, in the step of determining the inspection result of the target tunnel based on the disease identification result of the target tunnel, a comprehensive disease score of the target tunnel can be determined based on the disease identification result of the target tunnel; and the inspection result of the target tunnel can be determined based on the comprehensive disease score.

[0095] In an embodiment of the present invention, the defect identification result of the target tunnel may include defect information such as the number of defects, the type of defects, the location of the defects, and the degree of the defects. After obtaining the defect identification result of the target tunnel, the server or edge end may calculate a comprehensive defect score for the target tunnel based on the number of defects, the type of defects, the location of the defects, and the degree of the defects, and obtain a comprehensive defect score for the target tunnel. The target tunnel may be determined to be qualified based on the comprehensive defect score. If the comprehensive defect score of the target tunnel is greater than or equal to a preset score threshold, it may be indicated that the inspection result of the target tunnel is unqualified. If the comprehensive defect score of the target tunnel is less than the preset score threshold, it may be indicated that the inspection result of the target tunnel is qualified. Specifically, the calculation of the above-mentioned comprehensive defect score S may be performed by the following formula:

[0096]

[0097] Among them, the above a u,i represents the empirical coefficient corresponding to the disease type u of the ith disease. The greater the impact of the disease type on traffic safety, the greater the empirical coefficient corresponding to the disease type. i represents the severity of the i-th disease, and n is the number of diseases. (x i ,y i ) represents the defect position of the ith defect, (x0, y0) represents the center of the road surface or the center of the arch surface, and the above-mentioned road surface center or the arch surface center can be obtained by performing image recognition processing on the target image, and specifically can be obtained by outputting a trained defect recognition model. At this time, the output defect detection frame can be (x, y, x0, y0, w, h, r, u, v). If the defect type in the defect detection frame is a road surface defect, then (x0, y0) represents the center of the road surface; if the defect type in the defect detection frame is an arch surface defect, then (x0, y0) represents the center of the arch surface. It can be seen from the above formula that the calculation of the comprehensive score of the defect is positively correlated with the empirical coefficient of the defect type and the degree of the defect, and is negatively correlated with the distance from the defect position to the center of the road surface or the center of the arch surface.

[0098] It should be noted that the tunnel inspection method provided in the embodiment of the present invention can be applied to photographing equipment, smart phones, computers, servers and other equipment that can perform tunnel inspections.

[0099] like Figure 2 As shown, an embodiment of the present invention provides a tunnel inspection device, the tunnel inspection device comprising:

[0100] The first acquisition module 201 is used to acquire the optical flow component and the brightness component corresponding to the image captured in the target tunnel;

[0101] A combining module 202, configured to combine the optical flow component and the brightness component with the color component of the captured image to obtain a target image;

[0102] The first processing module 203 is used to process the target image by using a trained defect recognition model to obtain a defect recognition result of the target tunnel;

[0103] The first determination module 204 is used to determine the inspection result of the target tunnel based on the disease identification result of the target tunnel.

[0104] Optionally, the device further comprises:

[0105] A collection module, used to collect road sign information in front of the target tunnel;

[0106] A second determination module, configured to determine the shooting parameters of the target tunnel according to the road sign information;

[0107] A shooting module is used to shoot the target tunnel with the shooting parameters after entering the target tunnel.

[0108] Optionally, the first acquisition module 201 includes:

[0109] An acquisition submodule is used to acquire images shot in the target tunnel and images shot previously;

[0110] An extraction submodule, used to extract the optical flow component between the previous captured image and the captured image to obtain the optical flow component corresponding to the captured image;

[0111] The first determination submodule is used to convert the captured image into an HSV color mode, and determine a brightness component corresponding to the captured image based on the HSV color mode.

[0112] Optionally, the first determining submodule includes:

[0113] A first determining unit, configured to determine a first brightness component of the captured image based on the HSV color model;

[0114] The second determining unit is configured to predict a second brightness component of the captured image according to the first brightness component, and determine the second brightness component as a brightness component corresponding to the captured image.

[0115] Optionally, the combining module 202 includes:

[0116] A stitching submodule, used for stitching the optical flow component and the brightness component to the color component of the captured image to obtain a stitched image;

[0117] The processing submodule is used to perform convolution fusion processing on the stitched image through a preset convolution kernel to obtain a target image.

[0118] Optionally, the device further comprises:

[0119] A second acquisition module is used to acquire a first data set and a disease recognition model to be trained, wherein the first data set includes a sample image and a disease label corresponding to the sample image, and the sample image is acquired in the same manner as the target image;

[0120] A training module, used for performing supervised training on the disease recognition model to be trained using the first data set;

[0121] The second processing module is used to complete the training to obtain the trained disease recognition model.

[0122] Optionally, the first determining module 204 includes:

[0123] A second determination submodule is used to determine a comprehensive disease score of the target tunnel based on the disease identification result of the target tunnel;

[0124] The third determination submodule is used to determine the inspection result of the target tunnel according to the comprehensive disease score.

[0125] It should be noted that the tunnel inspection device provided in the embodiment of the present invention can be applied to photographing equipment, smart phones, computers, servers and other equipment that can perform tunnel inspections.

[0126] The tunnel inspection device provided in the embodiment of the present invention can implement each process implemented by the tunnel inspection method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0127] See also Figure 3 , Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 3 As shown, it includes: a memory 302, a processor 301, and a computer program of a tunnel inspection method stored in the memory 302 and executable on the processor 301, wherein:

[0128] The processor 301 is used to call the computer program stored in the memory 302 and execute the following steps:

[0129] Obtain the optical flow component and brightness component corresponding to the image taken in the target tunnel;

[0130] Combining the optical flow component and the brightness component with the color component of the captured image to obtain a target image;

[0131] Processing the target image by using a trained defect recognition model to obtain a defect recognition result of the target tunnel;

[0132] The inspection result of the target tunnel is determined based on the disease identification result of the target tunnel.

[0133] Optionally, before acquiring the optical flow component and the brightness component of the image captured in the target tunnel, the method executed by the processor 301 further includes:

[0134] Collect road sign information before the target tunnel;

[0135] Determining shooting parameters of the target tunnel according to the road sign information;

[0136] After entering the target tunnel, the target tunnel is photographed using the photographing parameters.

[0137] Optionally, the acquiring of the optical flow component and the brightness component of the image captured in the target tunnel performed by the processor 301 includes:

[0138] Acquire images shot inside the target tunnel and images shot before;

[0139] Extracting an optical flow component between the previous captured image and the captured image to obtain an optical flow component corresponding to the captured image;

[0140] The captured image is converted into an HSV color mode, and a brightness component corresponding to the captured image is determined based on the HSV color mode.

[0141] Optionally, the determining, by the processor 301, of a brightness component corresponding to the captured image based on the HSV color model includes:

[0142] determining a first brightness component of the captured image based on the HSV color model;

[0143] A second brightness component of the captured image is predicted according to the first brightness component, and the second brightness component is determined as a brightness component corresponding to the captured image.

[0144] Optionally, the captured image is in RGB color mode, and the processor 301 performs the step of combining the optical flow component and the brightness component with the color component of the captured image to obtain a target image, including:

[0145] splicing the optical flow component and the brightness component to the color component of the captured image to obtain a spliced ​​image;

[0146] The spliced ​​images are subjected to convolution fusion processing using a preset convolution kernel to obtain a target image.

[0147] Optionally, before the target image is processed by the trained disease recognition model to obtain the disease recognition result of the target tunnel, the method executed by the processor 301 further includes:

[0148] Acquire a first data set and a disease recognition model to be trained, wherein the first data set includes sample images and disease labels corresponding to the sample images, and the sample images are acquired in the same manner as the target images;

[0149] Performing supervised training on the disease recognition model to be trained using the first data set;

[0150] After the training is completed, the trained disease recognition model is obtained.

[0151] Optionally, the determining of the inspection result of the target tunnel based on the disease identification result of the target tunnel performed by the processor 301 includes:

[0152] Determining a comprehensive disease score of the target tunnel based on the disease identification result of the target tunnel;

[0153] The inspection result of the target tunnel is determined according to the comprehensive disease score.

[0154] It should be noted that the electronic device provided in the embodiment of the present invention can be applied to smart phones, computers, servers and other devices that can perform tunnel inspection methods.

[0155] The electronic device provided in the embodiment of the present invention can implement each process implemented by the tunnel inspection method in the above method embodiment and can achieve the same beneficial effect. To avoid repetition, it will not be described here.

[0156] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the tunnel inspection method or the application-end tunnel inspection method provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0157] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0158] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A tunnel inspection method, characterized in that: The method comprises the following steps: Obtain the image shot in the target tunnel and the previous image; the color mode of the shot image is RGB color mode; Extracting the optical flow component between the previous captured image and the captured image to obtain the optical flow component corresponding to the captured image, wherein the color component of the captured image includes an R color component, a G color component, and a B color component; Convert the captured image into an HSV color mode, and determine a first brightness component of the captured image based on the HSV color mode; Inputting the first brightness component into a trained generator, outputting a corresponding generation result through the generator, using the generation result as a second brightness component of the captured image, determining the second brightness component as a brightness component corresponding to the captured image, and the brightness value of each pixel in the second brightness component is greater than the brightness value of each pixel in the first brightness component; A generative adversarial network is constructed, the generative adversarial network includes a generator and a discriminator, a third data set is constructed, the third data set includes a sample brightness component and a target brightness component, the target brightness component is a brightness component after the sample brightness component is brightness enhanced, the sample brightness component and the target brightness component have the same resolution size, and the brightness value of each pixel in the sample brightness component is less than the brightness value of each pixel in the target brightness component; during the training process, the sample brightness component is input into the generator for generation processing to obtain a generation result, the generation result is input into the discriminator for discrimination processing with the target brightness component, if the discrimination result is false, the error loss between the generation result and the target brightness component is calculated, and the parameters of the generator are adjusted with the minimum error loss as the optimization goal; if the discrimination result is true, the error loss between the generation result and the target brightness component is calculated, and the parameters of the discriminator are adjusted with the minimum error loss as the optimization goal, the parameter adjustment process of the generator and the discriminator is iterated until the number of iterations reaches a preset number, the iteration is stopped, and a trained generator is obtained; The optical flow component and the brightness component are spliced ​​before or after the R color component, the G color component and the B color component to obtain a spliced ​​image, and the R color component, the G color component, the B color component, the optical flow component and the brightness component in the spliced ​​image are convoluted and fused by a preset convolution kernel to obtain a target image; Processing the target image by using a trained defect recognition model to obtain a defect recognition result of the target tunnel; The inspection result of the target tunnel is determined based on the disease identification result of the target tunnel.

2. The tunnel inspection method according to claim 1, characterized in that: Before acquiring the optical flow component and the brightness component of the image captured in the target tunnel, the method further includes: Collect road sign information before the target tunnel; Determining shooting parameters of the target tunnel according to the road sign information; After entering the target tunnel, the target tunnel is photographed using the photographing parameters.

3. The tunnel inspection method according to claim 1 or 2, characterized in that: Before the target image is processed by the trained disease recognition model to obtain the disease recognition result of the target tunnel, the method further includes: Acquire a first data set and a disease recognition model to be trained, wherein the first data set includes sample images and disease labels corresponding to the sample images, and the sample images are acquired in the same manner as the target images; Performing supervised training on the disease recognition model to be trained using the first data set; After the training is completed, the trained disease recognition model is obtained.

4. The tunnel inspection method according to claim 3, characterized in that: The determining the inspection result of the target tunnel based on the disease identification result of the target tunnel includes: Determining a comprehensive disease score of the target tunnel based on the disease identification result of the target tunnel; The inspection result of the target tunnel is determined according to the comprehensive disease score.

5. A tunnel inspection device, characterized in that: The tunnel inspection device comprises: The first acquisition module is used to acquire the captured image in the target tunnel and the previous captured image; the color mode of the captured image is RGB color mode, and the color components of the captured image include R color component, G color component and B color component; the optical flow component between the previous captured image and the captured image is extracted to obtain the optical flow component corresponding to the captured image; the captured image is converted into HSV color mode, and the first brightness component of the captured image is determined based on the HSV color mode; the first brightness component is input into the trained generator, and the corresponding generation result is output by the generator, and the generation result is used as the second brightness component of the captured image, and the second brightness component is determined as the brightness component corresponding to the captured image, and the brightness value of each pixel in the second brightness component is greater than the brightness value of each pixel in the first brightness component; a generative adversarial network is constructed, and the generative adversarial network includes a generator and a discriminator, and a third data The third data set includes a sample brightness component and a target brightness component, the target brightness component is a brightness component after the sample brightness component is brightness enhanced, the sample brightness component and the target brightness component have the same resolution size, and the brightness value of each pixel in the sample brightness component is less than the brightness value of each pixel in the target brightness component; in the training process, the sample brightness component is input into the generator for generation processing to obtain a generation result, and the generation result is input into the discriminator for discrimination processing with the target brightness component. If the discrimination result is false, the error loss between the generation result and the target brightness component is calculated, and the parameters of the generator are adjusted with the minimum error loss as the optimization goal; if the discrimination result is true, the error loss between the generation result and the target brightness component is calculated, and the parameters of the discriminator are adjusted with the minimum error loss as the optimization goal, and the parameter adjustment process of the generator and the discriminator is iterated until the number of iterations reaches a preset number, and the iteration is stopped to obtain a trained generator; A combining module is used to splice the optical flow component and the brightness component before or after the R color component, the G color component and the B color component to obtain a spliced ​​image, and to perform convolution fusion processing on the R color component, the G color component, the B color component, the optical flow component and the brightness component in the spliced ​​image through a preset convolution kernel to obtain a target image; A first processing module is used to process the target image by using a trained disease recognition model to obtain a disease recognition result of the target tunnel; The first determination module is used to determine the inspection result of the target tunnel based on the disease identification result of the target tunnel.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the tunnel inspection method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the tunnel inspection method according to any one of claims 1 to 4 are implemented.

Images