A runway surface glue detection method and device, computer equipment and storage medium

Abstract

This invention provides a method for detecting adhesive buildup on the surface of a running track, comprising: acquiring an HSV image corresponding to the running track to be tested; performing a first image processing on the HSV image to obtain a first intermediate image identifying a feature region; the feature region consisting of undifferentiated adhesive buildup areas and interference areas; performing a second image processing on the HSV image to obtain a second intermediate image identifying interference areas; and determining the target information of the adhesive buildup on the surface of the running track to be tested based on the first and second intermediate images. A running track surface adhesive buildup detection device, a computer device, and a storage medium are also provided. Using the HSV color model, and through relevant image processing on the HSV image, the target information of the adhesive buildup on the surface of the running track to be tested is finally determined; compared with manual visual inspection, this method can greatly improve detection efficiency, reduce labor costs, and accurately determine the target information such as the area of ​​the adhesive buildup on the surface.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a runway surface rubber deposit detection method and device, computer equipment and a storage medium. BACKGROUND

[0002] The surface rubber deposit on the runway refers to the black mark left by the airplane tire on the runway. When the rubber deposit is too much, it will affect the anti-skid performance of the runway, and the airplane tire is difficult to effectively grip the ground, resulting in skidding and other situations, and eventually causing serious accidents. Therefore, it is necessary to frequently detect the surface rubber deposit on the runway to clean it in time and avoid serious accidents.

[0003] At present, the detection of the surface rubber deposit mostly adopts the manual visual method, but this method is low in efficiency, high in cost, and cannot determine the area of the surface rubber deposit. SUMMARY

[0004] In view of the deficiencies in the prior art, the present application provides a runway surface rubber deposit detection method and device, computer equipment and a storage medium.

[0005] In a first aspect, in one embodiment, the present application provides a runway surface rubber deposit detection method, comprising:

[0006] obtaining an HSV image corresponding to a runway to be detected;

[0007] performing first image processing on the HSV image to obtain a first intermediate image in which a feature region is determined; the feature region is composed of an undifferentiated rubber deposit region and an interference region;

[0008] performing second image processing on the HSV image to obtain a second intermediate image in which the interference region is determined;

[0009] determining target information of the surface rubber deposit of the runway to be detected according to the first intermediate image and the second intermediate image.

[0010] In one embodiment, the first image processing on the HSV image comprises:

[0011] performing first binarization processing on the HSV image according to brightness.

[0012] In one embodiment, the second image processing on the HSV image to obtain the second intermediate image in which the interference region is determined comprises:

[0013] performing second binarization processing on the HSV image according to hue, saturation and brightness to obtain the second intermediate image.

[0014] In one embodiment, the second binarization processing on the HSV image to obtain the second intermediate image comprises:

[0015] performing second binarization processing on the HSV image to obtain a third intermediate image;

[0016] performing denoising processing on the third intermediate image to obtain a second intermediate image.

[0017] In one embodiment, the denoising processing on the third intermediate image to obtain the second intermediate image comprises:

[0018] obtaining a trained denoising model;

[0019] inputting the third intermediate image into the denoising model to obtain the second intermediate image output by the denoising model.

[0020] In one embodiment, the target information of the surface rubber of the runway to be detected is determined according to the first intermediate image and the second intermediate image, comprising:

[0021] obtaining a target image in which the rubber accumulation area is determined according to the first intermediate image and the second intermediate image;

[0022] determining the target information of the surface rubber of the runway to be detected according to the target image.

[0023] In one embodiment, the target image in which the rubber accumulation area is determined is obtained according to the first intermediate image and the second intermediate image, comprising:

[0024] determining a pixel point difference value image of the first intermediate image and the second intermediate image;

[0025] obtaining the target image according to the pixel point difference value image.

[0026] In a second aspect, in one embodiment, the present application provides a runway surface rubber detection device, comprising:

[0027] an image acquisition module configured to acquire an HSV image corresponding to a runway to be detected;

[0028] a first processing module configured to perform first image processing on the HSV image to obtain a first intermediate image in which a feature area is determined; the feature area is composed of an undifferentiated rubber accumulation area and an interference area;

[0029] a second processing module configured to perform second image processing on the HSV image to obtain a second intermediate image in which the interference area is determined;

[0030] an information determination module configured to determine target information of surface rubber of the runway to be detected according to the first intermediate image and the second intermediate image.

[0031] In a third aspect, in one embodiment, the present application provides a computer device, comprising a memory and a processor; the memory stores a computer program, and the processor is configured to execute the computer program stored in the memory to perform the steps in the runway surface rubber detection method in any one of the above embodiments.

[0032] In a fourth aspect, in one embodiment, the present application provides a storage medium, which stores a computer program, and the computer program is loaded by a processor to perform the steps in the runway surface rubber detection method in any one of the above embodiments.

[0033] By the above runway surface rubber detection method, device, computer device and storage medium, the HSV color model is adopted, the target information of the runway surface rubber to be detected is finally determined through relevant image processing on the HSV image, and compared with the manual visual method, the detection efficiency can be greatly improved, the labor cost can be reduced, and the target information such as the area of the surface rubber can be accurately determined. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0035] Figure 1 The flowchart of the runway surface rubber detection method in one embodiment of the present application is shown in the figure.

[0036] Figure 2 The specific flowchart of the first image processing in one embodiment of the present application is shown in the figure.

[0037] Figure 3 The schematic diagram of the first gray scale image in one embodiment of the present application is shown in the figure.

[0038] Figure 4 The specific flowchart of the second image processing in one embodiment of the present application is shown in the figure.

[0039] Figure 5 The schematic diagram of the second gray scale image in one embodiment of the present application is shown in the figure.

[0040] Figure 6 The specific flowchart of the second image processing containing the denoising processing in one embodiment of the present application is shown in the figure.

[0041] Figure 7 The schematic diagram of the third gray scale image in one embodiment of the present application is shown in the figure.

[0042] Figure 8A specific flowchart of the de-noising process in one embodiment of the present application is shown in the figure below.

[0043] Figure 9 A specific flowchart of the target information determination in one embodiment of the present application is shown in the figure below.

[0044] Figure 10 A schematic diagram of the fourth gray scale map in one embodiment of the present application is shown in the figure below.

[0045] Figure 11 A specific flowchart of the target image obtaining in one embodiment of the present application is shown in the figure below.

[0046] Figure 12 A structural schematic diagram of the runway surface rubber accumulation detection device in one embodiment of the present application is shown in the figure below.

[0047] Figure 13 An internal structural schematic diagram of the computer device in one embodiment of the present application is shown in the figure below.

[0048] In the above figures, 1 is a feature area, 11 is a rubber accumulation area, 12 is an interference area, 2 is a first other area, 3 is a second other area, and 4 is a third other area. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0050] In addition, the terms “first” and “second” are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with “first” and “second” can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of “multiple” is two or more, unless otherwise specifically limited.

[0051] In one embodiment, the present application provides a runway surface rubber accumulation detection method, as shown in the figure below. Figure 1

[0052] ​Step 101: Obtain the HSV image corresponding to the runway to be detected. Generally, images obtained directly from a camera are based on the RGB (Red, Green, Blue) color model. However, in the RGB color model, the color gamut changes with light intensity; specifically, the higher the light intensity, the greater the corresponding color gamut deviation. For example, in image areas with high light intensity, the color contrast is high and the segmentation is clear; in image areas with low light intensity, the color contrast is low, appearing dark, or even completely black. In the HSV (Hue, Saturation, Value) color model, light intensity only affects brightness, not hue or saturation. Therefore, the HSV color model can greatly reduce the impact of light.

[0053] In this embodiment, the main focus is on processing images of airport runways. Since airport runways are located outdoors and are largely unobstructed, they are subject to significant light exposure. If the RGB color model is used, the color gamut of the image varies at different times. This leads to multiple different processing results when using the same color gamut for image segmentation under different light intensities during subsequent processing. This demonstrates that the RGB color model cannot be used correctly in this scenario. Therefore, this embodiment requires an HSV image based on the HSV color model as the basis for subsequent image processing. Specifically, if the directly captured image is an RGB image, then... To further convert it to an HSV image (the principle of converting RGB color model to HSV color model: for any coordinate point in the image, its RGB color space is (R,G,B) and its HSV color space is (H,S,V). First, the R, G, and B values ​​need to be converted to between 0 and 1. Then, the H, S, and V values ​​are calculated. If the calculated H value is less than 0, 360 is added to the value to get the final H value. Since OpenCV needs to visualize HSV images, each value also needs to be converted to between 0 and 255). If, under certain conditions, the directly captured image can be made into an HSV image, then no conversion is needed.

[0054] Step 102: Perform first image processing on the HSV image to obtain a first intermediate image with the feature regions identified; the feature regions consist of undifferentiated glue accumulation regions and interference regions.

[0055] In this step, due to the black marks left by the rubber-coated aircraft tires on the surface, it is impossible to find a parameter range specifically for the rubber-coated area in the HSV image. Therefore, it is necessary to determine a relatively broad first parameter range in this step, so as to find as many rubber-coated areas as possible while allowing for interference areas. Specifically, all pixels in the HSV image are filtered and classified according to the determined first parameter range. The region corresponding to the set of pixels whose parameter values ​​are within the first parameter range is determined as the feature region, which is to obtain a first intermediate image with the feature region determined. Since the rubber-coated area and the interference area in the feature region are both based on the same parameter range, the rubber-coated area and the interference area are not distinguished at this time.

[0056] It is important to note that when acquiring images of the track to be inspected, a mobile detection device (including a mobile cart or robot) equipped with a camera is typically used. This device moves along the track and acquires images during this process (images can be acquired at fixed intervals). Inevitably, interference areas (mainly the grassy areas along the edges of the track) will be captured during this process. Therefore, the identified feature regions will include interference areas, and these interference areas need to be segmented out in subsequent steps. It should also be noted that the HSV image may only correspond to the entire track. A local area of ​​an airport runway, or the entire area, can be captured by a mobile detection device at a time. To make an HSV image correspond to the entire area, multiple captured images need to be stitched together. Then, an HSV image is obtained from the stitched image. (If the captured images are directly based on the HSV color model, the stitched image will directly correspond to the HSV image mentioned above. If the captured images are not based on the HSV color model, such as the RGB color model, a color model conversion is required after stitching, or a color model conversion is performed first and then stitched together to obtain the HSV image mentioned above.)

[0057] Step 103: Perform a second image processing on the HSV image to obtain a second intermediate image in which the interference region is determined;

[0058] The interference region mainly refers to the grassland area. In the HSV image, a second parameter range specific to the interference region can be found. Therefore, in this step, the individual interference region can be determined based on the second parameter range specific to the interference region. Specifically, all pixels in the HSV image are filtered and classified according to the determined second parameter range. The region corresponding to the set of pixels whose parameter values ​​are within the second parameter range is determined as the interference region, which is to obtain a second intermediate image with the interference region determined.

[0059] Step 104: Determine the target information of the surface adhesive on the runway to be detected based on the first intermediate image and the second intermediate image;

[0060] The first intermediate image identifies undifferentiated glue accumulation areas and interference areas, while the second intermediate image identifies individual interference areas. Therefore, individual glue accumulation areas can be determined based on the first and second intermediate images. Specifically, the first and second intermediate images are sequentially matched with the acquired original image. Undifferentiated glue accumulation areas and interference areas are retained first, and then interference areas are excluded. This allows for the determination of target information regarding the surface glue accumulation on the track to be detected based on the individual glue accumulation areas. Target information includes glue accumulation area and distribution. Specifically, determining the glue accumulation area requires determining the initial area of ​​the glue accumulation region in the original image and the size ratio between the original image and the acquired real-world scene. The actual glue accumulation area is obtained based on the initial area and size ratio. The glue accumulation distribution mainly refers to the positional relationship (orientation and distance) between various glue accumulation areas. Therefore, determining the glue accumulation distribution requires determining the initial positional relationship of each glue accumulation area in the original image and the size ratio between the original image and the acquired real-world scene. The actual positional relationship, i.e., the glue accumulation distribution, is obtained based on the initial positional relationship and size ratio.

[0061] In this embodiment, the image processing does not include cropping, stretching, or shrinking, so as to ensure that the size of the obtained intermediate image is consistent with that of the original image, thereby enabling the corresponding target information to be obtained based on the finally determined glue accumulation area.

[0062] The above-mentioned method for detecting adhesive residue on the runway surface employs the HSV color model. By performing relevant image processing on the HSV image, the target information of adhesive residue on the surface of the runway under test can be determined. Compared with manual visual inspection, this method can greatly improve detection efficiency, reduce labor costs, and accurately determine target information such as the area of ​​adhesive residue on the surface.

[0063] like Figure 2 As shown, in one embodiment, the HSV image undergoes a first image processing step, including:

[0064] Step 201: Perform a first binarization process on the HSV image based on brightness;

[0065] While light intensity affects brightness, the difference in brightness between the glue-covered area and other areas is more pronounced. Only the interference area, primarily composed of grass, shows less difference. Therefore, when segmenting the image using the brightness channel, even with a larger brightness range chosen to account for light intensity, the segmented areas will only include the glue-covered area and the interference area primarily composed of grass. Based on the above analysis, binarization can be performed based on brightness (binarization is for better visualization of the segmented areas and facilitates further image processing). Specifically, all pixels in the HSV image are filtered and classified according to the brightness range. Pixels with brightness within the range are assigned a grayscale value of 255, while pixels outside the range are assigned a grayscale value of 0, thus obtaining the first grayscale image (e.g., ...). Figure 3 (As shown), and use the first grayscale image as the first intermediate image; in Figure 3 The main features include a white (corresponding to a grayscale value of 255) feature region 1 and a black (corresponding to a grayscale value of 0) first other region 2. Feature region 1 contains an undifferentiated glue accumulation region 11 and an interference region 12.

[0066] By directly analyzing the luminance channel in the HSV color model, we can efficiently identify the feature regions containing glue buildup.

[0067] like Figure 4 As shown, in one embodiment, a second image processing step is performed on the HSV image to obtain a second intermediate image in which the interference region is determined, including:

[0068] Step 401: Perform a second binarization process on the HSV image based on hue, saturation, and brightness to obtain a second intermediate image;

[0069] Since the interference area, primarily composed of grassland, has a corresponding parameter range, it can be binarized based on the definition of green, considering hue, saturation, and brightness (the definition of green mainly focuses on hue and corresponding saturation, while brightness serves as a further constraint). Specifically, all pixels in the HSV image are filtered and classified according to the corresponding hue, saturation, and brightness ranges. Pixels whose hue, saturation, and brightness all fall within their respective hue, saturation, and brightness ranges are assigned a grayscale value of 255. Pixels whose hue, saturation, and brightness are outside their respective hue, saturation, and brightness ranges are assigned a grayscale value of 0, thus obtaining a second grayscale image (e.g., ...). Figure 5 (As shown), and use the second grayscale image as the second intermediate image; in Figure 5The main components include a white interference area 12 (corresponding to a grayscale value of 255) and a second, other area 3 (corresponding to a black grayscale value of 0), which are compared to... Figure 3 and Figure 5 The shape and / or area of ​​the second other region 3 differ from that of the first other region 2 because, in the process of determining the separate interference region 12, the glue accumulation region 11 was also classified into the second other region 3.

[0070] like Figure 6 As shown, in one embodiment, the HSV image undergoes a second binarization process to obtain a second intermediate image, including:

[0071] Step 601: Perform a second binarization process on the HSV image to obtain a third intermediate image;

[0072] Step 602: Denoise the third intermediate image to obtain the second intermediate image;

[0073] In this case, because the boundaries of the interference region, mainly composed of grassland, are not smooth, noise is easily generated at the boundaries of the second grayscale image obtained after binarization. Therefore, to improve the accuracy of subsequent processing results, it is necessary to denoise the second grayscale image (by judging whether the colors and distances are similar). That is, the second grayscale image is used as the third intermediate image, and then denoising is performed on the third intermediate image to obtain the third grayscale image (e.g., ...). Figure 7 (As shown), and use the third grayscale image as the second intermediate image; in Figure 7 The image mainly includes a white interference area 12 (corresponding to a grayscale value of 255) and a second other area 3 (corresponding to a black grayscale value of 0), compared with a second grayscale image (e.g., ...). Figure 5 (as shown) and the third grayscale image (as shown) Figure 7 As shown in the figure, it is clear that the boundaries of the third grayscale image are smoother and there is less noise.

[0074] The denoising process can employ common denoising algorithms found in existing technologies, which will not be elaborated upon here.

[0075] Denoising can improve the accuracy of subsequent processing results.

[0076] like Figure 8 As shown, in one embodiment, denoising the third intermediate image to obtain the second intermediate image includes:

[0077] Step 801: Obtain the trained denoising model;

[0078] Step 802: Input the third intermediate image into the denoising model to obtain the second intermediate image output by the denoising model;

[0079] By leveraging the advantages of AI (Artificial Intelligence) algorithms and training and learning with a large amount of data, the denoising model can automatically avoid some errors, thereby greatly improving the denoising accuracy.

[0080] In one embodiment, the training steps of the denoising model include:

[0081] Obtain a training sample set, which includes multiple training samples. Each training sample includes a third intermediate training image and a second intermediate training image. The second intermediate training image is obtained by manually processing the third intermediate training image.

[0082] Obtain a training sample, use the third intermediate training image as the input to the denoising model, use the third intermediate training image as the expected output of the denoising model, and train the denoising model to complete one training cycle.

[0083] Determine the comparison result between the actual output and the expected output of the denoising model. If the comparison result does not meet the requirements, update the model parameters of the denoising model according to the comparison result.

[0084] Obtain the next training sample, and then re-enter the process of using the third intermediate training image as the input to the denoising model and the third intermediate training image as the expected output of the denoising model to train the denoising model until the obtained comparison results meet the requirements. Then stop training and obtain the trained denoising model.

[0085] In one embodiment, the comparison result between the actual output and the expected output of the denoising model is determined. If the comparison result does not meet the requirements, the model parameters of the denoising model are updated according to the comparison result, including:

[0086] Determine the comparison difference between the actual output and the expected output of the denoising model, and calculate the loss value based on the comparison difference;

[0087] If the loss value does not meet the preset convergence condition, the model parameters of the denoising model are updated according to the loss value.

[0088] In one embodiment, after the denoising model has been trained, the method further includes:

[0089] Obtain a test sample set, which includes multiple test samples, each of which includes a third intermediate test image;

[0090] Each test sample is sequentially input into the trained denoising model for testing. The testing process is the same as the actual usage process described above, and will not be repeated here.

[0091] If the test results are unsatisfactory, the model parameters of the denoising model need to be further adjusted.

[0092] In one embodiment, the trained denoising model can be based on a conditional random field model and then trained; in other embodiments, it can also be based on other machine learning models.

[0093] Among them, the Conditional Random Field (CRF) model is a typical discriminative model. It models the target sequence based on the observed sequence and focuses on solving the problem of serialization labeling. The CRF model has the advantages of discriminative models and the characteristics of generative models, which take into account the transition probability between context labels and perform global parameter optimization and decoding in serialization form. It solves the label bias problem that other discriminative models (such as the maximum entropy Markov model) cannot avoid. Therefore, its application in the image denoising step of this embodiment has good performance.

[0094] like Figure 9 As shown, in one embodiment, determining the target information of the surface adhesive on the runway to be detected based on the first intermediate image and the second intermediate image includes:

[0095] Step 901: Based on the first intermediate image and the second intermediate image, obtain the target image of the determined glue accumulation area;

[0096] Among them, the first grayscale image (e.g.) Figure 3 As shown) as the first intermediate image, the third grayscale image (as shown) Figure 7 Taking the second intermediate image as an example, the gray values ​​of each pixel in the first grayscale image and the gray values ​​of each pixel in the third grayscale image can be correlated to obtain the fourth grayscale image (as shown). Figure 10 As shown), and the fourth grayscale image is used as the target image; in Figure 10 The main components include a white (corresponding to a grayscale value of 255) glue accumulation area 11 and a black (corresponding to a grayscale value of 0) third other area 4, for comparison. Figure 3 , Figure 7 and Figure 10 Third other areas 4 ( Figure 10 The shape and / or area of ​​the black portion in the first region 2 are different from those of the other regions. Figure 3 The black part in the middle) and the second other area 3 ( Figure 7 The black part in the text is because, in the process of identifying the individual glue accumulation area 11, the interference area 12 was also classified into the third other area 4;

[0097] Step 902: Determine the target information of the surface adhesive on the runway to be detected based on the target image;

[0098] Since individual gel accumulation regions have already been identified in the target image, the corresponding target information can be directly determined from the target image without further matching with the original image. Specifically, when determining the gel accumulation area, it is necessary to determine the initial area of ​​the gel accumulation region in the target image, as well as the size ratio between the target image and the acquired real scene. Finally, the actual gel accumulation area is obtained based on the initial area and size ratio. The gel accumulation distribution mainly refers to the positional relationship (orientation and distance, etc.) between each gel accumulation region. Therefore, when determining the gel accumulation distribution, it is necessary to determine the initial positional relationship of each gel accumulation region in the target image, as well as the size ratio between the target image and the acquired real scene. Finally, the actual positional relationship, i.e., the gel accumulation distribution, is obtained based on the initial positional relationship and size ratio.

[0099] Determining target information directly from the target image is more efficient and simpler than matching it with the original image.

[0100] like Figure 11 As shown, in one embodiment, obtaining a target image that identifies the glue accumulation area based on a first intermediate image and a second intermediate image includes:

[0101] Step 1101: Determine the pixel difference map between the first intermediate image and the second intermediate image;

[0102] Among them, the first grayscale image (e.g.) Figure 3 As shown) as the first intermediate image, the third grayscale image (as shown) Figure 7 Taking the second intermediate image as an example, the gray values ​​of each pixel in the first grayscale image are subtracted from the gray values ​​of each pixel in the third grayscale image. For example, if a pixel in the first grayscale image is white and the corresponding pixel in the third grayscale image is white, the difference is 255 - 255 = 0; if a pixel in the first grayscale image is white and the corresponding pixel in the third grayscale image is black, the difference is 255 - 0 = 255; if a pixel in the first grayscale image is black and the corresponding pixel in the third grayscale image is white, the difference is 0 - 255 = -255; if a pixel in the first grayscale image is black and the corresponding pixel in the third grayscale image is black, the difference is 0 - 0 = 0. Combining these differences with the corresponding positions yields the pixel difference map.

[0103] Step 1102: Obtain the target image based on the pixel difference map;

[0104] In the pixel difference image, there are negative values. In this case, all negative values ​​in the pixel difference image need to be replaced with 0. Then, the replaced pixel difference image is converted into a corresponding grayscale image according to the corresponding numerical value, i.e., the fourth grayscale image (e.g., ...). Figure 10(as shown), and the fourth grayscale image is used as the target image;

[0105] It should be noted that, in the process of subtracting the third grayscale image from the first grayscale image, since the glue accumulation area 11 in the first grayscale image is white, the corresponding area in the third grayscale image is basically black. Therefore, after subtraction, the difference is 255, which corresponds to white in grayscale value, and the glue accumulation area 11 is still retained. Since the interference area 12 in the first grayscale image is white, the corresponding area (i.e., the interference area) in the third grayscale image is basically white. Therefore, after subtraction, the difference is 0, which corresponds to black in grayscale value, and the interference area 12 is removed. The same applies to other areas, which will not be elaborated here.

[0106] The specific operations involved in this example can be represented by the following formula:

[0107] Pixel difference map = max(0, (grayscale values ​​of each pixel in the first grayscale image) minus (grayscale values ​​of each pixel in the third grayscale image)).

[0108] In one embodiment, the target information includes the area of ​​adhesive buildup, and after the step of determining the target information of the adhesive buildup on the surface of the track to be tested, the method further includes:

[0109] Determine whether the area of ​​accumulated glue exceeds a preset area threshold;

[0110] If the limit is exceeded, a prompt message will be generated and sent.

[0111] By automatically determining the size of the accumulated adhesive area and issuing alarms when abnormalities occur, relevant personnel can promptly clean up the adhesive on the surface, preventing serious accidents.

[0112] Secondly, such as Figure 12 As shown, in one embodiment, the present invention provides a running track surface adhesive detection device, comprising:

[0113] Image acquisition module 1201 is used to acquire the HSV image corresponding to the runway to be detected;

[0114] The first processing module 1202 is used to perform first image processing on the HSV image to obtain a first intermediate image with the feature region determined; the feature region consists of undifferentiated glue accumulation region and interference region;

[0115] The second processing module 1203 is used to perform second image processing on the HSV image to obtain a second intermediate image in which the interference region is determined.

[0116] The information determination module 1204 is used to determine the target information of the surface adhesive of the runway to be detected based on the first intermediate image and the second intermediate image.

[0117] The aforementioned runway surface adhesive detection device employs the HSV color model and performs relevant image processing on the HSV image to ultimately determine the target information of the adhesive on the surface of the runway under test. Compared with manual visual inspection, this method can greatly improve detection efficiency, reduce labor costs, and accurately determine target information such as the area of ​​the adhesive on the surface.

[0118] In one embodiment, the first processing module is specifically used to perform a first binarization process on the HSV image based on brightness.

[0119] In one embodiment, the second processing module is specifically used to perform a second binarization process on the HSV image based on hue, saturation, and brightness to obtain a second intermediate image.

[0120] In one embodiment, the second processing module is specifically used to perform a second binarization process on the HSV image to obtain a third intermediate image; and to perform denoising processing on the third intermediate image to obtain a second intermediate image.

[0121] In one embodiment, the second processing module is specifically used to obtain the trained denoising model; input the third intermediate image into the denoising model to obtain the second intermediate image output by the denoising model.

[0122] In one embodiment, the information determination module is specifically used to obtain a target image of the identified adhesive accumulation area based on a first intermediate image and a second intermediate image; and to determine the target information of the adhesive accumulation on the surface of the runway to be inspected based on the target image.

[0123] In one embodiment, the information determination module is specifically used to determine the pixel difference map of the first intermediate image and the second intermediate image; and to obtain the target image based on the pixel difference map.

[0124] Thirdly, in one embodiment, the present invention provides a computer device, such as... Figure 13 As shown, it illustrates the structure of the computer device involved in this invention, specifically:

[0125] The computer device may include components such as a processor 1301 with one or more processing cores, a memory 1302 with one or more computer-readable storage media, a power supply 1303, and an input unit 1304. Those skilled in the art will understand that... Figure 13 The structure of the computer device shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0126] The processor 1301 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 1302, and by calling data stored in the memory 1302, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, the processor 1301 may include one or more processing cores; preferably, the processor 1301 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and computer programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1301.

[0127] The memory 1302 can be used to store software programs and modules. The processor 1301 executes various functional applications and data processing by running the software programs and modules stored in the memory 1302. The memory 1302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, computer programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 1302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1302 may also include a memory controller to provide the processor 1301 with access to the memory 1302.

[0128] The computer device also includes a power supply 1303 that supplies power to various components. Preferably, the power supply 1303 can be logically connected to the processor 1301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 1303 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0129] The computer device may also include an input unit 1304, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0130] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1301 in the computer device loads the executable files corresponding to the processes of one or more computer programs into the memory 1302 according to the following instructions, and the processor 1301 runs the computer programs stored in the memory 1302 to perform the following steps:

[0131] Acquire the HSV image corresponding to the runway to be detected;

[0132] The HSV image is subjected to a first image processing step to obtain a first intermediate image in which the feature regions are determined; the feature regions consist of undifferentiated glue accumulation regions and interference regions.

[0133] A second image processing step is performed on the HSV image to obtain a second intermediate image in which the interference region is determined.

[0134] Based on the first intermediate image and the second intermediate image, the target information of the surface adhesive on the runway to be detected is determined.

[0135] Using the aforementioned computer equipment and the HSV color model, the target information of the adhesive layer on the surface of the track under test is determined by performing relevant image processing on the HSV image. Compared with manual visual inspection, this method can greatly improve detection efficiency, reduce labor costs, and accurately determine target information such as the area of ​​the adhesive layer on the surface.

[0136] In one embodiment, performing a first image processing step on an HSV image includes:

[0137] The HSV image is first binarized based on its brightness.

[0138] In one embodiment, performing a second image processing on an HSV image to obtain a second intermediate image with the interference region identified includes:

[0139] Based on hue, saturation, and brightness, the HSV image undergoes a second binarization process to obtain a second intermediate image.

[0140] In one embodiment, performing a second binarization process on the HSV image to obtain a second intermediate image includes:

[0141] The HSV image is subjected to a second binarization process to obtain a third intermediate image;

[0142] The third intermediate image is denoised to obtain the second intermediate image.

[0143] In one embodiment, denoising the third intermediate image to obtain the second intermediate image includes:

[0144] Obtain the trained denoising model;

[0145] The third intermediate image is input into the denoising model to obtain the second intermediate image output by the denoising model.

[0146] In one embodiment, determining the target information of the surface adhesive on the runway to be detected based on the first intermediate image and the second intermediate image includes:

[0147] Based on the first intermediate image and the second intermediate image, the target image of the glue accumulation area is obtained;

[0148] Based on the target image, determine the target information of the surface adhesive on the runway to be inspected.

[0149] In one embodiment, obtaining a target image that identifies the glue accumulation area based on a first intermediate image and a second intermediate image includes:

[0150] Determine the pixel difference map between the first intermediate image and the second intermediate image;

[0151] The target image is obtained based on the pixel difference map.

[0152] Those skilled in the art will understand that all or part of the steps in any of the methods in the above embodiments can be performed by a computer program or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0153] Fourthly, in one embodiment, the present invention provides a storage medium storing a plurality of computer programs that can be loaded by a processor to perform the following steps:

[0154] Acquire the HSV image corresponding to the runway to be detected;

[0155] The HSV image is subjected to a first image processing step to obtain a first intermediate image in which the feature regions are determined; the feature regions consist of undifferentiated glue accumulation regions and interference regions.

[0156] A second image processing step is performed on the HSV image to obtain a second intermediate image in which the interference region is determined.

[0157] Based on the first intermediate image and the second intermediate image, the target information of the surface adhesive on the runway to be detected is determined.

[0158] Using the aforementioned storage medium and employing the HSV color model, the target information of the adhesive layer on the surface of the runway under test is ultimately determined through relevant image processing of the HSV image. Compared to manual visual inspection, this method can greatly improve detection efficiency, reduce labor costs, and accurately determine target information such as the area of ​​the adhesive layer on the surface.

[0159] In one embodiment, performing a first image processing step on an HSV image includes:

[0160] The HSV image is first binarized based on its brightness.

[0161] In one embodiment, performing a second image processing on an HSV image to obtain a second intermediate image with the interference region identified includes:

[0162] Based on hue, saturation, and brightness, the HSV image undergoes a second binarization process to obtain a second intermediate image.

[0163] In one embodiment, performing a second binarization process on the HSV image to obtain a second intermediate image includes:

[0164] The HSV image is subjected to a second binarization process to obtain a third intermediate image;

[0165] The third intermediate image is denoised to obtain the second intermediate image.

[0166] In one embodiment, denoising the third intermediate image to obtain the second intermediate image includes:

[0167] Obtain the trained denoising model;

[0168] The third intermediate image is input into the denoising model to obtain the second intermediate image output by the denoising model.

[0169] In one embodiment, determining the target information of the surface adhesive on the runway to be detected based on the first intermediate image and the second intermediate image includes:

[0170] Based on the first intermediate image and the second intermediate image, the target image of the glue accumulation area is obtained;

[0171] Based on the target image, determine the target information of the surface adhesive on the runway to be inspected.

[0172] In one embodiment, obtaining a target image that identifies the glue accumulation area based on a first intermediate image and a second intermediate image includes:

[0173] Determine the pixel difference map between the first intermediate image and the second intermediate image;

[0174] The target image is obtained based on the pixel difference map.

[0175] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0176] Since the computer program stored in the storage medium can execute the steps in any of the runway surface adhesive detection methods provided in the embodiments of the present invention, the beneficial effects that any of the runway surface adhesive detection methods provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0177] It will be understood by those skilled in the art that any references to memory, storage, database, or other media used in the embodiments provided by this invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0178] Fifthly, in one embodiment, the present invention provides a track surface adhesive application detection system, comprising:

[0179] The pavement inspection robot and the computer device in any of the above embodiments are communicatively connected;

[0180] The pavement inspection robot includes a robot body, a motion device located at the bottom of the robot body, and a camera device located on the robot body.

[0181] The camera device is used to acquire images of the runway to be inspected; the acquired images are sent to a computer device; the computer device is used to process the acquired images.

[0182] The computer equipment can be set up remotely or directly on the pavement inspection robot.

[0183] The above-mentioned runway surface adhesive detection system uses the HSV color model and performs relevant image processing on the HSV image to finally determine the target information of the adhesive on the surface of the runway to be tested. Compared with manual visual inspection, it can greatly improve detection efficiency, reduce labor costs, and accurately determine the target information such as the area of ​​adhesive on the surface.

[0184] In one embodiment, the camera device is an industrial camera.

[0185] Industrial cameras include those based on CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) chips, which feature high image stability, high transmission capability, and high anti-interference capability.

[0186] In one embodiment, the above-mentioned runway surface adhesive detection system further includes a cloud management system (such as a server) and / or an administrator terminal (such as a mobile terminal such as a mobile phone or smartwatch) that is connected to a computer device.

[0187] The computer equipment is used to generate a prompt message and send it to the cloud management system and / or administrator terminal to provide an alarm when the area of ​​accumulated glue exceeds a preset area threshold.

[0188] The foregoing has provided a detailed description of a method, apparatus, and system for detecting adhesive residue on the surface of a running track, as provided in the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0189] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for detecting adhesive residue on the surface of a running track, characterized in that, include: Acquire the HSV image corresponding to the runway to be detected; The HSV image is subjected to a first image processing step to obtain a first intermediate image in which the feature regions are determined. The feature region consists of undifferentiated glue accumulation areas and interference areas; The HSV image is subjected to a second image processing step to obtain a second intermediate image in which the interference region is determined. Based on the first intermediate image and the second intermediate image, determine the target information of the surface adhesive on the runway to be detected; The first image processing of the HSV image includes: The HSV image is subjected to a first binarization process based on its brightness. The second image processing of the HSV image to obtain a second intermediate image that identifies the interference region includes: The HSV image is subjected to a second binarization process based on hue, saturation, and brightness to obtain the second intermediate image; The step of determining the target information of the surface adhesive on the track to be detected based on the first intermediate image and the second intermediate image includes: Determine the pixel difference map between the first intermediate image and the second intermediate image; The target image is obtained based on the pixel difference map; Based on the target image, the target information of the surface adhesive of the runway to be detected is determined.

2. The method for detecting adhesive residue on the surface of a running track according to claim 1, characterized in that, The second binarization processing of the HSV image to obtain the second intermediate image includes: The HSV image is subjected to the second binarization process to obtain the third intermediate image; The third intermediate image is denoised to obtain the second intermediate image.

3. The method for detecting adhesive residue on the surface of a running track according to claim 2, characterized in that, The step of denoising the third intermediate image to obtain the second intermediate image includes: Obtain the trained denoising model; The third intermediate image is input into the denoising model to obtain the second intermediate image output by the denoising model.

4. A device for detecting adhesive residue on the surface of a running track, characterized in that, include: The image acquisition module is used to acquire the HSV image corresponding to the runway to be detected; The first processing module is used to perform first image processing on the HSV image to obtain a first intermediate image with the feature region determined. The feature region consists of undifferentiated glue accumulation areas and interference areas; The second processing module is used to perform second image processing on the HSV image to obtain a second intermediate image that identifies the interference region; The information determination module is used to determine the target information of the surface adhesive of the runway to be detected based on the first intermediate image and the second intermediate image; The first processing module is specifically used to perform a first binarization process on the HSV image based on its brightness. The second processing module is specifically used to perform a second binarization process on the HSV image based on hue, saturation, and brightness to obtain the second intermediate image; The information determination module is specifically used to determine the pixel difference map between the first intermediate image and the second intermediate image, obtain the target image based on the pixel difference map, and determine the target information of the surface adhesive of the runway to be detected based on the target image.

5. A computer device, characterized in that, It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the steps in the runway surface adhesive detection method according to any one of claims 1 to 3.

6. A storage medium, characterized in that, The storage medium stores a computer program, which is loaded by a processor to execute the steps of the runway surface adhesive detection method according to any one of claims 1 to 3.

Images