Markstone quality inspection method and device for long-distance directly-buried optical cable

Abstract

The invention provides a long-distance directly-buried optical cable markstone quality inspection method and device, and the method comprises the steps: converting the image data of a to-be-detected optical cable markstone into a gray image, and segmenting a markstone candidate region from the gray image; classifying the markstone candidate areas based on a deep convolutional neural network classifier, and detecting and classifying the markstone candidate areas based on an image detection model according to feature data of a penultimate layer of a deep neural network in the deep convolutional neural network classifier, obtaining a markstone area and a markstone defect type in the image data; the deep convolutional neural network classifier and the image detection model are obtained by taking image data of an optical cable markstone sample as sample data and taking markstone areas and markstone defect types marked by the sample data as labels for training. According to the invention, automatic quality inspection of the optical cable markstone based on image deep learning is realized, and the quality inspection efficiency, timeliness and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of communication line inspection, and in particular to a method and device for quality inspection of long-distance direct-buried optical cable markers. Background Art

[0002] Due to the inherent fragility of optical fiber cables, they are very susceptible to external force damage and communication blockage. Once the optical cable is blocked, it will cause significant economic losses. Reducing line obstacles is the main task to ensure the safety and smoothness of communications. In a certain sense, the safety and smoothness of optical cable lines mainly depend on the daily maintenance and management of the lines and the level of obstacle prevention. Maintenance regulations stipulate that line inspectors should regularly patrol the optical cable lines. The number of patrols should be increased in special areas such as urban areas, villages, mining areas, and construction areas, after heavy rain, during important communications, and in seasons with more groundbreaking.

[0003] At present, the fiber optic cable patrol mainly adopts two methods: full patrol on foot and key patrol, which adopts the working method of manual patrol and manual camera recording. This method has many human factors and low work efficiency. A large number of work records cannot be transmitted back in real time and need to be manually judged one by one afterwards, which has poor timeliness, large workload for defect judgment, and inconsistent judgment standards. It is often easy to cause delays in processing time, resulting in delays in the discovery and processing of line obstacles. Summary of the invention

[0004] The present invention provides a long-distance direct-buried optical cable marker quality inspection method and device, which are used to solve the defects of low efficiency, poor timeliness and inaccuracy of manual quality inspection of optical cable markers in the prior art, realize automatic quality inspection of optical cable markers based on image deep learning, and improve quality inspection efficiency, timeliness and accuracy.

[0005] The present invention provides a long-distance direct-buried optical cable benchmark quality inspection method, comprising:

[0006] Converting the image data of the optical cable landmark to be detected into a grayscale image, and segmenting the landmark candidate area from the grayscale image;

[0007] Classifying the candidate area of ​​the landmark stone based on a deep convolutional neural network classifier, and detecting and classifying the candidate area of ​​the landmark stone based on an image detection model according to feature data of the penultimate layer of the deep neural network in the deep convolutional neural network classifier, to obtain the landmark stone area and the landmark stone defect type in the image data;

[0008] The deep convolutional neural network classifier and the image detection model are obtained by training using the image data of the optical cable marker sample as sample data and using the marker area and marker defect type annotated on the sample data as labels.

[0009] According to a long-distance direct buried optical cable marker quality inspection method provided by the present invention, image data of the optical cable marker to be inspected is converted into a grayscale image, and a marker candidate area is segmented from the grayscale image, including:

[0010] Compressing the image data of the optical cable marker to be detected, and converting the compressed image data into a grayscale image;

[0011] Calculate the standard deviation of pixels surrounding each pixel in the grayscale image;

[0012] Each pixel whose standard deviation is less than the first preset threshold is taken as a pixel of the landmark candidate area.

[0013] According to a method for quality inspection of long-distance direct-buried optical cable markers provided by the present invention, the image data of the optical cable marker sample includes a forward sample image with a marker and a reverse sample image without a marker;

[0014] The image data of the optical cable benchmark sample also includes image data obtained by an image augmentation method.

[0015] According to a method for quality inspection of long-distance direct buried optical cable markers provided by the present invention, before classifying the marker candidate areas based on a deep convolutional neural network classifier, the method further comprises:

[0016] Converting the sample data into a grayscale image, and segmenting a landmark candidate region from the grayscale image of the sample data;

[0017] Classifying the landmark candidate region in the sample data based on a deep convolutional neural network classifier to obtain a first landmark defect type of the sample data, and detecting and classifying the landmark candidate region in the sample data based on an image detection model according to feature data of a penultimate layer of a deep neural network in the deep convolutional neural network classifier to obtain a landmark region and a second landmark defect type in the sample data;

[0018] Calculating the IoU loss between the landmark area predicted by the sample data and the marked area;

[0019] Calculating a first Focal Loss between a first benchmark defect type predicted by the sample data and a marked benchmark defect type;

[0020] Calculating a second Focal Loss between a second landmark defect type predicted by the sample data and an annotated landmark defect type;

[0021] According to the IoU loss, the first Focal Loss loss and the second Focal Loss loss, parameters of the deep convolutional neural network classifier and the image detection model are adjusted.

[0022] According to a long-distance direct-buried optical cable marker quality inspection method provided by the present invention, the marker defect types marked in the sample data include one or more of abnormal marker excavation height, missing marker cap, marker writing defects, marker weed coverage, marker breakage and marker tilt.

[0023] According to a method for quality inspection of long-distance direct buried optical cable markers provided by the present invention, before classifying the marker candidate areas based on a deep convolutional neural network classifier, the method further comprises:

[0024] Calculate the height and width of the contour line of the landmark area marked in the sample data;

[0025] Calculating the ratio between the outline width and the logo design width;

[0026] According to the ratio and the height of the contour line, the actual height of the mark in the sample data is calculated as the height of the marker stone unearthed;

[0027] When the height of the landmark stone unearthed is greater than the second preset threshold or less than the third preset threshold, marking the sample data as abnormal height of the landmark stone unearthed;

[0028] In the case where the contour line of the landmark area is a non-diamond structure, marking the sample data as a damaged landmark, and determining a damaged area in the landmark area according to the contour line of the landmark area;

[0029] If the damaged area is above the contour line and the gray value standard deviation of the damaged area is greater than the first preset threshold, the sample data is marked as a landmark with missing cap;

[0030] When the outline of the landmark area is a diamond structure and the width of the landmark area is consistent with the width of the outline, the landmark area above the outline is regarded as a cap-missing area, and the sample data is marked as a landmark-cap-missing area;

[0031] When a weed area is identified from the landmark area, the weed area is below the contour line, and the grayscale value standard deviation of the weed area is greater than the first preset threshold, marking the sample data as landmark weed coverage;

[0032] Performing text recognition on the monument area, comparing the recognized text with the phrases in the monument surface spray text library, and determining whether to mark the sample data as a monument text defect according to the comparison result;

[0033] The verticality of the contour line is calculated, and when the verticality is greater than a fourth preset threshold, the sample data is marked as a landmark tilt.

[0034] A method for quality inspection of long-distance direct buried optical cable markers provided by the present invention also includes:

[0035] According to the relationship between the coordinate system of the image data and the camera coordinate system of the intelligent inspection device that took the image data, and the positioning deviation of the optical cable marker to be detected relative to the intelligent inspection device in the world coordinate system, the absolute positioning information of the optical cable marker to be detected in the world coordinate system is calculated based on the GPS of the intelligent inspection device.

[0036] The present invention also provides a long-distance direct buried optical cable benchmark quality inspection device, comprising:

[0037] A grayscale segmentation module, used to convert the image data of the optical cable landmark to be detected into a grayscale image, and segment the landmark candidate area from the grayscale image;

[0038] A classification detection module is used to classify the candidate area of ​​the landmark stone based on a deep convolutional neural network classifier, and detect and classify the candidate area of ​​the landmark stone based on an image detection model according to feature data of the penultimate layer of the deep neural network in the deep convolutional neural network classifier, so as to obtain the landmark stone area and the landmark stone defect type in the image data;

[0039] The deep convolutional neural network classifier and the image detection model are obtained by training using the image data of the optical cable marker sample as sample data and using the marker area and marker defect type annotated on the sample data as labels.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for quality inspection of a long-distance direct buried optical cable marker as described above is implemented.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for the long-distance direct-buried optical cable marker quality inspection device.

[0042] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned long-distance direct-buried optical cable marker quality inspection methods.

[0043] The long-distance direct-buried optical cable marker quality inspection method and device provided by the present invention realize the intelligent recognition and analysis of important locations such as long-distance direct-buried optical cable markers, potential danger points, protection and publicity facilities, and optical cable joints through the intelligent recognition technology of communication facilities based on deep learning, thereby improving the efficiency and timeliness of analysis and processing. The candidate area of ​​the marker is first segmented from the grayscale image of the image data of the optical cable marker to be inspected, with the purpose of eliminating the interference of irrelevant background on the defect feature extraction of the target object. Then, detection and classification are performed based on the deep convolutional neural network classifier and the image detection model, thereby improving the defect feature extraction capability and defect recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 This is one of the flow diagrams of the long-distance direct buried optical cable benchmark quality inspection method provided by the present invention;

[0046] Figure 2 This is the second flow chart of the method for quality inspection of long-distance direct buried optical cable marker provided by the present invention;

[0047] Figure 3 It is a structural schematic diagram of the long-distance direct-buried optical cable benchmark quality inspection device provided by the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Combine the following Figure 1 A method for quality inspection of a long-distance direct buried optical cable marker of the present invention is described, comprising:

[0050] Step 101, converting the image data of the optical cable landmark to be detected into a grayscale image, and segmenting the landmark candidate area from the grayscale image;

[0051] The execution subject in this embodiment can be a long-distance resource management system, which is a full-service long-distance network resource information management system with resource management functions such as optical cables, systems, equipment, and services of transmission trunks. It opens an interface with the transmission integrated network management to provide a name standard for data association between the transmission equipment network management and the transmission integrated network management.

[0052] The marker stone is an important part of the direct buried optical cable line. The marker stone is made of reinforced concrete and is divided into ordinary marker stones and monitoring marker stones. Ordinary marker stones should be set at the optical cable joints, optical cable turning points, drainage line start and end points, optical cable special reserved points, crossing points with other cables, crossing obstacles and straight sections every 50 to 100 meters to indicate the optical cable route. Monitoring marker stones should be set at the joint points where the optical cable metal sheath insulation and potential need to be monitored.

[0053] Maintenance personnel can follow the task work order navigation and use intelligent inspection equipment to collect RGB image data of the front, back, left and right sides of the long-distance direct buried optical cable marker, collect the lens tilt angle, azimuth angle, roll angle and WGS84 coordinate system coordinate values ​​during shooting, and record the marker name, coordinates and type information on the task work order.

[0054] Convert RGB image data into grayscale images. In the complex background of the wild, due to the obvious differences in the grayscale values ​​of different targets in the image, the grayscale image can be used to identify and segment image landmark targets based on the grayscale gradient features, and the possible landmark target areas can be used as candidate landmark areas.

[0055] To improve the recognition accuracy, RGB image target recognition and grayscale image gradient segmentation target recognition can be combined. Based on image classification and morphological processing, the rectangular graphic features of the marker can be detected, the long-distance direct buried optical cable marker can be located, and the coordinates of the four vertices of the target image area can be returned.

[0056] RGB color space is the color space of the image, using three channels to represent a pixel, namely red (Red, R), green (Green, G) and blue (Blue, B). The three color channel data of the RGB image is converted into a single-channel grayscale image. The calculation formula is as follows:

[0057] Gray=(3000*R+5000*G+1100*B) / 10000

[0058] Among them, Gray is the grayscale value, R is the red value of the pixel in the image, G is the green value of the pixel in the image, and B is the blue value of the pixel in the image.

[0059] Step 102, classifying the candidate landmark area based on a deep convolutional neural network classifier, and detecting and classifying the candidate landmark area based on an image detection model according to feature data of the penultimate layer of the deep neural network in the deep convolutional neural network classifier, to obtain the landmark area and landmark defect type in the image data;

[0060] The deep convolutional neural network classifier and the image detection model are obtained by training using the image data of the optical cable marker sample as sample data and using the marker area and marker defect type annotated on the sample data as labels.

[0061] Multiple modules that can improve the accuracy of the algorithm can be added to the deep convolutional neural network classifier, such as attention mechanism, feature fusion module, more efficient activation function, etc., to improve the feature extraction capability of the model in complex scenarios and thus improve the model accuracy.

[0062] The attention mechanism allocates resources between different dimensions of the model, learns the model's dependence on different dimensions, and adjusts the corresponding weights to achieve the purpose of retaining important features and suppressing features.

[0063] The feature fusion module effectively fuses the information obtained from different positions of the model. Its output not only retains the low-level underlying visual features, but also adds high-level semantic visual features, which greatly enhances the model's recognition accuracy for objects at different scales. This is beneficial for the detection and extraction of target features at different distances in online equipment inspection scenarios with inconsistent scales.

[0064] The activation function is the key to determining the performance of the neural network. Choosing a suitable neural network is not only conducive to improving the accuracy of the model, but also can reduce the complexity of the model, reduce the running time and resource usage of the model. Therefore, the lightweight activation function Mish is selected. The expression of this function is as follows:

[0065] Mish = x*tanh(ln(1+e x ))

[0066] Among them, x represents the input, which is transformed by a series of logarithmic transformations and then superimposed on the original input through the tanh function transformation.

[0067] The YOLOv8 algorithm can be used as an image detection model to detect and classify candidate areas of landmarks. Based on the trained deep convolutional neural network classifier and image detection model, the candidate areas of landmarks are classified and detected to distinguish between background areas and landmark areas. The landmark areas can include normal landmark areas and landmark defect areas. The landmark defect areas include weed areas, damaged areas, and landmark cap missing areas.

[0068] Specifically, the candidate area of ​​the landmark stone is processed through the trained deep convolutional neural network classifier, and the feature data of the penultimate layer of the deep neural network in the deep convolutional neural network classifier is extracted; the feature data is classified and detected through the image detection model to distinguish the background area and the landmark stone area, and identify the type of marking defect.

[0069] Since the defect of the landmark is located on the surface of the landmark, the rectangular area of ​​the landmark defect area will overlap with the rectangular area of ​​the landmark body. If they do not overlap, the landmark defect area is incorrectly identified and deleted.

[0070] This embodiment uses deep learning-based intelligent recognition technology for communication facilities to achieve intelligent recognition and analysis of important locations such as long-distance direct-buried optical cable markers, potential hazard points, protection and publicity facilities, and optical cable joints, thereby improving the efficiency and timeliness of analysis and processing. The candidate marker area is first segmented from the grayscale image of the image data of the optical cable marker to be detected, with the aim of eliminating the interference of irrelevant background on the defect feature extraction of the target object. Then, detection and classification are performed based on a deep convolutional neural network classifier and an image detection model, thereby improving the defect feature extraction capability and defect recognition accuracy.

[0071] On the basis of the above embodiment, in this embodiment, the image data of the optical cable landmark to be detected is converted into a grayscale image, and the landmark candidate area is segmented from the grayscale image, including:

[0072] Compressing the image data of the optical cable marker to be detected, and converting the compressed image data into a grayscale image;

[0073] Calculate the standard deviation of pixels surrounding each pixel in the grayscale image;

[0074] Each pixel whose standard deviation is less than the first preset threshold is taken as a pixel of the landmark candidate area.

[0075] In order to improve the accuracy of using image data to identify long-distance direct buried optical cable landmark targets, a series of preprocessing of the image data is required.

[0076] First, the original image data collected by the intelligent inspection equipment can be compressed into a preset size, such as 1600*1200 pixels.

[0077] In the complex outdoor environment, the landmark is usually a white cuboid with a large area of ​​uniform white, so the standard deviation in the grayscale image is small. The characters and the landmark cover on the surface of the landmark are a fixed concentrated color, such as blue, red, and yellow. The background color is complex, and the outdoor environment is mostly grassland, roads, etc., with uneven grayscale values ​​and large standard deviation. Therefore, the landmark area can be quickly identified based on the standard deviation.

[0078] The standard deviation is calculated as follows, where A represents the average value, xi is the gray value of a pixel in the image data, and N is the number of pixels.

[0079]

[0080] Traverse each pixel of the grayscale image to obtain the standard deviation of the pixels in the 5*5 area around the pixel. Set a first preset threshold, and pixels with a standard deviation greater than the first preset threshold are considered to be background areas, which are non-marker stone target areas; pixels with a standard deviation less than the first preset threshold are likely to be marker stone target areas, and the area will be subsequently verified with the deep learning recognition area to improve recognition accuracy.

[0081] On the basis of the above-mentioned embodiment, the image data of the optical cable marker sample in this embodiment includes a positive sample image with a marker and a reverse sample image without a marker;

[0082] The image data of the optical cable benchmark sample also includes image data obtained by an image augmentation method.

[0083] The long-distance direct buried optical cable markers in natural scenes are photographed by intelligent inspection equipment, and semantic segmentation annotation data is obtained manually to form an optical cable marker image dataset. The original dataset is divided into training set, test set and validation set. The training set and validation set data are annotated to mark whether the optical cable marker exists. For images with optical cable markers, the marker area is annotated for subsequent segmentation tasks that facilitate deep learning processing of images. The marker defect types in the marker area are also annotated, such as whether there are weeds, whether the marker is missing a cap, and whether the marker has a damaged hole. The deep convolutional neural network classifier and image detection model are trained with the training images. The optical cable marker facility group that needs to be inspected is photographed by the intelligent inspection equipment, and the image detection model is used to identify whether there are defects in the optical cable facility group.

[0084] Image data without landmarks in natural scenes captured by intelligent inspection equipment are also used to establish a background data set to enhance the accuracy of sample training. The sample images in the data set include positive sample images marked with landmarks and negative sample images that determine that there are no landmarks. The forward sample images and the negative sample images are trained in a deep convolutional neural network to obtain a trained deep convolutional neural network classifier. The number of positive sample images obtained is basically consistent with the number of negative sample images, and the ratio of the number of positive sample images to the number of negative sample images is controlled to be 1:1.

[0085] In order to be suitable for the generalization ability of the recognition model of target objects such as landmarks, data augmentation methods are used, including geometric space transformation methods (scaling, cropping, translation, rotation, etc.), illumination transformation methods, adding occlusion (such as Cutout), weather transformation methods (such as sunny, rainy, foggy, etc.), etc. to effectively expand the number of samples. Diversified data is conducive to model robustness training and reasoning. And ensure that the image always presents the most significant area after data enhancement, that is, the detected target is unobstructed and the detected background is eliminated or transformed.

[0086] GridMask and random erasing algorithms can be used as basic methods for image augmentation, and one of them is randomly selected for image augmentation with equal probability. The specific methods include the following:

[0087] (1) Brightness and saturation adjustment strategy

[0088] By simulating the differences in weather, camera angle selection and exposure control during different inspection periods, the collected images present diverse brightness and saturation characteristics. By finely adjusting the brightness and saturation of a single image, a variety of lighting conditions from strong direct sunlight at noon (or shooting from above) to soft light at dusk (or shooting from below) are simulated, thereby significantly enhancing the deep learning model's ability to adapt to lighting changes and improving the accuracy of the image recognition model under different lighting environments.

[0089] (2) Mosaic technology

[0090] Each image is randomly cropped, and the cropped parts are randomly scaled and seamlessly spliced ​​into a new training sample, which increases the complexity of the data background. In addition, incomplete or partially occluded targets are introduced through cropping operations to further enhance the robustness of the model. The landmark area in the image is randomly scaled to increase the number of targets in the sample data and improve the accuracy of the small target detection scene model.

[0091] (3) CutMix technology

[0092] Cropped blending merges randomly cropped areas of one type of image with another image to create new training samples, simulating the mutual occlusion phenomenon between objects in actual scenes and improving the model's ability to detect overlapping objects in complex scenes.

[0093] (4) Rotation, scaling and flipping transformations

[0094] Image transformation technologies such as rotation, rescaling and flipping are used, and up and down flipping, 90-degree rotation and random rotation are used to simulate the angle and distance changes in the actual shooting process, improve the deep learning network's ability to learn the key features of the target during training, and improve the model's detection accuracy for targets of different postures and sizes.

[0095] Based on the above embodiment, before classifying the landmark candidate area based on the deep convolutional neural network classifier, this embodiment further includes:

[0096] Converting the sample data into a grayscale image, and segmenting a landmark candidate region from the grayscale image of the sample data;

[0097] Classifying the landmark candidate region in the sample data based on a deep convolutional neural network classifier to obtain a first landmark defect type of the sample data, and detecting and classifying the landmark candidate region in the sample data based on an image detection model according to feature data of a penultimate layer of a deep neural network in the deep convolutional neural network classifier to obtain a landmark region and a second landmark defect type in the sample data;

[0098] Calculating the IoU loss between the landmark area predicted by the sample data and the marked area;

[0099] Calculating a first Focal Loss between a first benchmark defect type predicted by the sample data and a marked benchmark defect type;

[0100] Calculating a second Focal Loss between a second landmark defect type predicted by the sample data and an annotated landmark defect type;

[0101] According to the IoU loss, the first Focal Loss loss and the second Focal Loss loss, parameters of the deep convolutional neural network classifier and the image detection model are adjusted.

[0102] The deep convolutional neural network classifier and image detection model are used for training. The loss gap between the model output and the true value of various types of landmark defects is calculated to provide guidance for model optimization. When the loss drops to a certain ratio and is relatively stable, the training ends and a long-distance direct buried optical cable landmark target detection model parameter file is generated. This file records a series of parameters used to calculate the target recognition category of the long-distance direct buried optical cable landmark image. Through this model parameter file, the target classification verification and prediction of the long-distance direct buried optical cable landmark image data can be performed.

[0103] Specifically, the IoU loss function is calculated by incorporating the relative position relationship between the model's predicted bounding box and the true box into the scope of its loss calculation. By refining the relative position of the two boxes, the overlap between the predicted bounding box and the true box is continuously improved during the model training process, guiding the training process to reach the convergence point faster and more smoothly. The IoU loss calculation formula is as follows:

[0104]

[0105] Among them, B represents the prediction box, B gt Represents a label box.

[0106] On the other hand, the classification accuracy of the model can also be improved appropriately. By introducing Focal Loss, the classification weight of the model for a small number of data labels can be increased, and the classification accuracy of the model can be improved when the amount of data labels is unbalanced. The calculation formula of Focal Loss is as follows:

[0107]

[0108] Among them, α is the weight of sample balance, γ is the weight given to difficult samples, and y t is the predicted probability of the model.

[0109] Based on the above embodiments, the types of marker stone defects annotated for the sample data in this embodiment include one or more of abnormal marker stone excavation height, missing marker stone cap, marker stone writing defects, marker stone covered with weeds, marker stone damage and marker tilt.

[0110] Among them, the missing cap of the marker stone means that the top cover of the marker stone facility is lost, the tilt of the marker stone means that the marker stone facility is not perpendicular to the horizontal, the damaged marker stone means that the marker stone facility is incomplete as a whole and there are some structural defects, the abnormal unearthed height of the marker stone means that the height from the top cover of the marker stone to the ground is abnormal, the weed coverage of the marker stone means that there are a large number of weeds near the marker stone facility that need to be manually cleared, and the defect of the marker stone's handwriting means that the text on the surface of the marker stone facility is missing or blurred.

[0111] Based on the above embodiment, before classifying the landmark candidate area based on the deep convolutional neural network classifier, this embodiment further includes:

[0112] Calculate the height and width of the contour line of the landmark area marked in the sample data;

[0113] Calculating the ratio between the outline width and the logo design width;

[0114] According to the ratio and the height of the contour line, the actual height of the mark in the sample data is calculated as the height of the marker stone unearthed;

[0115] When the height of the landmark stone unearthed is greater than the second preset threshold or less than the third preset threshold, marking the sample data as abnormal height of the landmark stone unearthed;

[0116] In the case where the contour line of the landmark area is a non-diamond structure, marking the sample data as a damaged landmark, and determining a damaged area in the landmark area according to the contour line of the landmark area;

[0117] If the damaged area is above the contour line and the gray value standard deviation of the damaged area is greater than the first preset threshold, the sample data is marked as a landmark with missing cap;

[0118] When the outline of the landmark area is a diamond structure and the width of the landmark area is consistent with the width of the outline, the landmark area above the outline is regarded as a cap-missing area, and the sample data is marked as a landmark-cap-missing area;

[0119] When a weed area is identified from the landmark area, the weed area is below the contour line, and the grayscale value standard deviation of the weed area is greater than the first preset threshold, marking the sample data as landmark weed coverage;

[0120] Performing text recognition on the monument area, comparing the recognized text with the phrases in the monument surface spray text library, and determining whether to mark the sample data as a monument text defect according to the comparison result;

[0121] The verticality of the contour line is calculated, and when the verticality is greater than a fourth preset threshold, the sample data is marked as a landmark tilt.

[0122] The determination of the type of identification defect mainly includes the following aspects:

[0123] (1) Determination of the height of the marker stone

[0124] The marker contour is obtained based on the boundary of the marker area, the contour height and width are calculated, and the actual height of the marker in the image is calculated using the marker design width value as the marker height. If the marker height is greater than or less than the threshold, it is determined that the marker height is abnormal.

[0125] (2) Determination of missing cap of marker stone

[0126] Calculate the defect area features and obtain the area contour line. If the contour line is a non-diamond structure, determine the damaged area of ​​the marker stone based on the contour line. The diamond structure may be a cap-missing area. Further calculate the width of the marker stone area. If it is consistent with the width of the marker stone area contour line, the marker stone area above the marker stone contour line is considered as a cap-missing area and is determined to be a marker stone cap-missing area.

[0127] (3) Determination of monument number and character defects

[0128] The surface of the monument is sprayed with text information to indicate the property rights and warning information of the monument. Identifying the text on the monument is used to determine whether the monument has text damage defects and to determine the identification number. A library of sprayed text on the monument surface is established to count commonly used sprayed text phrases. The identification number is a 3 to 5-digit string consisting of numbers and letters.

[0129] Using the OCR (Optical Character Recognition) text recognition model framework, the sprayed text phrases are imported into the framework through artificial experience knowledge, and the recognition rate will be improved by constraining the recognition content to the phrase range. The OCR algorithm is used to recognize the text in the image within the contour range of the monument. The extraction accuracy is greater than the set value. For example, more than 95% of the text is compared with the phrases in the sprayed text library on the surface of the monument. If there is a partial match, it is determined to be a monument handwriting defect. The digital numbered text is obtained and compared with the monument number of the inspection work order to determine whether the monument number is correct.

[0130] (4) Determination of weeds on landmark stones

[0131] If the defective area is identified as weeds and is below the marker stone contour, and the image marker stone target identification and segmentation steps based on the grayscale gradient feature obtain a large standard deviation of the regional grayscale value, it is determined that the marker stone is covered with weeds.

[0132] (5) Determination of damage to the marker stone

[0133] If the defective area is identified as a damaged area, it is located in the upper middle of the marker stone contour line, and the image marker stone target recognition and segmentation steps based on the grayscale gradient characteristics obtain a large standard deviation of the regional grayscale value, and it is determined that the marker stone is missing a cap.

[0134] (6) Determination of the inclination of the landmark

[0135] The verticality of the contour line is calculated to obtain the inclination angle of the marker stone. If it is greater than the preset value, the marker stone is judged to be tilted.

[0136] Within the contour of the marking area, the image landmark target recognition and segmentation based on the grayscale gradient feature obtains the grayscale value of the marking area surface, and further uses the area for binary segmentation to obtain the foreground area of ​​the landmark surface. The formula is as follows:

[0137]

[0138] Where x is the current pixel, p[x] is the binarization result of the current pixel x, and R x , G x , B x It is the value corresponding to the pixel point x in the three channels of R, G, and B. R 、h R , l G 、h G , l B 、h B is the threshold value.

[0139] For the pixel points in the foreground area of ​​the extracted surface of the landmark, a linear regression equation is used to fit the boundary points to four landmark boundary lines. The intersection of the four boundary lines is the four endpoints A, B, C, and D of the landmark segmentation. The endpoint coordinates are (xA, yA, zA), (xB, yB, zB), (xC, yC, zC), and (xD, yD, zD). The distance between two adjacent points in A, B, C, and D is calculated respectively, and the line connecting the two points with the longest distance is obtained and the angle with the vertical line is calculated to obtain the inclination angle of the landmark image. This angle is subtracted from the roll angle of the camera lens when the intelligent inspection equipment collects the photo to obtain the verticality of the landmark contour line. If it is greater than the set threshold, it is considered that the landmark is tilted.

[0140] Based on the above embodiment, this embodiment also includes:

[0141] According to the relationship between the coordinate system of the image data and the camera coordinate system of the intelligent inspection device that took the image data, and the positioning deviation of the optical cable marker to be detected relative to the intelligent inspection device in the world coordinate system, the absolute positioning information of the optical cable marker to be detected in the world coordinate system is calculated based on the GPS (Global Positioning System) of the intelligent inspection device.

[0142] The camera image target positioning technology is used to ensure the accurate relationship between the image and the camera positioning. The image of the intelligent inspection equipment captures a physical point in the three-dimensional world. The coordinates in the world coordinate system are (X, Y, Z). After the camera takes the picture, the pixel coordinates of the picture are (u, v), the coordinates in the image coordinate system are (x, y), and the coordinates in the camera coordinate system are (X C , Y C , Z C ), and three transformation relationships between coordinates are calculated mathematically.

[0143] The physical size of each pixel on the X-axis and Y-axis is dx and dy, and the center position is (u0, v0). The relationship between any pixel in the image in these two image coordinate systems is shown as follows:

[0144]

[0145] The relationship between the image physical coordinate system and the camera coordinate system is shown in the following formula:

[0146]

[0147] The image pixel coordinates of the target object are (u t , v t ), the center position is (u0, v0), and the position deviations are Δx and Δy respectively. The positioning deviation of the target object relative to the intelligent inspection equipment in the world coordinate system is shown as follows:

[0148]

[0149] The absolute coordinates are obtained by using the GPS of the intelligent inspection equipment, and the above positioning deviation is added to obtain the absolute positioning information of the target object in the world coordinate system.

[0150] Using deep convolutional neural network classifiers and image detection models, we automatically audit the result data taken on-site by patrol personnel and send them back. We mainly conduct image quality inspection on long-distance direct buried optical cable marker scenes. At the same time, we proofread the data through automatic OCR recognition, analyze the accuracy of facility information data, determine whether the buried marker exists, obtain the number, and whether it is tilted or cracked. We also determine whether the four sides of the marker have propaganda slogans, identification numbers, etc. as required, and whether the handwriting is clear. We also compare the data collection latitude and longitude verification position with the platform database position, and verify whether the identification plate number and optical cable information extracted from the on-site photos match the database information. If the handwriting is unclear, we prompt you to manually supplement and fill in the report.

[0151] The complete flow chart of the long-distance direct buried optical cable benchmark quality inspection method is as follows Figure 2 As shown, combined with the work order issuance data and the line patrol return results data, on the one hand, the patrol personnel's work completion degree is analyzed, counted and evaluated; on the other hand, it is used to statistically analyze the equipment's integrity rate and the overall situation of damage or non-compliance, providing data basis for subsequent work evaluation and implementation of rectification.

[0152] The main purpose of the present invention is to provide a method for quality inspection and rapid positioning of long-distance direct-buried optical cable markers based on image deep learning. The invention can use various handheld devices to intelligently determine the precise spatial coordinate information of targets such as optical cable facilities in real time through visible light, greatly improving the efficiency of staff in locating optical cable facilities for defect diagnosis. The use of intelligent wearable devices for human patrols improves work efficiency, reduces patrol costs, promptly detects faults and defects, and ensures equipment safety. At the same time, it combines various technical means such as the Internet of Things and GPS to achieve remote monitoring, guide on-site operations, realize closed-loop business management, improve the operation and maintenance mode of communication lines, effectively improve operation and maintenance efficiency, ensure communication reliability, and avoid economic losses.

[0153] The present invention utilizes intelligent quality inspection equipment technology to realize rapid collection of on-site data, synchronous feedback, real-time analysis and processing, and guide patrol personnel in navigation and positioning; based on image deep learning, the intelligent recognition technology of communication facilities can realize the intelligent recognition, analysis and audit of long-distance direct buried optical cable markers and potential danger points, improve the efficiency of analysis and processing, and for the quality inspection of long-distance direct buried optical cable markers, provide the number and proportion of equipment with unclear handwriting, the number and proportion of tilted equipment, the number and proportion of cracked equipment, the number and proportion of broken equipment, and the number and proportion of equipment with missing caps, thereby improving the inspection efficiency.

[0154] In order to realize real-time data collection through the edge computing unit at the collection end, the intelligent computing unit is light in weight and weak in function, which meets the requirements of lightweight and high precision. The low-resolution information after multiple downsampling can provide contextual semantic information in the entire image for the segmented target, which can be understood as reflecting the relationship between the target and the surrounding environment. This feature helps to improve classification judgment; extracting high-resolution features from the corresponding position of the feature extraction part for feature fusion can provide more refined features for the segmented target, such as gradient information, which helps to divide the boundary of the segmented target. It realizes the rapid deployment of intelligent quality inspection equipment, meets the requirements of lightweight and high precision, and has the characteristics of lightweight and low resource occupation. It is very suitable for deployment in the collection system of patrol equipment.

[0155] The long-distance direct-buried optical cable marker quality inspection device provided by the present invention is described below. The long-distance direct-buried optical cable marker quality inspection device described below and the long-distance direct-buried optical cable marker quality inspection method described above can be referenced to each other.

[0156] like Figure 3 As shown, the device includes a grayscale segmentation module 301 and a classification detection module 302, wherein:

[0157] The grayscale segmentation module 301 is used to convert the image data of the optical cable landmark to be detected into a grayscale image, and segment the landmark candidate area from the grayscale image;

[0158] The classification detection module 302 is used to classify the landmark candidate area based on the deep convolutional neural network classifier, and detect and classify the landmark candidate area based on the image detection model according to the feature data of the penultimate layer of the deep neural network in the deep convolutional neural network classifier, so as to obtain the landmark area and landmark defect type in the image data;

[0159] The deep convolutional neural network classifier and the image detection model are obtained by training using the image data of the optical cable marker sample as sample data and using the marker area and marker defect type annotated on the sample data as labels.

[0160] This embodiment uses deep learning-based intelligent recognition technology for communication facilities to achieve intelligent recognition and analysis of important locations such as long-distance direct-buried optical cable markers, potential hazard points, protection and publicity facilities, and optical cable joints, thereby improving the efficiency and timeliness of analysis and processing. The candidate marker area is first segmented from the grayscale image of the image data of the optical cable marker to be detected, with the aim of eliminating the interference of irrelevant background on the defect feature extraction of the target object. Then, detection and classification are performed based on a deep convolutional neural network classifier and an image detection model, thereby improving the defect feature extraction capability and defect recognition accuracy.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for quality inspection of long-distance direct buried optical cable markers, characterized in that: include: Converting the image data of the optical cable landmark to be detected into a grayscale image, and segmenting the landmark candidate area from the grayscale image; Classifying the candidate area of ​​the landmark stone based on a deep convolutional neural network classifier, and detecting and classifying the candidate area of ​​the landmark stone based on an image detection model according to feature data of the penultimate layer of the deep neural network in the deep convolutional neural network classifier, to obtain the landmark stone area and the landmark stone defect type in the image data; The deep convolutional neural network classifier and the image detection model are obtained by training using the image data of the optical cable marker sample as sample data and using the marker area and marker defect type annotated on the sample data as labels.

2. The long-distance direct buried optical cable benchmark quality inspection method according to claim 1 is characterized in that: Converting the image data of the optical cable landmark to be detected into a grayscale image, and segmenting the landmark candidate area from the grayscale image, including: Compressing the image data of the optical cable marker to be detected, and converting the compressed image data into a grayscale image; Calculate the standard deviation of pixels surrounding each pixel in the grayscale image; Each pixel whose standard deviation is less than the first preset threshold is taken as a pixel of the landmark candidate area.

3. The long-distance direct buried optical cable benchmark quality inspection method according to claim 1 is characterized in that: The image data of the optical cable marker sample includes a forward sample image with a marker and a reverse sample image without a marker; The image data of the optical cable marker sample also includes image data obtained by an image augmentation method.

4. The long-distance direct buried optical cable benchmark quality inspection method according to claim 1 is characterized in that: Before classifying the landmark candidate area based on the deep convolutional neural network classifier, the method further includes: Converting the sample data into a grayscale image, and segmenting a landmark candidate region from the grayscale image of the sample data; Classifying the landmark candidate region in the sample data based on a deep convolutional neural network classifier to obtain a first landmark defect type of the sample data, and detecting and classifying the landmark candidate region in the sample data based on an image detection model according to feature data of a penultimate layer of a deep neural network in the deep convolutional neural network classifier to obtain a landmark region and a second landmark defect type in the sample data; Calculating the IoU loss between the landmark area predicted by the sample data and the marked area; Calculating a first FocalLoss loss between a first landmark defect type predicted by the sample data and a marked landmark defect type; Calculating a second FocalLoss loss between a second landmark defect type predicted by the sample data and annotated landmark defect type; According to the IoU loss, the first Focal Loss loss and the second Focal Loss loss, parameters of the deep convolutional neural network classifier and the image detection model are adjusted.

5. The long-distance direct buried optical cable benchmark quality inspection method according to claim 1 is characterized in that: The types of monument defects annotated for the sample data include one or more of abnormal monument excavation height, missing monument cap, monument writing defects, monument weed coverage, monument breakage and marker tilt.

6. The long-distance direct buried optical cable benchmark quality inspection method according to claim 5 is characterized in that: Before classifying the landmark candidate area based on the deep convolutional neural network classifier, the method further includes: Calculate the height and width of the contour line of the landmark area marked in the sample data; Calculating the ratio between the outline width and the logo design width; According to the ratio and the height of the contour line, the actual height of the mark in the sample data is calculated as the height of the marker stone unearthed; When the height of the landmark stone unearthed is greater than the second preset threshold or less than the third preset threshold, marking the sample data as abnormal height of the landmark stone unearthed; In the case where the contour line of the landmark area is a non-diamond structure, marking the sample data as a damaged landmark, and determining a damaged area in the landmark area according to the contour line of the landmark area; If the damaged area is above the contour line and the gray value standard deviation of the damaged area is greater than the first preset threshold, the sample data is marked as a landmark with missing cap; When the outline of the landmark area is a diamond structure and the width of the landmark area is consistent with the width of the outline, the landmark area above the outline is regarded as a cap-missing area, and the sample data is marked as a landmark-cap-missing area; When a weed area is identified from the landmark area, the weed area is below the contour line, and the grayscale value standard deviation of the weed area is greater than the first preset threshold, marking the sample data as landmark weed coverage; Performing text recognition on the monument area, comparing the recognized text with the phrases in the monument surface spray text library, and determining whether to mark the sample data as a monument text defect according to the comparison result; The verticality of the contour line is calculated, and when the verticality is greater than a fourth preset threshold, the sample data is marked as a landmark tilt.

7. The long-distance direct buried optical cable benchmark quality inspection method according to claim 1 is characterized in that: Also includes: According to the relationship between the coordinate system of the image data and the camera coordinate system of the intelligent inspection device that took the image data, and the positioning deviation of the optical cable marker to be detected relative to the intelligent inspection device in the world coordinate system, the absolute positioning information of the optical cable marker to be detected in the world coordinate system is calculated based on the GPS of the intelligent inspection device.

8. A long-distance direct buried optical cable marker quality inspection device, characterized in that: include: A grayscale segmentation module, used to convert the image data of the optical cable landmark to be detected into a grayscale image, and segment the landmark candidate area from the grayscale image; A classification detection module is used to classify the candidate area of ​​the landmark stone based on a deep convolutional neural network classifier, and detect and classify the candidate area of ​​the landmark stone based on an image detection model according to feature data of the penultimate layer of the deep neural network in the deep convolutional neural network classifier, so as to obtain the landmark stone area and the landmark stone defect type in the image data; The deep convolutional neural network classifier and the image detection model are obtained by training using the image data of the optical cable marker sample as sample data and using the marker area and marker defect type annotated on the sample data as labels.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the long-distance direct-buried optical cable marker quality inspection method as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the long-distance direct-buried optical cable marker quality inspection method as described in any one of claims 1 to 7 is implemented.

Images