Method and apparatus for self-identifying status of optical cross-connect port

Abstract

A self-correctable optical cross-box port state self-identification method and device, the method comprising: dividing the optical cross-box image in the data set into a training set, a validation set and a test set; constructing a calculation of the angle of the straight line where each row of ports is located, the scale of each port, and drawing the target heat map of the port; constructing a convolutional neural network. The convolutional neural network is trained through the images of the training set, and the parameters of the model are optimized; the hyperparameters of the model training are determined through the validation effect of the validation set; the target heat map of the port and the port related information prediction are generated; the coordinates, categories, angles and scales of the ports obtained by the port prediction denoising module are input into the tilt correction of the image through the tilt correction algorithm; whether there is a missing port in the row is determined through row and column matching, and the missing port is completed. The present application not only can automatically complete the tilt correction of the optical cross-box port image, but also can realize the port identification and row and column division of the optical cross-box after the correction of the optical cross-box port image.

Description

Technical Field

[0001] This application relates to the field of image detection, and in particular, to image detection of fiber optic junction boxes. Background Technology

[0002] With the development of communication and network technologies, the number and ports of fiber optic junction boxes, as one of the supporting devices, will increase rapidly. Fiber optic junction boxes are port devices used at the junction of backbone optical cables and distribution optical cables in fiber optic access networks. They are considered "dumb" resources, meaning the equipment itself cannot be subject to information-based monitoring. Furthermore, their diverse types make automated detection difficult through standardized pattern recognition. Currently, the main front-end maintenance method involves technicians periodically conducting field inspections to troubleshoot faulty ports and recording the status of each port in a table. However, each fiber optic junction box typically contains 200-400 fiber optic ports, and each province generates over 100,000 images annually for routine maintenance and resource surveys. Manual statistics and identification require significant manpower and resources. Therefore, operators and maintenance companies urgently need an automatic fiber optic port status detection system for fiber optic junction boxes.

[0003] Due to the large number of optical cable junction boxes, frequent status changes, and numerous ports in each box, manual identification is labor-intensive and prone to errors. Currently, some scholars both domestically and internationally have conducted research to achieve information-based management of optical cable junction boxes. Existing technologies can be mainly divided into two schemes: information-based retrofitting and automatic port status detection. In 2013, Shen Yue proposed an identification method using intelligent electronic tags to assist in the information-based management of optical cable junction box ports. In 2018, Chen Xiaobing proposed adding snap-on digital particles to the optical fibers of the splitter and then using a convolutional neural network to implement port resource management. However, these schemes require retrofitting and upgrading each optical cable junction box, making retrofitting and maintenance costs difficult to control. In 2021, He Jinghui proposed a visual detection method for port positioning and identification to automate the port detection process; however, a large amount of manual preprocessing is required before system detection. In 2022, Ge Xiong proposed a method and device for identifying the port status of optical cable junction boxes. This device combines character recognition in images with port region segmentation to achieve port status identification of optical cable junction boxes. In production environments, there are many types of optical distribution boxes, and most of their labels lack a standardized format and content, generally using a mix of handwritten and printed text. Upgrading to an information-based solution using the above methods is too costly. Intelligent detection solutions can only address the identification of port status for some optical distribution boxes with relatively unique labels, making it difficult to replace manual statistical identification work in production environments. Therefore, existing technologies require manual correction and cropping of optical distribution box port images, which consumes significant manpower and time.

[0004] Therefore, new technical solutions are urgently needed to address the shortcomings of existing technologies. Summary of the Invention

[0005] The purpose of this invention is to provide a self-correcting method and apparatus for identifying the port status of optical distribution boxes, which can not only automatically correct the tilt of the port image of the optical distribution box, but also identify the port and divide the rows and columns of the optical distribution box after the port image is corrected.

[0006] To achieve the above objectives, this invention provides a self-identifying method for the port status of optical distribution boxes with self-calibrating images, comprising: Step S1: dividing the optical distribution box images in the dataset into three parts: a training set, a validation set, and a test set; Step S2: during training and testing, constructing and calculating the straight-line angle of each port, the scale of each port, and drawing the target heatmap of the port by calling the port target heatmap function in real time based on the labeled port position and row / column; Step S3: constructing a convolutional neural network based on image characteristics. This convolutional neural network is used to detect the position, category, angle, and scale of the port. The port categories include ports at the beginning, end, and edge of a line; Step S4: Train a convolutional neural network using images from the training set to optimize the parameters of the optical distribution box port status self-recognition model; and determine the hyperparameters for model training by using the validation effect of the validation set; Step S5: Generate a heatmap and port-related information predictions for the optical distribution box port status self-recognition model by inputting test set images into the model; the port-related information includes the port's coordinates, category, angle, and scale; and, after removing heatmap noise using a heatmap denoising algorithm, extract the port's coordinates, category, angle, and scale; Step S6: Perform tilt correction on the input image using a tilt correction algorithm on the port coordinates, category, angle, and scale obtained from the port prediction denoising module; and input the corrected image into the optical distribution box port status self-recognition model to obtain the heatmap and port-related information predictions output by the model; the port-related information includes the port's coordinates, category, angle, and scale; and, after removing heatmap noise using a heatmap denoising algorithm, extract the port coordinates, category, angle, and scale again. Step S7: Use a row and column matching algorithm to comprehensively match the coordinates, category, and scale of each independent port, thereby connecting the independent ports into multiple rows; and use row and column matching to determine whether there are any missing ports in a row, and to fill in the missing ports.

[0007] In the method of this invention, step S1 includes: calculating the angle and scale of the port, and generating a regression heatmap. Furthermore, after data augmentation, the angle and scale of the port are recalculated, and a regression heatmap is generated again: Step S11: The sample set is divided, with one portion of the samples forming a test set and a validation set, and the other portion forming a training set. Step S12: Labeling information is added to each port in the sample. The labeling information includes port coordinates, port category, edge port attributes, and port row number.

[0008] In the method of the present invention, step S2 includes: Step S21, constructing and calculating the straight-line angle of each row port, including: taking the first and last port points A and B of the current row, where the coordinates of A are (x, y, y). a ,y a The coordinates of B are (x, y). b ,y b If ), then the slope of the straight line is... The angle between the line and the x-axis is θ0 = arctank, then the range of θ0 is [0°, 180°]. Let... Where θ is the angle value of the straight line where the port is located, and the value of θ ranges from (-90°, 90°). Step S22, construct and calculate the scale of each port, including: for each port, the distance between the port and the port to its left is W. left The distance between the port and its right-hand port is W. right The port's scale is Scale. If the first and last ports of a row have only one adjacent port on each side, then Scale = W. left Or Scale = W right Step S23, drawing the target heatmap includes: placing the Gaussian kernel corresponding to the port on a proportional coordinate system of the port category channel, where the size of the Gaussian kernel is proportional to the port scale. A regression heatmap is generated based on the size of the Gaussian kernel and the port angle and scale. Step S24, after data augmentation, the port angle and scale are recalculated, and a regression heatmap is generated again.

[0009] In the method of this invention, step S3 includes: Step S31: The optical distribution box port status self-identification model is composed of two cascaded hourglass networks, and the loss of each hourglass network is calculated. Step S32: 4x downsampling regression of the target heatmap and offset, generating the port location, classification, and edge port attributes through the target heatmap, and generating location information through the offset. Step S33: 16x downsampling regression of the straight line angle and scale. Step S34: The port target heatmap output of the optical distribution box port status self-identification model is normalized to between 0 and 1 using an S-shaped growth curve, the angle regression feature map is normalized to between -π / 2 and π / 2 after being multiplied by π / 2 using the tanh function, and other regression feature maps are directly output without activation functions.

[0010] In the method of this invention, step S4 includes: Step S41: Setting network training parameters. This involves training a convolutional neural network using an adaptive matrix estimation function with weight decay to obtain optimal parameters. Step S42: During training, randomly cropping images of appropriate sizes as training base images and normalizing them. After data augmentation, the port target heatmap function is called in real time to recalculate the angle and scale, and then fed into the optical distribution box port status self-recognition model for training.

[0011] In the method of this invention, generating a heatmap of the optical distribution box port status self-identification model and predicting port-related information includes: Step S51: Generating a port heatmap by accumulating category prediction heatmaps. Foreground and background are distinguished by a preset threshold. In the target heatmap, areas greater than or equal to the threshold are foreground, and areas less than the threshold are background. Step S52: Performing a morphological opening operation on the foreground to remove burrs and fine adhesions. Step S53: Extracting all connected regions and calculating the area of ​​each connected region. Removing connected regions with excessively large or small areas. Step S54: Removing connected regions with outliers. Step S55: Extracting the center points of all connected regions. Extracting the offset from the offset prediction map based on the center point coordinates, and generating port coordinates by adding the offset to the center point coordinates.

[0012] In the method of this invention, step S6 includes: Step S61: Extracting the category confidence, size information, and angle information of all ports from the corresponding feature map using port coordinates to generate an overall scale mean. Step S62: Performing a four-sided search, wherein the four-sided search includes searching the top and bottom sides and searching the left and right sides respectively. Step S63: Finding the intersection points of the four sides. Step S64: Determining the perspective transformation required to decorrect the image. Step S65: Applying the projection transformation obtained in step S64 to the image of the optical distribution box to obtain the corrected image. Step S66: Forwarding the corrected image through a deep learning optical distribution box port state self-recognition model to generate an output result. Step S67: Repeating the heatmap denoising algorithm in step S5 to remove heatmap noise to generate port coordinates.

[0013] In the method of this invention, the top and bottom edge search includes: simultaneously performing window sliding detection from the top and bottom ports as the starting positions of the sliding windows. The top window slides down from top to bottom and the bottom window slides up from bottom to top. During the sliding process, the ports within the window are detected. If both the top and bottom sliding windows meet the detection conditions or the two sliding windows overlap, the overall sliding window detection ends. Furthermore, the port lines within the windows that meet the conditions are extracted from both the top and bottom sliding windows. If no port line within the window meets the conditions, the line containing the most port points among the recorded lines of that window is extracted. After the top and bottom edge search is completed, the line consisting of one port point each at the top and bottom is output and stored. The left and right edge search includes: simultaneously performing window sliding detection from the leftmost and rightmost ports as the starting positions of the sliding windows. The left window slides to the right and the right window slides to the left. Furthermore, the lines within the window are detected during the sliding process. If both the left and right sliding windows meet the detection conditions or the two sliding windows overlap, the overall sliding window detection ends. Furthermore, the port lines within the windows that meet the conditions are extracted from both the left and right sliding windows. If no line within the window meets the criteria, extract the line with the most port points among the recorded lines in that window. Output and store the line consisting of one port point on each side.

[0014] In the method of this invention, the row-column matching algorithm includes: Step S71, row clustering: For all port points, points with a vertical distance less than a preset threshold are grouped into the same row. Step S72, column estimation: For all port points, points with a horizontal distance less than a preset threshold are grouped into the same column, and columns with only 2 ports are removed. Step S73, in-row scale estimation and filling: For rows with 4 or more ports, the estimated mean of the port size within the row is calculated. A port interval value is generated using the estimated mean, and the port interval value is compared with the actual port position to generate a matching degree. The port interval value corresponding to the highest matching degree is taken. Furthermore, if the interval between two rows is greater than double the port interval value, equal-interval port filling is performed within the interval. Step S74, row-column completion: For rows with less than 4 ports, the matching degree between the in-row ports and the column estimates is calculated. Furthermore, if the matching degree meets a preset matching degree threshold, the estimated port positions in the column are directly filled into the corresponding positions within the row.

[0015] This invention discloses a self-correcting optical distribution box (ODC) port status self-identification device. The device includes: a dataset type partitioning module, used to divide ODC images within the dataset into three parts: a training set, a validation set, and a test set; a port parameter construction and drawing module, during training and testing, constructing and calculating the straight-line angle of each port, the scale of each port, and drawing the target heatmap of the port by real-time invocation of the port target heatmap function based on the labeled port positions and rows / columns; a convolutional neural network (CNN) construction module, used to construct a CNN based on image characteristics. This CNN is used to detect the position, category, angle, and scale of the ports. Port categories include ports at the beginning, end, and edge of rows; a parameter optimization and extraction module, which trains the CNN using images from the training set to optimize the parameters of the ODC port status self-identification model; and a parameter optimization and extraction module, which determines the hyperparameters for model training based on the validation results of the validation set; and a port prediction and denoising module, which inputs test set images into the ODC port status self-identification model to obtain the output heatmap and port-related information predictions from the model. Port-related information includes the port's coordinates, category, angle, and scale. The port prediction denoising module removes heatmap noise using a heatmap denoising algorithm and then extracts the port's coordinates, category, angle, and scale. The port tilt correction denoising module corrects the tilt of the input image using a tilt correction algorithm based on the port coordinates, category, angle, and scale obtained from the port prediction denoising module. This corrected image is then input into the optical distribution box port status self-recognition model to obtain the heatmap and port-related information predictions output by the model. Port-related information includes the port's coordinates, category, angle, and scale. The port tilt correction denoising module removes heatmap noise using a heatmap denoising algorithm and then extracts the port coordinates, category, angle, and scale again. The missing port completion module uses a row-column matching algorithm to comprehensively match the coordinates, category, and scale of each independent port, connecting the independent ports into multiple rows. The missing port completion module uses row-column matching to determine if there are any missing ports within a row and completes the missing ports.

[0016] The present invention provides a self-correcting optical distribution box port status self-identification method and device that can not only automatically complete the tilt correction of the optical distribution box port image, but also realize the port identification and row and column division of the optical distribution box after the optical distribution box port image is corrected. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the method of the present invention.

[0018] Figure 2 This is a schematic diagram of the target heat map and the four-sided detection results during the intermediate process of tilt correction in this invention.

[0019] Figure 3 This is a target heat map of the intermediate process of tilt correction in this invention and a schematic diagram of the result after four-sided correction.

[0020] Figure 4 This is a heatmap diagram of the row division result after row and column completion in this invention.

[0021] Figure 5 This is a heatmap diagram of the column division result after row and column completion in this invention.

[0022] Figure 6 The table shows the port detection accuracy of different methods on the test set. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments described herein are specific implementations of the present invention, used to illustrate the concept of the present invention, and are illustrative and exemplary, and should not be construed as limiting the embodiments or scope of the present invention. In addition to the embodiments described herein, those skilled in the art can employ other obvious technical solutions based on the content disclosed in the claims and specification of this application. These technical solutions include those that make any obvious substitutions and modifications to the embodiments described herein.

[0024] The accompanying drawings in this specification are schematic diagrams used to help illustrate the concept of the invention and to schematically show the interrelationships of the various parts.

[0025] This invention uses a dataset of on-site photos of fiber optic junction boxes as an example dataset, containing 7550 images taken on-site. Each port in these images is labeled with its coordinates, port type, whether it is an edge port, and its row number. Port coordinates are in (x, y) format; port types include three categories: occupied port, idle port, and obstructed port; whether a port is an edge port is indicated by a Boolean (True / False); and the row number is an integer (int), with ports having the same row number belonging to the same row. Based on a constructed convolutional neural network, this invention outputs a corresponding heatmap, removes noise from the heatmap using a denoising algorithm, extracts port coordinates from the category heatmap, connects independent ports into multiple rows using a row matching algorithm, fills in missed ports using an inline completion algorithm, and finally outputs the port detection results.

[0026] Figure 1 A flowchart of the method of the present invention is shown, as follows. Figure 1 As shown, the self-identification method for the port status of an optical distribution box with self-calibration capability disclosed in this invention includes:

[0027] Step S1: Divide the optical cross-box images in the dataset into three parts: training set, validation set, and test set.

[0028] Specifically, 70% of the 7,550 photos of fiber optic junction boxes were selected for constructing the training set, 10% for constructing the validation set, and the remaining 20% ​​for constructing the test set.

[0029] Step S2: During training and testing, the port target heatmap function is called in real time to construct and calculate the straight line angle of each port, the scale of each port, and draw the target heatmap of the port based on the labeled port position and row and column.

[0030] Specifically, first, a function is constructed to calculate the port angle. This function reads all points within the same row of the graph and returns the angle formed by the line containing that row and the horizontal line, with the angle ranging from -90° to 90°. Second, a function is constructed to calculate the port scale. This function primarily returns the port scale by calculating the distance between the port and its neighboring points. Finally, a function is constructed to generate a heatmap, producing a 6-dimensional heatmap with 4x downsampling. Three dimensions are for port category heatmaps, two dimensions are for offset regression, and one dimension is for edge port heatmaps.

[0031] Step S3: Construct a convolutional neural network based on image characteristics. This convolutional neural network is used to detect the position, category, angle, and scale of ports. Port categories include ports at the beginning, end, and edge of a line.

[0032] Step S4: Train a convolutional neural network using images from the training set to optimize the parameters of the optical junction box port status self-identification model. Determine the hyperparameters for model training based on the validation results on the validation set.

[0033] Specifically, in step S4, the network parameters are first initialized, with a training learning rate of 10⁻³, momentum of 0.9, a training batch size of 16, and 200 training iterations. Next, the network is forward-propagated: training images are randomly cropped to 512×512 size after data augmentation (rotation, scaling, brightness adjustment), normalized, and then input into the network for forward propagation, outputting relevant predicted heatmaps. Simultaneously, labeled heatmaps for angles, scales, and port categories are generated based on the input images. Finally, the error is backpropagated: the category heatmap and edge port heatmap are activated using the sigmoid function, and the loss is calculated using the MSE loss function. The angle and scale losses are calculated using the smooth L1 loss function. The loss functions of each part are summed to obtain the final loss function. Error backpropagation updates the network parameters. After each training round, the model calculates the overall loss function on the validation set. After 200 training iterations, the model with the smallest loss function on the validation set is selected as the optimal neural network.

[0034] Step S5: By inputting the test set images into the optical distribution box port status self-identification model, a heatmap and port-related information predictions for the model are generated. Port-related information includes the port's coordinates, category, angle, and scale. Furthermore, after removing noise from the heatmap using a heatmap denoising algorithm, the port's coordinates, category, angle, and scale are extracted.

[0035] Specifically, test set images are fed into the model, which outputs predicted heatmaps. The category heatmaps are summed along the category dimension to obtain the target heatmap. Foreground is extracted from the target heatmap using a thresholding method. Small noise and contiguous areas are removed from the foreground using image morphological opening operations. Then, regions that are too large or too small are removed by connected region area statistics, and the center point of each region is used as the port location. Based on the region center point location, the offset is obtained from the offset prediction map, and these are then superimposed to obtain the port coordinates.

[0036] Step S6: Figure 2-3 A diagram illustrating the intermediate process of tilt correction is shown. (See diagram for example.) Figure 2-3 As shown, a tilt correction algorithm is used to correct the tilt of the input image based on the port coordinates, category, angle, and scale obtained by the port prediction and denoising module. The corrected image is then input into the optical distribution box port state self-identification model to obtain the heatmap and port-related information predictions output by the model. The port-related information includes the port's coordinates, category, angle, and scale. Furthermore, after removing noise from the heatmap using a heatmap denoising algorithm, the port coordinates, category, angle, and scale are extracted again.

[0037] Specifically, the corrected image is input into the optical distribution box port status self-identification model and passed through the model again for heatmap denoising.

[0038] Step S7: A row-column matching algorithm is used to comprehensively match the coordinates, category, and scale of each independent port, thereby connecting the independent ports into multiple rows. Furthermore, row-column matching is used to determine if any ports are missing within a row, and to complete the missing ports.

[0039] In a further preferred embodiment, step S1 includes:

[0040] The angle and scale of the ports are calculated, and a regression heatmap is generated. Furthermore, after data augmentation, the angle and scale of the ports are recalculated, and a regression heatmap is generated again.

[0041] Step S11: Divide the sample set into two parts: select a portion of the samples to form the test set and the validation set, and select another portion of the samples to form the training set.

[0042] Step S12: Add annotation information for each port in the sample. The annotation information includes port coordinates, port category, edge port attributes, and port row number.

[0043] In a further preferred embodiment, step S2 includes...

[0044] Step S21, construct and calculate the straight-line angle of each row's port, including: taking the first and last port points A and B of the current row, where the coordinates of A are (x, y, y). a ,y a The coordinates of B are (x, y). b ,y b If ), then the slope of the straight line is... The angle between the line and the x-axis is θ0 = arctank, then the range of θ0 is [0°, 180°]. Let...

[0045] in,

[0046] θ is the angle of the line containing the port, and its range is (-90°, 90°). The actual regression uses radians, with a range of...

[0047] Step S22, construct and calculate the scale for each port, including: for each port, the distance between the port and its left port is W. left The distance between the port and its right-hand port is W. right The port's scale is Scale. If the first and last ports of a row have only one adjacent port on each side, then Scale = W. left Or Scale = W right .

[0048] Step S23, drawing the target heatmap includes:

[0049] A Gaussian kernel corresponding to each port is placed on a coordinate system proportional to the port category's channel size, where the kernel size is proportional to the port's scale. A regression heatmap is generated based on the kernel size and the port's angle and scale.

[0050] Specifically, the target heatmap is constructed at a 4x downsampling resolution with 6 channels, corresponding to 3 categories. Two channels represent the offsets in the x and y directions, and one channel represents whether a port is an edge port. The edge port regression takes the same form as the port category, but with an additional label independent of the port category. The offset value is the difference between the actual port coordinates and the rounded coordinates in the 4x downsampling heatmap, representing the coordinate accuracy loss due to the 4x downsampling rounding, and includes components in both the x and y directions.

[0051] Step S24: After data augmentation, the angle and scale of the ports are recalculated, and a regression heatmap is generated.

[0052] Specifically, data augmentation employs techniques such as rotation, flipping, and affine transformation.

[0053] In a further preferred embodiment, step S3 includes:

[0054] Step S31: The optical distribution box port status self-identification model is composed of two cascaded hourglass networks. The loss of the two hourglass networks is calculated respectively.

[0055] Specifically, both hourglass networks have 4x downsampled heatmap outputs and 16x downsampled heatmap outputs.

[0056] Step S32: 4x downsampling regression of target heatmap and offset, generating port location, classification and edge port attributes through target heatmap, and generating location information through offset.

[0057] Specifically, the target heatmap can provide a rough location of the port, its classification, and whether it is an edge port, while the offset can provide more precise location information.

[0058] Step S33: Regression line angle and scale under 16x sampling.

[0059] Step S34: The port target heat map output of the optical distribution box port status self-identification model is normalized to between 0 and 1 by the S-shaped growth curve. The angle regression feature map is normalized to between -π / 2 and π / 2 by multiplying it by π / 2 after the tanh function. Other regression feature maps are directly output without activation function.

[0060] In a further preferred embodiment, step S4 includes:

[0061] Step S41: Set the network training parameters. Use an adaptive matrix estimation function with weight decay to train the convolutional neural network to obtain the optimal parameters.

[0062] Specifically, when setting the network training parameters, the heatmap loss function is the mean squared error loss (MSE loss), and the offset, angle, and scale loss functions are the smooth L1 norm loss. The convolutional neural network is trained using the Adam with decoupled weight decay function to obtain the optimal parameters.

[0063] Step S42: During training, images of corresponding sizes are randomly cropped as training base images and normalized. After data augmentation, the port target heatmap function is called in real time to recalculate the angle and scale, and then fed into the optical distribution box port status self-recognition model for training.

[0064] Specifically, during training, images of 512×512 size are randomly cropped as the training base images and normalized. After data augmentation using methods such as rotation, scaling, and brightness adjustment, the port target heatmap function is called according to the method in S2 to recalculate the angle and scale, and the training base images are then fed into the model for training.

[0065] In a further preferred embodiment, the generation of the heatmap and port-related information prediction of the optical distribution box port status self-identification model includes:

[0066] Step S51: Generate a port heatmap by accumulating category prediction heatmaps. Foreground and background are distinguished using a preset threshold. In the target heatmap, values ​​greater than or equal to the threshold are considered foreground, and values ​​less than the threshold are considered background. The threshold used is th = max(th...). osu ,0.4) where th osu To obtain the Otsu threshold using the Otsu method.

[0067] Step S52: Perform an opening operation on the foreground to remove burrs and small adhesions.

[0068] Step S53: Extract all connected components and calculate the area of ​​each connected component. Remove connected components with excessively large or small areas.

[0069] Specifically, connected regions with an area less than 1 / 3 of the average area and an area greater than twice the average area are removed.

[0070] Step S54: Remove outlier connected components.

[0071] Step S55: Extract the center points of all connected regions. Extract the offsets from the offset prediction map based on the center point coordinates, and generate port coordinates by adding the offsets to the center point coordinates.

[0072] In a further preferred embodiment, step S6 includes:

[0073] Step S61: Using the port coordinates, extract the category confidence, size information, and angle information of all ports from the corresponding feature map to generate the overall scale mean S. mean .

[0074] Step S62: Perform a four-sided search, which includes searching the top and bottom edges and searching the left and right edges respectively.

[0075] Step S63: Find the intersection of the four sides.

[0076] Specifically, to find L up L down L left L rightThe four intersection points of the four lines are denoted as p0, p1, p2, and p3, where p0 is the line length of line L. up With L left The intersection point, p1 is L up With L right The intersection point, p2 is L down With L right The intersection point, p3 is L down With L left The intersection point. p i The coordinates are (x i ,y i ).

[0077] Step S64: Determine the perspective transformation required for the decorrected image.

[0078] Specifically, calculate the new points q0, q1, q2, and q3 corresponding to p0, p1, p2, and p3. Then, find x. left =min(x0,x1,x2,x3), that is, x left Let x be the leftmost x-value. Find x. right =max(x0,x1,x2,x3), that is, x right Let x be the rightmost x-value. Find y. up =min(y0,y1,y2,y3), that is, y up Let y be the value at the top. Calculate y. down =max(y0,y1,y2,y3), that is, y down Let y be the bottommost y value. Then the coordinates of q0 are (x... left ,y up ), q1 coordinates are (x right ,y up ),q2 coordinates are (x right ,y down ),q3 coordinates are (x left ,y down Based on the correspondence between (p0, p1, p2, p3) and (q0, q1, q2, q3), the perspective transformation required for correcting the image is solved. The perspective transformation has eight parameters, which can be solved based on four pairs of corresponding points.

[0079] Step S65: Perform the transformation by applying the projection transformation obtained in step S64 to the image of the optical distribution box to obtain the corrected image.

[0080] Step S66: The corrected image is forwarded through the deep learning optical distribution box port state self-recognition model to generate the output result.

[0081] Step S67: Repeat the heatmap denoising algorithm in step S5 to remove heatmap noise in order to generate port coordinates.

[0082] In a further preferred embodiment, the top and bottom edge search includes:

[0083] Simultaneously, sliding window detection is performed from both the top and bottom ports as the starting positions. The top window slides down from top to bottom, and the bottom window slides up from bottom to top. (2*S) mean For the window width, use 0.4*S mean The step size is used to detect the ports within the window during the sliding window process.

[0084] Specifically, extract ports D = {d1, d2, ..., d} within the window. n}, the angular regression value is ang i The scaling regression value is s i The category confidence level is c. i Extract all combinations of 4 elements from D. The total number of combinations is . Regress the equations of the lines containing all the points where the points are combined, and assign all points whose distance to the line is less than 0.15*S. mean All port points belong to this line. Regress the line equation based on the port points of each line, and calculate the average distance between the port points and the line. Check the following conditions:

[0085] (a) There are more than 6 ports on the straight line.

[0086] (b) The average distance between the port point and the line is less than 0.08*S. mean hour,

[0087] (c) The regression mean of the port point angle differs from the actual angle of the line by less than 30°.

[0088] The detection stops when all three conditions (a), (b), and (c) are met.

[0089] Otherwise, record that the average distance between all port points and the line is less than 0.1*S. mean A straight line. If a sliding window meets the condition, then the sliding window stops.

[0090] If both the upper and lower sliding windows meet the detection conditions, or if the two sliding windows overlap, the overall sliding window detection ends. Furthermore, the port lines within the windows that meet the conditions are extracted from both the upper and lower sliding windows respectively.

[0091] If no line within the window meets the conditions, then extract the line with the most port points among the recorded lines in that window.

[0092] Specifically, if no line within the window meets the criteria, the line containing the most port points among the lines already recorded in that window is extracted. After searching the top and bottom edges, a line consisting of one port point at the top and one at the bottom is obtained, denoted as L. up L down .

[0093] After completing the top and bottom edge search, output and store the straight line formed by one port point at the top and one at the bottom.

[0094] In a further preferred embodiment, the left and right side search includes:

[0095] Simultaneously, sliding window detection is performed starting from the leftmost and rightmost ports. This includes detecting when the left window slides to the right and the right window slides to the left. Furthermore, straight lines within the window are detected during the sliding process.

[0096] Specifically, the left and right side search detects straight lines within the window using a method similar to sliding windows on the top and bottom sides.

[0097] If both left and right sliding windows meet the detection conditions or the two sliding windows overlap, the overall sliding window detection ends. Furthermore, the port lines within the windows that meet the conditions are extracted from both the left and right sliding windows respectively.

[0098] If no line within the window meets the conditions, then extract the line with the most port points among the recorded lines in that window.

[0099] Output and store the straight line formed by one port point on each side, denoted as L. left L right .

[0100] In a further preferred embodiment, the row-column matching algorithm includes:

[0101] Step S71, row clustering: For all port points, group points with a vertical distance less than a preset threshold into the same row.

[0102] Specifically, for all port points, according to the ordinate, with 0.5*S mean Using a threshold, points with a vertical distance less than the threshold are grouped into the same row.

[0103] Step S72, column estimation: For all port points, group points with a horizontal distance less than a preset threshold into the same column, and remove columns with only 2 ports.

[0104] Specifically, for all port points, according to the x-coordinate, with 0.3*S mean Using a threshold, points with a horizontal distance less than the threshold are grouped into the same column, and columns containing only ports less than 2 are removed.

[0105] Step S73, In-row Scale Estimation and Filling: For rows with 4 or more ports, calculate the estimated mean of the port sizes within the row. Generate port spacing values ​​using the estimated mean, compare the port spacing values ​​with the actual port positions to generate a conformity score, and take the port spacing value corresponding to the highest conformity score. Furthermore, if there are two rows with a spacing greater than double the port spacing value, then perform equal-spaced port filling within that spacing.

[0106] Specifically, for rows with 4 or more ports, calculate the mean estimated port size S within the row. line_mean , at 0.8*S line_mean Up to 1.2*S line_mean Take 20 values ​​at equal intervals, calculate the degree of agreement between each port interval value and the actual port location, and take the value with the highest degree of agreement as the S value for that row. line For lines with an inline distance greater than 2*S line The process is performed at intervals, let the interval distance be S. d The number of ports to fill, n = round(S) d / S line -0.2)-1, where round means rounding. Then, evenly spaced ports are filled in between, and the port type for filling is set to "obstructed".

[0107] Step S74, Row and Column Completion: For rows with fewer than 4 ports, calculate the matching degree between the ports in the row and the estimated columns. Furthermore, if the matching degree meets a preset matching degree threshold, the estimated port positions in the columns are directly filled into the corresponding positions in the row.

[0108] Specifically, after directly filling the estimated port locations in the columns into the corresponding positions in the rows, the port categories filled are all set to "occluded".

[0109] This invention provides a self-calibrating optical distribution box port status self-identification device, comprising:

[0110] The dataset type partitioning module is used to divide the optical cross-box images in the dataset into three parts: training set, validation set, and test set.

[0111] During training and testing, the port parameter construction and drawing module constructs and calculates the straight-line angle of each row of ports, the scale of each port, and draws the target heatmap of the ports by calling the port target heatmap function in real time based on the labeled port positions and rows and columns.

[0112] The convolutional neural network (CNN) construction module is used to construct a CNN based on image characteristics. This CNN is used to detect the position, category, angle, and scale of ports. Port categories include ports at the beginning, end, and edge of a line.

[0113] The parameter optimization extraction module trains a convolutional neural network using images from the training set, thereby optimizing the parameters of the optical junction box port status self-identification model. The module also determines the hyperparameters for model training based on the validation results on the validation set.

[0114] The port prediction and denoising module inputs test set images into the optical distribution box port status self-identification model to obtain the heatmap and port-related information predictions output by the model. The port-related information includes the port's coordinates, category, angle, and scale. Furthermore, the module removes noise from the heatmap using a heatmap denoising algorithm and then extracts the port's coordinates, category, angle, and scale.

[0115] The port tilt correction and denoising module corrects the tilt of the input image by applying a tilt correction algorithm to the port coordinates, category, angle, and scale obtained from the port prediction and denoising module. This corrected image is then input into the optical distribution box port status self-recognition model to obtain the heatmap and port-related information predictions output by the model. The port-related information includes the port's coordinates, category, angle, and scale. Furthermore, after removing heatmap noise using a heatmap denoising algorithm, the port tilt correction and denoising module extracts the port coordinates, category, angle, and scale again.

[0116] The missing port completion module uses a row-column matching algorithm to comprehensively match the coordinates, category, and scale of each individual port, thereby connecting the individual ports into multiple rows. Furthermore, the missing port completion module uses row-column matching to determine if there are any missing ports within a row and then completes the missing ports.

[0117] For a comparative statistical analysis of the self-identification method of this invention and the baseline algorithm, please refer to [link / reference]. Figure 6 The table shows the port detection accuracy of different methods on the test set. Figure 6 It can be seen that the accuracy of the stacked hourglass network method is 0.81, while the accuracy of the stacked hourglass network + additional regression + tilt correction algorithm method used in this invention is as high as 0.95.

[0118] Existing technologies require manual image correction and cropping, which are labor-intensive. In contrast, this invention provides a self-correcting method and apparatus for identifying the port status of optical fiber junction boxes, capable of automatically correcting tilt through algorithms. After tilt correction, combined with post-image processing algorithms, port identification and row / column division of the corrected optical fiber junction box image are achieved. This invention can automatically correct image tilt without relying on character recognition, making it a widely applicable method for detecting port images of optical fiber junction boxes.

[0119] The present invention has the following advantages over the prior art:

[0120] First, this invention eliminates the need for manual image correction and cropping. By regressing the coordinates, angles, and scales of each port and using a tilt correction algorithm, the tilt correction of the image can be completed.

[0121] Secondly, based on tilt correction, this invention uses a post-processing algorithm to complete port identification and row / column division of the fiber optic junction box image.

[0122] The embodiments of the self-calibrating optical distribution box port status self-identification method and apparatus of the present invention have been described above, with the aim of explaining the spirit of the present invention. Please note that those skilled in the art can modify and combine the features of the above embodiments without departing from the spirit of the present invention; therefore, the present invention is not limited to the above embodiments. Specific features of the self-calibrating optical distribution box port status self-identification apparatus of the present invention, such as shape, size, and position, can be specifically designed based on the functions of the features disclosed above, and these designs are all achievable by those skilled in the art. Furthermore, the disclosed technical features are not limited to the disclosed combinations with other features; those skilled in the art can also make other combinations between the technical features according to the purpose of the invention to achieve the purpose of the present invention.

Claims

1. A self-correctable optical cross-connect port status self-identifying method, characterized in that, The method comprises the following steps: Step S1: dividing the optical crossbox images in the data set into three parts of a training set, a validation set and a test set; Step S2: in the process of training and testing, a port target heat map function is called in real time to construct the angle of the straight line where each row of ports is located, the size of each port, and draw the target heat map of the port according to the labeled port position and the row and column; Step S3: constructing a convolutional neural network according to the image characteristics; wherein the convolutional neural network is used to detect the position, category, angle and size of the port; the category of the port includes the port at the beginning, end and edge of the row; Step S4: training the convolutional neural network by the images of the training set, so as to optimize the parameters of the optical crossbox port state self-recognition model; and determining the hyperparameters of the model training by the validation effect of the validation set; Step S5: inputting the test set images into the optical crossbox port state self-recognition model to generate the heat map of the optical crossbox port state self-recognition model and the prediction of the port related information; wherein the port related information includes the coordinates, category, angle and size of the port; and after removing the heat map noise by the heat map denoising algorithm, the coordinates, category, angle and size of the port are extracted; Step S6: inputting the coordinates, category, angle and size of the port obtained by the port prediction denoising module into the tilt correction algorithm for tilt correction of the input image; and inputting the corrected image into the optical crossbox port state self-recognition model, so as to obtain the heat map output by the optical crossbox port state self-recognition model and the prediction of the port related information; wherein the port related information includes the coordinates, category, angle and size of the port; and after removing the heat map noise by the heat map denoising algorithm, the coordinates, category, angle and size of the port are extracted again; Step S7: comprehensively matching the coordinates, category and size of each independent port by the row and column matching algorithm, so as to connect the independent ports into multiple rows; and determining whether there is a missing port in the row by the row and column matching, and completing the missing port, Wherein, step S2 comprises: Step S21, constructing calculating the angle of the straight line where each row of ports is located, comprising: taking the current row head and tail port points A, B, wherein the coordinates of A are (x a , y a ), the coordinates of B are (x b , y b ), the slope of the straight line is , the angle between the straight line and the x-axis is , then the value range of is [0°, 180°], and it is assumed that , wherein, is an angle value of a straight line where the port is located, and the value range of is ; Step S22, constructing the scale of each port, including: for each port, the port is W distance from its left port left , the port is W distance from its right port right , the scale of the port is Scale, , the first and last ports of a row only have adjacent ports on one side, then or ; Step S23, drawing a target heat map comprises: placing a Gaussian kernel corresponding to the port on the same scale coordinates of the port category channel, wherein the size of the Gaussian kernel is proportional to the size of the port; and generating a regression heat map according to the size of the Gaussian kernel and the angle and size of the port; Step S24, after data augmentation, the angle and size of the port are recalculated, and a regression heat map is generated.

2. The method of claim 1, wherein, Wherein, The step S1 comprises: calculating the angle and size of the port, and generating a regression heat map; and after data augmentation, the angle and size of the port are recalculated, and a regression heat map is generated: Step S11: dividing the sample set, selecting a part of the samples to constitute a test set and a validation set, and another part of the samples to constitute a training set; Step S12: adding label information to each port in the sample; wherein the label information includes port coordinates, port category, edge port attribute and port row number.

3. The method of claim 1, wherein, Wherein, The step S3 comprises: Step S31: the optical crossbox port state self-recognition model is composed of two hourglass network cascades, and the losses of the two hourglass networks are calculated respectively; Step S32: 4 times down-sampling regression target heat map and offset, generating the position, classification and edge port attribute of the target heat map, and generating the position information through the offset; Step S33: 16 times down-sampling regression straight line angle and scale; Step S34: the port target heat map output of the optical cross box port state self-recognition model is normalized to 0~1 through an S-shaped growth curve, the angle regression feature map is multiplied by π / 2 after passing through a tanh function and is normalized to -π / 2~π / 2, and other regression feature maps are directly outputted without passing through an activation function.

4. The method of claim 1, wherein, Wherein, The step S4 comprises: Step S41: setting network training parameters; and using an adaptive matrix estimation function with weight decay to train the convolutional neural network, so as to obtain optimal parameters; Step S42: during training, randomly cropping an image of a corresponding size as a training basic image, performing normalization; and after data enhancement, calling a port target heat map function in real time to recalculate the angle and scale, and inputting the angle and scale into the optical cross box port state self-recognition model for training.

5. The method of claim 1, wherein, Wherein, The heat map and port related information prediction of the optical cross box port state self-recognition model comprises: Step S51: generating a port heat map through accumulated category prediction heat maps; and dividing foreground and background through a preset threshold; wherein, in the target heat map, the foreground is greater than or equal to the threshold, and the background is less than the threshold; Step S52: performing an open operation of morphological processing on the foreground, so as to remove burrs and small adhesions; Step S53: extracting all connected regions, and counting the area of each connected region; removing corresponding connected regions with excessively large and small areas; Step S54: removing connected regions with position outliers; Step S55: extracting the center point of all connected regions; and extracting the offset through the center point coordinate in the offset prediction map, and generating the port coordinate through the center point coordinate plus the offset.

6. The method of claim 1, wherein, Wherein, The step S6 comprises: Step S61: extracting the category confidence, size information and angle information of all ports on the corresponding feature map through the port coordinate, to generate an overall scale mean; Step S62: performing a four-corner search, wherein the four-corner search comprises performing an up-down edge search and a left-right edge search respectively; Step S63: calculating the intersection of four corners; Step S64: determining the perspective transformation required for correcting the image; Step S65: applying the projection transformation obtained in step S64 to the image of the optical cross box, to obtain a corrected image; Step S66: forwarding the corrected image through the optical cross box port state self-recognition model of deep learning, to generate an output result; Step S67: repeating the heat map denoising algorithm in step S5 to remove heat map noise, to generate a port coordinate.

7. The method according to claim 6, wherein, The up-down edge search comprises: Simultaneously taking the ports at the upper and lower ends as the starting positions of the sliding window to perform sliding window detection; sliding the upper end window from top to bottom and sliding the lower end window from bottom to top; and detecting the ports in the window during the sliding. If both the upper and lower sliding windows meet the detection condition or the upper and lower sliding windows overlap each other, the overall sliding window detection is ended; and a straight line formed by port points in the window meeting the condition in the upper and lower sliding windows is extracted, respectively; If there is no straight line in the window meeting the condition, a straight line containing the most port points in the recorded straight line of the window is extracted; After the upper and lower edge search, a straight line formed by port points in the upper and lower edges is output and stored; The left and right edge search includes: Simultaneously, the leftmost and rightmost ports are taken as the starting positions of the sliding windows for the sliding window detection; and the left window slides to the right and the right window slides to the left; and in the process of the sliding window, the straight lines in the window are detected; If both the left and right sliding windows meet the detection condition or the left and right sliding windows overlap each other, the overall sliding window detection is ended; and a straight line formed by port points in the window meeting the condition in the left and right sliding windows is extracted, respectively; If there is no straight line in the window meeting the condition, a straight line containing the most port points in the recorded straight line of the window is extracted; A straight line formed by port points in the left and right edges is output and stored.

8. The method of claim 1, wherein, Wherein, The row and column matching algorithm includes: Step S71, row clustering: for all port points, points with a vertical distance less than a preset threshold value are classified into the same row; Step S72, column estimation: for all port points, points with a horizontal distance less than a preset threshold value are classified into the same column, and columns with less than 2 ports are removed; Step S73, intra-row scale estimation and filling: for rows with more than or equal to 4 ports, the estimated mean of the intra-row port size is calculated; and the estimated mean is used to generate a port spacing value, the port spacing value is compared with the actual position of the port to generate a degree of conformity, and the port spacing value corresponding to the highest degree of conformity is taken; and if the spacing between two rows is greater than twice the port spacing value, equally spaced ports are filled in the spacing; Step S74, row and column completion: for rows with less than 4 ports, the matching degree of the intra-row port and the column estimation is calculated; and if the matching degree meets a preset matching degree threshold value, the port position of the column estimation is directly filled into the corresponding position in the row.

9. A self-correcting optical cross-connect port status self-identifying device for use in the method of any of claims 1-8, characterized by, It includes: The data set type division module is used to divide the optical cross-box images in the data set into three parts of a training set, a validation set and a test set; The port parameter construction and drawing module constructs and calculates the angle of the straight line where each row of ports is located, the scale of each port, and draws the target heat map of the port by calling the port target heat map function in real time according to the labeled port position and the row and column during the training and testing; The convolutional neural network construction module is used to construct a convolutional neural network according to the image characteristics; wherein the convolutional neural network is used to detect the position, category, angle and scale of the port; the category of the port includes the port at the head, tail and edge of the row; The parameter optimization extraction module trains the convolutional neural network through the images of the training set, thereby optimizing the parameters of the optical cross-box port state self-recognition model; and the parameter optimization extraction module determines the hyperparameters of the model training through the validation effect of the validation set; The row and column matching algorithm includes: The port prediction denoising module obtains the heat map output by the optical crossbox port state self-recognition model and port-related information prediction by inputting the test set image into the optical crossbox port state self-recognition model; the port-related information includes the coordinates, category, angle and scale of the port; and the port prediction denoising module extracts the coordinates, category, angle and scale of the port after removing the heat map noise by the heat map denoising algorithm. The port tilt correction denoising module performs tilt correction on the coordinates, category, angle and scale of the port obtained by the port prediction denoising module by using a tilt correction algorithm; and the port tilt correction denoising module inputs the corrected image into the optical crossbox port state self-recognition model, thereby obtaining the heat map output by the optical crossbox port state self-recognition model and port-related information prediction; the port-related information includes the coordinates, category, angle and scale of the port; and the port tilt correction denoising module extracts the coordinates, category, angle and scale of the port again after removing the heat map noise by the heat map denoising algorithm. The missing port completion module comprehensively matches the coordinates, category and scale of each independent port by using a row-column matching algorithm, thereby connecting the independent ports into multiple rows; and the missing port completion module determines whether there is a missing port in the row and completes the missing port by using row-column matching.

Images