Method and device for identifying the port status of optical fiber junction boxes

Abstract

This invention provides a method and apparatus for identifying the status of ports in an optical cable junction box. The method includes: acquiring an image of the optical cable junction box; determining the character recognition result of the image; wherein the character recognition result indicates the position of multiple port characters of the optical cable junction box in the image; cropping multiple rows of ports in the image that correspond one-to-one with the multiple port character images based on the character recognition result, to determine multiple row port images that correspond one-to-one with the multiple rows of ports; performing port positioning on each of the multiple row port images, to determine the port positioning result; wherein the port positioning result indicates the position of each port in the multiple rows of ports in the row port image; cropping single ports on each of the multiple row port images based on the port positioning result, to determine multiple single port images; and performing status recognition on each of the multiple single port images to determine the status of each of the multiple single port images. This method can improve the accuracy of single port segmentation, thereby improving the accuracy of status recognition.

Description

Technical Field

[0001] This invention relates to the field of intelligent identification technology, and in particular to a method and apparatus for identifying the status of ports in optical cable junction boxes. Background Technology

[0002] With the rapid development of internet technology, fiber optic networks are widely used in cities. To improve the utilization efficiency of backbone fiber and reduce the number of pairs in the backbone, fiber optic transmission boxes are needed to broadcast to multiple fiber cores through physical channels. Nowadays, fiber optic transmission boxes are ubiquitous on streets. To maintain their proper functioning, telecommunications companies need to regularly arrange for staff to inspect the port status of each transmission box. Due to the large number of ports in each box, coupled with interference from messy fiber optic lines, manual inspection becomes extremely inefficient and may lead to visual fatigue and inaccurate results.

[0003] Currently, computer vision technology is used to process images of fiber optic transmission boxes to quickly determine the port status. Existing object detection methods can be broadly categorized into two types based on feature extraction methods. The first type is traditional object detection methods, which first generate a batch of candidate regions on a given image, then manually design and extract features from these regions, and finally use a classifier to classify the feature vectors. Traditional object detection methods suffer from problems such as non-specific region selection strategies, high time complexity, and poor robustness of manually designed features to diverse variations. The second type is object detection methods based on deep learning. These methods mainly utilize the powerful automatic feature extraction capabilities and backpropagation mechanism of Convolutional Neural Networks (CNNs) to implement a classifier that can be trained end-to-end. Compared to previous multilayer perceptrons, the local connectivity and weight sharing features of CNNs reduce the complexity of the network model and the number of parameters that need to be learned. Relying on high-performance computing hardware, representative second-order networks such as SSP-Net, Faster R-CNN, and R-FCN, and first-order networks such as SSD, Yolov3, and Yolov4, can maintain high detection accuracy and fast detection speed for targets of normal size on computers with GPUs.

[0004] However, the above method may not be able to accurately identify the status of the ports in the fiber optic junction box. Summary of the Invention

[0005] This invention provides a method, apparatus, computer-readable storage medium, and electronic device for identifying the port status of an optical cable junction box. By using the correspondence between port characters and row ports, row port images are cropped from an image. Then, the row port images are positioned and cropped to obtain single-port images that can better reflect the port status, thereby enabling more accurate identification of the port status.

[0006] In a first aspect, the present invention provides a method for identifying the status of a port in an optical cable junction box, comprising:

[0007] Acquire an image of the optical cable junction box; wherein the image includes multiple port characters on the optical cable junction box and multiple rows of ports that correspond one-to-one with the multiple port characters;

[0008] Determine the character recognition result of the image; wherein the character recognition result indicates the position of the plurality of port characters in the image;

[0009] Based on the character recognition results, the multi-row ports in the image are cropped to determine multiple row port images that correspond one-to-one with the multi-row ports;

[0010] Port positioning is performed on each of the multiple row port images to determine the port positioning result; wherein, the port positioning result indicates the position of each port in the multiple row port images;

[0011] Based on the port positioning results, the multiple row port images are cropped into single ports to determine multiple single-port images;

[0012] The states of the multiple single-port images are identified to determine their respective states.

[0013] Secondly, the present invention provides a port status identification device for an optical cable junction box, comprising:

[0014] An image acquisition module is used to acquire an image of an optical cable junction box; wherein, the image includes multiple port characters on the optical cable junction box and multiple rows of ports that correspond one-to-one with the multiple port characters;

[0015] A character recognition module is used to determine the character recognition result of the image; wherein the character recognition result indicates the position of the plurality of port characters in the image;

[0016] The first cropping module is used to crop the multi-row ports in the image based on the character recognition result, and determine multiple row port images that correspond one-to-one with the multi-row ports;

[0017] The positioning module is used to perform port positioning on the multiple row port images respectively and determine the port positioning result; wherein, the port positioning result indicates the position of each port in the multiple row ports in the multiple row port images;

[0018] The second cropping module is used to crop the multiple row port images into single ports based on the port positioning results, thereby determining multiple single port images;

[0019] The status recognition module is used to perform status recognition on the multiple single-port images respectively, and determine the status of each of the multiple single-port images.

[0020] Thirdly, the present invention provides a computer-readable storage medium including executable instructions, which, when executed by a processor of an electronic device, perform the method as described in any of the first aspects.

[0021] Fourthly, the present invention provides an electronic device including a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor performs the method as described in any of the first aspects.

[0022] This invention provides a method, apparatus, computer-readable storage medium, and electronic device for identifying the port status of an optical cable junction box. The method involves acquiring an image of the optical cable junction box, the image including multiple port characters on the box and multiple rows of ports corresponding one-to-one with each port character; then, determining the character recognition result of the image, where the character recognition result indicates the position of the multiple port characters in the image; then, cropping the multiple rows of ports in the image based on the character recognition result to determine multiple row port images corresponding one-to-one with each row of ports; then, performing port positioning on each of the multiple row port images to determine the port positioning result, where the port positioning result indicates the position of each port in the multiple row port images; then, based on the port positioning result, cropping each row port image individually to determine multiple individual port images; and finally, performing status recognition on each of the multiple individual port images to determine the status of each individual port image. In summary, the technical solution of the present invention uses the correspondence between port characters and line ports to crop line port images from an image. Then, the line port images are positioned and cropped to obtain single port images that can better reflect the port status, thereby enabling more accurate identification of the port status.

[0023] The further effects of the aforementioned non-conventional preferred method will be explained below in conjunction with specific embodiments. Attached Figure Description

[0024] To more clearly illustrate the embodiments of the present invention or the existing technical solutions, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1An architecture diagram of an image processing system provided in an embodiment of the present invention;

[0026] Figure 2a A schematic diagram of an optical cable junction box provided in an embodiment of the present invention. Figure 1 ;

[0027] Figure 2b Schematic diagram 2 of an optical cable junction box provided in an embodiment of the present invention;

[0028] Figure 2c A schematic diagram of an optical cable junction box provided in an embodiment of the present invention. Figure 3 ;

[0029] Figure 2d A schematic diagram of an image and a corrected image of an optical cable junction box provided in an embodiment of the present invention;

[0030] Figure 3 A schematic diagram illustrating the process of identifying the port status of an optical cable junction box according to an embodiment of the present invention;

[0031] Figure 4a This is a schematic diagram of the character recognition results of the image and character image of the optical cable junction box provided in an embodiment of the present invention. Figure 1 ;

[0032] Figure 4b Schematic diagram 2 showing the character recognition results of the image and character image of the optical cable junction box provided in an embodiment of the present invention;

[0033] Figure 4c This is a schematic diagram of the character recognition results of the image and character image of the optical cable junction box provided in an embodiment of the present invention. Figure 3 ;

[0034] Figure 5a A schematic diagram of a row port image provided for an embodiment of the present invention. Figure 1 ;

[0035] Figure 5b Schematic diagram 2 of the row port image provided in an embodiment of the present invention;

[0036] Figure 6a A schematic diagram of the positioning results of the row port image provided in the embodiments of the present invention. Figure 1 ;

[0037] Figure 6b Schematic diagram 2 showing the positioning result of the row port image provided in an embodiment of the present invention;

[0038] Figure 6c A schematic diagram of the positioning results of the row port image provided in the embodiments of the present invention. Figure 3 ;

[0039] Figure 6d Schematic diagram four showing the positioning results of the row port image provided in an embodiment of the present invention;

[0040] Figure 7a A schematic diagram of the port status provided in the embodiments of the present invention. Figure 1 ;

[0041] Figure 7b A second schematic diagram illustrating the state of a port provided in an embodiment of the present invention;

[0042] Figure 7c A schematic diagram of the port status provided in the embodiments of the present invention. Figure 3 ;

[0043] Figure 7d A schematic diagram of the port status provided in an embodiment of the present invention;

[0044] Figure 7e Schematic diagram five illustrating the state of a port provided in an embodiment of the present invention;

[0045] Figure 8a This is a schematic diagram of the structure of the stacked hourglass network provided in an embodiment of the present invention;

[0046] Figure 8b This is a schematic diagram of the structure of a second-order hourglass network provided in an embodiment of the present invention;

[0047] Figure 8c for Figure 8b A schematic diagram of the residual module is provided.

[0048] Figure 9 A flowchart illustrating a method for identifying the port status of an optical cable junction box according to an embodiment of the present invention;

[0049] Figure 10 A schematic diagram of the structure of a port status identification device for an optical cable junction box provided in an embodiment of the present invention;

[0050] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0052] Figure 1This is an architecture diagram of the image processing system used in this embodiment. Figure 1 As shown, system 100 includes image acquisition device 110 and image processing device 120.

[0053] The image acquisition device 110 is used to capture images of the ports of the optical cable junction box. The image includes multiple port characters of the optical cable junction box and a row of ports corresponding to each port character. For example, the port characters are port letters, and each port letter corresponds to a row of ports; in other words, the number of port letters is the same as the number of rows of ports. The port letters can be, for example, ... Figures 2a to 2d The rightmost letter shown represents all ports within the same row as the port letter; Figure 2a Taking this as an example, the port letters from top to bottom are X, W, V, U, T, S, R, Q, P, M, N, L, K, a total of 13 port letters. Correspondingly, there are 13 rows of ports, and each row of ports has 12 ports. Therefore, the optical cable junction box has a total of 156 ports.

[0054] In one example, a user takes a picture of the port of the optical cable junction box using the image acquisition device 110 to obtain an image, and then uploads the image to the image processing device 120.

[0055] For example, Figure 2a , Figure 2b , Figure 2c and Figure 2d Figure (1) shows an image obtained by the image acquisition device 110 taking a picture of the port of the optical cable junction box.

[0056] Furthermore, the image acquisition device 110 can perform image preprocessing on the image before uploading it to the image processing device 120.

[0057] Image preprocessing can include noise reduction, angle correction, and enhancement.

[0058] For example, Gaussian filtering algorithms can be used to eliminate noise pixels that cause significant interference in an image; the principle of projection mapping can be used to eliminate image tilt and distortion to correct the image; and image enhancement techniques can be used to adjust image brightness.

[0059] Furthermore, image preprocessing can also include image cropping to remove interfering areas around the image. This addresses issues such as tilted shooting angles when photographing fiber optic junction boxes with a mobile phone, which can lead to image tilt, or interference from other factors around the box, which can hinder target detection and recognition. For example, such as... Figure 2dAs shown, Figure (1) shows the image obtained by the image acquisition device 110 taking a picture of the port of the optical cable junction box, and Figure (2) shows the image after image correction and cropping of Figure (1). Obviously, Figure (2) is more convenient for target detection and recognition than Figure (1).

[0060] In one example, the image acquisition device 110 is a device with image acquisition capabilities. For example, the image acquisition device 110 can be a mobile phone.

[0061] Please refer to Figure 3 The image processing device 120 can sequentially perform image ROI segmentation, image character detection, ROI segmentation rows, row port location, port segmentation and status recognition on the image.

[0062] Image ROI segmentation mainly refers to performing ROI segmentation on an image to separate the portion containing port characters, resulting in a port character image. This port character image contains port characters, and may further include portions of ports.

[0063] In practical applications, image character detection is achieved based on object detection models. These models require training on a training set. Correspondingly, image ROI segmentation primarily involves segmenting the input fiber optic junction box image based on the size and content of the images in the training set. This ensures the accuracy of character detection, further removes port interference, and accurately locates the line character positions. For example, the segmented image has a width of 1 / 6 of the original image, and the same height.

[0064] Image character detection can be understood as recognizing port characters in a segmented image and determining the location of the recognized port characters.

[0065] In one example, image character detection can be achieved using an object detection model. For instance, the structure of this object detection model could be YOLO-v4. The YOLO-v4 example described above is merely an example of an object detection model structure and does not constitute a specific limitation; the specific structure of the object detection model needs to be determined based on the actual situation.

[0066] like Figure 4a and Figure 4c As shown, Figure A presents the character detection results of an image without ROI segmentation using an object detection model, while Figure B presents the character detection results of a port character image obtained after ROI segmentation using the same object detection model. Clearly, by specifically targeting the regions where port characters are located, the accuracy of character detection can be improved.

[0067] In this context, ROI segmentation can be understood as dividing each row of ports based on the characteristic that each port character in the optical cable junction box corresponds to a row of ports, and obtaining multiple row port images by dividing each row of ports according to the position of the port characters.

[0068] In one example, based on the characteristic that each port character in the fiber optic junction box corresponds to a row of ports, and the fact that the segmented image has the same height as the original image, the height position of the port character in the segmented image is determined, thus inferring the position of the port character in the original image. Then, based on the characteristic that the port character corresponds to a row of ports, to avoid omissions, the height can be increased to segment the image of each row of ports.

[0069] For example, Figure 5a and Figure 5b The images shown are of the row ports. It is worth noting that these two row port images are from different fiber optic junction boxes.

[0070] Row port localization can be understood as locating ports within multiple row port images to obtain the positions of the ports contained in each row port image. For example, Figures 6a to 6d The black dots in the image indicate the location of the port.

[0071] In one example, the positions of the multiple ports contained in each row of port images are obtained through a stacked hourglass network aligned with key points.

[0072] For example, a stacked hourglass network can be trained using an open-source human pose dataset to obtain a trained stacked hourglass network. Then, the number of predicted keypoints in the output of the trained stacked hourglass network is modified to the number of ports in the row ports. Transfer learning is then performed on the modified stacked hourglass network based on the training dataset to obtain a keypoint-aligned stacked hourglass network. The MPII dataset is a human pose prediction dataset containing over 25,000 labeled images of more than 40,000 people. The training dataset includes multiple row port images and their respective labeled port positions.

[0073] For example, by training a stacked hourglass network using the MPII dataset, after multiple iterations, such as 100, the predicted keypoints of the stacked hourglass network are modified to the number of ports contained in the row port image. Then, after freezing the network parameters of the first 50 layers, after multiple iterations, such as 100, all the parameters are unfrozen and trained until termination, resulting in a keypoint-aligned stacked hourglass network.

[0074] It should be noted that the stacked hourglass network can be any existing network, and the specific model structure can be selected according to actual needs. For example, the order of the hourglass network can be selected, such as 2nd order, and the number of stacked hourglass networks can be selected.

[0075] Figure 8a The structure of a stacked hourglass network is shown. (Example) Figure 8a As shown, each hourglass network is called an hourglass network. Multiple hourglass networks stacked together are called stacked hourglass networks. The input image of the stacked hourglass network is passed through a fully convolutional network (FCN) to obtain features, and then passed through multiple stacked hourglasses to obtain the final heatmap.

[0076] Figure 8b A second-order hourglass network is shown. (Example:) Figure 8b As shown, the larger rounded rectangles in the figure represent residual modules. These modules are independent of the input size and only require the number of input and output channels. The first row within the rounded rectangles represents the number of input channels, and the second row represents the number of output channels. This allows for the gradual extraction of deeper features. The area within the dashed box represents a first-order hourglass network. The upper path extracts features at the original scale, while the lower path extracts features at a downsampled ( / 2) scale space. Then, the features are upsampled (*2) to the same size as the upper path and fused. Specifically, downsampling uses max pooling, upsampling uses nearest neighbor upsampling, and finally, the two different feature sets are added element-wise. The size OriSize of the feature map output by the first branch is twice the size OriSize / 2 of the feature map output by the second branch.

[0077] Figure 8c The structure of the residual module is shown. For example... Figure 8cAs shown, the first row is a convolutional path, consisting of three concatenated convolutional operations interspersed with Batch Normalization (BN) and ReLU activation functions. The second row is a skip path, containing only a 1x1 convolutional layer responsible for changing the number of input channels to be added to the convolution result of the first row. If the number of input and output channels of the skip path is the same, then this path is an identity mapping. All convolutional layers have a stride of 1 and padding of 1, without changing the data size, only the data depth (channels). Therefore, the residual module is controlled by only two parameters: the number of input channels (Channel = M) and the number of output channels (Channel = N), and is independent of the image sizes H and W. Furthermore, the first row in the rounded rectangle represents the number of input channels, the second row represents the kernel size, and the third row represents the number of output channels. The residual module extracts higher-level features (convolutional path) while preserving information from the original layers (skip path), without changing the data size, only changing the data depth, and also solving the gradient vanishing problem that may occur during backpropagation.

[0078] Port segmentation and status recognition can be understood as cutting the ports in the optical cable junction box image after determining the port location to obtain multiple port images, and then classifying the ports in the port images.

[0079] In one example, port classification can be achieved using a lightweight deep separable network. Considering the large number of ports, a lightweight deep separable network can quickly classify the ports of multiple port images.

[0080] There are five types of ports: red cap, white cap, plugged in, damaged, and obstructed. For example, Figure 7a A red hat was shown. Figure 7b A white hat was shown. Figure 7c The wiring is shown. Figure 7d Damage was indicated. Figure 7e Occlusion is shown. Considering the need to recognize colors, the input to the lightweight deep separable network is a color image, with one channel per color.

[0081] For example, the structure of a lightweight, deep separable network is shown in Table 1 below:

[0082]

[0083] Table 1

[0084] In this table, `depthwise_conv_block` is a channel-wise convolutional layer, abbreviated as DWConv; `res_block` is a residual layer; `add` can be understood as fusion, that is, merging 7 groups of 14*14*512 into 14*14*512; `GlobalAveragePooling2D` is a two-dimensional global pooling method used to apply average pooling to the spatial dimensions until each spatial dimension is one, while other dimensions remain unchanged; `Dropout` can be used as a trick to train convolutional neural networks. In each training batch, by ignoring half of the feature detectors (making half of the hidden layer nodes have a value of 0), overfitting can be significantly reduced. It should be noted that `Dropout` needs to be removed from the trained network; `Activation` can be understood as an activation layer; `Conv2D` can be understood as a two-dimensional convolution used for channel dimensionality reduction. Additionally, the input in the first row of Table 1 is a three-channel color image, and the input in each subsequent row is the output of the structure shown in the previous row.

[0085] like Figure 9 The image shows a method provided by an embodiment of the present invention. The method provided by this embodiment can be applied to electronic devices, specifically to servers or general computers. In this embodiment, the method specifically includes the following steps:

[0086] Step 101: Obtain an image of the optical cable junction box; wherein the image includes multiple port characters on the optical cable junction box and multiple rows of ports corresponding one-to-one with the multiple port characters.

[0087] As a feasible approach, the photographer takes a picture of the port of the optical cable junction box using a terminal with image capture capabilities, obtains an image of the optical cable junction box, and uploads the image of the optical cable junction box to an electronic device, thereby enabling the electronic device to acquire the image of the optical cable junction box.

[0088] The image of the fiber optic junction box includes multiple port characters and multiple rows of ports. For example, the port characters are port letters, with each port letter corresponding to a row of ports; in other words, the number of port letters is the same as the number of rows of ports. See the description above for details on the port letters and row ports. Figures 2a to 2d The rightmost letter shown is the port letter, and multiple ports located in the same horizontal area as the port letter form a row of ports.

[0089] Furthermore, considering the potential for image tilt due to tilted shooting angles when photographing fiber optic junction boxes using devices such as mobile phones, or the presence of interfering elements around the boxes that hinder detection and identification, in practical applications, the images of fiber optic junction boxes undergo angle correction and image cropping to remove interfering areas and correct the image angle, thus facilitating target detection and identification. Additionally, further preprocessing such as noise reduction and image enhancement can be performed to further ensure target detection and identification. The processed images are then subjected to subsequent processing.

[0090] In one example, the photographer takes a picture of the port of the fiber optic junction box using a terminal with image capture capabilities, obtaining an image of the junction box. By manually selecting the four corner points of the junction box in the image, and then using transmission transformation, the image correction problem is transformed into the transformation of an image on one object plane to an image on another object plane. This leads to the derivation of the correction method for perspective distortion images, that is, transforming a point on one plane to a point on another plane. This corrects the image tilt that may be caused by the shooting angle, while removing other factors around the box, thus obtaining the image of the fiber optic junction box that needs to be processed later.

[0091] Step 102: Determine the character recognition result of the image; wherein, the character recognition result indicates the position of multiple port characters in the image.

[0092] Considering that the image mainly includes the ports of the fiber optic junction box, the port characters are a small part of the content; such as Figure 4a As shown in Figure 4c, the accuracy of port character recognition is relatively low if the entire image is recognized. Recognizing characters within the region where the port character is located can improve accuracy. Therefore, as a feasible implementation method, the character recognition result can be determined as follows:

[0093] The image of the optical cable junction box is cropped to determine the port character image, which includes multiple port characters. The port character image is then used for character recognition and localization to determine the character recognition result.

[0094] For example, the port character image can be Figures 4a to 4c Figure B is shown for each. It should be noted that the port character image must include all port characters within the fiber optic junction box image; however, in some cases, it may include only a few ports. For example, the width of the port character image may be 1 / 6 of the width of the fiber optic junction box image, and the height of the port character image may be the same as the height of the fiber optic junction box image.

[0095] The character recognition results indicate the positions of multiple port characters of the optical cable junction box in the image of the optical cable junction box. For example, it can be the position of the center point of the port character or the position of the area occupied by the port character.

[0096] In one example, the port character image is input into a trained object detection model to determine the character recognition result output by the trained object detection model. Correspondingly, the character recognition result includes the positions of multiple bounding boxes in the image of the fiber optic junction box, and each bounding box corresponds one-to-one with multiple port characters of the fiber optic junction box.

[0097] For example, Figures 4a to 4c The target bounding boxes and the probability that the target bounding boxes belong to port characters in Figure B are the character recognition results.

[0098] Step 103: Based on the character recognition results, crop the multi-row ports in the image to determine multiple row port images that correspond one-to-one with the multi-row ports.

[0099] In this embodiment, each row of port images includes the same number of ports, typically 12.

[0100] As a feasible implementation method, multiple row port images can be determined as follows:

[0101] Based on the character recognition results, the row regions corresponding to each of the plurality of port characters are determined. The row region corresponding to the port character includes a row of ports corresponding to the port character. The row region indicates the area extending horizontally from the position of the port character in the image. For each of the plurality of port characters, the row region corresponding to the port character in the image is cropped to determine the row port image corresponding to the port character.

[0102] Since the character recognition results indicate the position of the port characters in the image of the fiber optic junction box, the regions of multiple ports in the row containing the port characters can be deduced from the original image. To avoid omissions, the row height needs to be increased to ensure that the row region includes all ports in the row containing the port characters, and that these ports are complete. Then, the row region in the image of the fiber optic junction box is cropped to obtain the row port image. For example, the character recognition results include the target bounding boxes of the port characters, the row height of the row region can be 1.5 times the height of the target bounding boxes of the port characters, and the width is the same as the width of the image of the fiber optic junction box, thus segmenting each row of ports.

[0103] For example, Figure 5a and Figure 5b Each shows a row port image.

[0104] Step 104: Perform port location on multiple row port images respectively, and determine the port location results; wherein, the port location results indicate the position of each port in the multiple row port images.

[0105] As a feasible implementation, for each of the multiple row port images, the row port image is substituted into a trained port localization model to determine the localization result output by the trained port localization model. The localization result indicates the position of each port in the row port image corresponding to the row port image. Correspondingly, the port localization result includes the localization results for each of the multiple row port images.

[0106] In one example, the trained port localization model can be determined using the following method;

[0107] A human pose prediction model is trained using an open-source human pose dataset to determine a well-trained human pose prediction model. The number of predicted keypoints in the output of the well-trained human pose prediction model is modified to the number of ports included in the row port image to determine a port localization model to be trained. Transfer learning is performed on the port localization model to be trained based on the training dataset to determine a well-trained port localization model. The training dataset includes row port image samples and port position labels of the row port image samples.

[0108] Given the limited number of samples in the training dataset, transfer learning was employed to obtain a better-performing model. Details can be found above and will not be elaborated upon here.

[0109] Considering that the ports in this embodiment may have different states such as red hat, white hat, occluded, occupied, and damaged, the target detection model may not be able to accurately detect ports in different states. Therefore, this embodiment selects a human pose prediction model for transfer learning to output the center point of the port. For example, the structure of the human pose prediction model can be a stacked hourglass network. Of course, this network is only an example and does not constitute a specific limitation. The structure of the model can be selected and designed based on actual needs.

[0110] Step 105: Based on the port location results, crop the multiple row port images into single ports to determine multiple single-port images.

[0111] Since the port location result indicates the position of the center point of each port in the corresponding row port image of the optical cable junction box, for each of the multiple row port images, the position of the center point of each port in the row port image can be determined based on the port location result. Then, based on the position of the center point, the region is expanded to obtain the port region corresponding to each port in the row port image. The port region is then cropped from the row port image to obtain multiple single port images.

[0112] Step 106: Perform state recognition on multiple single-port images to determine the state of each single-port image.

[0113] In this step, the state of a single-port image is identified using a trained port state recognition model to obtain the single-port image.

[0114] In one example, to improve computational efficiency, the port status recognition model is a lightweight model, which can be the lightweight deep separable network mentioned above. The structure can be seen in Table 1 above, and will not be elaborated further here. Of course, Table 1 is only an example and does not constitute a specific limitation. The specific structure of the model needs to be determined based on the actual requirements.

[0115] As can be seen from the above technical solutions, the beneficial effects of this embodiment are:

[0116] By mapping port characters to row ports, row port images are cropped from the image. Then, the row port images are positioned and cropped to obtain single-port images that can better reflect the port status, which can further identify the port status more accurately.

[0117] Based on the same concept as the embodiments of the method of the present invention, please refer to Figure 10 This invention also provides a status identification device for the port of an optical cable junction box, comprising:

[0118] Image acquisition module 201 is used to acquire an image of the optical cable junction box; wherein, the image includes multiple port characters on the optical cable junction box and multiple rows of ports that correspond one-to-one with the multiple port characters;

[0119] The character recognition module 202 is used to determine the character recognition result of the image; wherein the character recognition result indicates the position of the plurality of port characters in the image;

[0120] The first cropping module 203 is used to crop the multi-row ports in the image based on the character recognition result, and determine multiple row port images that correspond one-to-one with the multi-row ports;

[0121] The positioning module 204 is used to perform port positioning on the plurality of row port images respectively and determine the port positioning result; wherein, the port positioning result indicates the position of each port in the plurality of row ports in the plurality of row port images;

[0122] The second cropping module 205 is used to crop the multiple row port images into single ports based on the port positioning results, thereby determining multiple single port images;

[0123] The state recognition module 206 is used to perform state recognition on the multiple single-port images respectively, and determine the state of each of the multiple single-port images.

[0124] As one possible implementation, the states are red hat, white hat, plugged in, damaged, or blocked.

[0125] As a feasible implementation, the image of the optical cable junction box is an image obtained by cropping the port and port characters of the optical cable junction box after transmission transformation.

[0126] As a feasible implementation, the multiple port characters in the image are arranged in multiple rows. The first cropping module 203 includes: a region determination unit and a cropping unit; wherein,

[0127] The region determination unit is used to determine the row region corresponding to each of the plurality of port characters based on the character recognition result. The row region corresponding to the port character includes a row of ports corresponding to the port character. The row region indicates the region in the image where the port character is located extends horizontally towards the image.

[0128] The first cropping unit is used to crop the row region corresponding to the port character in the image for each character of the plurality of port characters, and to determine the row port image corresponding to the port character.

[0129] As a feasible implementation, the character recognition module 202 includes: a second cutting unit and a character recognition unit; wherein,

[0130] The second cropping unit is used to crop the area where the port characters are located in the image to determine the port character image, wherein the port character image includes the plurality of port characters;

[0131] The character recognition unit is used to perform character recognition and positioning on the port character image and determine the character recognition result.

[0132] As a feasible implementation, the character recognition unit is used to input the port character image into a trained target detection model and determine the character recognition result output by the trained target detection model.

[0133] As a feasible implementation, the positioning module 204 includes: a model positioning unit; wherein,

[0134] The model localization unit is used to substitute each of the multiple row port images into the trained port localization model to determine the localization result output by the trained port localization model.

[0135] As a feasible implementation method, the trained port localization model is determined by the following approach;

[0136] The human pose prediction model was trained using an open-source human pose dataset, and the trained human pose prediction model was determined.

[0137] The number of predicted keypoints in the output of the trained human pose prediction model is modified to the number of ports included in the row port image to determine the port localization model to be trained.

[0138] The port location model to be trained is transferred to the training dataset to determine the trained port location model. The training dataset includes row port image samples and port position labels of the row port image samples.

[0139] As a feasible implementation, the state recognition module 206 is used to substitute each of the multiple single-port images into a trained port state determination model to determine the state of the single-port image output by the trained port state determination model.

[0140] Figure 11This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. At the hardware level, the electronic device includes a processor 301 and a memory 302 storing execution instructions, and optionally also includes an internal bus 303 and a network interface 304. The memory 302 may include main memory 3021, such as high-speed random-access memory (RAM), and may also include non-volatile memory 3022, such as at least one disk storage device. The processor 301, network interface 304, and memory 302 can be interconnected via the internal bus 303, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The internal bus 303 can be divided into an address bus, a data bus, a control bus, etc., for ease of representation. Figure 11 The symbol is represented by only one double-headed arrow, but this does not mean that there is only one bus or one state bus. Of course, the electronic device may also include other hardware required for other services. When the processor 301 executes the execution instructions stored in the memory 302, the processor 301 executes the method in any embodiment of the present invention, and is at least used to perform the following: Figure 3 or Figure 9 The method shown.

[0141] In one possible implementation, the processor reads the corresponding execution instructions from non-volatile memory into main memory and then executes them. Alternatively, it can obtain the corresponding execution instructions from other devices to logically form a port status identification device for an optical cable junction box. The processor executes the execution instructions stored in the memory to implement the port status identification method for an optical cable junction box provided in any embodiment of the present invention.

[0142] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above methods can be completed through integrated logic circuits in the processor's hardware or through software instructions. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0143] This invention also provides a computer-readable storage medium including executable instructions. When a processor of an electronic device executes the executable instructions, the processor performs the method provided in any embodiment of this invention. Specifically, the electronic device may be as follows: Figure 11 The electronic device shown; the execution instruction is the computer program corresponding to the port status identification device of a fiber optic junction box.

[0144] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can be implemented in a completely hardware embodiment, a completely software embodiment, or a combination of software and hardware.

[0145] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0146] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0147] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for identifying the status of a port in an optical cable junction box, characterized in that, include: Acquire an image of the optical cable junction box; wherein the image includes multiple port characters on the optical cable junction box and multiple rows of ports that correspond one-to-one with the multiple port characters; Determine the character recognition result of the image; wherein the character recognition result indicates the position of the plurality of port characters in the image; Based on the character recognition results, the multi-row ports in the image are cropped to determine multiple row port images that correspond one-to-one with the multi-row ports; Port localization is performed on the multiple row port images respectively to determine the port localization result; wherein, the port localization result indicates the position of each port in the multiple row port images, the position of the port in the multiple row port images is the center point of the port, and the port localization is implemented using a trained port localization model; Based on the port positioning results, the multiple row port images are cropped into single ports to determine multiple single-port images. The single-port images are obtained by expanding the region of the center point. The states of the multiple single-port images are identified to determine their respective states.

2. The method according to claim 1, characterized in that, The states are red cap, white cap, plugged in, damaged, or blocked.

3. The method according to claim 1, characterized in that, The image of the optical cable junction box is obtained by cropping the port and port characters of the optical cable junction box after transmission transformation.

4. The method according to claim 1, characterized in that, The image contains multiple port characters arranged in multiple rows. The step of cropping each row of ports corresponding to the multiple port characters in the image based on the character recognition result, to determine multiple row port images that correspond one-to-one with the multiple rows of ports, includes: Based on the character recognition results, the row regions corresponding to each of the plurality of port characters are determined. The row region corresponding to the port character includes a row of ports corresponding to the port character. The row region indicates the area in the image where the port character is located extends horizontally towards the image. For each character of the plurality of port characters, the row region corresponding to the port character in the image is cropped to determine the row port image corresponding to the port character.

5. The method according to claim 1, characterized in that, The determination of the character recognition result of the image includes: The image is cropped to include the port characters, thus determining a port character image, which comprises the plurality of port characters. The port character image is subjected to character recognition and localization to determine the character recognition result.

6. The method according to claim 5, characterized in that, The step of performing character recognition and localization on the port character image and determining the character recognition result includes: The port character image is input into the trained target detection model to determine the character recognition result output by the trained target detection model.

7. The method according to claim 1, characterized in that, The step of locating ports in the multiple row port images and determining the port location results includes: For each of the multiple row port images, the row port image is substituted into the trained port localization model to determine the localization result output by the trained port localization model.

8. The method according to claim 7, characterized in that, The trained port localization model is determined using the following method; The human pose prediction model was trained using an open-source human pose dataset, and the trained human pose prediction model was determined. The number of predicted keypoints in the output of the trained human pose prediction model is modified to the number of ports included in the row port image to determine the port localization model to be trained. The port location model to be trained is transferred to the training dataset to determine the trained port location model. The training dataset includes row port image samples and port position labels of the row port image samples.

9. The method according to claim 1, characterized in that, The step of performing state recognition on the plurality of single-port images respectively to determine the state of each of the plurality of single-port images includes: For each of the multiple single-port images, the single-port image is substituted into the trained port state determination model to determine the state of the single-port image output by the trained port state determination model.

10. A status identification device for the port of an optical cable junction box, characterized in that, include: An image acquisition module is used to acquire an image of an optical cable junction box; wherein, the image includes multiple port characters on the optical cable junction box and multiple rows of ports that correspond one-to-one with the multiple port characters; A character recognition module is used to determine the character recognition result of the image; wherein the character recognition result indicates the position of the plurality of port characters in the image; The first cropping module is used to crop the multi-row ports in the image based on the character recognition result, and determine multiple row port images that correspond one-to-one with the multi-row ports; The positioning module is used to perform port positioning on the multiple row port images respectively and determine the port positioning result; wherein, the port positioning result indicates the position of each port in the multiple row port images, the position of the port in the multiple row port images is the center point of the port, and the port positioning is implemented using a trained port positioning model; The second cropping module is used to crop the multiple row port images individually based on the port positioning results, thereby determining multiple individual port images, wherein the individual port images are obtained by expanding the region of the center point; The status recognition module is used to perform status recognition on the multiple single-port images respectively, and determine the status of each of the multiple single-port images.

Images