Method and system for detecting cable body label of fiber distribution box

Abstract

The invention provides a method and system for detecting a cable body label of a fiber distribution box, and relates to the technical field of image detection. The method comprises the following steps: acquiring a target image containing a label on each line body in a target fiber distribution box; inputting the target image into a feature extraction network to extract multi-scale feature information of different dimensions, performing feature map size alignment through a convolution and / or interpolation method, and performing feature information fusion of different scales by using a cross channel self-attention module to obtain first feature information; inputting the first feature information into a visual state keeping module to calculate global feature information and distribute different weight information according to the distance of the features so as to obtain second feature information; and inputting the second feature information into a rotary detection head module to generate size, category and angle information of the target so as to obtain label detection result information. According to the scheme, the line body label in the fiber distribution box with a complex line body environment can be efficiently and accurately identified.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a method and system for detecting line body labels of a fiber distribution box. Background Art

[0002] In the optical fiber communication network, the fiber splitter box is the management and distribution center of the optical fiber line. The efficient detection of the internal line labels is crucial to improve the efficiency of fiber splitter box data recording by the installation and maintenance personnel. The accurate identification of labels not only helps the maintenance and management of the optical fiber network, but also effectively reduces the error rate of manual operation.

[0003] Although there are some solutions for detecting cable labels in fiber splitter boxes, these solutions still face many challenges in practical applications. The cable environment of the fiber splitter box is complex and chaotic. The main problems include dense accumulation of cables, label obstruction, and large changes in label angles. The dense accumulation and entanglement of cables cause the labels to be partially or completely covered, increasing the difficulty of identification. At the same time, the rotation angle and irregular shape of the label further increase the complexity of identification. In other words, in view of the complex environment and high precision requirements of cable label detection in the fiber splitter box, a more efficient and accurate detection method is urgently needed to deal with it. Summary of the invention

[0004] The purpose of the present application is to provide a method and system for detecting line labels in a fiber splitter box, which can efficiently and accurately identify line labels in a fiber splitter box with a complex line environment.

[0005] This application is implemented as follows:

[0006] In a first aspect, the present application provides a method for detecting line labels in a fiber splitter box, comprising the following steps: obtaining a target image containing labels on each line in a target fiber splitter box; inputting the target image into a feature extraction network to extract multi-scale feature information of different dimensions, and aligning the feature map size through convolution and / or interpolation, and using a cross-channel self-attention module to fuse feature information of different scales to obtain first feature information; inputting the first feature information into a visual state maintenance module to calculate global feature information and assign different weight information according to the distance size of the feature, thereby obtaining second feature information; inputting the second feature information into a rotating detection head module to generate target size, category and angle information to obtain label detection result information.

[0007] In some implementations, before inputting the target image into the feature extraction network, it also includes deleting dark scenes and blurred target images, and adjusting the size of the target image and normalizing the target image.

[0008] In some implementations, the backbone network of the feature extraction network includes ordinary convolution layers and inverse residual multi-scale convolution layers with different convolution kernel sizes, which are used to capture information of different receptive field sizes; wherein, the shallow feature information contains local detail information and target position information, and the deep feature information contains global abstract information.

[0009] In some implementations, the cross-channel self-attention module obtains a weight mask by calculating the similarity between pairwise channel weights to adjust the weights of feature maps at different layers, thereby fusing feature information at different scales.

[0010] In some implementations, the visual state maintenance module includes a feature segmentation submodule, a parallel calculation submodule and a loop submodule; wherein the feature segmentation submodule is used to divide the large feature map in the first characteristic information into multiple small feature blocks, the parallel calculation submodule is used to adjust the weight of the feature block by calculating the degree of correlation between each small feature block, and the loop submodule is used to calculate the connection between the current feature block and the previous feature block to selectively retain important feature blocks.

[0011] In some implementations, the visual state maintenance module divides the feature map in the first feature information into multiple small blocks of features of the same size in an evenly distributed manner, uses a parallel computing submodule to perform calculations within the block, and uses a loop submodule to perform calculations between different blocks.

[0012] In some implementations, the rotation detection head module includes a position detection network, a classification detection network, and an angle detection network, which are respectively used to predict the position, category, and angle information of the target.

[0013] In some implementations, the target image is input into a feature extraction network to extract multi-scale feature information of different dimensions, and feature map size alignment is performed through convolution and / or interpolation, including: inputting the target image into the feature extraction network, extracting features P1 and P2 using a normal convolution layer, and extracting features P3, P4, and P5 using an inverse residual multi-scale convolution layer with different convolution kernel sizes; aligning the feature map sizes of features P3, P4, and P5 by convolution downsampling and difference upsampling according to a top-to-bottom and bottom-to-top path aggregation network; wherein, in the top-to-bottom path aggregation network, feature P5 is fused with feature P5 through bilinear difference upsampling to obtain feature C4, and feature C4 is fused with feature P3 through convolution downsampling to obtain feature C3; in the bottom-to-top path aggregation network, feature C4 is downsampled and aggregated by convolution to obtain feature C5 to obtain feature S5, and feature C3 and feature C4 are fused to obtain feature S4.

[0014] In some implementations, the method further includes comparing the predicted position with the actual position based on the tag detection information results to calculate the loss to optimize the model.

[0015] In the second aspect, the present application provides a system for detecting line labels in a fiber splitter box, comprising: an image acquisition unit, for acquiring a target image containing labels on each line in a target fiber splitter box; a first processing unit, for inputting the target image into a feature extraction network to extract multi-scale feature information of different dimensions, and aligning the size of feature maps through convolution and / or interpolation, and using a cross-channel self-attention module to fuse feature information of different scales to obtain first feature information; a second processing unit, for inputting the first feature information into a visual state maintenance module to calculate global feature information and assign different weight information according to the distance size of the feature, thereby obtaining second feature information; a result generation unit, for inputting the second feature information into a rotating detection head module to generate target size, category and angle information to obtain label detection result information.

[0016] In a third aspect, the present application provides an electronic device, comprising a memory for storing one or more programs; a processor; when the one or more programs are executed by the processor, a method as described in any one of the first aspects is implemented.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method as described in any one of the above-mentioned first aspects.

[0018] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:

[0019] This application proposes a method for detecting the body label of a fiber splitter box. Through multi-scale feature extraction and fusion, global feature calculation and weight distribution, and the application of a rotating detection head module, it can effectively deal with the complex environmental problems in the detection of the body label of a fiber splitter box. This method can not only improve the accuracy and robustness of label detection, but also effectively deal with challenges such as dense lines, label occlusion, and large changes in label angles. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1This is a flow chart of an embodiment of a method for detecting a line label of a fiber distribution box according to the present application;

[0022] Figure 2 This is a schematic diagram of the structure of the first processing unit in one embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of the structure of the second processing unit in one embodiment of the present application;

[0024] Figure 4 This is a structural diagram of a visual state maintaining module in one embodiment of the present application;

[0025] Figure 5 This is a schematic diagram of the structure of a parallel computing submodule in an embodiment of the present application;

[0026] Figure 6 This is a schematic diagram of the structure of the circulation submodule in one embodiment of the present application;

[0027] Figure 7 This is a structural block diagram of an embodiment of a system for detecting line labels in a fiber distribution box according to the present application;

[0028] Figure 8 A structural block diagram of an electronic device provided in an embodiment of the present application.

[0029] Icon: 201, processor; 202, memory; 203, communication interface. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0031] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0032] Example 1

[0033] Efficient detection of wire labels in the fiber splitter box can significantly improve the efficiency of fiber splitter box data recording by installation and maintenance personnel. The wire environment of the fiber splitter box is complex and chaotic, with major problems such as dense accumulation of wires, label obstruction, and large angle changes. Since cables usually leave a certain margin to facilitate the installation of optical fibers, cables often get tangled in the fiber splitter box. In some areas of the image, cables may be densely stacked, covering some or all of the labels. Overlapping cables cause labels to only show part of the label, and labels may be sandwiched between two or more cables, causing the label to appear as multiple segments on the image. The method of winding the wire causes the label on the wire to also be a label with a rotation angle. Irregular labels cause the shape and direction of the characters to change, increasing the difficulty of recognition.

[0034] The rapid development of artificial intelligence has made automated note detection methods possible. However, the detection environments corresponding to the existing detection methods are relatively regular and simple, without complex occlusion problems, and cannot quickly and effectively detect the body labels of fiber splitter boxes. In response to this, an embodiment of the present application provides a method for detecting body labels of fiber splitter boxes, which can also efficiently and accurately detect the complex environment and high-precision requirements of body label detection of fiber splitter boxes.

[0035] See also Figure 1 The method for detecting the label of the fiber distribution box comprises the following steps:

[0036] Step S101: acquiring a target image including labels on each wire body in a target fiber distribution box;

[0037] It should be noted that step S101 is the starting point of the entire detection process, and a clear image containing labels on each line in the target fiber distribution box can be obtained through a camera or other image acquisition device. The key to this step is to ensure the image quality so that the subsequent steps can accurately extract label features, reducing detection failures or errors caused by factors such as image blur and insufficient light. Therefore, in some implementations of the present application, before inputting the target image into the feature extraction network, it also includes deleting dark scenes and blurred target images, adjusting the size of the target image and normalizing the target image.

[0038] For example, when acquiring the target image, the fiber splitter box can be opened correctly under the guidance of the installation and maintenance personnel to capture the fiber label data containing the fiber information. The light should be as sufficient as possible, the label should not be blocked as much as possible, and the image should be clear. Then, data cleaning is performed to delete low-quality data images. For example, blurry, unclear, dark, and unlabeled images are removed from the data set. Next, data preprocessing is performed to adjust the image size and normalize it. H and W represent the input image X imgThe length and width of the image are adjusted to the size of H' and W' using the resize function to obtain the image X res , exemplarily set it to 640 and 640. Then set the image X res Each pixel is normalized by dividing it by 255 to obtain the input X of the model in , each pixel x ij The values ​​of are all between [0,1]. The above steps are shown in the following formula: res = resize(X img ); x ij ∈[0,1].

[0039] Step S102: inputting the target image into a feature extraction network to extract multi-scale feature information of different dimensions, aligning the feature map size by convolution and / or interpolation, and fusing feature information of different scales by using a cross-channel self-attention module to obtain first feature information;

[0040] It should be noted that in step S102, the target image is input into the feature extraction network. This network can extract multi-scale feature information of different dimensions in the image, and these feature information covers key information such as the shape, color, and texture of the label. Then, the feature maps of different scales are aligned in size through convolution and / or interpolation to ensure the accuracy of subsequent processing. Finally, the feature information of different scales is fused using the cross-channel self-attention module to obtain the first feature information containing rich label features. Among them, multi-scale feature extraction can capture the detailed features of the label at different scales and improve the robustness of detection; feature map size alignment and feature information fusion ensure the accuracy and efficiency of subsequent processing; the application of the cross-channel self-attention module enhances the feature representation capability and improves the accuracy of detection.

[0041] Step S103: inputting the first feature information into a visual state maintenance module to calculate global feature information and assign different weight information according to the distance of the features, thereby obtaining second feature information;

[0042] It should be noted that in step S103, the first feature information is input into the visual state maintenance module. This module can calculate the global feature information, that is, consider the correlation and difference between the label features in the entire image range. Then, different weights are assigned to different feature information according to the distance size of the features (such as Euclidean distance, cosine similarity, etc.). The size of the weight reflects the importance of the feature information for label detection. Through this step, the weighted second feature information is obtained. Among them, the global feature calculation helps to distinguish the subtle differences between different labels; the weight allocation reduces the influence of background noise and occlusions and improves the accuracy of detection; through the visual state maintenance module, the adaptability of the model to complex environments is enhanced.

[0043] Step S104: input the second feature information into the rotation detection head module to generate the size, category and angle information of the target and obtain the label detection result information.

[0044] It should be noted that in step S104, the second feature information is input into the rotation detection head module. This module can use deep learning technology (such as convolutional neural network, regression model, etc.) to generate the size, category and angle information of the target according to the weighted feature information. These information together constitute the label detection result information for subsequent recognition and processing.

[0045] In summary, in the above embodiments, on the one hand, through multi-scale feature extraction and fusion, as well as global feature calculation and weight distribution, it is possible to capture partially or fully obscured label features, effectively dealing with the problems of dense lines and label occlusion. On the other hand, the application of the rotating detection head module enables accurate identification of the rotation angle and irregular shape of the label, solving the recognition problem of the prior art when the label angle changes greatly. In addition, through a series of advanced deep learning technologies, it is possible to extract rich feature information, and effectively fuse and weight it, thereby enhancing the robustness and accuracy of detection.

[0046] In other words, the above-mentioned embodiments can effectively deal with complex environmental problems in the detection of fiber optic cable labels through the application of multi-scale feature extraction and fusion, global feature calculation and weight distribution, and rotating detection head module. This method can not only improve the accuracy and robustness of label detection, but also effectively deal with challenges such as dense cables, label occlusion, and large changes in label angles. Therefore, this technical solution is of great significance for improving the efficiency of fiber optic cable box data recording by installation and maintenance personnel, reducing the error rate of manual operations, and optimizing the maintenance and management of optical fiber networks.

[0047] Based on the aforementioned scheme, in some implementations of the present application, the backbone network of the feature extraction network includes ordinary convolutional layers and inverse residual multi-scale convolutional layers with different convolution kernel sizes, which are used to capture information of different receptive field sizes; wherein, the shallow feature information contains local detail information and target position information, and the deep feature information contains global abstract information.

[0048] In the above implementation, the backbone network of the feature extraction network is designed to include ordinary convolution layers and inverted residual multi-scale convolution layers with different convolution kernel sizes. This structural design is intended to capture information of different receptive field sizes, so as to more comprehensively understand the label features of each line in the target fiber distribution box.

[0049] Among them, the ordinary convolution layer, as the basis of feature extraction, extracts local features in the image through convolution operations. These local features are crucial for identifying key information such as the shape, color, and texture of the label. The ordinary convolution layer is located in the shallow layer of the network and can capture detailed information in the image, such as the edges and corners of the label. The inverted residual multi-scale convolution layer uses convolution kernels of different sizes to capture information under different receptive fields. Through convolution kernels of different sizes, the inverted residual multi-scale convolution layer can extract a variety of feature information from local to global. In the shallow layer of the network, the inverted residual multi-scale convolution layer can capture rich local detail information and target location information. This information is crucial for accurately locating the location and shape of the label. As the network goes deeper, the inverted residual multi-scale convolution layer can extract more global and abstract feature information. This information helps to distinguish the subtle differences between different labels and improve the accuracy of detection.

[0050] Based on the aforementioned scheme, in some implementations of the present application, the cross-channel self-attention module obtains a weight mask by calculating the similarity between the weights of each pair of channels, which is used to adjust the weights of feature maps of different layers, thereby fusing feature information of different scales.

[0051] In the above implementation, the cross-channel self-attention module adjusts the weights of feature maps of different layers by calculating the similarity between channels and generating a weight mask, thereby promoting the effective fusion of feature information of different scales. The working principle of the cross-channel self-attention module may include: (1) Feature map extraction: First, feature maps are extracted from different layers of the network. These feature maps contain information of different scales and levels, which is the basis for subsequent feature fusion. (2) Channel weight calculation: Then, the cross-channel self-attention module calculates the similarity between the weights of each pair of channels. This can be achieved through some form of dot product, cosine similarity or other measurement methods. The calculation of similarity aims to capture the correlation between channels, that is, which channels are similar or complementary in feature expression. (3) Weight mask generation: Based on the calculated channel weight similarity, the cross-channel self-attention module generates a weight mask. This mask is a two-dimensional matrix whose element values ​​represent the relative importance of different channels. Through this mask, the channels of the feature map can be weighted to emphasize or suppress the information of certain channels. (4) Feature map adjustment and fusion: Finally, the extracted feature map is adjusted using the generated weight mask. Specifically, the weight mask is multiplied (or element-wise multiplied) with the channel dimension of the feature map to adjust the weight of each channel. This process achieves effective fusion of feature information at different scales because each channel is assigned a new weight based on its similarity with other channels.

[0052] Based on the aforementioned scheme, in some implementations of the present application, the visual state maintenance module includes a feature blocking submodule, a parallel calculation submodule and a loop submodule; wherein the feature blocking submodule is used to divide the large feature map in the first characteristic information into multiple small feature blocks, the parallel calculation submodule is used to adjust the weight of the feature block by calculating the degree of correlation between each small feature block, and the loop submodule is used to calculate the connection between the current feature block and the previous feature block to selectively retain important feature blocks.

[0053] It should be noted that the visual state preservation module is designed to efficiently process and retain key feature information in the image. Among them, the main task of the feature block submodule is to divide the input first feature information (usually a large feature map) into multiple small feature blocks. The purpose of this step is to decompose the complex feature space into small blocks that are easier to manage and process, thereby reducing computational complexity and improving processing efficiency. Feature block division can be achieved through simple grid division, that is, dividing the feature map into multiple small blocks according to a predetermined size (such as 8x8, 16x16, etc.). Each small block retains part of the information of the original feature map, but is smaller in scale, which is convenient for subsequent processing.

[0054] The parallel computing submodule is responsible for calculating the degree of correlation between each small feature block and adjusting the weights of the feature blocks based on these correlations. This step aims to emphasize those feature blocks that are highly correlated with other feature blocks (i.e. contain important information), while reducing the weights of those feature blocks with low correlation (i.e. information redundancy). Parallel computing can be achieved in a variety of ways, such as using attention mechanisms, convolutional neural networks (CNNs), or graph neural networks (GNNs). These techniques can efficiently calculate the similarities between feature blocks and adjust the weights accordingly. Specifically, a lightweight network can be designed to calculate the correlation matrix between feature blocks, and then use this matrix to update the weights of the feature blocks.

[0055] The task of the recurrent submodule is to calculate the connection between the current feature block and the previous feature block, and to selectively retain important feature blocks based on these connections. This step aims to maintain the consistency of the feature map in the time or sequence dimension, thereby retaining key visual state information. The recurrent submodule can be implemented based on a recurrent neural network (RNN) or its variants (such as long short-term memory network LSTM, gated recurrent unit GRU, etc.). These networks can capture the temporal dependencies in sequence data and are therefore suitable for processing temporal connections between feature blocks. By designing a recurrent network to process a sequence of feature blocks, the weight of the current feature block can be updated based on the previous state, thereby selectively retaining important information.

[0056] Based on the aforementioned scheme, in some implementations of the present application, the visual state maintenance module divides the feature map in the first feature information into multiple small feature blocks of the same size in an evenly distributed manner, uses a parallel computing submodule to perform calculations within the block, and uses a loop submodule to perform calculations between different blocks.

[0057] In the above implementation, the visual state maintenance module first divides the input first feature information (feature map) into multiple small blocks of features of the same size in an evenly divided manner. The purpose of this step is to decompose the complex feature space into small units that are easier to manage and analyze, so as to perform efficient parallel calculation and loop processing later. Then, the parallel calculation submodule is responsible for processing the information in each small block feature. It uses efficient parallel computing technology (such as GPU acceleration, multi-threaded processing, etc.) to independently calculate and analyze each small block feature. The loop submodule is responsible for processing the relationship between different small block features. It uses a recurrent neural network (RNN) or its variants (such as LSTM, GRU, etc.) to capture the temporal or spatial dependencies between small block features. Thus, the relationship between different small block features is processed, the state is updated, and important features are selectively retained. That is, the visual state maintenance module achieves effective capture and retention of key visual information through fine feature map segmentation and efficient parallel and loop calculation strategies. This design not only improves computational efficiency, but also enhances the model's ability to understand and analyze image content.

[0058] Based on the aforementioned scheme, in some implementations of the present application, the rotation detection head module includes a position detection network, a classification detection network and an angle detection network, which are respectively used to predict the position, category and angle information of the target.

[0059] In the above implementation, the rotation detection head module realizes accurate detection of rotating targets by introducing a position detection network, a classification detection network and an angle detection network. This design not only improves the detection accuracy of the model, but also enhances the robustness of the model to rotating targets. Among them, the main task of the position detection network is to predict the position information of the target in the image. It can include the bounding box of the target, that is, the coordinates of the upper left corner and lower right corner of the target, or more complex representations, such as the center point coordinates, width and height. The position detection network can adopt structures such as convolutional neural network (CNN) or fully connected network (FCN) to predict the target position by extracting image features and applying regression algorithms. In order to improve accuracy, the network can be designed as a multi-stage or cascade structure to gradually refine the prediction results. The classification detection network can adopt a convolutional neural network (CNN) structure, extract image features through components such as convolutional layers, pooling layers and fully connected layers, and apply softmax function or cross entropy loss function to calculate category probabilities. In order to improve classification accuracy, the network can introduce advanced technologies such as attention mechanism and feature pyramid. The angle detection network can use a regression algorithm to predict the rotation angle of the target. In order to improve the prediction accuracy, the network can be designed as a multi-branch structure, with each branch responsible for predicting targets within a different angle range. In addition, angle constraints or regularization terms can be introduced to limit the range of predicted angles to avoid ambiguity.

[0060] Based on the aforementioned scheme, in some implementations of the present application, the target image is input into a feature extraction network to extract multi-scale feature information of different dimensions, and feature map size alignment is performed through convolution and / or interpolation, including: inputting the target image into the feature extraction network, extracting features P1 and P2 using a normal convolution layer, and extracting features P3, P4, and P5 using an inverse residual multi-scale convolution layer with different convolution kernel sizes; according to a top-to-bottom and bottom-to-top path aggregation network, feature P3, feature P4, and feature P5 are aligned in size by convolution downsampling and difference upsampling; wherein, in the top-to-bottom path aggregation network, feature P5 is fused with feature P5 through bilinear difference upsampling to obtain feature C4, and feature C4 is fused with feature P3 through convolution downsampling to obtain feature C3; in the bottom-to-top path aggregation network, feature C4 is downsampled and aggregated by convolution to obtain feature C5 to obtain feature S5, and feature C3 and feature C4 are fused to obtain feature S4.

[0061] In the above implementation, multi-scale feature information is extracted through the feature extraction network, and the feature map is aligned using the top-to-bottom and bottom-to-top path aggregation networks. This not only improves the accuracy of feature extraction, but also enhances the complementarity between features, providing a more reliable feature representation for subsequent target detection or classification tasks.

[0062] Based on the above scheme, in some implementations of the present application, the method further includes comparing the predicted position with the actual position according to the label detection information results to calculate the loss to optimize the model. That is, by comparing the predicted position with the actual position according to the label detection information results, the label detection information is introduced as a supervisory signal so that the model can learn the actual position and category information of the target, thereby improving the accuracy of target detection. At the same time, by calculating the loss and optimizing the model accordingly, the performance of the model can be gradually improved to better meet the needs of actual application scenarios.

[0063] In order to make the technical personnel in this field understand the present application more intuitively, a specific example will be used here to illustrate. In this example, the system for detecting the label of the fiber distribution box includes an image acquisition unit, a first processing unit, a second processing unit and a result generation module. The information interaction between the various units is as follows:

[0064] The image acquisition unit is used to acquire a target image including labels on each wire body in the target fiber distribution box, and before inputting the target image into the feature extraction network, it also includes deleting dark scenes and blurred target images, adjusting the size of the target image and normalizing the target image.

[0065] The first processing unit is used to fuse feature information of different scales and perform feature encoding. The backbone feature extraction network extracts feature information. The target image is input into the feature extraction network to obtain multi-scale feature information of different dimensions. The feature information of different scales is first aligned by convolution or interpolation method, and then input into the cross-channel self-attention module to fuse feature information of different scales.

[0066] The second processing unit is used to input the first feature information into the visual state maintenance module to calculate the global feature information and assign different weight information according to the distance size of the feature, so as to obtain the second feature information, thereby decoding the high-dimensional feature information. The visual state maintenance module can efficiently calculate the global feature information and assign different weight information according to the distance size of the feature. The visual state maintenance module includes a feature block submodule, a parallel calculation submodule and a loop submodule; wherein the feature block submodule is used to divide the large feature map in the first feature information into multiple small feature blocks, the parallel calculation submodule is used to adjust the weight of the feature block by calculating the degree of correlation between each small feature block, and the loop submodule is used to calculate the connection between the current feature block and the previous feature block to selectively retain important feature blocks.

[0067] The result generating unit is used to input the second feature information into the rotating detection head module to generate the size, category and angle information of the target and obtain the label detection result information. That is, the feature information decoded by the visual state maintenance module is input into the rotating detection head module to generate the label detection result information. At the same time, the model predicted position and the actual position can also be compared to calculate the loss optimization model.

[0068] Furthermore, the data processing flow in the image acquisition unit, the first processing unit, the second processing unit and the result generation module may be as follows:

[0069] (1) The working principle of the image acquisition unit can be as follows: when acquiring the target image, under the guidance of the installation and maintenance personnel, the fiber splitter box can be opened correctly to capture the fiber label data containing the fiber information. The light should be as sufficient as possible, the label should not be blocked as much as possible, and the image should be clear. Then, data cleaning is performed to delete low-quality data images. For example, blurry, unclear, dark and unlabeled images are removed from the data set. Next, data preprocessing is performed to adjust the image size and normalize it. H and W represent the input image X img The length and width of the image are adjusted to the size of H' and W' using the resize function to obtain the image X res , exemplarily set it to 640 and 640. Then set the image X res Each pixel is normalized by dividing it by 255 to obtain the input X of the model in , each pixel x ij The values ​​of are all between [0,1]. The above steps are shown in the following formula: res = resize(X img ); x ij ∈[0,1].

[0070] (2) The working principle of the first processing unit may be as follows:

[0071] Step 201, extract feature information using the backbone feature extraction network. Figure 2 As shown in the figure, the feature map X is input into the convolutional neural network backbone network for feature extraction of different scales. The backbone network is a normal convolution layer Conv and an inverse residual multi-scale convolution layer Ms with different convolution kernel sizes. The inverse residual multi-scale convolution layer Ms can reduce the convolution layer parameters and capture information of different receptive field sizes to obtain features P1, P2, P3, P4 and P5. The shallow feature information contains local detail information and target location information, and the deep feature information contains global abstract information. Feature information at different levels is input into subsequent modules to fuse feature information of different scales. The above steps are shown in the following formula:

[0072] P1=Conv1(X)

[0073] P2=Conv2(P1)

[0074] P3=Ms3(P2)

[0075] P4=Ms4(P3)

[0076] P5=Ms5(P4)

[0077] Where X represents the input feature map; Conv1() and Conv2() represent processing by ordinary convolutional layers; Ms3(), Ms4(), and Ms5() represent processing using inverted residual multi-scale convolutional layers; P i Represents the features obtained after feature extraction by the convolution layer Conv or the inverse residual multi-scale convolution layer Ms.

[0078] Step 202, align feature maps of different scales. Based on the features P3, P4, and P5 obtained in step 201, align the feature map sizes by convolution downsampling Conv and interpolation upsampling Upsample, and build a top-to-bottom and bottom-to-top path aggregation network. In the top-to-bottom path aggregation network, the high-level feature P5 is fused with the feature P5 through bilinear difference upsampling to obtain feature C4, where CLFM is a cross-level feature fusion module. C4 is fused with the feature P3 through convolution downsampling to obtain C3. In the bottom-to-top path aggregation network, C4 is downsampled and aggregated by convolution to obtain C5 features to obtain S5, and C3 and C4 are fused to obtain S4. 1×1 For point convolution, the above steps are shown in the following formula:

[0079] C5=f 1×1 (P5)

[0080] C4=CLFM(P4,Upsample(P5))

[0081] C3=CLFM(P3,Upsample(C4))

[0082] S5=CLFM(C5,Conv(C4))

[0083] S4=CLFM(C4,Conv(C3))

[0084] S3=f 1×1 (C3)

[0085] Among them, f 1×1 represents a point convolution with a kernel size of 1; CLFM represents a cross-level feature fusion module; Upsample represents upsampling, which is used to align feature layers of different sizes; C i It represents the features obtained by top-down feature fusion in the first stage; S i Represents the features obtained through bottom-up feature fusion in the second stage.

[0086] Step 203, adjust the weights using cross-channel self-attention. Cross-level feature fusion is mainly performed in the form of cross-channel self-attention. Figure 3 As shown in Figure 1, the features of different layers are first aligned in size by Conv or Upsample. Taking features P4 and P5 as an example, the two layers of features are first passed through the linear mapping layer fc and then concatenated along the channel dimension to obtain P c1 , P c1 It is input into the convolution to aggregate the information of channel features at different levels to obtain P c2 .P c1 and P c2 Calculate the similarity between the two channel weights to get the weight mask P c_mask The features of P4 and P5 are directly concatenated along the channel dimension and the outer product is performed with the weight mask to obtain the fused features. The above steps are shown in the following formula:

[0087] P c1 =Concat(fc(P4),fc(Upsample(P5)))

[0088] P c2 =f 1×1 (P c1 )

[0089] P c_mask =P c1 ×P c2

[0090] S4=Concat(P4,P5)P c_mask

[0091] Among them, fc() means processing using a fully connected layer; Concat() means concatenating features along the channel dimension; P c1 ∈R C×N represents the features obtained by splicing, C represents the channel dimension size, and N represents the size of each dimension; P c2 ∈R N×C represents the feature map obtained after channel weight adjustment; P c_mask ∈R C×C Indicates P c1 and P c2 The cross-channel self-attention feature map is obtained by performing matrix outer product to calculate the similarity relationship between the features of each channel; S4 represents the feature obtained after cross-level feature fusion of feature P4 and feature P5.

[0092] Step 204, cross-level multi-scale feature encoding. Based on the different scale feature maps S3, S4 and S5 obtained in step 10202, the feature map H1 is directly fused according to the channel dimension. The above steps are shown in the following formula:

[0093] H1=Concat(S3,S4,S5)

[0094] Among them, Concat() represents concatenating features along the channel dimension; S3, S4 and S5 represent features obtained after cross-level fusion; H1 represents the feature map obtained by fusion of different level features S3, S4 and S5 according to the channel dimension.

[0095] (3) The working principle of the second processing unit may be as follows:

[0096] Step 301: feature segmentation. Figure 4 As shown, based on the feature X obtained by the first processing unit, the feature map is split into multiple small blocks X of the same size according to the equal division method. i , the parallel computing submodule PRM is used for calculation within the block, and the loop submodule RRM is used for calculation between different blocks. The specific steps are as follows:

[0097] X i =Split(X)

[0098] Y i =PRM(X i )

[0099] Z i =RRM(Y i )

[0100] Among them, Split() indicates feature segmentation along the channel dimension; PRM() indicates calculation using the parallel computing submodule; PRM() indicates calculation using the loop submodule; the input feature X is segmented along the channel dimension to obtain the feature subgraph X i ; Feature X i Then input it into the parallel computing submodule to perform data calculation in parallel to obtain the feature subgraph Y i ; The characteristic subgraph is input into the loop submodule for loop holding calculation to obtain the characteristic subgraph Z i .

[0101] Step 302: Extract features. n Represents the model state information at time n, s n The state information s at time n-1 n-1 and key vector With the value vector v n Linear attention summation gives o n Represents the feature information at the nth moment. n By query vector Q n With status information n The matrix outer product is obtained. n Substitute the calculation formula into o n The equation can be obtained This formula represents the characteristic information o at time n n The query vector Q from time 1 to time n n , position embedding matrix A n-m , and v m The matrix outer product is obtained. A is the position embedding matrix. γ n-m is the attenuation coefficient. The farther away from the current position, the more serious the attenuation. Λ is a diagonal matrix, which is reversible. Substitute the position code A into o n Get the final feature Finally, the feature n The attenuation coefficient γ and the position information encoding e inθ , query feature Q n , key feature K m Sum value feature v m The composition is affected by the characteristics from the initial time m to the current time n. The specific steps are as follows:

[0102]

[0103] A=Λ(γe iθ )Λ -1

[0104]

[0105] Step 303: Processing is performed using the parallel computing submodule PRM. The parallel computing submodule is used to maintain the parallel computing mode of the network. Figure 5 As shown, the feature map X passes through the linear layer f 1×1 Mapping and position encoding xPos are performed to obtain the query vector Q, key vector K and value vector V. Position encoding xPos and Indicates that the module is kept inθ and e imθ , the position code xPos of Q and the position code of K is a conjugate form. D represents the attenuation coefficient, D nm The formula is D nm Indicates the influence of the feature of pixel point m on the feature of current pixel point n. The farther away from the current pixel point, the smaller the weight of the pixel point. nm Gamma in the module for visual state preservation n-m Q and K are matrix multiplied to get the self-attention matrix QK T , and then multiply it by D with the weight coefficient matrix to get the self-attention mask. The value vector V is multiplied by the self-attention mask to adjust the feature map weight. The parallel calculation submodule adjusts the feature weight according to the distance. The above steps are shown in the following formula:

[0106] Q=xPos(f 1×1 (X)

[0107]

[0108] V=f 1×1 (X)

[0109] xPos=e inθ

[0110]

[0111] Y = PRM (X) = (Softmax (QK T ⊙D))V+X

[0112] Among them, Q, K and V are the query vector, key vector and value vector obtained by feature X after linear layer and position encoding; inθ represents position coding, i is the imaginary unit, n is the nth moment, and θ is the angle information; is the attenuation coefficient, x n Represents the horizontal axis feature information, y n Represents the vertical axis feature information.

[0113] Step 304: Processing is performed using a loop submodule. The loop submodule is a loop calculation submodule of the visual state maintenance module. The loop submodule considers the correlation between blocks and retains the state information of the previous block. Figure 6 As shown, s i-1 Indicates the status information of the i-1th block, X i Represents the i-th feature map. The state s at the previous moment i-1 After the attenuation coefficient γ is calculated, it is compared with the current linear self-attention mask Add together to get the current state s i .X i After the parallel computing submodule PRM calculates the correlation between each pixel in the block, it combines the current state with the query vector Q i , after linear mapping, we get the block feature Y i The feature map Y of each block i Aggregate as a whole and input it into the lower network. The above steps are shown in the following formula:

[0114]

[0115] Y i =f 1×1 (PRM(X i )+Q i s i )

[0116] Y=f 1×1 (Concat(Y1,Y2,...,Y i ,...Y n ))

[0117] Among them, s i-1 and i is the state information of the i-1th block and the i-th block; the query vector, key vector and value vector obtained by the linear layer and position encoding of Q, K and V features X; e inθ represents position coding, i is the imaginary unit, n is the nth moment, and θ is the angle information; is the attenuation coefficient, x n Represents the horizontal axis feature information, y n Represents the vertical axis feature information.

[0118] Step 305: Decoding is performed using the visual state preservation module. The overall calculation method is to calculate each pixel point X in the block i The correlation between blocks, the inter-block calculation state information s i-1 After aggregating the information of each sub-block, it is input into the forward feedback network FFN for information decoding. The above steps are shown in the following formula:

[0119]

[0120] Among them, PRM() means using parallel computing submodules to perform calculations and parallel calculations of intra-block feature information; RRM() means using loop submodules to perform calculations and calculate inter-block feature information; FFN() means using feedforward neural network for processing; n means the feature information of the nth block; X i Represents the i-th block of feature information.

[0121] (4) The working principle of the result generation unit can be as follows:

[0122] Step 401, use the rotation detection head module for processing. The feature layer Y obtained by the second processing unit is input into the position detection network, the classification detection network and the angle detection network respectively to predict the position and category of the target. The learning parameter lengths of the last layer of the detection head are different, representing the number of corresponding prediction parameters. The above steps are shown in the following formula:

[0123] Pos=f pos Conv(Conv(Y))

[0124] Class = f class Conv(Conv(Y))

[0125] Angle = f angle Conv(Conv(Y))

[0126] Among them, Conv() means processing using the convolutional layer, f pos 、f class and f angle Represents position linear mapping, category linear mapping and angle linear mapping; Pos, Class and Angle represent the predicted position information, category information and angle information respectively.

[0127] Step 402, compare the predicted position with the actual position. Obtaining the actual position information includes using the rolabelimg program to label the line body data, select the minimum circumscribed rectangle of the label, and record the position and angle information of its center point. Calculate the positions of the initial four points, (x c ,y c ) is the position of the center point, (x, y) is the position of the four points of the minimum circumscribed rectangle, w and h are the width and height of the image respectively. The above steps are shown in the following formula:

[0128]

[0129] (d x ,d y ) is the original four points (x, y) and the center point (x c ,yc ), (x', y') is the actual position of the four points of the rotated rectangle. The above steps are shown in the following formula:

[0130] d x =xx c

[0131] d y =yy c

[0132] x'=x c +x o cosθ+y o sinθ

[0133] y'=y c -x o sinθ+y o cosθ

[0134] The loss information calculated by the model prediction position information Pos and the real position information (x', y') is used to optimize the model, where c is the number of samples. The specific calculation method is as follows:

[0135]

[0136] Example 2

[0137] See also Figure 7 , the embodiment of the present application provides a system for detecting a line label of a fiber distribution box, which includes:

[0138] An image acquisition unit is used to acquire a target image including labels on each wire body in a target fiber distribution box; a first processing unit is used to input the target image into a feature extraction network to extract multi-scale feature information of different dimensions, and perform feature map size alignment through convolution and / or interpolation, and use a cross-channel self-attention module to fuse feature information of different scales to obtain first feature information; a second processing unit is used to input the first feature information into a visual state maintenance module to calculate global feature information and assign different weight information according to the distance size of the feature, so as to obtain second feature information; a result generation unit is used to input the second feature information into a rotation detection head module to generate target size, category and angle information to obtain label detection result information.

[0139] For the specific implementation process of the above system, please refer to the method for detecting the line label of the fiber distribution box provided in Example 1, which will not be repeated here.

[0140] Example 3

[0141] See also Figure 8, the embodiment of the present application provides an electronic device, which includes at least one processor 201 and at least one memory 202; wherein the processor 201 and the memory 202 are directly connected to each other, or communicate with each other through a communication interface 203, or are electrically connected through one or more communication buses or signal lines to achieve data transmission or interaction; the memory 202 stores program instructions that can be executed by the processor 201, and the processor 201 calls the program instructions to execute a method for detecting a line label of a fiber distribution box. For example, the following is realized:

[0142] A target image including labels on each wire in a target fiber distribution box is obtained; the target image is input into a feature extraction network to extract multi-scale feature information of different dimensions, and feature map size alignment is performed through convolution and / or interpolation, and feature information of different scales is fused using a cross-channel self-attention module to obtain first feature information; the first feature information is input into a visual state maintenance module to calculate global feature information and assign different weight information according to the distance of the feature, thereby obtaining second feature information; the second feature information is input into a rotation detection head module to generate target size, category and angle information, thereby obtaining label detection result information.

[0143] Among them, the memory 202 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0144] The processor 201 may be an integrated circuit chip with signal processing capability. The processor 201 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0145] Understandably, Figure 8 The structure shown is for illustration only. The electronic device may also include Figure 8 More or fewer components as shown, or with Figure 8 Different configurations shown. Figure 8 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0146] Example 4

[0147] The present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by the processor 201, a method for detecting a line label of a fiber distribution box is implemented.

[0148] For example, to implement:

[0149] A target image including labels on each wire in a target fiber distribution box is obtained; the target image is input into a feature extraction network to extract multi-scale feature information of different dimensions, and feature map size alignment is performed through convolution and / or interpolation, and feature information of different scales is fused using a cross-channel self-attention module to obtain first feature information; the first feature information is input into a visual state maintenance module to calculate global feature information and assign different weight information according to the distance of the feature, thereby obtaining second feature information; the second feature information is input into a rotation detection head module to generate target size, category and angle information, thereby obtaining label detection result information.

[0150] If the above functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0151] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for detecting a label on a fiber distribution box, characterized in that: The following steps are involved: Acquire a target image including labels on each wire body in a target fiber distribution box; Input the target image into the feature extraction network to extract multi-scale feature information of different dimensions, align the feature map size by convolution and / or interpolation, and fuse the feature information of different scales by using a cross-channel self-attention module to obtain the first feature information; Inputting the first feature information into a visual state maintenance module to calculate global feature information and assign different weight information according to the distance of the features, thereby obtaining second feature information; The second feature information is input into the rotating detection head module to generate the size, category and angle information of the target and obtain the label detection result information.

2. The method according to claim 1, characterized in that It also includes removing dark scenes and blurred target images, as well as resizing and normalizing the target images before inputting them into the feature extraction network.

3. The method according to claim 1, characterized in that The backbone network of the feature extraction network includes ordinary convolutional layers and inverted residual multi-scale convolutional layers with different convolution kernel sizes, which are used to capture information of different receptive field sizes; wherein, the shallow feature information contains local detail information and target position information, and the deep feature information contains global abstract information.

4. The method according to claim 1, characterized in that: The cross-channel self-attention module obtains a weight mask by calculating the similarity between the weights of each pair of channels, which is used to adjust the weights of feature maps of different layers, thereby fusing feature information of different scales.

5. The method according to claim 1, characterized in that The visual state maintenance module includes a feature block submodule, a parallel calculation submodule and a loop submodule; wherein the feature block submodule is used to divide the large feature map in the first characteristic information into multiple small feature blocks, the parallel calculation submodule is used to adjust the weight of the feature block by calculating the degree of correlation between each small feature block, and the loop submodule is used to calculate the connection between the current feature block and the previous feature block to selectively retain important feature blocks.

6. The method according to claim 5, characterized in that The visual state preservation module divides the feature map in the first feature information into multiple small blocks of features of the same size in an evenly divided manner, uses a parallel computing submodule to perform calculations within the block, and uses a loop submodule to perform calculations between different blocks.

7. The method according to claim 1, characterized in that The rotation detection head module includes a position detection network, a classification detection network and an angle detection network, which are used to predict the position, category and angle information of the target respectively.

8. The method according to claim 1, characterized in that: The target image is input into the feature extraction network to extract multi-scale feature information of different dimensions, and the size of the feature map is aligned by convolution and / or interpolation, including: Input the target image into the feature extraction network, use the ordinary convolution layer to extract features P1 and P2, and use the inverse residual multi-scale convolution layer with different convolution kernel sizes to extract features P3, P4 and P5; According to the top-to-bottom and bottom-to-top path aggregation networks, the feature maps of features P3, P4 and P5 are aligned in size by convolution downsampling and difference upsampling. In the top-to-bottom path aggregation network, feature P5 is fused with feature P5 by bilinear difference upsampling to obtain feature C4, and feature C4 is fused with feature P3 by convolution downsampling to obtain feature C3. In the bottom-to-top path aggregation network, feature C4 is downsampled and aggregated by convolution to obtain feature C5 to obtain feature S5, and feature C3 and feature C4 are fused to obtain feature S4.

9. The method according to claim 1, characterized in that: The method also includes comparing the predicted position with the actual position based on the label detection information results to calculate the loss to optimize the model.

10. A system for detecting label on a fiber distribution box, characterized in that: include: An image acquisition unit, used to acquire a target image including labels on each wire body in a target fiber distribution box; A first processing unit is used to input the target image into a feature extraction network to extract multi-scale feature information of different dimensions, align the size of the feature map by convolution and / or interpolation, and fuse the feature information of different scales by using a cross-channel self-attention module to obtain first feature information; A second processing unit, used for inputting the first feature information into a visual state maintaining module to calculate global feature information and assign different weight information according to the distance of the features, so as to obtain second feature information; The result generating unit is used to input the second feature information into the rotation detection head module to generate the size, category and angle information of the target and obtain the label detection result information.

Images