Method, device and equipment for detecting status of optical cross-connect port and storage medium

Abstract

The application discloses a method, device and equipment for detecting the state of a port of an optical crossbox and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a port detection image of the optical crossbox; inputting the port detection image into a pre-trained port state detection model, performing multi-scale feature extraction on the port detection image through the port state detection model to obtain a multi-scale initial feature map, and performing feature enhancement on the initial feature map according to image channel weights and image space weights to obtain port feature maps of different scales; performing feature fusion on the port feature maps of different scales through a feature fusion network of the port state detection model to obtain a plurality of first fusion features of different scales, performing feature fusion on the first fusion features to obtain a second fusion feature; and determining the port state of the optical crossbox through the second fusion feature of the port state detection model and preset weight parameters. The application improves the accuracy of the detection of the port state of the optical crossbox.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method, apparatus, device and storage medium for detecting the port status of an optical distribution box. Background Technology

[0002] As a key device in optical fiber access networks used to connect trunk optical cables and branch optical cables, the optical cable junction box undertakes the core functions of optical cable aggregation, distribution, scheduling and management. The accurate identification of its port connection status is a critical task for operators' network operation and maintenance. This identification work can not only help operation and maintenance personnel to rationally plan optical fiber resources and improve network expansion efficiency, but also quickly locate problem nodes when a fault occurs to shorten the repair time. At the same time, it can avoid resource idleness and redundant construction, realize refined management of network resources, and ensure the efficient operation of communication networks and the improvement of user service quality.

[0003] Existing technical solutions often segment the port portion of an image to obtain multiple port images, and then use a trained convolutional neural network model to classify each port image to obtain the recognition result. However, in actual engineering applications, due to the dense arrangement of pigtails, a certain proportion of the ports of the optical distribution box are often partially blocked by pigtails, which can easily lead to misjudgment or omission during the judgment, resulting in a decrease in recognition accuracy. Therefore, the accuracy of optical distribution box port status detection is low. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for detecting the port status of an optical distribution box, in order to solve the problem that existing methods for detecting the port status of optical distribution boxes have low accuracy.

[0005] In a first aspect, embodiments of this application disclose a method for detecting the port status of an optical distribution box, the method comprising:

[0006] Acquire port detection images of the optical distribution box;

[0007] The port detection image is input into the pre-trained port state detection model. The feature extraction network of the port state detection model performs multi-scale feature extraction on the port detection image to obtain multi-scale initial feature maps. The extracted multi-scale initial feature maps are then enhanced according to the image channel weights and image spatial weights to obtain port feature maps of different scales.

[0008] The feature fusion network of the port state detection model performs feature fusion on port feature maps of different scales to obtain multiple first fusion features of different scales, and then fuses the multiple first fusion features of different scales to obtain a second fusion feature.

[0009] The port status of the optical distribution box is determined by the second fusion feature of the port status detection model and the preset weight parameters.

[0010] Secondly, embodiments of this application disclose a device for detecting the port status of an optical distribution box, the device comprising:

[0011] The acquisition module is used to acquire port detection images of the optical distribution box;

[0012] The extraction module is used to input the port detection image into the pre-trained port state detection model. The feature extraction network of the port state detection model performs multi-scale feature extraction on the port detection image to obtain multi-scale initial feature maps. The extracted multi-scale initial feature maps are then enhanced according to the image channel weights and image spatial weights to obtain port feature maps of different scales.

[0013] The fusion module is used to perform feature fusion on port feature maps of different scales through the feature fusion network of the port state detection model to obtain multiple first fusion features of different scales, and then fuse the multiple first fusion features of different scales to obtain a second fusion feature.

[0014] The determination module is used to determine the port status of the optical distribution box by using the second fusion feature of the port status detection model and preset weight parameters.

[0015] Thirdly, embodiments of this application provide a terminal device, which includes: a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the method for detecting the status of the optical distribution box port as described in the first aspect.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the method for detecting the status of an optical distribution box port as described in the first aspect.

[0017] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the optical distribution box port status detection method as described in the first aspect.

[0018] This application provides a method, apparatus, device, and storage medium for detecting the port status of an optical distribution box. The method first acquires a port detection image of the optical distribution box; then, it inputs the port detection image into a pre-trained port status detection model. The feature extraction network of the port status detection model performs multi-scale feature extraction on the port detection image to obtain multi-scale initial feature maps. The extracted multi-scale initial feature maps are then enhanced according to image channel weights and image spatial weights to obtain port feature maps of different scales. By extracting port features at different scales, fixed segmentation is not required, avoiding port feature loss. The enhancement of port features through channel weights and spatial weights improves detection accuracy. The feature fusion network of the port status detection model fuses the port feature maps of different scales separately to obtain multiple first fused features of different scales. These first fused features of different scales are then fused to obtain a second fused feature. By fusing features of the same scale, details and semantic features of the same scale are preserved. By fusing features of different scales, details and semantic features of different scales can be combined, improving the accuracy of port detection. The port status of the optical distribution box is determined using the second fused feature of the port status detection model and preset weight parameters. This application embodiment improves the accuracy of optical distribution box port status detection by performing multi-scale feature extraction on port detection images and fusing details and semantic features at different scales, while ensuring the semantic integrity of port category judgment and the detailed accuracy of location positioning. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the method for detecting the port status of an optical distribution box provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of a capped port provided in an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of a capless port provided in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of a plug-in port provided in an embodiment of this application;

[0024] Figure 5 This is a port diagram of a damaged port provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the backbone network that introduces the convolutional block attention mechanism provided in the embodiments of this application;

[0026] Figure 7 This is a schematic diagram of the process for determining the port feature map provided in the embodiments of this application;

[0027] Figure 8 This is a schematic diagram of the process for determining the second fusion feature provided in the embodiments of this application;

[0028] Figure 9 This is a schematic diagram of the structure of the optical distribution box port status detection device provided in the embodiments of this application;

[0029] Figure 10 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0030] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0031] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0032] Existing technologies rely on chessboard algorithms to segment the ports of optical distribution boxes and use classifiers to determine the port connection status in the segmented port images, or rely on training convolutional neural networks to output port status heatmaps and then identify the port connection status based on these heatmaps. However, in practical engineering applications, optical distribution boxes distributed in many counties and rural areas often suffer from misalignment of the fiber optic trays due to improper installation by workers, leading to misalignment of the upper and lower ports. For images of such optical distribution boxes, this method suffers from missing port information because the chessboard algorithm cannot segment complete port images, resulting in a significant decrease in the accuracy of port connection status identification. Furthermore, in scenarios with high port occupancy rates, if a certain proportion of ports are partially obstructed by pigtails, the heatmap-based port connection status determination may easily misclassify partially obstructed ports as missed, further reducing the accuracy of port connection status identification and affecting the reliability and effectiveness of this technology in practical applications.

[0033] To address the problems of existing technologies, this application provides a method, apparatus, device, and storage medium for detecting the port status of an optical distribution box. The method first acquires a port detection image of the optical distribution box; then, it inputs the port detection image into a pre-trained port status detection model. The feature extraction network of the port status detection model performs multi-scale feature extraction on the port detection image to obtain multi-scale initial feature maps. The extracted multi-scale initial feature maps are then enhanced based on image channel weights and image spatial weights to obtain port feature maps of different scales. By extracting port features at different scales, fixed segmentation is not required, avoiding port feature loss. The enhancement of port features through channel and spatial weights improves detection accuracy. The feature fusion network of the port status detection model fuses the port feature maps of different scales separately to obtain multiple first fused features of different scales. These first fused features of different scales are then fused to obtain a second fused feature. By fusing features of the same scale, details and semantic features of the same scale are preserved. By fusing features of different scales, details and semantic features of different scales can be combined, improving the accuracy of port detection. The port status of the optical distribution box is determined using the second fused feature of the port status detection model and preset weight parameters. This application embodiment improves the accuracy of optical distribution box port status detection by performing multi-scale feature extraction on port detection images and fusing details and semantic features at different scales, while ensuring the semantic integrity of port category judgment and the detailed accuracy of location positioning.

[0034] The following is a description of a method for detecting the port status of an optical distribution box provided in an embodiment of this application.

[0035] Figure 1This illustration shows a flowchart of a method for detecting the port status of an optical distribution box according to an embodiment of this application. Figure 1 As shown, the method may include the following steps: S101 to S104.

[0036] S101, acquire the port detection image of the optical distribution box.

[0037] Among them, the port detection image of the optical distribution box includes a visual image of the identifiable ports inside the optical distribution box.

[0038] In some embodiments, port detection images of the optical distribution box can be collected by staff using a smart terminal.

[0039] This application embodiment relies on the portability of smart terminals, which can be easily carried by installation and maintenance personnel to any optical distribution box site without complicated equipment installation and debugging procedures or additional purchase of dedicated acquisition equipment, thus greatly reducing equipment investment costs; and it is adapted to the real-time acquisition needs of different outdoor scenarios, which can improve the flexibility of on-site acquisition operations.

[0040] In some embodiments of this application, a camera is set at a preset position in the optical distribution box, such as on the bracket after the optical distribution box is opened or inside the optical distribution box, and a port image is captured once every preset time interval.

[0041] This application embodiment sets up a camera at a preset position in the optical distribution box and takes timed pictures at fixed intervals, eliminating the need for manual on-site operation, greatly reducing labor costs and improving the automation of optical distribution box port status detection.

[0042] S102, the port detection image is input into the pre-trained port state detection model. The feature extraction network of the port state detection model performs multi-scale feature extraction on the port detection image to obtain multi-scale initial feature maps. The extracted multi-scale initial feature maps are then enhanced according to the image channel weights and image spatial weights to obtain port feature maps of different scales.

[0043] Among them, the pre-trained port state detection model is a port state detection model trained in advance using the optical cross-box labeled dataset; the multi-scale initial feature map is a feature map extracted at different resolutions; the image channel weight is a weight used to enhance the key feature channels of the port and suppress the background channels, and the spatial weight is a weight used to focus on the pixel region where the port is located and suppress the background pixels.

[0044] In one example, the initial feature maps at multiple scales can be 80×80, 40×40, and 20×20 in size. Larger initial feature maps are used to capture port details, while smaller initial feature maps are used to capture the global semantic distribution of the ports. The size of the feature map refers to its spatial dimension, i.e., the number of pixels in its width and height. The scale of the feature map refers to the information level of the original image corresponding to the feature map, i.e., the degree of abstraction of the feature or the size of the receptive field. Larger sizes result in higher spatial resolution, smaller scales, and richer spatial details, which is beneficial for accurately capturing port details. Conversely, smaller sizes result in lower spatial resolution, larger scales, and stronger semantic information, which is beneficial for capturing global port information.

[0045] This application embodiment obtains initial feature maps of different resolutions through multi-scale feature extraction, avoiding the omission of small ports or the deviation in judging the overall distribution of ports by single-scale features. By strengthening key feature channels and suppressing background channels through image channel weights, and focusing on port pixel areas by combining spatial weights, interference information such as box background and screws can be effectively filtered out, providing more accurate features for subsequent detection.

[0046] In some embodiments, the port status detection model is a deep learning model (Transformer model) based on a self-attention mechanism, including a backbone network (feature extraction network), a neck network (feature fusion network), and a head network. The backbone network extracts features from the ports in the optical distribution box image, the neck network performs feature fusion, and the head network outputs the port status based on the fused features. By selecting the Transformer model, its self-attention characteristic can effectively capture the correlation between ports, avoiding the problem of traditional detection models' insufficient capture of long-distance feature dependencies, and improving the adaptability to complex optical distribution box port scenarios.

[0047] In some embodiments, the backbone network is a real-time end-to-end object detection model (RT-DETR), in which a Convolutional Block Attention Module (CBAM) is introduced and added to the final basic residual block, BasicBlock, of the backbone network. The CBAM attention mechanism enables the model to automatically learn attention weights in space and channel by sequentially integrating spatial and channel attention, thereby guiding the model to focus on information-rich parts of the image. This enhances the model's ability to perceive image details and generalize to different scenes and complex data, improving the accuracy of the model in detecting the port connection status of optical junction boxes. By integrating spatial and channel attention in the final basic residual block, BasicBlock, the model can specifically enhance port details during feature extraction while suppressing background interference, thus improving the accuracy of port status detection.

[0048] S103, the feature fusion network of the port state detection model performs feature fusion on the port feature maps of different scales respectively to obtain multiple first fused features of different scales, and then fuses the multiple first fused features of different scales to obtain the second fused features.

[0049] The first fusion feature is a single-scale feature, which strengthens the correlation between features within the same scale; the second fusion feature is a multi-scale fusion feature, which strengthens the correlation between features within different scales.

[0050] This application embodiment first strengthens the correlation of single-scale features to ensure that the first fused feature can accurately capture the local correlation rules of ports within a single scale; the cross-scale fusion of the second fused feature is based on the single-scale features that have already undergone feature fusion, rather than directly fusing the original multi-scale port feature map. This can preserve the details of small-scale features and incorporate the semantics of large-scale features, avoiding the problems of detail loss or semantic confusion caused by direct cross-scale fusion.

[0051] In some embodiments, the feature fusion network is a hybrid encoder, including an attention-based intra-scale feature interaction (AIFI) module and a CNN-based cross-scale feature fusion (CCFM) module. AIFI performs feature fusion on port feature maps of different scales through the encoder of the Transformer to obtain multiple first fused features of different scales. CCFM then fuses the multiple first fused features of different scales to obtain a second fused feature.

[0052] In one example, the first fusion feature includes (Dimensions are 80×80, number of channels is set to C, focusing on port detail features) (Dimensions are 40×40, number of channels is C, emphasis is placed on port profile features) (Size is 20×20, number of channels is C, focusing on global port arrangement features).

[0053] Due to the largest size of the first fusion feature To retain the most port details, CCFM upsamples all small-sized first-fusion features to 80×80, ensuring that subsequent features can interact within the same spatial dimension. (40×40) and Upsample (20×20) to 80×80 to obtain (80×80, number of channels C) and (80×80, number of channels C). At this point, the three first fusion features are unified to the same size. After size unification, CCFM is processed through a CNN layer... , , The feature map is concatenated along the channel dimension to obtain an "80×80×3C" feature map. Finally, a 3×3 convolution kernel (with the number set to C) is used to perform a convolution operation on the "80×80×3C" feature map to finally output a second fusion feature of "80×80×C".

[0054] The AIFI implementation in this application can strengthen the correlation of features at the same scale through attention weights, weaken irrelevant interference, and make the first fused features better reflect the feature correlation of ports within a single scale. CCFM can quickly process and fuse multi-scale first fused features, thereby efficiently realizing cross-scale feature integration.

[0055] In some embodiments, before being input to the encoder, the port features need to be adjusted for channel number through 1×1 convolution and flattened into a one-dimensional vector to adapt to Transformer processing. The features are then added to the positional encoding before being input to the Transformer encoder for feature encoding. The feature map processed by the Transformer module needs to be resized back to the size of the two-dimensional feature map before input to the Transformer module to obtain the first fused features for subsequent cross-scale feature fusion.

[0056] S104. The port status of the optical distribution box is determined by the second fusion feature of the port status detection model and the preset weight parameters.

[0057] The preset weight parameters are the parameters learned by the port status detection model during training; the port status of the optical distribution box is the preset port category.

[0058] In some embodiments, the port status of an optical distribution box is determined using a second fusion feature and preset weight parameters through the head network of the port status detection model. Specifically, the head network is a Transformer decoder architecture. Based on the second fusion feature and preset weight parameters, the head network generates several candidate query vectors corresponding to potential port regions in the optical distribution box image. Each candidate query is associated with an image region where a port may exist. An IoU-aware query selection mechanism is used to calculate the intersection-union ratio (IoU) between each candidate query and the real port region, selecting a fixed number of highly matched candidate queries as the initial target queries for the Transformer decoder. The initial target queries are input into the Transformer decoder. Based on the preset weight parameters, the decoder dynamically focuses on key regions related to the port in the second fusion feature through multi-head cross-attention, obtaining an attention output. The attention output is fed into a feedforward neural network, which further optimizes the port features according to the preset weight parameters. The output features processed by the decoder are sent to the category prediction head and the location prediction head. Through preset classification and regression layer weights, the features are converted into the port category, confidence score, and bounding box coordinates corresponding to each target query. Finally, the port status of the optical distribution box is determined based on the port type, confidence level, and bounding box coordinates.

[0059] The candidate query vectors generated in this application are associated with image regions that may contain ports, rather than indiscriminately covering the entire image. This reduces invalid queries in background regions from the source and improves feature processing efficiency. By calculating the intersection-union ratio through IoU-aware query selection, high-matching queries are selected as initial target queries, which can effectively eliminate queries with low overlap with real ports, providing more accurate feature input to the decoder and reducing subsequent processing costs. The decoder dynamically focuses on key port regions in the second fused features through multi-head cross-attention, and then optimizes the features based on preset weights through a feedforward neural network to further enhance port features and suppress noise. Finally, the port status is determined by integrating multi-dimensional information, improving the accuracy and reliability of the detection results.

[0060] In some embodiments, the preset port categories of the optical distribution box may include capped ports, uncapped ports, plug-in ports, and damaged ports; capped ports, such as... Figure 2 The image shows a port with a protective cap and no fiber optic cable inserted; a capless port is shown below. Figure 3 As shown, this is a port without a protective cap and without a fiber optic cable insertion; the plug-in port is as follows. Figure 4 The image shows the port where the fiber optic cable is inserted; a damaged port is shown below. Figure 5 The image shows a port that is physically damaged.

[0061] This application embodiment extracts port features at different scales through multi-scale extraction, eliminating the need for fixed segmentation and avoiding feature loss. It also enhances port features through channel weights and spatial weights, thereby improving detection accuracy. By fusing features at the same scale, details and semantic features of the same scale are preserved. Furthermore, by fusing features at different scales, details and semantic features at different scales can be combined, further improving port detection accuracy. In other words, this application embodiment, by performing multi-scale feature extraction on port detection images and fusing details and semantic features at different scales, ensures both the semantic integrity of port category judgment and the detailed accuracy of location positioning, thus improving the accuracy of optical junction box port status detection.

[0062] In some embodiments, port feature maps of different scales may include port feature maps with a size not less than a preset size threshold.

[0063] In some embodiments, the preset size threshold is a size threshold of the feature map set according to the actual physical size of the optical distribution box port.

[0064] In some embodiments, the preset size threshold can be 160×160.

[0065] In existing RT-DETR models, to extract multi-scale features from shallow to deep layers, the outputs of the last three BasicBlocks of the backbone network are selected as multi-scale features. These three features are used to detect large, medium, and small targets in the image, respectively. Specifically, they correspond to low-resolution, high-semantic features of size 20×20, and high-resolution detail features of sizes 40×40 and 80×80, respectively. However, for the tiny and densely distributed ports in the optical distribution box image, the model needs to acquire higher-resolution detail features to convey more detail and location information to the subsequent neck network. Therefore, this embodiment adds an additional BasicBlock output feature map of size 160×160, which is used as input to the hybrid encoder. Correspondingly, the CBAM attention mechanism is added to the last four BasicBlocks of the backbone network to enhance the model's ability to perceive image details and its generalization ability to different scenes and complex data, thereby improving the accuracy of the model in detecting the connection status of the optical distribution box ports.

[0066] In one example, the backbone network structure diagram for introducing the CBAM attention mechanism is as follows: Figure 6 As shown, it includes an input layer 601, a first convolutional layer 602, a first batch normalization layer 603, a max pooling layer 604, and multiple basic residual blocks 605 that introduce convolutional block attention mechanisms. Each basic residual block 605 that introduces convolutional block attention mechanisms includes an activation function layer 6051, a convolutional block attention mechanism layer 6052, a second batch normalization layer 6053, and a second convolutional layer 6054.

[0067] The input layer 601 receives the original image; the first convolutional layer 602 performs initial convolution on the input image; the first batch normalization layer 603 performs batch normalization on the convolution output, making the distribution of each batch of features more stable, accelerating training convergence and reducing variable bias; the max pooling layer 604 downsamples the feature map, further reducing its size and extracting more abstract features, while reducing the computational cost of subsequent layers; multiple basic residual blocks 605 with convolutional block attention mechanism are stacked sequentially, with each basic residual block outputting feature maps of different scales. The basic residual block 605 with convolutional block attention mechanism at the front outputs a high-resolution feature map containing more detailed information, while the basic residual block 605 with convolutional block attention mechanism at the back processes low-resolution feature maps containing more abstract semantic information. In the basic residual block 605 that introduces the convolutional block attention mechanism, the input features first undergo preliminary feature transformation through the second convolutional layer 6054, the second batch normalization layer 6053, and the activation function layer 6051. Then, they undergo feature enhancement through the subsequent second convolutional layer 6054, the second batch normalization layer 6053, and the convolutional block attention mechanism layer 6052. Finally, the features are merged with the initial input through the residual connection and pass through the activation function layer 6051 to obtain the final output.

[0068] The embodiments of this application, through initial transformation, attention enhancement, and residual merging, retain the advantages of residual networks in terms of ease of training and strong deep expression capabilities, while also specifically strengthening key features.

[0069] This application embodiment, by setting a preset size threshold for feature maps of that size, can avoid the loss of key details such as port edges, colors, and textures due to excessively small feature map sizes, ensuring that the feature extraction process retains port detail information with sufficient resolution.

[0070] In some embodiments, such as Figure 7 As shown, the extracted multi-scale initial feature maps are enhanced according to the image channel weights and image spatial weights to obtain port feature maps of different scales, which may include: S701 to S704.

[0071] S701: For initial feature maps of different scales, calculate image channel weights based on the pixel values ​​of different channels in the initial feature maps.

[0072] Here, the channels of the initial feature map represent the dimensions of the initial feature map; the image channel weights are values ​​used to quantify the importance of each channel.

[0073] In some embodiments, the formula for calculating image channel weights can be:

[0074]

[0075] in, Here, F represents the image channel weights, F is the initial feature map of the input, AvgPool and MaxPool represent global average pooling and max pooling operations, respectively, and MLP represents a multilayer perceptron. This represents the Sigmoid activation function.

[0076] S702, multiply the initial feature map by the image channel weights to obtain the first feature map.

[0077] In some embodiments, the formula for calculating the first feature map can be:

[0078]

[0079] in, Let F be the first feature map, and F be the initial feature map input. This indicates element-wise multiplication. These are the image channel weights.

[0080] S703, the first feature map is stitched together along the channel dimension, and the image spatial weight is calculated based on the pixel values ​​of the stitched first feature map.

[0081] In some embodiments, the formula for calculating image spatial weights can be:

[0082]

[0083] in, Image spatial weights, This is the first feature map. This represents an n×n convolution operation. This means concatenating the results of average pooling and max pooling of the first feature map along the channel dimension.

[0084] S704 multiplies the first feature map with the image spatial weights to obtain port feature maps at different scales.

[0085] In some embodiments, the formula for calculating the port feature map can be:

[0086]

[0087] in, Port feature map, Image spatial weights, This is the first feature map.

[0088] The channel weight calculation in this application embodiment can filter out the most critical feature channels for port detection and suppress background noise channels; the spatial weight calculation can further locate the spatial region where the port is located and weaken irrelevant background, thereby providing higher quality input for subsequent feature fusion and improving the model's ability to recognize ports.

[0089] In some embodiments, the feature fusion network of the port state detection model performs feature fusion on port feature maps of different scales respectively to obtain multiple first fused features of different scales, including:

[0090] Feature fusion is performed only on port feature maps whose size is not less than a preset size threshold to obtain the first fused feature of the port feature map whose size is not less than the preset size threshold.

[0091] Feature maps smaller than a preset threshold typically contain limited port details, making fusion of them not only difficult to improve detection performance but also increasing the computational burden on the model. This application's embodiments limit fusion to port feature maps with a size not smaller than the threshold, which can significantly improve computational efficiency while ensuring detection accuracy.

[0092] In some embodiments, the feature fusion network of the port state detection model performs feature fusion on port feature maps of different scales to obtain multiple first fused features of different scales, which may include:

[0093] Linear transformations are performed on the first fusion features at different scales to obtain query matrices, key matrices, and value matrices at different scales.

[0094] The association weight matrix is ​​calculated based on the query matrix and the key matrix;

[0095] Multiplying the correlation weight matrix with the value matrix yields the first fusion features at multiple different scales.

[0096] In this embodiment, the association weight matrix is ​​calculated using the query matrix and the key matrix to quantify the similarity between features, thereby enabling adaptive acquisition of the association relationship between port features. This allows the model to learn autonomously and strengthen meaningful feature associations, weaken the interference of irrelevant features, and improve the ability of the first fused feature to acquire port features.

[0097] In some embodiments, such as Figure 8 As shown, multiple first fusion features of different scales are fused to obtain second fusion features, which may include: S801 to S803.

[0098] S801, the first fusion feature with the largest scale among the first fusion features of different scales is upsampled with the other first fusion features in descending order of scale to obtain semantic features; wherein, the first fusion feature with the largest scale is the feature with the highest resolution among the first fusion features of multiple different scales; upsampling is to enlarge the low-resolution feature map to a high resolution; the semantic features are features that contain higher-level semantic information, such as features that distinguish ports and backgrounds.

[0099] In some embodiments, this application first determines the largest feature among the first fusion features of different scales, and then fuses the largest feature with other smaller scale features in descending order of scale. During the fusion process, the smaller scale features are upsampled to make their size consistent with the currently processed larger scale features, and then feature fusion is performed to finally obtain semantic features containing rich semantic information. The upsampling process can be interpolation.

[0100] The embodiments of this application can enlarge the size of small-scale features through upsampling operations, thereby achieving effective fusion of features at different scales. The fusion order from large to small is conducive to transferring detailed information from large-scale features to small-scale features. At the same time, semantic information is integrated, and the resulting semantic features contain semantic information from features at different scales, thereby improving the expressive power of the features.

[0101] S802, the smallest first fusion feature among the first fusion features of different scales is downsampled with the remaining first fusion features in order of increasing scale to obtain detail features; where the smallest first fusion feature is the feature with the lowest resolution among multiple first fusion features of different scales; downsampling is the operation of reducing the high-resolution feature map to a low resolution; detail features are features that contain detailed information about objects, such as edges and textures.

[0102] In some embodiments, this application first determines the smallest feature among the first fusion features of different scales, and then fuses the smallest feature with other larger scale features in ascending order of scale. During the fusion process, the larger scale features are downsampled to make their size consistent with the currently processed smaller scale features, and then feature fusion is performed to finally obtain detailed features containing rich detail information. The downsampling process can be pooling.

[0103] The embodiments of this application can reduce the size of large-scale features through downsampling operations, thereby achieving effective fusion of features at different scales. The fusion order from small to large is conducive to transferring semantic information from small-scale features to large-scale features. At the same time, it integrates detailed information, and the resulting detailed features contain detailed information from features at different scales, thus improving the ability to capture port details.

[0104] S803 concatenates semantic features and detail features to obtain the second fused feature.

[0105] Among them, the second fusion feature across scales is a feature obtained by concatenating semantic features and detail features, which contains information at different scales, including both semantic and detail information.

[0106] In some embodiments, concatenating semantic features and detail features may include directly concatenating semantic features and detail features in the channel dimension. For example, if the semantic features are 200×200×64 and the detail features are 200×200×64, the concatenated features will result in a cross-scale second fusion feature of 200×200×128.

[0107] The semantic features in this application retain the high-level semantics of large-scale features, while the detailed features retain the low-level details of small-scale features. The second fused feature, obtained by concatenating the two, not only solves the problem of insufficient detail in large-scale features but also compensates for the semantic ambiguity of small-scale features, thereby improving the comprehensiveness and accuracy of port status determination.

[0108] In some embodiments, the method may further include, before inputting the port detection image into a pre-trained port state detection model:

[0109] Obtain the training dataset for the optical distribution box, including training images of the optical distribution box and corresponding labeled data. The training images are multiple photos taken in complex scenarios with complex backgrounds, varying lighting, fiber optic cable installation offsets, high port occupancy, and obstructions, capturing images of the complete optical distribution box. The labeled data consists of each identifiable port of the optical distribution box in each image labeled using the image labeling tool LabelImg. The labeled data must include the port connection status category and the coordinates of the bounding box where the port is located.

[0110] Preprocessing of the training images of the optical cross-section can include resizing, normalization, and data augmentation.

[0111] The preprocessed training image of the optical distribution box and the corresponding labeled data are input into the initial port state detection model to obtain the prediction data;

[0112] The loss value is calculated using the loss function, predicted data, and labeled data. The model parameters of the initial port state detection model are then optimized based on the loss value until the accuracy of the predicted data reaches a set threshold, thus obtaining a pre-trained port state detection model.

[0113] The training dataset in this application consists of optical distribution box images acquired under complex scenarios such as complex backgrounds, changing lighting, fiber optic cable installation offsets, high port occupancy rates, and occlusions. These images are then trained using a loss function, enabling the model to learn complex features in real-world scenarios and significantly improving its generalization ability in practical applications.

[0114] In some embodiments, resizing the optical cross-section training image may include maintaining the original aspect ratio, adjusting the size of the optical cross-section training image to a preset size, and padding any insufficient parts with zeros.

[0115] Since the images input to the neural network model are often fixed-size images, such as 640×640, while the images of optical distribution boxes are mainly collected by installation and maintenance personnel on-site using their mobile phones, the resolution of these images is not uniform. Therefore, it is necessary to resize the dataset images. In this embodiment, the original aspect ratio is maintained when resizing the images, and any insufficient portion is padded with zeros to avoid distortion of the port targets. This method is also applied before inputting the port detection images into the pre-trained port state detection model.

[0116] In some embodiments, normalizing the optical distribution box training image may include dividing each pixel value of the image by 255 to normalize it to the range [0, 1], and then subtracting the mean and dividing by the standard deviation of each channel of the optical distribution box training image using the global mean and standard deviation of the pixel values ​​of the optical distribution box training images in the optical distribution box training dataset to complete the image normalization process. This method is also applied before inputting the port detection image into the pre-trained port state detection model.

[0117] This application embodiment normalizes the pixel values ​​of the training images of the optical cross-section box to a smaller range, making the grayscale distribution of the image more uniform, reducing the difference in image contrast and brightness, and enabling data to be compared and calculated on the same scale, thereby improving the convergence speed and generalization ability of the model.

[0118] In some embodiments, data augmentation of the training images of optical junction boxes may include randomly cropping local regions of the images to enable the model to learn target features at different locations; randomly rotating the images to improve the robustness of the model to the recognition of optical junction box images taken at different angles; and randomly adjusting the image brightness, contrast, and saturation to enable the model to adapt to optical junction box images taken under different lighting conditions.

[0119] In some embodiments, data augmentation of the optical distribution box training images may further include mosaic data augmentation. Mosaic data augmentation increases the diversity of training data by combining images with different viewpoints, backgrounds, and target sizes, thereby improving the model's ability to detect targets in various complex environments. For example, embodiments of this application may randomly select four different training images, scale them, and stitch the scaled images together into a large image as new input data to enrich the scene background and improve the model's detection capability.

[0120] In some embodiments, the loss function includes classification loss, bounding box regression loss, and auxiliary bounding box loss. The classification loss function is cross-entropy loss, and the formula for calculating the classification loss function value can be:

[0121]

[0122] in, Here, N represents the classification loss function value, and C represents the number of samples and the number of target classification categories. This represents the true label value of the i-th sample belonging to the c-th category. This represents the probability predicted by the model that the i-th sample belongs to the c-th category.

[0123] The bounding box regression loss function is the L1 loss function, and the formula for calculating the bounding box regression loss function value is as follows:

[0124]

[0125] in, The value of the L1 loss function. and represents the coordinates of the predicted bounding box and the ground truth bounding box of the i-th sample, respectively.

[0126] The formula for calculating the auxiliary bounding box loss function value can be:

[0127]

[0128]

[0129] in, To assist in determining the bounding box loss function value, Intersection over Union (IoU) is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, inter is the intersection area of ​​the predicted bounding box and the ground truth bounding box, and union is the union area of ​​the predicted bounding box and the ground truth bounding box.

[0130] The formula for calculating the loss function value can be:

[0131]

[0132] in, The value of the loss function. , and The preset balancing weights.

[0133] Figure 9 This application illustrates an apparatus 900 for detecting the port status of an optical distribution box, which may include:

[0134] The acquisition module 901 is used to acquire port detection images of the optical distribution box;

[0135] The extraction module 902 is used to input the port detection image into the pre-trained port state detection model, and to extract multi-scale features from the port detection image through the feature extraction network of the port state detection model to obtain multi-scale initial feature maps. The extracted multi-scale initial feature maps are then enhanced according to the image channel weights and image spatial weights to obtain port feature maps of different scales.

[0136] The fusion module 903 is used to perform feature fusion on port feature maps of different scales through the feature fusion network of the port state detection model to obtain multiple first fusion features of different scales, and to fuse the multiple first fusion features of different scales to obtain a second fusion feature.

[0137] The determination module 904 is used to determine the port status of the optical distribution box by using the second fusion feature of the port status detection model and preset weight parameters.

[0138] In some embodiments, the port feature maps extracted by the extraction module 902 at different scales include port feature maps with a size not less than a preset size threshold.

[0139] In some embodiments, the device 900 for detecting the port status of the optical distribution box may further include:

[0140] The calculation module is used to calculate the image channel weights based on the pixel values ​​of different channels of the initial feature maps at different scales.

[0141] The calculation module is also used to multiply the initial feature map with the image channel weights to obtain the first feature map;

[0142] The stitching module is used to stitch the first feature map along the channel dimension and calculate the image spatial weights based on the pixel values ​​of the stitched first feature map.

[0143] The calculation module is also used to multiply the first feature map with the image spatial weights to obtain port feature maps at different scales.

[0144] In some embodiments, the feature fusion module 903 is further configured to perform feature fusion only on port feature maps whose size is not less than a preset size threshold, to obtain a first fused feature of the port feature map whose size is not less than the preset size threshold.

[0145] In some embodiments, the device 900 for detecting the port status of the optical distribution box may further include:

[0146] The transformation module is used to perform linear transformations on the first fused features at different scales to obtain query matrices, key matrices, and value matrices at different scales, respectively.

[0147] The calculation module is also used to calculate the association weight matrix based on the query matrix and the key matrix;

[0148] The calculation module is also used to multiply the correlation weight matrix with the value matrix to obtain the first fusion features at multiple different scales.

[0149] In some embodiments, the device 900 for detecting the port status of the optical distribution box may further include:

[0150] The sampling module is used to upsample the largest first fusion feature among the first fusion features of different scales, in descending order of scale, with the remaining first fusion features to obtain semantic features;

[0151] The sampling module is also used to downsample the smallest first fusion feature among the first fusion features of different scales, in order of increasing scale, with the remaining first fusion features to obtain detailed features;

[0152] The splicing module is also used to splice semantic features and detail features to obtain a second fused feature.

[0153] Figure 9 The various modules in the illustrated device can achieve Figure 1 The various steps involved, and the corresponding technical effects achieved, will not be elaborated upon here for the sake of brevity.

[0154] Figure 10 A schematic diagram of the hardware structure of the terminal device provided in an embodiment of this application is shown.

[0155] The terminal device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0156] Specifically, the processor 1001 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0157] Memory 1002 may include mass storage for data or instructions. For example, and not limitingly, memory 1002 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 1002 may include removable or non-removable (or fixed) media, or memory 1002 may be non-volatile solid-state memory. Memory 1002 may be internal or external to the integrated gateway disaster recovery device.

[0158] In one example, memory 1002 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method for detecting the port status of an optical distribution box according to this disclosure.

[0159] The processor 1001 reads and executes computer program instructions stored in the memory 1002 to achieve... Figure 1The method for detecting the port status of the optical distribution box in the illustrated embodiment.

[0160] In one example, the terminal device may further include a communication interface 1003 and a bus 1004. For example, Figure 10 As shown, the processor 1001, memory 1002, and communication interface 1003 are connected through bus 1004 and complete communication with each other.

[0161] The communication interface 1003 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0162] Bus 1004 includes hardware, software, or both, that couples components of an end device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1004 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0163] Furthermore, in conjunction with the optical distribution box port status detection method in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the optical distribution box port status detection methods in the above embodiments.

[0164] This application also provides a computer program product, including a computer program, which, when executed, implements any of the methods for detecting the port status of optical distribution boxes in the above embodiments.

[0165] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0166] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or text segments used to perform the required tasks. Programs or text segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Text segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0167] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0168] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0169] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for detecting the port status of an optical distribution box, characterized in that, The method includes: Acquire port detection images of the optical distribution box; The port detection image is input into a pre-trained port state detection model. The feature extraction network of the port state detection model performs multi-scale feature extraction on the port detection image to obtain multi-scale initial feature maps. The extracted multi-scale initial feature maps are then enhanced according to image channel weights and image spatial weights to obtain port feature maps of different scales. The multi-scale initial feature maps include high-resolution initial feature maps with a size not less than a preset size threshold. The feature extraction network includes multiple basic residual blocks connected in sequence. The basic residual blocks are equipped with a convolutional block attention mechanism based on spatial attention and channel attention. The feature fusion network of the port state detection model performs feature fusion on the port feature maps at different scales to obtain multiple first fusion features at different scales, and then fuses the multiple first fusion features at different scales to obtain a second fusion feature. The port status of the optical distribution box is determined by the second fusion feature and preset weight parameters of the port status detection model. The step of fusing the multiple first fusion features at different scales to obtain the second fusion feature includes: The first fusion feature with the largest scale among the first fusion features of different scales is upsampled with the remaining first fusion features in descending order of scale to obtain semantic features; The smallest first fusion feature among the first fusion features of different scales is downsampled with the remaining first fusion features in order of increasing scale to obtain detailed features; The semantic features and detail features are concatenated to obtain the second fused feature.

2. The method for detecting the port status of an optical distribution box according to claim 1, characterized in that, The port feature maps of different scales include port feature maps with a size not less than a preset size threshold.

3. The method for detecting the port status of an optical distribution box according to claim 1, characterized in that, The step of performing feature enhancement on the extracted multi-scale initial feature maps based on image channel weights and image spatial weights to obtain port feature maps of different scales includes: For initial feature maps of different scales, image channel weights are calculated based on the pixel values ​​of different channels in the initial feature maps. The initial feature map is multiplied by the image channel weights to obtain the first feature map; The first feature map is stitched together along the channel dimension, and the image spatial weight is calculated based on the pixel values ​​of the stitched first feature map. Multiply the first feature map by the image spatial weights to obtain port feature maps of different scales.

4. The method for detecting the port status of an optical distribution box according to claim 2, characterized in that, The feature fusion network of the port state detection model performs feature fusion on the port feature maps at different scales to obtain multiple first fused features at different scales, including: Feature fusion is performed only on port feature maps whose size is not less than a preset size threshold to obtain a first fused feature of the port feature map whose size is not less than the preset size threshold.

5. The method for detecting the port status of an optical distribution box according to claim 1, characterized in that, The feature fusion network of the port state detection model performs feature fusion on the port feature maps at different scales to obtain multiple first fused features at different scales, including: Linear transformations are performed on the first fusion features at different scales to obtain query matrices, key matrices, and value matrices at different scales. The association weight matrix is ​​calculated based on the query matrix and the key matrix. Multiplying the correlation weight matrix with the value matrix yields first fusion features at multiple different scales.

6. A device for detecting the port status of an optical distribution box, characterized in that, The device includes: The acquisition module is used to acquire port detection images of the optical distribution box; An extraction module is used to input the port detection image into a pre-trained port state detection model, and to extract multi-scale features from the port detection image through the feature extraction network of the port state detection model to obtain multi-scale initial feature maps. The extracted multi-scale initial feature maps are then enhanced based on image channel weights and image spatial weights to obtain port feature maps of different scales. The multi-scale initial feature maps include high-resolution initial feature maps with a size not less than a preset size threshold. The feature extraction network includes multiple sequentially connected basic residual blocks, each of which incorporates a convolutional block attention mechanism based on spatial and channel attention. The fusion module is used to perform feature fusion on the port feature maps of different scales through the feature fusion network of the port state detection model to obtain multiple first fusion features of different scales, and to fuse the multiple first fusion features of different scales to obtain a second fusion feature; The determination module is used to determine the port status of the optical distribution box by using the second fusion feature of the port status detection model and preset weight parameters; The step of fusing the multiple first fusion features at different scales to obtain the second fusion feature includes: The first fusion feature with the largest scale among the first fusion features of different scales is upsampled with the remaining first fusion features in descending order of scale to obtain semantic features; The smallest first fusion feature among the first fusion features of different scales is downsampled with the remaining first fusion features in order of increasing scale to obtain detailed features; The semantic features and detail features are concatenated to obtain the second fused feature.

7. A terminal device, characterized in that, The device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the method for detecting the port status of an optical distribution box as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method for detecting the port status of an optical distribution box as described in any one of claims 1-5.

9. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the method for detecting the port status of the optical distribution box as described in any one of claims 1-5.

Images