Port state identification method, device, equipment, medium and product

Abstract

The invention discloses a port state identification method and device, equipment, a medium and a product. The method comprises the following steps: acquiring optical fiber distribution frame (ODF) picture data; the ODF picture data is recognized through a pre-trained final improved Yov5 model, a recognition result is determined, and the final improved Yov5 model comprises an improved input layer, a backbone network layer fused with a self-attention mechanism, a neck feature fusion layer and a prediction layer; the identification result comprises states of all ODF ports in the ODF picture data, and the states comprise an idle state, an occupied state and an unknown state. According to the method, the sensitive capture capability of a multi-head self-attention mechanism on remote space information is integrated into a backbone network layer, the performance of a network on a target detection task is enhanced, then ODF port recognition is carried out through an improved Yolov5 algorithm, and the state of an ODF port is quickly and accurately recognized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical communication technology, and in particular to a port state recognition method, device, equipment, medium and product. BACKGROUND

[0002] The optical distribution frame (ODF) is a node device connecting various optical cable sections in the optical cable network, and it completes the coupling of optical fibers on both sides in an optical manner. It is the core equipment in optical fiber communication. At present, there are some problems in ODF port detection. In actual application, the following problems exist: serious shielding is common; the ODF port itself is small and densely arranged, the target frame overlap frequency is high in the recognition process, and thus the missed detection probability is high; the shooting problem caused by insufficient light in the machine room makes the ODF port detection very difficult; the plug types connected to the port are various, and the colors are not uniform and irregular, which increases the difficulty of feature extraction; due to the non-standard collection operation, the quality of part of the samples is not high, and the detection difficulty is increased.

[0003] At present, the management of ODF port resources mostly adopts the traditional manual on-site verification collection and background system input mode. In addition to manual statistics, a photograph confirmation mode is added, and a cable marking mode through RFID (radio frequency electronic tag) is also used to verify the completion of the work.

[0004] However, due to the long on-site verification collection cycle, frequent changes of port resources, non-standard operation of disassembling machines without disassembling lines, and other problems, the inconsistency between the system and the on-site port resources is very serious, which has serious lag, and the manual verification workload is large and the efficiency is low. The RFID mode has a high marking cost in the early stage and is not suitable for large-scale development. SUMMARY

[0005] The present application provides a port state recognition method, device, equipment, medium and product to realize fast and accurate recognition of the state of ODF port.

[0006] According to a first aspect of the present application, a port state recognition method is provided, comprising:

[0007] Obtaining optical distribution frame (ODF) picture data;

[0008] Recognizing the ODF picture data through a pre-trained final improved Yolov5 model to determine a recognition result, wherein the final improved Yolov5 model comprises an improved input layer, a backbone network layer fused with a self-attention mechanism, a neck feature fusion layer and a prediction layer; and the recognition result comprises the state of all ODF ports in the ODF picture data, and the state comprises an idle state, an occupied state and an unknown state.

[0009] According to a second aspect of the present application, there is provided a port state recognition device, comprising:

[0010] a data acquisition module configured to acquire ODF picture data;

[0011] a state recognition module configured to recognize the ODF picture data by a pre-trained final improved Yolov5 model, determine a recognition result, wherein the final improved Yolov5 model comprises an improved input layer, a backbone network layer fused with a self-attention mechanism, a neck feature fusion layer and a prediction layer; and the recognition result comprises states of all ODF ports in the ODF picture data, and the states comprise an idle state, an occupied state and an unknown state.

[0012] According to a third aspect of the present application, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory connected to the at least one processor in communication; wherein

[0015] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the port state recognition method according to any one of the embodiments of the present application.

[0016] According to a fourth aspect of the present application, there is provided a computer readable storage medium, which stores computer instructions for enabling a processor to implement the port state recognition method according to any one of the embodiments of the present application when executed by the processor.

[0017] According to a fifth aspect of the present application, the embodiments of the present application further provide a computer program product, which comprises a computer program, and the computer program implements the port state recognition method according to any one of the embodiments of the present application when executed by a processor.

[0018] The technical scheme of the embodiment of the application comprises the following steps: obtaining optical distribution frame (ODF) picture data; identifying the ODF picture data by using a pre-trained improved Yolov5 model to determine an identification result, wherein the improved Yolov5 model comprises an improved input layer, a backbone network layer with a fused self-attention mechanism, a neck feature fusion layer and a prediction layer; and the identification result comprises the states of all ODF ports in the ODF picture data, and the states comprise an idle state, an occupied state and an unknown state. The multi-head self-attention mechanism is integrated into the backbone network layer to enhance the performance of the network in a target detection task, and then the improved Yolov5 algorithm is used to identify the ODF ports, so that the states of the ODF ports can be quickly and accurately identified.

[0019] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a flow chart of a port state identification method according to an embodiment of the application;

[0022] Figure 2 is a structural schematic diagram of a port state identification device according to an embodiment of the application;

[0023] Figure 3 is a structural schematic diagram of an electronic device for implementing the embodiment of the application. DETAILED DESCRIPTION

[0024] In order to make the person skilled in the art better understand the application scheme, the following will combine the drawings in the embodiments of the application to clearly and completely describe the technical solutions in the embodiments of the application. Obviously, the described embodiments are only some embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment one

[0027] Figure 1 A flowchart of a port state recognition method is provided for the first embodiment of the present application. The present embodiment can be applied to the determination of the port state of an ODF. The method can be executed by a port state recognition device, which can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in Figure 1 , the method comprises:

[0028] S110, obtaining optical fiber distribution frame (ODF) picture data.

[0029] In the present embodiment, the ODF picture data can be understood as an ODF picture containing all ports.

[0030] Specifically, the processor can obtain ODF picture data, which can be data obtained by regularly inspecting the specific port usage change according to the field inspection task order and timely photographing to complete system update.

[0031] S120, recognizing the ODF picture data through a pre-trained final improved Yolov5 model to determine the recognition result, wherein the final improved Yolov5 model comprises an improved input layer, a backbone network layer fused with a self-attention mechanism, a neck feature fusion layer and a prediction layer.

[0032] In the embodiment, the pre-trained final improved Yolov5 model can be understood as an improved Yolov5 model pre-trained, and the self-attention mechanism is added to the Yolov5 model, and the initial parameters of the clustered anchor box are more suitable for ODF. The improved input layer, the backbone network layer fused with the self-attention mechanism, the neck feature fusion layer and the prediction layer are included. The improved input layer is used for automatically adjusting the picture size of the input data. The backbone network layer fused with the self-attention mechanism is used for feature extraction. The neck feature fusion layer is used for feature fusion. The prediction layer is used for prediction.

[0033] Specifically, the processor can input the ODF picture data as input data into the final improved Yolov5 model. The input data first enters the Input input layer of the final improved Yolov5 model. The layer applies an automatic picture size adjustment method to process the input ODF picture data, increases the diversity of the data set, reduces the CPU load, and improves the accuracy and stability of target detection. The processed picture data flows into the backbone network layer fused with the self-attention mechanism for feature extraction, and the low-level detail features and high-level abstract feature results are obtained to obtain enhanced feature maps. Through the neck feature fusion layer, different levels of feature fusion are performed on the feature maps to obtain fused features. Through the prediction layer, the fused features are processed to obtain the recognition result. The recognition result includes the state of all ODF ports in the ODF picture data. The state can be represented by a number. The state includes an idle state, an occupied state and an unknown state. The unknown state is a state that cannot be determined due to occlusion and the like. For example, the text number "0" in the recognition result represents an idle port, "1" represents an occupied port, and "2" represents an unknown port. The row and column information is represented in the format of "row number_column number", for example, "3_5" represents the port located at the 3rd row and the 5th column.

[0034] The technical scheme of the embodiment of the application acquires the ODF picture data of the fiber distribution frame ODF; identifies the ODF picture data through the pre-trained final improved Yolov5 model to determine the recognition result. The final improved Yolov5 model includes an improved input layer, a backbone network layer fused with a self-attention mechanism, a neck feature fusion layer and a prediction layer. The recognition result includes the state of all ODF ports in the ODF picture data. The state includes an idle state, an occupied state and an unknown state. The sharp capture ability of the multi-head self-attention mechanism for remote spatial information is integrated into the backbone network layer to enhance the performance of the network in the target detection task, and then the ODF port recognition is performed through the improved Yolov5 algorithm to realize fast and accurate recognition of the state of the ODF port.

[0035] Further, on the basis of the above-mentioned embodiment, the training steps of the final improved Yolov5 model can be refined as follows:

[0036] obtain a training data set and an initial improved Yolov5 model, the initial improved Yolov5 model comprising an improved input layer, an initial backbone network layer fused with a self-attention mechanism, an initial neck feature fusion layer, and an initial prediction layer; input the training data set into the initial improved Yolov5 model to obtain a prediction result; determine a loss function according to the prediction result; if the improved loss function does not satisfy a training end condition, adjust model parameters and continue iterative training; otherwise, obtain a final improved Yolov5 model.

[0037] In the embodiment, the training data set can be understood as a data set used for model training, which comprises annotation information (such as target bounding box coordinates, class labels) and original ODF pictures. The prediction result is a data detection result of the model, which comprises a predicted target bounding box, a target confidence, and a class probability (the probability that the target belongs to a certain class). The loss function is a function used to measure the error between the prediction result and the true result. The training end condition is a judgment standard for stopping model training, which can include, for example, that the loss function value converges to a preset threshold or the number of training iterations reaches a maximum. The model parameters are variables that can be learned in the model.

[0038] Specifically, the processor can obtain a training data set and an initial improved Yolov5 model, the initial improved Yolov5 model comprising an improved input layer, an initial backbone network layer fused with a self-attention mechanism, an initial neck feature fusion layer, and an initial prediction layer; input training data in the training data set into the initial improved Yolov5 model to obtain a prediction result. The processor can determine a loss function according to the prediction result and the corresponding true result; if the improved loss function does not satisfy a training end condition, adjust model parameters and continue iterative training; otherwise, obtain a final improved Yolov5 model.

[0039] Further, the step of inputting the training data set into the initial improved Yolov5 model to obtain a prediction result can be refined as follows:

[0040] The input data is obtained by clustering the training data set through the improved input layer; the enhanced features are obtained by processing the input data through the initial backbone network layer; the fusion features are obtained by performing multi-scale fusion on the enhanced features through the initial neck feature fusion layer; and the prediction result is obtained by predicting the fusion features through the initial prediction layer.

[0041] In the embodiment, the input data can be understood as data input into the model for processing. The enhanced features can be understood as features captured from multiple aspects. The fusion features can be understood as the result of multi-scale feature fusion.

[0042] Specifically, the training data set can be expanded and clustered by improving the input layer to obtain anchor box parameters corresponding to the cluster centers, and the expanded data set and the anchor box parameters are jointly used as input data. The input data is processed by the initial backbone network layer to obtain enhanced features; the enhanced features are multi-scale fused by the initial neck feature fusion layer to obtain fused features; and the fused features are predicted by the initial prediction layer to obtain prediction results.

[0043] For example, the initial neck feature fusion layer can use SPPF and PANet structures, which mainly enhance the feature expression ability of the model and further process and compress the feature maps extracted by the backbone layer. Using SPPF and PANet structures, the feature maps output by the backbone layer are further processed and compressed. The output data has stronger feature expression ability and can better capture multi-scale information of the ODF port and reduce feature loss. At the same time, by fusing features of different levels, the adaptability to small targets and complex scenes is improved.

[0044] Among them, the SPPF structure processes the input features through multiple parallel maximum pooling layers, and then splices the results, effectively expanding the receptive field and reducing the computational amount. The PANet structure enhances through bottom-up and top-down paths, fuses features of different levels, and improves the adaptability to small targets and complex scenes. Compared with common feature fusion structures such as FPN (Feature Pyramid Network), SPPF and PANet structures can better capture multi-scale information and reduce feature loss when processing ODF ports and other targets with complex structures and diversified features, thereby improving the performance of the model.

[0045] For example, the initial prediction layer can detect targets of different sizes while avoiding the problem of repeated detection of the same object, improving the accuracy and efficiency of detection, and outputting the prediction results of the model as the results of target detection. The specific process is as follows:

[0046] Target probability: for each target box, the probability of containing a target is calculated by an activation function (such as a sigmoid function). Assuming that the feature vector of the target box is , after a series of linear transformations and activation function processing, the target probability , wherein is a weight matrix, is a bias term, is a sigmoid function. Class probability: for each target box, calculate its probability belonging to each predefined port type class. Suppose there are n classes, and the feature vector of the target box is x, after a series of linear transformations and softmax function processing, the class probability where and are the weight matrix and bias term of the i-th class, respectively.

[0047] According to the anchor box initial parameters in the input data, a plurality of target boxes are predicted for each grid, and each target box contains the position and size information of the target. For each target box, the probability of containing the target is calculated, and if it is greater than a set threshold, it is judged that the target box contains the target. The probability of belonging to each class is calculated, and the class is a predefined port type class. The non-maximum suppression algorithm is performed on all detected target boxes to remove duplicate target boxes, and the final prediction result is obtained.

[0048] wherein the step of obtaining the input data by improving the input layer to cluster the training data set can be refined as:

[0049] The training data set is expanded to obtain an expanded training data set; a training data is randomly selected from the expanded training data set as an initial clustering center; the shortest distance between the initial clustering center and each training data is calculated, and the probability of determining the next clustering center is determined; the point with the highest probability is taken as the new clustering center to obtain a clustering center set; the distance between each training data and each clustering center in the clustering center set is combined with a termination condition to determine the anchor box initial parameters after clustering; and the anchor box initial parameters and the expanded training data set are taken as the input data.

[0050] In the embodiment, the expanded training data set can be understood as a set formed after the training data is expanded. The termination condition is used to stop clustering, such as no (or minimum number) of objects being reassigned to different clusters, no (or minimum number) of clustering centers changing again, or local minimum of error sum of squares. The clustering center can be understood as the representative core point of each group (cluster) after a batch of similar data / characteristics are grouped by a clustering algorithm (such as K-Means). The anchor box initial parameter can be understood as the combination of the width (w) and height (h) pixel values of the rectangular box generated after clustering.

[0051] Specifically, due to the relatively cumbersome and difficult collection of ODF, only the operator maintenance personnel with unlocking permission in the corresponding area can complete the collection during the collection, and the collection difficulty and cost consumption are large. Because the current ODF data set is small in size, when building the model, the input layer is improved to expand the input data, including flipping, rotating, cropping, shifting, adjusting saturation, adjusting exposure, adjusting contrast, adjusting brightness, and adding noise, etc. operations, thereby increasing the number of samples. The collected and positioned port positions are cut and classified to form a training data set of occupied ports, idle ports and unknown ports (obstructed cases). When performing data expansion operations, the specific parameter range and application ratio of each operation can be selected as follows: when performing data expansion operations, the flipping operation is mainly horizontal flipping with a probability of 30%; the rotation angle is randomly selected between -15° and 15°, and the application ratio is 40%; the cropping ratio is set to 20%-50% of the original image, and the use probability is 50%; the shift operation moves in the horizontal and vertical directions by 10% of the image side length, and the application ratio is 30%; the saturation adjustment range is ±20%, and the use probability is 25%; the exposure adjustment range is ±15%, and the use probability is 20%; the contrast adjustment range is ±25%, and the use probability is 30%; the brightness adjustment range is ±10%, and the use probability is 20%; the added noise is Gaussian noise, the noise intensity is randomly between 0-0.05, and the use probability is 15%, to obtain an expanded training data set. The processor can randomly select a training data in the expanded training data set as an initial clustering center; determine the probability of the next clustering center according to the shortest distance between the initial clustering center and each training data; take the point with the highest probability as the new clustering center to obtain a clustering center set; determine the initial parameters of the clustered anchor frame according to the distance between each training data and each clustering center in the clustering center set combined with the termination condition; and take the initial parameters of the anchor frame and the expanded training data set as input data.

[0052] For example, a point is randomly selected in the expanded training data set as an initial clustering center, and the calculation probability of the next clustering center is as follows:

[0053]

[0054] In the above formula, is the probability of the clustering center, and the shortest distance between the clustering center and each sample is , is the probability of the clustering center.

[0055] By continuously selecting the point with the highest probability as the new clustering center, and selecting K clustering centers to obtain a clustering center set. The distance between each training data and each clustering center is calculated, and each training data is assigned to the nearest clustering center. This process will be repeated until the termination condition is met.

[0056] Compared with the traditional mean clustering method, the clustering center of the traditional mean clustering method is randomly selected, while the selection of the clustering center of the application is calculated by the shortest distance formula. This method improves the clustering algorithm for ODF port identification scenarios. According to the characteristics of the ODF port, such as the fixed aspect ratio not affected by any factor, the improved clustering algorithm can obtain the number of anchor boxes with a certain width and height, minimize the width and height difference between the target box and the anchor box, and thus produce faster convergence speed and better positioning effect. For example, when processing a set of ODF picture data, the improved clustering algorithm can find the most suitable anchor box size for the data set, improving the detection accuracy of the port.

[0057] Among them, the initial backbone network layer includes a down-sampling aggregation module Focus module, a cross-stage local network module CSP module, a self-attention mechanism module, a basic convolution module CBL module and a spatial pyramid pooling module SPP module. Correspondingly, the step of processing the input data through the initial backbone network layer to obtain the enhanced features can be refined as:

[0058] The input data is down-sampled through the Focus module to obtain a plurality of first feature maps; the first feature maps are down-sampled through the CBL module to obtain second feature maps; the second feature maps are fused through the CSP module to obtain third feature maps; the third feature maps are weighted and fused through the self-attention mechanism module to obtain global feature maps; and the global feature maps are multi-scale aggregated through the SPP module to obtain enhanced features.

[0059] In this embodiment,

[0060] Specifically, in the initial backbone network Backbone layer, the input image can be operated layer by layer, and the size of the feature map is gradually reduced, and the feature information of the image is extracted, wherein the Backbone layer includes five structures of Focus, CBL, CSP, self-attention mechanism and SPP, which combines low-level detail features and high-level abstract features. The specific processing process is as follows: the pixel value of the input image is extracted according to a certain interval through the Focus module, thereby generating four independent feature layers, which are stacked to convert the width and height dimension information of the image into the channel dimension to obtain the first feature map. The Focus structure output converts the width and height dimension information of the image into the channel dimension to generate four independent feature layers, which have high spatial resolution but relatively few channels, mainly containing low-level detail features of the input image. The image is down-sampled through the CBL module, which is an independent component composed entirely of 3x3 convolution, mainly realizing the down-sampling operation of the image, ensuring that the size of the first feature map is reduced to half of the original size each time it passes through the CBL module, and the second feature map is obtained. The CBL module can significantly reduce the required computing resources when processing data while maintaining its feature recognition function. Assuming that the size of the input image is (width x height x channel number), after one CBL module processing, the size becomes , wherein is the channel number after convolution processing. The CBL module output is down-sampled to half the original size, and the channel number may increase. The output data has low spatial resolution but contains more abstract feature information, which can capture local patterns and textures of the image.

[0061] Specifically, the second feature map is divided into two parts through the CSP module, and fusion is performed through the cross-stage hierarchical structure, while reducing the computational cost, more diversified gradient mixing is obtained, and the third feature map is obtained. The CSP module divides the base feature map into two parts and fuses them, and the output data has more diversified gradient mixing while maintaining a certain spatial resolution, which can better represent the overall structure and features of the image.

[0062] Specifically, the application introduces a self-attention mechanism on the basis of the original Backbone backbone network layer, reduces the dependence of the model on external information by setting Q, K and V, increases the internal relevance of data, and obtains a global feature map. Under the condition of not increasing the total amount of parameters, the application divides the query, key and value three parameters into h groups of independent units. Each group of units independently projects the corresponding parameters into different high-dimensional space subsets and independently performs attention operation within these subspaces to obtain the corresponding weight. This makes the multi-head attention mechanism able to capture and fuse information in multiple levels in parallel, and then obtain more comprehensive results, and the calculation formula is as follows;

[0063]

[0064] In the above formula, Q, K and V represent the query, key and value matrices when calculating the self-attention, is the self-attention matrix, the V matrix is weighted processed, represents the dimension of the key.

[0065] Calculate the adaptive of each space (SDA), as shown in the following formula:

[0066]

[0067] In the above formula, , h represents h groups of independent units.

[0068] Integrate the attention information of each subspace, and the specific calculation process is as follows:

[0069]

[0070] In the above formula, , , , represents the parameter matrix in the linear transformation process, represents the i-th head in the multi-head attention mechanism. represents the splicing operation, represents the final output result, that is, the global feature map.

[0071] Specifically, the SPP module combines global feature maps of various sizes according to the max-pooling technique to obtain enhanced features, thereby realizing integration of multi-scale features. The spatial pyramid network structure adopted by the SPP module matches the spatial information, and different pooling operations are performed on multiple sizes to obtain standard feature vector outputs, thereby ensuring that images of any scale can be directly input. This method effectively reduces the characteristics of the feature map and avoids the phenomenon that the traditional pooling operation easily loses important information and easily overfits. Through multi-scale feature integration, the SPP module outputs data with different scale feature information, which can adapt to target detection tasks of different sizes. Meanwhile, the characteristics of the feature map are effectively reduced, and the phenomenon that the traditional pooling operation easily loses information and easily overfits is avoided.

[0072] The self-attention mechanism cooperates with other structures of the Backbone layer to further capture remote spatial information on the data processed by the Focus, CBL, CSP and SPP, and enhance the performance of the network in the target detection task. For example, applying the self-attention mechanism to the data output by the CSP module can better fuse the features of different parts and improve the extraction capability of the ODF port features. The present application uses the sharp capture capability of the multi-head self-attention mechanism for remote spatial information and integrates it into the Backbone network architecture, which can enhance the performance of the network in the target detection task.

[0073] Further, on the basis of the above-mentioned embodiments, the step of determining the loss function according to the prediction result can be refined as:

[0074] According to the prediction box in the prediction result and the real box in the training data set, the intersection over union is determined; according to the prediction box and the real box, the minimum bounding rectangle area and the maximum bounding rectangle area are determined; and according to the intersection over union, the minimum bounding rectangle area and the maximum bounding rectangle area, the loss function is determined.

[0075] In the present embodiment, the prediction box can be understood as the target bounding box output by the prediction layer. The real box can be understood as the target bounding box manually labeled in the training data set, which is the real label of the "actual position and size of the target". The intersection over union is used to quantify the degree of overlap between the prediction box and the real box. The minimum bounding rectangle area is used to represent the area of the smallest rectangle that can contain both the prediction box and the real box. The maximum bounding rectangle area is used to represent the maximum rectangle area among the prediction box area and the real box area.

[0076] Specifically, according to the prediction box in the prediction result and the real box in the training data set, the intersection over union is determined; according to the prediction box and the real box, the minimum bounding rectangle area and the maximum bounding rectangle area are determined; and according to the intersection over union, the minimum bounding rectangle area and the maximum bounding rectangle area, the loss function is determined.

[0077] Wherein, the loss function is a key component of the Yolov5 model, and the loss function in the application is as follows:

[0078]

[0079] In the above formula, is the intersection over union, represents the minimum circumscribed rectangle area, and U is the maximum circumscribed rectangle area.

[0080] The technical scheme of the embodiment of the application expands the training data set to obtain an expanded training data set, and then optimizes the selection method of the initial clustering center through an optimized clustering method, which can ensure that the initial clustering centers are far away from each other, greatly improves the stability of the clustering result, and significantly improves the final error of the classification result, thereby obtaining input data containing anchor box initial parameters and the expanded training data set. The sharp capture ability of long-range spatial information of the multi-head self-attention mechanism is integrated into the backbone network architecture, which can enhance the performance of the network in the target detection task, and then train the initial improved Yolov5 model to obtain the final improved Yolov5 model. The ODF port recognition model based on the improved Yolov5 algorithm has the characteristics of fast speed and small size, and can be deployed and applied more quickly under the premise of ensuring a certain accuracy, thereby improving the overall performance and efficiency. The ODF port state can be remotely and automatically detected, and the ODF port positioning and occupancy state can be quickly and accurately collected, realizing the transition from traditional manual detection to machine detection, liberating labor and greatly improving the port inspection efficiency.

[0081] Embodiment three

[0082] Figure 2 A structural schematic diagram of a port state recognition device provided by the embodiment three of the application is shown in FIG. 1. Figure 2 As shown in the figure, the device comprises:

[0083] A data acquisition module 21 is configured to acquire ODF picture data.

[0084] A state recognition module 22 is configured to recognize the ODF picture data through a pre-trained final improved Yolov5 model to determine a recognition result, wherein the final improved Yolov5 model comprises an improved input layer, a backbone network layer fused with a self-attention mechanism, a neck feature fusion layer, and a prediction layer; the recognition result comprises the states of all ODF ports in the ODF picture data, and the states include an idle state, an occupied state, and an unknown state.

[0085] The technical scheme of the embodiment of the application comprises the following steps: obtaining optical distribution frame (ODF) picture data; identifying the ODF picture data by using a pre-trained final improved Yolov5 model to determine an identification result, wherein the final improved Yolov5 model comprises an improved input layer, a backbone network layer with a fused self-attention mechanism, a neck feature fusion layer and a prediction layer; and the identification result comprises the states of all ODF ports in the ODF picture data, and the states include an idle state, an occupied state and an unknown state. The multi-head self-attention mechanism is integrated into the backbone network layer to enhance the performance of the network in the target detection task, and then the improved Yolov5 algorithm is used to identify the ODF ports, so that the states of the ODF ports can be quickly and accurately identified.

[0086] Further, the apparatus further comprises a model training module.

[0087] The model training module comprises:

[0088] a first determining unit configured to obtain a training data set and an initial improved Yolov5 model, wherein the initial improved Yolov5 model comprises an improved input layer, an initial backbone network layer with a fused self-attention mechanism, an initial neck feature fusion layer and an initial prediction layer;

[0089] a second determining unit configured to input the training data set into the initial improved Yolov5 model to obtain a prediction result;

[0090] a third determining unit configured to determine a loss function according to the prediction result;

[0091] a fourth determining unit configured to, if the improved loss function does not satisfy a training end condition, adjust model parameters and continue iterative training;

[0092] a fifth determining unit configured to, otherwise, obtain a final improved Yolov5 model.

[0093] The second determining unit comprises:

[0094] a first determining subunit configured to cluster the training data set by using the improved input layer to obtain input data;

[0095] a second determining subunit configured to process the input data by using the initial backbone network layer to obtain enhanced features;

[0096] a third determining subunit configured to perform multi-scale fusion on the enhanced features by using the initial neck feature fusion layer to obtain fused features;

[0097] a fourth determining subunit configured to perform prediction on the fused features by using the initial prediction layer to obtain a prediction result.

[0098] The first determining subunit is specifically configured to:

[0099] The training data set is expanded to obtain an expanded training data set;

[0100] A training data is randomly selected from the expanded training data set as an initial clustering center;

[0101] According to the shortest distance between the initial clustering center and each training data, the probability of the next clustering center is determined;

[0102] The point with the highest probability is taken as a new clustering center to obtain a clustering center set;

[0103] According to the distance between each training data and each clustering center in the clustering center set and a termination condition, an initial parameter of a clustered anchor frame is determined;

[0104] The initial parameter of the anchor frame and the expanded training data set are taken as input data.

[0105] The initial backbone network layer includes a down-sampling aggregation module Focus module, a cross-stage local network module CSP module, a self-attention mechanism module, a basic convolution module CBL module and a spatial pyramid pooling module SPP module, and correspondingly, the second determining subunit is specifically configured to:

[0106] The input data is down-sampled through the Focus module to obtain a plurality of first feature maps;

[0107] The first feature maps are down-sampled through the CBL module to obtain second feature maps;

[0108] The second feature maps are fused through the CSP module to obtain third feature maps;

[0109] The third feature maps are weighted and fused through the self-attention mechanism module to obtain global feature maps;

[0110] The global feature maps are multi-scale aggregated through the SPP module to obtain enhanced features.

[0111] The third determining unit is specifically configured to:

[0112] According to the predicted frame in the prediction result and the real frame in the training data set, an intersection over union is determined;

[0113] According to the predicted frame and the real frame, a minimum circumscribed rectangle area and a maximum circumscribed rectangle area are determined;

[0114] According to the intersection ratio, the minimum circumscribed rectangle area, and the maximum circumscribed rectangle area, a loss function is determined.

[0115] The port state recognition device provided by the embodiments of the present application can execute the port state recognition method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0116] Embodiment three

[0117] Figure 3 A structural schematic diagram of an electronic device 40 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0118] As shown in Figure 3 The electronic device 40 includes at least one processor 41, and a memory, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., which is communicatively connected to the at least one processor 41, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 into the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0119] A plurality of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, a speaker, etc.; a storage unit 48, such as a magnetic disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0120] The processor 41 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The processor 41 performs various methods and processes described above, such as the port state identification method.

[0121] In some embodiments, the port state identification method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded onto the RAM 43 and executed by the processor 41, one or more steps of the port state identification method described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to perform the port state identification method by any other suitable means, such as by means of firmware.

[0122] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0123] Computer programs used to implement the methods of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a machine or a remote machine or a server.

[0124] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0126] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0127] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. Servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are a host product in the cloud computing service system to solve the defects of large management difficulty and weak business scalability in traditional physical hosts and VPS services.

[0128] In an embodiment, the present embodiment further includes a computer program product comprising a computer program which, when executed by a processor, implements the port state identification method of any of the embodiments of the present application.

[0129] In implementing the computer program product, the computer program code for carrying out operations of the present application can be written in one or more programming languages or combinations of languages including object oriented programming languages such as Java, Smalltalk, C++ and conventional procedural programming languages such as "C" or the like. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0130] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the spirit and scope of the present application. For example, the steps recited in the present application can be performed in parallel, in series, or in a different order, and the present application is not limited in this regard.

[0131] The specific embodiments described above are not intended to limit the scope of the present application. Those skilled in the art will understand that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present application. Any further modifications, equivalents or alternatives within the spirit and principles of the present application are to be considered as falling within the scope of the present application.

Claims

1. A port status identification method, characterized in that, include: Obtain ODF image data of fiber optic patch panel; The ODF image data is identified by a pre-trained, improved Yolov5 model to determine the identification result. The improved Yolov5 model includes an improved input layer, a backbone network layer with a self-attention mechanism, a neck feature fusion layer, and a prediction layer. The identification result includes the state of all ODF ports in the ODF image data, including idle state, occupied state, and unknown state.

2. The method according to claim 1, characterized in that, The training steps for the final improved Yolov5 model include: Obtain the training dataset and the initial improved Yolov5 model, which includes an improved input layer, an initial backbone network layer with self-attention mechanism, an initial neck feature fusion layer, and an initial prediction layer. The training dataset is input into the initial improved Yolov5 model to obtain the prediction results; Based on the prediction results, determine the loss function; If the improved loss function does not meet the training termination condition, the model parameters are adjusted and iterative training continues. Otherwise, the final improved Yolov5 model is obtained.

3. The method according to claim 2, characterized in that, The step of inputting the training dataset into the initial improved Yolov5 model to obtain prediction results includes: The improved input layer is used to cluster the training dataset to obtain the input data; The input data is processed through the initial backbone network layer to obtain enhanced features; The enhanced features are fused at multiple scales through the initial neck feature fusion layer to obtain fused features; The fused features are predicted using the initial prediction layer to obtain the prediction result.

4. The method according to claim 3, characterized in that, The step of clustering the training dataset through the improved input layer to obtain input data includes: The training dataset is expanded to obtain an expanded training dataset; Randomly select a training data point from the expanded training dataset as the initial cluster center; The probability of the next cluster center is determined based on the shortest distance between the initial cluster center and each of the training data. The point with the highest probability is taken as the new cluster center, and the cluster center set is obtained; Based on the distance between each training data point and each cluster center in the cluster center set, combined with the termination condition, the initial parameters of the anchor box after clustering are determined; The initial parameters of the anchor frame and the expanded training dataset are used as input data.

5. The method according to claim 3, characterized in that, The initial backbone network layer includes a downsampling aggregation module (Focus module), a cross-stage local network module (CSP module), a self-attention mechanism module, a basic convolutional module (CBL module), and a spatial pyramid pooling module (SPP module). Correspondingly, the processing of the input data through the initial backbone network layer to obtain enhanced features includes: The input data is downsampled using the Focus module to obtain multiple first feature maps; The first feature map is downsampled using the CBL module to obtain the second feature map; The second feature map is fused using the CSP module to obtain the third feature map; The third feature map is weighted and fused using the self-attention mechanism module to obtain a global feature map. The global feature map is aggregated at multiple scales using the SPP module to obtain enhanced features.

6. The method according to claim 2, characterized in that, The step of determining the loss function based on the prediction result includes: The intersection-union ratio (IUU) is determined based on the predicted bounding boxes in the prediction results and the ground truth bounding boxes in the training dataset. Based on the predicted bounding box and the true bounding box, determine the area of ​​the minimum and maximum bounding rectangles; The loss function is determined based on the intersection-union ratio, the area of ​​the minimum bounding rectangle, and the area of ​​the maximum bounding rectangle.

7. A port status identification device, characterized in that, include: The data acquisition module is used to acquire ODF image data of fiber optic distribution frames; The state recognition module is used to identify the ODF image data using a pre-trained final improved Yolov5 model and determine the recognition result. The final improved Yolov5 model includes an improved input layer, a backbone network layer with a self-attention mechanism, a neck feature fusion layer, and a prediction layer. The recognition result includes the state of all ODF ports in the ODF image data, including idle state, occupied state, and unknown state.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the port status identification method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the port status identification method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the port status identification method according to any one of claims 1-6.

Images