Port identification method and device, equipment, medium and product

Abstract

The invention provides a port identification method and device, equipment, a medium and a product, and the method comprises the steps: obtaining a port video of an optical cable cross connecting cabinet; based on the port video, determining a target frame image by using a long-short term memory network; performing positioning segmentation processing on the target frame image, and determining a port segmentation image; based on the port segmentation image, utilizing a pre-training model to determine a fusion feature vector of the port; and determining an identification result of the port based on the fusion feature vector. According to the method, the target frame image is extracted from the port video, the problem that the port turntable cannot be accurately recognized due to the fact that the picture shot by the inspection personnel has problems can be avoided, the shooting operation difficulty of the inspection personnel is reduced, meanwhile, the feature fusion of the port in the target frame image is realized by determining the fusion feature vector, and the accuracy of port feature fusion is improved. And the port identification accuracy is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to a port identification method, device, equipment, medium and product. Background Art

[0002] With the rapid development of computers in recent years, digital image processing technology is developing towards high speed, high resolution, three-dimensional, intelligent and standardized, and has begun to be applied to all walks of life. Among them, optical cross-box port image recognition technology has emerged. However, due to the messy wiring inside the optical cross-box, it is easy to cause cable occlusion problems on the ports, so the inspection personnel need to shoot multiple times to take images of all the optical cross-box ports for identification, which greatly increases the complexity of the work. In addition, the inspection personnel's shooting skills are easy to reduce the recognition accuracy. Summary of the invention

[0003] The present disclosure provides a port identification method, device, equipment, medium and product to solve the problems in related technologies, reduce the operation difficulty of inspection personnel, and improve the accuracy of port identification.

[0004] The first aspect of the embodiment of the present disclosure proposes a port recognition method, which includes: obtaining a port video of an optical cross-box; based on the port video, using a long short-term memory network to determine a target frame image; performing positioning and segmentation processing on the target frame image to determine a port segmentation image; based on the port segmentation image, using a pre-trained model, determining a fused feature vector of the port; and determining a recognition result of the port based on the fused feature vector.

[0005] In some embodiments of the present disclosure, determining a target frame image based on a port video using a long short-term memory network includes: extracting timing information and global information of the port video using a long short-term memory network; and determining the target frame image based on the timing information and the global information.

[0006] In some embodiments of the present disclosure, based on the target frame image, using an image segmentation algorithm, determining a port segmentation image includes: preprocessing the target frame image to determine a preprocessed target frame image; using a Hough circle detection algorithm to determine the coordinates of a first port in the preprocessed target frame image; based on the coordinates of the first port, using a chessboard segmentation algorithm to determine the coordinates of a second port, the first port being different from the second port; and determining a port segmentation image based on the coordinates of the first port and the coordinates of the second port.

[0007] In some embodiments of the present disclosure, based on the coordinates of the first port, the coordinates of the second port are determined using a chessboard segmentation algorithm, including: based on the coordinates of the first port in the target row, determining the minimum difference between the coordinates of adjacent first ports in the target row; based on the horizontal coordinate relationship between the first first port of the target row and the first first port of the non-target row, determining the coordinates of the first port of the target row; based on the coordinates of the first port in the target row, the coordinates of the first port, and the minimum difference, determining the number of second ports; based on the coordinates of the first port in the target row, the coordinates of the first port, and the minimum difference, determining the coordinates of the second port based on the coordinates of the first port of the target row, the minimum difference, and the number of second ports.

[0008] In some embodiments of the present disclosure, based on the port segmentation image, using a pre-trained model, determining the fused feature vector of the port includes: based on the port segmentation image, using the first output convolution layer and the second output convolution layer of the pre-trained model to respectively determine the first position feature image and the second position feature image; based on the first position feature image and the second position feature image, using a principal component analysis algorithm to determine the first feature vector; based on the port segmentation image, determining the second feature vector through the first fully connected layer of the pre-trained model; based on the first feature vector and the second feature vector, determining the fused feature vector.

[0009] In some embodiments of the present disclosure, the method also includes: preprocessing the training image set to obtain a preprocessed image set; based on the preprocessed image set, using the model to be trained, determining a training fusion vector; based on the training fusion vector, using a support vector machine algorithm, training the model to be trained to obtain a pretrained model.

[0010] The second aspect of the present disclosure proposes a port identification device, which includes: an acquisition unit, used to acquire a port video of an optical cross-box; a first determination unit, used to determine a target frame image based on the port video using a long short-term memory network; a segmentation unit, used to perform positioning and segmentation processing on the target frame image to determine a port segmentation image; a second determination unit, used to determine a fused feature vector of the port based on the port segmentation image using a pre-trained model; and an identification unit, used to determine an identification result of the port based on the fused feature vector.

[0011] The third aspect embodiment of the present disclosure proposes an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when used to run the computer program, executes the method described in the first aspect embodiment of the present disclosure.

[0012] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0013] The fifth aspect embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method described in the first aspect embodiment of the present disclosure when executed by a processor.

[0014] In summary, according to a port identification method proposed in the present disclosure, the method includes: obtaining a port video of an optical cross-connection box; based on the port video, using a long short-term memory network to determine a target frame image; performing positioning and segmentation processing on the target frame image to determine a port segmentation image; based on the port segmentation image, using a pre-trained model, determining a fusion feature vector of the port; based on the fusion feature vector, determining the recognition result of the port. The method disclosed in the present disclosure can avoid problems in the pictures taken by patrol personnel and fail to accurately identify the port turntable by extracting the target frame image from the port video, thereby reducing the difficulty of the patrol personnel's shooting operation. At the same time, by determining the fusion feature vector, the feature fusion of the port in the target frame image is realized, thereby improving the accuracy of port identification.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0017] Figure 1 A flow chart of a port identification method provided by an embodiment of the present disclosure;

[0018] Figure 2 A flowchart of another port identification method provided by an embodiment of the present disclosure;

[0019] Figure 3 An example diagram of a brightness adjustment curve provided by an embodiment of the present disclosure;

[0020] Figure 4 A flow chart of another port identification method provided by an embodiment of the present disclosure;

[0021] Figure 5 An example diagram of a model structure provided in an embodiment of the present disclosure;

[0022] Figure 6 A flowchart of a port identification method provided by an embodiment of the present disclosure;

[0023] Figure 7 A flowchart diagram of another port identification method provided by an embodiment of the present disclosure;

[0024] Figure 8An example diagram of a flow chart of another port identification method provided by an embodiment of the present disclosure;

[0025] Fig. 9 A flowchart diagram of another port identification method provided by an embodiment of the present disclosure;

[0026] Fig.10 An example diagram of another model structure provided in an embodiment of the present disclosure;

[0027] Fig.11 A schematic diagram of the structure of a port identification device provided in an embodiment of the present disclosure;

[0028] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0030] With the rapid development of computers in recent years, digital image processing technology is developing towards high speed, high resolution, three-dimensional, intelligent and standardized, and has begun to be applied to all walks of life. Among them, optical cross-box port image recognition technology has emerged. However, due to the messy wiring inside the optical cross-box, it is easy to cause cable occlusion problems on the ports, so the inspection personnel need to shoot multiple times to take images of all the optical cross-box ports for identification, which greatly increases the complexity of the work. In addition, the inspection personnel's shooting skills are easy to reduce the recognition accuracy.

[0031] In the related art, the following two methods are usually used to identify the ports of the optical cross-connect box:

[0032] 1. The input optical intersection box image is subjected to feature extraction through the first convolutional neural network to obtain a feature map, and then the feature map is used to obtain an anchor point frame feature map in units of anchor points through the second convolutional neural network. Finally, the anchor point frame feature map is subjected to classification and recognition of optical intersection box components through the third convolutional neural network to obtain the detection result of the target optical intersection box.

[0033] 2. After preprocessing the image containing the optical cross-box port, the port part and the character part in the image are separated respectively to obtain multiple port images and multiple character images; then, each port image and character image is recognized to obtain corresponding recognition results.

[0034] Therefore, the related solutions have the following defects:

[0035] 1. Only a single image is input, and the amount of information obtained by the optical intersection port is limited. In addition, the model directly processes the entire image, which may reduce the recognition accuracy due to inaccurate positioning of the anchor box.

[0036] 2. Only a single image is input, and the amount of information obtained about the optical cross-box port is limited. For the optical cross-box port that cannot be identified, the user needs to manually add multiple optical cross-box port images for identification, which leads to the problem of low port identification efficiency.

[0037] The present disclosure is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] Figure 1 A flow chart of a port identification method provided by an embodiment of the present disclosure. Figure 1 As shown, steps 101-105 are included.

[0039] Step 101, obtaining the port video of the optical cross-connect box.

[0040] In some embodiments, the inspection personnel can use handheld video equipment to record the ports of the optical cross-connection box to obtain the port video of the optical cross-connection box; or the video equipment can be installed in the optical cross-connection box to obtain the port video of the optical cross-connection box through the video equipment. The present disclosure does not limit the method of obtaining the port video of the optical cross-connection box.

[0041] Step 102, based on the port video, using the long short-term memory network, determine the target frame image.

[0042] In some embodiments, by using a long short-term memory (LSTM) algorithm, a frame image with a relatively large amount of information is extracted from the port video as a target frame image, thereby achieving determination of the target frame image.

[0043] In an optional embodiment, a long short-term memory network can be used to extract the timing information and global information of the port video, and then the frame image in the video whose timing information and global information are greater than a preset value is determined as the target frame image.

[0044] Step 103, performing positioning and segmentation processing on the target frame image to determine the port segmentation image.

[0045] In some embodiments, different segmentation algorithms are used to segment and locate the ports in the target frame image, thereby obtaining a port segmentation image with port coordinate information. This method uses a composite positioning and segmentation processing method to determine the port coordinate information, thereby improving the accuracy of port positioning.

[0046] In some embodiments, the port segmentation image is a target frame image having all port coordinate information.

[0047] In an optional embodiment, the Hough circle detection algorithm can be first used to determine the coordinate information of the ports with prominent colors (such as the red hat ports) in the target frame image; then based on the coordinate information of the ports determined above, the chessboard segmentation algorithm can be used to determine the coordinate information of the remaining ports in the target frame image, thereby obtaining the coordinate information of all ports in the target frame image to achieve the determination of the port segmentation image.

[0048] Step 104 , based on the port segmentation image, using the pre-trained model, determine the fusion feature vector of the port.

[0049] In some embodiments, based on the port segmentation image, at least two feature vectors of each port in the port segmentation image can be extracted using a pre-trained model, and then the at least two feature vectors are merged to determine a fused feature vector of each port.

[0050] In an optional embodiment, the pre-trained model may be a neural network model including 13 convolutional layers, but is not limited thereto and the present disclosure is not limited thereto.

[0051] In an optional embodiment, the port segmentation image can be input into a pre-trained model, and the first position feature image and the second position feature image can be determined respectively through the first output convolution layer and the second output convolution layer of the pre-trained model; then the first position feature image and the second position feature image are subjected to dimensionality reduction processing using a principal component analysis algorithm to determine a first feature vector; and the second feature vectors of the first position feature image and the second position feature image are determined through the first fully connected layer of the pre-trained model; finally, the first feature vector and the second feature vector are fused to determine a fused feature vector.

[0052] Step 105: Determine the recognition result of the port based on the fused feature vector.

[0053] In some embodiments, the matching degree between each port and a preset port state is determined by determining the fused feature vector, thereby determining the identification result of each port.

[0054] In an optional embodiment, the identification result of the port may be at least one of the following: red hat, white hat, plugged in, damaged, and blocked.

[0055] In summary, according to the port identification method proposed in the present disclosure, the method includes: obtaining a port video of an optical cross-box; based on the port video, using a long short-term memory network to determine a target frame image; performing positioning and segmentation processing on the target frame image to determine a port segmentation image; based on the port segmentation image, using a pre-trained model, determining a fusion feature vector of the port; based on the fusion feature vector, determining the recognition result of the port. The method disclosed in the present disclosure can avoid problems in the pictures taken by patrol personnel and fail to accurately identify the port turntable by extracting the target frame image from the port video, thereby reducing the difficulty of the patrol personnel's shooting operation. At the same time, by determining the fusion feature vector, the feature fusion of the port in the target frame image is realized, thereby improving the accuracy of port identification.

[0056] Figure 2 A flow chart of a port identification method proposed in the present disclosure is further shown. Figure 1 The embodiment shown, Figure 2 The following steps are included.

[0057] Step 201, obtaining the port video of the optical cross-connect box.

[0058] In some embodiments, the principle of step 201 is the same as that of the embodiment shown in step 101, and reference may be made to the above related description, which will not be repeated here.

[0059] Step 202, using a long short-term memory network, extracting the timing information and global information of the port video.

[0060] In some embodiments, since the frames of the port video are continuous in time, the timing information of each frame image in the port video can be extracted through a recurrent neural network model including a long short-term memory network.

[0061] In some embodiments, the long short-term memory network can extract global information of each frame image in the video according to the scene transition, global structure, etc. of the port video.

[0062] Step 203: determine the target frame image based on the timing information and the global information.

[0063] In some embodiments, the target target frame image can be determined by a preset threshold. For example, when the timing information and global information of a frame image are greater than a preset threshold, the frame image is determined as the target frame image. A frame image containing specific timing information and global information can also be determined as the target frame image. The present disclosure does not limit the method for determining the target frame image.

[0064] Step 204: pre-process the target frame image to determine a pre-processed target frame image.

[0065] In some embodiments, the preprocessing may be processing the brightness of the target frame image to solve the problem of uneven exposure in the target frame image.

[0066] For example, the target frame image can be dynamically compressed by adjusting the gamma curve of the target frame image, and low pixels (underexposed parts) can be brightened and high-brightness pixels (overexposed parts) can be darkened through processing, and then the problem of uneven image exposure can be handled by fitting the brightness information of the image and the contour details of the image to determine the preprocessed target frame image.

[0067] In an optional embodiment, since the exposure conditions of different target frame images are usually different, different brightness adjustment schemes need to be adopted for different target frame images, that is, the target frame image is divided into several different brightness levels according to the brightness channel of the brightness-chrominance (YUV) color space of the target frame image, and then the target frame image is divided into multiple sub-regions and their brightness values ​​are optimized at the same time, and the brightness level change of the target frame image is statistically fitted into the brightness adjustment curve of the target frame image using the optimized information of these regions, and finally a pre-processed target frame image with a more visual effect is obtained. Among them, the brightness adjustment curve is, for example, Figure 3 As shown, the horizontal axis of the S curve is the brightness value of the input image, and the vertical axis is the brightness value of the output image.

[0068] Step 205 , using the Hough circle detection algorithm, determine the coordinates of the first port in the preprocessed target frame image.

[0069] In some embodiments, since the internal wiring of the optical cross-connect box is distributed in a disorderly manner and in large numbers, it is difficult to completely locate and identify the position and status of each port while ensuring good generalization if machine learning and deep learning methods are used directly. Therefore, this method first uses the Hough circle detection (Hough) algorithm to locate the first port (such as the red hat port) that is easy to identify in the pre-processed target frame image, so that the coordinates of the remaining ports can be determined based on the coordinates of the first port, which can improve the accuracy of the port coordinate determination.

[0070] Step 206: Based on the coordinates of the first port, the coordinates of the second port are determined using a chessboard segmentation algorithm.

[0071] In some embodiments, since the number of first ports in the optical cross-link box usually accounts for the majority of the total number of ports, the coordinates of the first ports can be used to determine the coordinate information of other ports (i.e., second ports) in the pre-processed target frame image through a chessboard segmentation algorithm.

[0072] It should be understood that the second port refers to other ports in the preprocessed target frame image except the first port.

[0073] Among them, the chessboard segmentation algorithm can be Figure 4 As shown, the following steps are included:

[0074] Step 301: Based on the coordinates of the first ports in the target row, determine the minimum difference between the coordinates of adjacent first ports in the target row.

[0075] The target row may be any port row in the preprocessed target frame image.

[0076] In particular, when there is only one first port in the target row, this step can be skipped.

[0077] In this step, the minimum difference between the coordinates of adjacent first ports in the target row can be determined through the coordinates of the first ports determined in step 206, so that the minimum difference is used as the distance between two adjacent ports in the target row.

[0078] Step 302 : determining the coordinates of the first port of the target row based on the horizontal coordinate relationship between the first first port of the target row and the first first port of the non-target row.

[0079] In this step, whether the first port of the target row is the first port can be determined by comparing the horizontal coordinate relationship (indentation, same column, protrusion, etc.) between the first first port of the target row and the first first port of the non-target row.

[0080] If it is determined that a certain first port in the target row is the first port of the target row, the coordinates of the first port are determined as the coordinates of the first port.

[0081] If it is determined that the first ports in the target row are not the first port, the coordinates of the first port in the target row can be determined by using a linear difference method based on the coordinates of the first ports in the adjacent row (ie, non-target row) of the target row.

[0082] Step 303, determining the number of second ports based on the coordinates of the first port in the target row, the coordinates of the first port, and the minimum difference.

[0083] Among them, the determined minimum difference can be determined as the distance between two adjacent ports in the optical cross-box; then, by subtracting the coordinates of the first port in the target row from the coordinates of the first port, the obtained difference is divided by the above minimum difference, so as to determine the number of second ports (other ports except the first port).

[0084] Step 304 : determining the coordinates of the second port based on the coordinates of the first port of the target row, the minimum difference, and the number of the second ports.

[0085] Step 207: Determine a port segmentation image based on the coordinates of the first port and the coordinates of the second port.

[0086] In some embodiments, the determined coordinates of the first port and the determined coordinates of the second port may be annotated into the pre-processed target frame image, thereby determining a port segmentation image.

[0087] In other words, the port interface segmentation image may be a pre-processed target frame image with port coordinate information.

[0088] Step 208, based on the port segmentation image, using the first output convolution layer and the second output convolution layer of the pre-trained model, respectively determine the first position feature image and the second position feature image.

[0089] In an optional embodiment, the neural network of the pre-trained model is as follows: Figure 5 As shown, the pre-trained model has a total of 13 convolutional layers. The port segmentation image is input into the pre-trained model, and the first position feature image and the second position feature image are output respectively through the positions of the first output convolutional layer (i.e., the second convolutional layer) and the second output convolutional layer (i.e., the fifth convolutional layer) of the pre-trained model, thereby realizing the determination of the first position feature image and the second position feature image.

[0090] Step 209 : Determine a first feature vector using a principal component analysis algorithm based on the first position feature image and the second position feature image.

[0091] In some embodiments, since the dimensions of the first position feature image and the second position feature image are relatively high and not conducive to calculation, the first position feature image and the second position feature image may be reduced in dimension using a principal component analysis (PCA) algorithm to obtain at least two first feature vectors.

[0092] It should be understood that the first feature vector is a feature vector obtained by performing a dimensionality reduction operation on the first position feature image and the second position feature image, and the present disclosure does not limit the number of the first feature vectors.

[0093] Step 210, based on the port segmentation image, determine the second feature vector through the first fully connected layer of the pre-trained model.

[0094] In some embodiments, the port segmentation image can be input into a pre-trained model, and the first fully connected layer of the pre-trained model outputs the second feature vector. In particular, the first fully connected layer can be the first fully connected layer of the pre-trained model.

[0095] The acquisition of the pre-trained model may include the following steps:

[0096] 1. Preprocess the training image set to obtain a preprocessed image set.

[0097] For preprocessing methods, please refer to Figure 2The preprocessing method of step 204 is not described in detail here.

[0098] 2. Based on the preprocessed image set, use the model to be trained to determine the training fusion vector.

[0099] In this step, in order to address the diverse forms of the optical cross-box port states, the PCA principal component analysis algorithm is used to extract representative feature vectors from the preprocessed image set. At the same time, this feature vector is combined with the feature vector obtained from the first fully connected layer of the convolutional neural network to obtain the final training fusion vector for training.

[0100] In particular, considering that the same type of port morphology will have diverse port state morphology in different optical cross-boxes due to different colors and shapes of the optical cross-boxes, this scheme chooses to use batch normalization to unify the scattered image eigenvalue data into a normal distribution with a mean of 0 and a variance of 1, so as to ensure that overfitting can be prevented while accelerating the network convergence speed.

[0101] 3. Based on the training fusion vector, use the support vector machine algorithm to train the model to be trained to obtain a pre-trained model.

[0102] In this step, the first feature vector and the second feature vector are trained using a support vector machine algorithm (SVM) to obtain a SVM port recognition classification model with strong robustness. The SVM-based image fusion feature vector training is to use the SVM support vector machine algorithm to train the extracted fusion feature vector to obtain a port classification recognition model with strong robustness and generalization performance, that is, a pre-trained model.

[0103] Step 211: Determine a fused feature vector based on the first feature vector and the second feature vector.

[0104] In some embodiments, weights may be set for the first feature vector and the second feature vector, and then the first feature vector and the second feature vector are weighted and summed to obtain a fused feature vector. However, this is not limited to this, and the present disclosure does not limit the fusion method of the first feature vector and the second feature vector.

[0105] Step 212: Determine the recognition result of the port based on the fused feature vector.

[0106] In some embodiments, the features of each port are determined based on the fused feature vector, and then the corresponding recognition results are matched with the features.

[0107] For example, if the fused feature vector of port 1 shows that the feature of the port matches the feature of a preset port with occlusion, the recognition result of port 1 is determined as port 1 with occlusion.

[0108] In summary, the port recognition method proposed in the present disclosure includes: obtaining a port video of an optical cross-box; using a long short-term memory network to extract timing information and global information of the port video; determining a target frame image based on the timing information and the global information; preprocessing the target frame image to determine a preprocessed target frame image; using a Hough circle detection algorithm to determine the coordinates of a first port in the preprocessed target frame image; based on the coordinates of the first port, using a chessboard segmentation algorithm to determine the coordinates of a second port, the first port being different from the second port; determining a port segmentation image based on the coordinates of the first port and the coordinates of the second port; based on the port segmentation image, using the first output convolution layer and the second output convolution layer of a pre-trained model to respectively determine a first position feature image and a second position feature image; based on the first position feature image and the second position feature image, using a principal component analysis algorithm to determine a first feature vector; based on the port segmentation image, determining a second feature vector through the first fully connected layer of the pre-trained model; determining a fused feature vector based on the first feature vector and the second feature vector; and determining a recognition result of the port based on the fused feature vector. The method disclosed in the present invention can avoid the problem that the pictures taken by the patrol personnel cannot accurately identify the port turntable by extracting the target frame image from the port video, thereby reducing the difficulty of the patrol personnel's shooting operation; by utilizing the Hough circle detection algorithm and the chessboard segmentation algorithm to determine the coordinates of each port, the accuracy of the port coordinate determination can be improved, and by determining the fusion feature vector, the feature fusion of the port in the target frame image is realized, thereby improving the accuracy of port identification.

[0109] Therefore, this solution has the following beneficial effects:

[0110] This solution adaptively selects key frame images of the optical cross-box video stream in real time, uses the SVM support vector machine algorithm to perform multi-classification on the multi-scale feature map features of the CNN convolutional neural network model of the optical cross-box different interface status images, and then uses its classifier model to classify and identify the optical cross-box interface, and finally performs online weighted fusion on the recognition results of multiple frames. This achieves intelligent recognition of the optical cross-box to solve the problem of time-consuming and labor-intensive inspection of the optical cross-box. After real-time recognition of the key frames, this solution uses the SVM port recognition and classification model to identify the status of each port, making the recognition process real-time, fast, and smooth, and improving the efficiency and accuracy of port recognition.

[0111] The following is an exemplary description of this scenario:

[0112] This program includes Figure 6The following steps are shown: video adaptive frame selection, key frame image preprocessing, port segmentation and positioning, port image convolutional neural network model pre-training, image multi-scale feature map principal component analysis, image comprehensive feature vector training based on SVM (Support Vector Machines), and multi-frame recognition result weighted fusion.

[0113] The above-mentioned video adaptive frame selection means selecting the key frame image with the richest information of the current port through the recurrent neural network LSTM (Long Short Term Memory). This solution proposes a real-time video adaptive frame selection model. The above-mentioned optical cross-connection box video stream is obtained from the real-time shooting of the patrol personnel's mobile phone, which can automatically identify the port status of the optical cross-connection box in real time.

[0114] The key frame image preprocessing mentioned above refers to using digital image processing technology to process a series of interference components that are not conducive to the image recognition process, such as overexposure and underexposure, which occur during the image acquisition process.

[0115] The segmentation and positioning of the above ports means that all the port parts in the optical intersection box are positioned and segmented by combining the physical structure of the optical intersection box with the Hough circle detection through the designed chessboard algorithm, so as to facilitate the subsequent classification and identification.

[0116] The pre-training of the port image convolutional neural network model means pre-training the optical cross-box port through a convolutional neural network CNN (Convolutional Neural Networks), and later extracting the feature vector of the image through the pre-trained model.

[0117] The principal component analysis of the multi-scale feature map of the above image represents the output feature maps of different scales in the CNN neural network pre-training model, and then the feature map is reduced in dimension by the PCA (Principal Component Analysis) algorithm to obtain the feature vector of the feature map.

[0118] The above-mentioned SVM-based image comprehensive feature vector training means using the SVM support vector machine algorithm to train the pre-trained model and the feature vector obtained by principal component analysis to obtain a port recognition model with strong robustness.

[0119] The weighted fusion of the multi-frame recognition results means that the key frame images with the richest port information are continuously selected through the real-time video stream for port recognition and the recognition results are weighted and classified according to the key frame weights to improve the dynamic recognition accuracy.

[0120] This plan may also include Figure 7The following recognition process is shown: LSTM adaptively selects key frames, SVM port recognition classification model, and online weighted fusion of recognition results.

[0121] The above LSTM adaptive key frame selection means selecting the key frame image with the most information on the current port through the recurrent neural network LSTM algorithm. In the LSTM adaptive key frame selection, the recurrent neural network LSTM model is mainly used to obtain the timing information and global information of the video stream, and the key frame image with the most information on the port in the current video stream is selected.

[0122] Preferably, the above-mentioned LSTM adaptive key frame selection is mainly used for the problem that the inspectors cannot shoot the complete image of the optical cross box due to operational problems. This solution uses the recurrent neural network LSTM model to adaptively select key frame images of the acquired video stream information in real time, so as to obtain the global information of the optical cross box through video recording, which saves time and effort for the inspectors.

[0123] The above-mentioned online weighted fusion of recognition results means that the key frame images with the richest port information are continuously selected through the real-time video stream for port recognition and the recognition results are weighted and classified to improve the recognition accuracy. With the information of the video stream, the recognition results obtained by each selected key frame are online weighted fused according to the key frame weight to obtain the final port recognition result.

[0124] Preferably, the above-mentioned online fusion of multiple frame recognition results is to perform weighted fusion on the recognition results obtained from multiple selected key frame images, thereby improving the recognition accuracy of the port.

[0125] As one of the embodiments of this solution, please refer to Figure 8 As shown, the above optical cross-connect box port dynamic identification method includes the following steps:

[0126] Step S810: Adaptively select key frames through the LSTM model of the recurrent neural network. The LSTM model of the recurrent neural network is used to obtain the timing information and global information of the current port video stream, and select the key frame image with the richest port information in the current video stream.

[0127] Step S820: Establish an SVM port recognition classification model, and input the key frame image into the SVM port recognition classification model to obtain a port recognition result. The recognition process of the SVM port recognition classification model includes: key frame image preprocessing, port location and segmentation, image port convolutional neural network model pretraining, image multi-scale feature map principal component analysis, and SVM-based image comprehensive feature vector training.

[0128] Please refer to Fig. 9 As shown, the SVM port identification classification model is established according to the following steps:

[0129] Sub-step S821: key frame image preprocessing. Use digital image processing technology to process a series of interference components that are not conducive to the image recognition process, such as overexposure and underexposure, that occur during the image acquisition process. In terms of image preprocessing, the gamma curve is adjusted to dynamically compress the image of the light cross box, so that low pixels (underexposed parts) become brighter and high-brightness pixels (overexposed parts) become darker. Then, the problem of uneven image exposure is handled by fitting the brightness information of the image and the contour details of the image.

[0130] Preferably, the image preprocessing is mainly used to deal with the phenomenon of overexposure or underexposure caused by light when the inspectors acquire the image. This solution uses the inverse S curve of the deformation of the γ curve to compress the dynamic range of the image, making the low pixels (underexposed parts) brighter and the high-brightness pixels (overexposed parts) darker, and then fitting the brightness information of the image and the contour details of the image to deal with the problem of uneven image exposure.

[0131] Please refer to Figure 3 As shown in the figure, this scheme adaptively finds the adjustment curve suitable for different light cross-box images, divides the image into several different brightness levels according to the brightness channel of the input image brightness-chrominance (YUV) color space, and then divides the image into multiple sub-regions and optimizes their brightness values ​​at the same time. The brightness level change of the input image is statistically fitted into the brightness adjustment curve of the output image using the optimized information of these regions, and finally obtains a light cross-box image with a more visual effect. The horizontal axis of the S curve is the brightness value of the input image, and the vertical axis is the brightness value of the output image.

[0132] Sub-step S822: Positioning and segmenting the ports. By combining the designed chessboard algorithm with the physical structure of the optical cross-box and the Hough circle detection, all the port parts in the optical cross-box are positioned and segmented to facilitate subsequent classification and identification.

[0133] Preferably, the image port segmentation is mainly to locate the five states of all ports (red cap, white cap, plugged in, damaged, blocked), and organize them into structured information as the state record of the optical cross-connect box. This solution takes into account the fact that the internal wiring of the optical cross-connect box is messy and numerous. If machine learning and deep learning methods are used directly, it is difficult to completely locate and identify the position and state of each port while ensuring good versatility. Therefore, the red cap port is first segmented by Hough circle detection, and then the designed chessboard algorithm and composite classification algorithm are combined to segment and identify ports in different states.

[0134] This scheme takes into account that most ports in most optical cross-connect boxes are circular red caps, so the YCrCb channel of the image is used to segment the pre-processed image, and the red circle (i.e., red cap) part in the image is extracted by integrating Hough circle detection, and then the remaining ports with unclear color features are segmented by the adaptive chessboard algorithm. This algorithm mainly targets a row of red caps, reorders them from left to right, and realizes the inter-row segmentation of all red caps. Then, the chessboard algorithm is cycled to achieve the segmentation of all ports in the box. The specific implementation of the chessboard algorithm is as follows:

[0135] 1) Calculate the minimum coordinate difference between adjacent red hats in the current row;

[0136] 2) According to the horizontal coordinate relationship between the first port of the current layer and the adjacent layer, determine the relationship between the first red-hat port of the layer and the adjacent layer indentation, same column, and protrusion, so as to determine whether the first red-hat port is the first of the 12 ports. If not, use the first port coordinates of the two layers that are the most adjacent and whose first port is a red hat to obtain the coordinates of the first port of the current layer by linear interpolation;

[0137] 3) Calculate the difference between the coordinates of the last red hat circle center of the current row and the coordinates of the first port. If there is only one red hat port in the row and it is the first port, skip this step and the average port offset is based on the nearest row.

[0138] 4) Calculate the coordinates of the remaining 11 chessboard ports based on the average interval between two adjacent ports and the coordinates of the first port.

[0139] This scheme aims at the five states of ports to be identified in the inspection task (red hat, white hat, plugged in, damaged, and blocked). The red hat ports of the optical cross-connect box are first segmented through Hough circle detection, and then the designed chessboard algorithm and conformity classification algorithm are combined to segment and identify the ports in different states.

[0140] Sub-step S823: Pre-training of image port convolutional neural network model. The optical cross-box port is pre-trained by the designed 13-layer convolutional neural network, and the first eigenvector of the image is extracted by the pre-trained model later, and feature maps of different scales are output.

[0141] Preferably, the pre-training of the image port convolutional neural network model is mainly to pre-train the segmented ports through the designed thirteen-layer convolutional neural network. At the same time, considering that the same type of port morphology will have diverse port state morphology in different optical cross-boxes due to different optical cross-box colors, shapes, etc., this scheme chooses to use batch normalization method to unify the scattered image eigenvalue data into a normal distribution with a mean of 0 and a variance of 1, so as to ensure that overfitting can be prevented while accelerating the network convergence speed.

[0142] Please refer to Fig.10 As shown in the figure, the segmented ports are pre-trained by the designed thirteen-layer convolutional neural network. Taking into account that the same type of port morphology will have different port state morphologies in different optical cross-boxes due to different colors and shapes of the optical cross-boxes, this scheme chooses to use batch standardization to unify the scattered image feature value data into a normal distribution with a mean of 0 and a variance of 1, so as to ensure that the overfitting phenomenon can be prevented while accelerating the network convergence speed.

[0143] Sub-step S824: principal component analysis of the multi-scale feature map of the image. According to the first feature map output by the neural network pre-training model, the feature map output by the neural network pre-training model is subjected to dimensionality reduction processing by the PCA principal component analysis algorithm to obtain the second feature vector of the feature map. The pre-training of the image port convolutional neural network model means that the optical cross-box port is pre-trained by the designed 13-layer convolutional neural network, and the feature vector of the image is extracted later by the pre-training model.

[0144] Preferably, the principal component analysis of the multi-scale feature map of the image is mainly aimed at the morphological diversity of the port states of the optical cross-box, and the PCA principal component analysis algorithm is proposed to extract representative feature vectors from the feature maps of different scales of the pre-trained convolutional neural network model. At the same time, this feature vector is combined with the feature vector obtained from the first fully connected layer of the convolutional neural network to obtain the final comprehensive feature vector used for training. Since the comprehensive feature vector increases the gap between ports of different states compared to the feature vector of the fully connected layer output by a single convolutional neural network, it makes the training model more robust and further improves the recognition accuracy.

[0145] Please refer to Figure 5 The scheme shown in the figure outputs feature maps at the second and fifth convolutional layers of the convolutional neural network model, and performs principal component analysis on the feature maps at these two positions through PCA to obtain representative feature vectors Eigenvector_PCA in the corresponding scales. At the same time, the feature vector Eigenvector_CNN of the entire thirteen-layer convolutional neural network is output at the first fully connected layer of the pre-trained model. Finally, all the acquired feature vectors are connected to generate a comprehensive feature vector C_Eigenvector.

[0146] This scheme pre-trains the structure of port images through the designed thirteen-layer CNN convolutional neural network.

[0147] This scheme proposes to use the PCA principal component analysis algorithm to extract representative feature vectors from feature maps of different scales of the pre-trained thirteen-layer convolutional neural network model.

[0148] Sub-step S825: SVM-based image comprehensive feature vector training. The first feature vector and the second feature vector are trained using the SVM support vector machine algorithm to obtain an SVM port recognition classification model with strong robustness. The SVM-based image comprehensive feature vector training is to use the SVM support vector machine algorithm to train the extracted comprehensive feature vector to obtain a port classification recognition model with strong robustness and generalization performance.

[0149] This scheme combines the feature vector extracted by principal component analysis with the feature vector obtained from the first fully connected layer of the convolutional neural network to obtain a comprehensive feature vector that is ultimately used for training the SVM support vector machine algorithm.

[0150] Step S830: Online weighted fusion of recognition results: The key frame images with the most abundant port information are continuously selected through the real-time video stream to perform port recognition and the recognition results are weighted and classified to improve the recognition accuracy.

[0151] This solution selects key frame images of the optical cross-box video stream in real time and adaptively, and uses the SVM support vector machine algorithm to perform multi-classification on the multi-scale feature map features of the CNN convolutional neural network model of the optical cross-box different interface status images, and then uses its classifier model to classify and identify the optical cross-box interface, and finally performs online weighted fusion of the recognition results of multiple frames. In this way, intelligent recognition of the optical cross-box is realized, which is used to solve the problem of time-consuming and labor-intensive inspection of the optical cross-box.

[0152] Fig.11 FIG. 1 is a schematic diagram of a port identification device 1100 provided in an embodiment of the present disclosure. Fig.11 As shown, the port identification device includes:

[0153] An acquisition unit 1110 is used to acquire a port video of an optical cross-connect box;

[0154] A first determining unit 1120 is used to determine a target frame image based on the port video using a long short-term memory network;

[0155] The segmentation unit 1130 is used to perform positioning segmentation processing on the target frame image and determine the port segmentation image;

[0156] A second determining unit 1140 is used to determine a fusion feature vector of a port based on the port segmentation image using a pre-trained model;

[0157] The identification unit 1150 is used to determine the identification result of the port based on the fused feature vector.

[0158] In some embodiments of the present disclosure, the first determination unit 1120 is further configured to extract timing information and global information of the port video using a long short-term memory network; and determine the target frame image based on the timing information and the global information.

[0159] In some embodiments of the present disclosure, the segmentation unit 1130 is also used to preprocess the target frame image to determine a preprocessed target frame image; use a Hough circle detection algorithm to determine the coordinates of a first port in the preprocessed target frame image; based on the coordinates of the first port, use a chessboard segmentation algorithm to determine the coordinates of the second port, the first port is different from the second port; based on the coordinates of the first port and the coordinates of the second port, determine a port segmentation image.

[0160] In some embodiments of the present disclosure, the segmentation unit 1130 is further used to determine the minimum difference in coordinates of adjacent first ports in the target row based on the coordinates of the first port in the target row; determine the coordinates of the first port of the target row based on the horizontal coordinate relationship between the first first port of the target row and the first first port of the non-target row; determine the number of second ports based on the coordinates of the first port in the target row, the coordinates of the first port, and the minimum difference; determine the coordinates of the second port based on the coordinates of the first port of the target row, the minimum difference, and the number of second ports.

[0161] In some embodiments of the present disclosure, the second determination unit 1140 is also used to determine the first position feature image and the second position feature image respectively based on the port segmentation image using the first output convolution layer and the second output convolution layer of the pre-trained model; determine the first feature vector based on the first position feature image and the second position feature image using the principal component analysis algorithm; determine the second feature vector based on the port segmentation image through the first fully connected layer of the pre-trained model; and determine the fused feature vector based on the first feature vector and the second feature vector.

[0162] In some embodiments of the present disclosure, the second determination unit 1140 is further used to preprocess the training image set to obtain a preprocessed image set; based on the preprocessed image set, using the model to be trained, determine the training fusion vector; based on the training fusion vector, using the support vector machine algorithm, train the model to be trained to obtain a pretrained model.

[0163] In summary, the port identification device proposed in the present disclosure includes: an acquisition unit for acquiring a port video of an optical cross-box; a first determination unit for determining a target frame image based on the port video using a long short-term memory network; a segmentation unit for positioning and segmenting the target frame image to determine a port segmentation image; a second determination unit for determining a fused feature vector of the port based on the port segmentation image using a pre-trained model; and an identification unit for determining a port identification result based on the fused feature vector. The device of the present disclosure can avoid problems with pictures taken by patrol personnel that prevent them from accurately identifying port turntables by extracting the target frame image from the port video, thereby reducing the difficulty of patrol personnel's shooting operations. At the same time, by determining the fused feature vector, the feature fusion of the port in the target frame image is achieved, thereby improving the accuracy of port identification.

[0164] It should be noted that: the port identification device provided in the above embodiment only uses the division of the above-mentioned program modules as an example when performing port identification. In actual applications, the above-mentioned processing can be assigned to different program modules as needed, that is, the internal structure of the port identification device is divided into different program modules to complete all or part of the processing described above.

[0165] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.

[0166] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0167] Fig.12 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure is shown in FIG. Fig.12 As shown, the electronic device 1200 includes at least one processor 1202; and a memory 1201 communicatively connected to the at least one processor 1202; wherein the memory 1201 stores instructions executable by the at least one processor 1202, and the instructions are executed by the at least one processor 1202 to implement the steps of the port identification method described in the embodiment of the present disclosure; or, the instructions are executed by the at least one processor 1202 to implement the steps of the port identification method described in the embodiment of the present disclosure.

[0168] It is understood that the electronic device also includes a communication interface. The various components in the electronic device are coupled together through a bus system. It is understood that the bus system is used to achieve connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus and a status signal bus.

[0169] It can be understood that the memory 1201 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDR AM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 1201 described in this embodiment of the present solution is intended to include, but is not limited to, these and any other suitable types of memory.

[0170] The method disclosed in the above embodiment of the present disclosure may be applied to the processor 1202, or implemented by the processor 1202. The processor 1202 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 1202 or an instruction in the form of software.

[0171] The above-mentioned processor 1202 can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1202 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this solution. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of this solution, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 1201. The processor 1202 reads the information in the memory 1201 and completes the steps of the aforementioned method in combination with its hardware.

[0172] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), FPGA, general purpose processor, controller, MCU, microprocessor, or other electronic components to execute the aforementioned method.

[0173] The embodiment of the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to implement the steps of the port identification method described in the embodiment of the present solution when the computer instructions are executed.

[0174] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0175] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present solution. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0176] Any process or method description in the flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiment of the present scheme includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present scheme belong.

[0177] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0178] It should be understood that the various parts of the implementation of the present solution can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned implementation, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another implementation, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0179] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0180] In addition, each functional unit in each embodiment of the present solution can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0181] Although the implementation methods of the present scheme have been shown and described above, it can be understood that the above implementation methods are exemplary and cannot be understood as limitations on the present scheme. Ordinary technicians in this field can change, modify, replace and modify the above implementation methods within the scope of the present scheme.

Claims

1. A port identification method, characterized in that: The method comprises: Get the port video of the optical cross-connect box; Based on the port video, a target frame image is determined using a long short-term memory network; Performing positioning and segmentation processing on the target frame image to determine a port segmentation image; Based on the port segmentation image, using a pre-trained model, determining a fusion feature vector of the port; Based on the fused feature vector, an identification result of the port is determined.

2. The method according to claim 1, characterized in that Determining the target frame image based on the port video using a long short-term memory network includes: Utilizing the long short-term memory network, extracting the timing information and global information of the port video; Based on the timing information and the global information, the target frame image is determined.

3. The method according to claim 1, characterized in that Determining the port segmentation image based on the target frame image by using an image segmentation algorithm includes: Preprocessing the target frame image to determine a preprocessed target frame image; Determine the coordinates of the first port in the preprocessed target frame image using a Hough circle detection algorithm; Based on the coordinates of the first port, using a chessboard segmentation algorithm, determining the coordinates of a second port, the first port being different from the second port; The port segmentation image is determined based on the coordinates of the first port and the coordinates of the second port.

4. The method according to claim 3, characterized in that Determining the coordinates of the second port based on the coordinates of the first port by using a chessboard segmentation algorithm includes: Based on the coordinates of the first ports in the target row, determining a minimum difference between the coordinates of adjacent first ports in the target row; Determine the coordinates of the first port of the target row based on the horizontal coordinate relationship between the first first port of the target row and the first first port of the non-target row; Determining the number of the second ports based on the coordinates of the first port in the target row, the coordinates of the first port, and the minimum difference; The coordinates of the second port are determined based on the coordinates of the first port of the target row, the minimum difference, and the number of the second ports.

5. The method according to claim 1, characterized in that The determining of the fusion feature vector of the port based on the port segmentation image and using a pre-trained model comprises: Based on the port segmentation image, using the first output convolution layer and the second output convolution layer of the pre-trained model, respectively determine a first position feature image and a second position feature image; Based on the first position feature image and the second position feature image, using a principal component analysis algorithm, determining a first feature vector; Based on the port segmentation image, determine a second feature vector through a first fully connected layer of the pre-trained model; The fused feature vector is determined based on the first feature vector and the second feature vector.

6. The method according to claim 1, characterized in that The method further comprises: Preprocessing the training image set to obtain a preprocessed image set; Based on the preprocessed image set, using the model to be trained, determining a training fusion vector; Based on the training fusion vector, the model to be trained is trained using a support vector machine algorithm to obtain a pre-trained model.

7. A port identification device, characterized in that: The device comprises: An acquisition unit, used for acquiring a port video of an optical cross-connect box; A first determination unit is used to determine a target frame image based on the port video using a long short-term memory network; A segmentation unit, used for performing positioning segmentation processing on the target frame image to determine a port segmentation image; A second determination unit, configured to determine a fusion feature vector of the port based on the port segmentation image and using a pre-trained model; The identification unit is used to determine the identification result of the port based on the fused feature vector.

8. An electronic device, characterized in that: include: a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the method described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images