Optical facility port management method, system and equipment based on target tracking, and storage medium

Abstract

The embodiment of the invention provides an optical facility port management method, system and device based on target tracking and a storage medium, and belongs to the technical field of computers. According to the method, port target detection is carried out on each frame of image in a port video stream in a mode of collecting the port video stream of a target optical facility to obtain a detection result of each frame of image, then port target tracking is carried out according to the port detection results of multiple frames of images in the port video stream to obtain a tracking identifier about a port in each frame of image, and the tracking identifier of the port in each frame of image is obtained. And sorting the optical facility ports according to the port detection result of each frame of image and the tracking identifier to obtain a video port list. The optical facility port management accuracy can be improved, and the optical facility port management cost can be reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a method, system, device, and storage medium for optical facility port management based on target tracking. Background Art

[0002] Optical facility ports, also known as optical resource ports (such as switch ports, fiber access ports, etc.), play a crucial role in data communication, switch devices, and network infrastructure. With the development of fiber optic communication technology, more and more optical resource ports are applied to efficient network architectures. In order to deploy the network orderly, accurate management of optical facility ports is required.

[0003] In related technologies, the detection and management method for optical facility ports usually adopts a static vision detection method, that is, a shooting device is fixed at the optical facility port for monitoring. However, this method may have the problem of target occlusion, resulting in low accuracy of optical facility port management. In addition, there are numerous optical facility ports, and cameras need to be set in each viewing area, resulting in high management costs for optical facility ports. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose a method, system, device, and storage medium for optical facility port management based on target tracking, aiming to improve the accuracy of optical facility port management and reduce the management cost of optical facility ports.

[0005] To achieve the above purpose, on the one hand, an embodiment of this application proposes a method for optical facility port management based on target tracking, including the following steps:

[0006] Obtain the port video stream of the target optical facility;

[0007] Perform port target detection on each frame image in the port video stream to obtain the detection result of each frame image, and the detection result includes multiple detection information about the port;

[0008] Perform port target tracking based on the port detection results of multiple frame images in the port video stream to obtain the tracking identifier about the port in each frame image;

[0009] Sort the optical facility ports according to the port detection results and tracking identifiers of each frame image to obtain a video port list.

[0010] In some embodiments, the performing port target detection on each frame image in the port video stream to obtain the detection result of each frame image includes the following steps:

[0011] Obtain a port target detection model trained based on the YOLO algorithm;

[0012] Input each frame image in the port video stream into the port target detection model in sequence to obtain the detection result of each frame image;

[0013] Among them, the detection result includes multiple detection information about the port, and the detection information includes the detection frame position, status classification, and confidence level.

[0014] In some embodiments, the port target tracking is performed based on the port detection results of multiple frame images in the port video stream to obtain the tracking identifier about the port in each frame image, including the following steps:

[0015] Select the detection information with a confidence level greater than the expected value according to the port detection result of the image, and determine the tracking port according to the selected detection information;

[0016] Perform correlation matching on the tracking ports of the current frame image and the previous frame image according to the detection information to obtain a matching result;

[0017] Judge whether the tracking port in the current frame image is successfully matched according to the matching result;

[0018] When the tracking port is successfully matched, assign the tracking identifier of the port that matches in the previous frame image to the tracking port of the current frame image;

[0019] When the tracking port fails to match, assign a new tracking identifier to the tracking port of the current frame image.

[0020] In some embodiments, the sorting of the optical facility ports is performed according to the port detection results and tracking identifiers of each frame image to obtain a video port list, including the following steps:

[0021] For each port of the current frame image, judge whether the tracking identifier of the port appears in the tracking identifiers of the historical frame images;

[0022] When the tracking identifier of the port appears in the tracking identifiers of the historical frame images, determine the order of the port in the current frame image according to the port sorting list of the historical frame images;

[0023] When the tracking identifier of the port does not appear in the tracking identifiers of the historical frame images, determine the adjacent ports in the opposite shooting direction in the current frame image according to the detection information of the port, and determine the order of the port according to the order of the adjacent ports;

[0024] Determine the port sorting list of the current frame image according to the order of each port of the current frame image;

[0025] Determine the video port list according to the port sorting lists of each frame image in the port video stream.

[0026] In some embodiments, the optical facility port management method based on target tracking further includes the following steps:

[0027] For each port in the current frame image, determine adjacent ports according to the port sorting list of the current frame image;

[0028] Calculate the position gap between the detection box position of the port and the detection box position of the adjacent port;

[0029] Determine whether the position of the port is abnormal according to the position gap.

[0030] In some embodiments, determining the video port list according to the port sorting list of each frame image in the port video stream includes the following steps:

[0031] Merge the port sorting lists of each frame image in the port video stream in order and remove duplicates to obtain a video port list;

[0032] Add a port index to each port in the video port list in order, and associate the detection information of the port to the corresponding port in the video port list.

[0033] In some embodiments, the optical facility port management method based on target tracking further includes the following steps:

[0034] Receive a network resource comparison request, where the network resource comparison request includes a target comparison device and device port information;

[0035] Determine the video port list of the corresponding device according to the target comparison device;

[0036] Compare the device port information according to the video port list to obtain a comparison result, and return the comparison result.

[0037] To achieve the above object, another aspect of the embodiments of the present application proposes an optical facility port management system based on target tracking, including:

[0038] A first module for obtaining the port video stream of the target optical facility;

[0039] A second module for performing port target detection on each frame image in the port video stream to obtain the detection result of each frame image, where the detection result includes multiple pieces of detection information about the port;

[0040] A third module for performing port target tracking according to the port detection results of multiple frame images in the port video stream to obtain the tracking identifier of the port in each frame image;

[0041] A fourth module, configured to sort optical facility ports according to the port detection results and tracking identifiers of each frame of image, so as to obtain a video port list.

[0042] To achieve the above object, another aspect of the embodiments of the present application provides an electronic device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory. When the program is executed by the processor, the method described in the above embodiments is implemented.

[0043] To achieve the above object, another aspect of the embodiments of the present application provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the above embodiments.

[0044] The optical facility port management method, system, device and storage medium based on target tracking proposed by the present application collect the port video stream of the target optical facility, perform port target detection on each frame of image in the port video stream to obtain the detection result of each frame of image, and then perform port target tracking according to the port detection results of multiple frames of images in the port video stream to obtain the tracking identifier of the port in each frame of image. The optical facility ports are sorted according to the port detection results and tracking identifiers of each frame of image to obtain a video port list. By dynamically collecting the port video of the target optical facility, the present application can reduce the problem of port occlusion, and use a dynamic tracking algorithm to track the ports, improve the accuracy of video port detection, thereby improving the accuracy of optical facility port management, and there is no need to set cameras in each viewing area of the optical facility equipment, reducing the optical facility port management cost. Description of the Drawings

[0045] Figure 1 is a flowchart of the optical facility port management method based on target tracking provided by the embodiments of the present application;

[0046] Figure 2 is Figure 1 a flowchart of step S102 in

[0047] Figure 3 is Figure 1 a flowchart of step S103 in

[0048] Figure 4 is Figure 1 a flowchart of step S104 in

[0049] Figure 5It is a flowchart of an optical facility port management method based on target tracking provided by another embodiment of the present application;

[0050] Figure 6 It is Figure 4 a flowchart of step S405 in

[0051] Figure 7 It is a flowchart of an optical facility port management method based on target tracking provided by another embodiment of the present application;

[0052] Figure 8 It is a schematic diagram of an optical facility port management system based on target tracking provided by an embodiment of the present application;

[0053] Figure 9 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

[0054] Figure 10 It is a schematic diagram of the overall implementation process of an optical facility port management method based on target tracking provided by an embodiment of the present application;

[0055] Figure 11 It is a schematic diagram of a video port list provided by an embodiment of the present application. Detailed implementation manners

[0056] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

[0057] It should be noted that although functional module division is performed in the system and the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order from the module division in the system or the order in the flowchart. Terms such as "first" and "second" in the specification, claims and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0059] First, several nouns involved in the present application are analyzed:

[0060] Object Detection is a core task in computer vision, aiming to identify the categories of all objects in an image or video and predict the specific positions of the objects (usually represented by bounding boxes).

[0061] The YOLO model (You Only Look Once: Unified, Real-Time Object Detection, an object detection system based on a single neural network) is a deep learning algorithm in computer vision technology that can be used for image recognition. The YOLO model transforms the object detection problem into a regression problem, that is, given an input image, it directly regresses the bounding box (i.e., the detection box) of the object and its classification category at multiple positions on the image. The YOLO model includes but is not limited to Yolov3, Yolov4, Yolov5 (all different versions of YOLO), etc. Different models have different weights, network structure diagrams, algorithms, and region sampling methods used.

[0062] YOLOv11 is one version of the YOLO model. YOLOv11 simplifies the object detection problem to a single forward propagation of a neural network, so it can achieve real-time detection.

[0063] Object Tracking refers to continuously tracking the detected objects in a video and recording their trajectories in time and space.

[0064] The ByteTrack algorithm is an efficient multi-object tracking (MOT) algorithm developed based on the YOLO series of object detection models. It significantly improves the continuity and stability of object tracking by making full use of low-score detection results in the detection boxes (such as occluded objects).

[0065] In related technologies, the detection and management method for optical facility ports usually adopts a static vision detection method, where the shooting device is fixed at the optical facility port for monitoring. However, this method may have the problem of target occlusion, resulting in low accuracy in the management of optical facility ports. In addition, there are many optical facility ports, and cameras need to be set in each viewing area, resulting in high management costs for optical facility ports.

[0066] Based on this, the embodiments of this application provide a method, system, device, and storage medium for managing optical facility ports based on object tracking, aiming to improve the accuracy of optical facility port management and reduce the management cost of optical facility ports.

[0067] The method, system, device, and storage medium for managing optical facility ports based on object tracking provided by the embodiments of this application are specifically described through the following embodiments. First, the method for managing optical facility ports based on object tracking in the embodiments of this application is described.

[0068] The optical facility port management method based on target tracking provided by the embodiments of the present application relates to the field of computer technology. The optical facility port management method based on target tracking provided by the embodiments of the present application can be applied to a terminal, or to a server side, or can be software running on a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the optical facility port management method based on target tracking, etc., but is not limited to the above forms.

[0069] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0070] Figure 1 is an optional flowchart of the optical facility port management method based on target tracking provided by the embodiments of the present application, Figure 1 The method in may include but is not limited to steps S101 to S104.

[0071] Step S101, obtaining a port video stream of a target optical facility;

[0072] Step S102, performing port target detection on each frame of image in the port video stream to obtain the detection result of each frame of image, and the detection result includes multiple detection information about the port;

[0073] Step S103, performing port target tracking according to the port detection results of multiple frames of images in the port video stream to obtain the tracking identifier about the port in each frame of image;

[0074] Step S104: Sort the optical facility ports according to the port detection results and tracking identifiers of each frame of image to obtain a video port list.

[0075] For steps S101 to S104 illustrated in the embodiments of the present application, by dynamically collecting the port videos of the target optical facility, the problem of port occlusion can be reduced, and a dynamic tracking algorithm is used to track the ports, improving the accuracy of video port detection, thereby improving the accuracy of optical facility port management. Moreover, it is not necessary to set cameras in each perspective area of the optical facility equipment, reducing the cost of optical facility port management.

[0076] In step S101 of some embodiments, the port video stream can be collected by a camera. In one example, a staff member can hold the camera to collect the video of the optical facility port, walking from one end of the optical facility to the other end during collection. In another example, an intelligent robot can be used to collect the video, that is, a patrol robot is deployed in the computer room, and a fixed patrol route is set for the robot. When the robot patrols to the optical facility equipment, the camera on the robot is started, and the port video stream of each optical facility is collected from one end to the other end according to the port layout method of the optical facility.

[0077] Further, before collecting the port video stream, the camera can be used to collect identifiers such as barcodes or QR codes of the optical facility first, so as to facilitate the corresponding management of the port video stream of each optical facility.

[0078] In step S102 of some embodiments, the port video stream is composed of multiple frames of temporally consecutive images. For each frame of image, a target detection model can be used to identify the ports, so as to determine the detection information about the ports in each frame of image. The detection information of multiple ports forms the detection result of this frame of image. Among them, each detection information can include the position of the detection box of the port, status classification, confidence level, etc. The status classification can be one of the idle state or the occupied state. The idle state means that there is no optical fiber inserted into this port, and the occupied state means that there is an optical fiber inserted into this port.

[0079] Please refer to Figure 2 , in some embodiments, step S102 may include but is not limited to steps S201 to S202:

[0080] Step S201: Obtain a trained port target detection model based on the YOLO algorithm;

[0081] Step S202: Input each frame of image in the port video stream into the port target detection model in turn to obtain the detection result of each frame of image.

[0082] In this embodiment, the port target detection model can be trained based on the YOLO algorithm, specifically the YOLO11 algorithm. By collecting port images of optical facilities and marking the ports in the images to obtain sample data, the YOLO11 model is trained using the sample data to obtain a trained port target detection model. Exemplarily, the training process of the port target detection model is as follows:

[0083] Data collection and annotation: Obtain optical facility images or video data containing ports, and disassemble the video into frame images. Use image annotation tools such as labelimg to perform data annotation on the positions and states of the optical facility ports, with the labels "on" and "off" representing the "idle" and "occupied" states.

[0084] Divide the dataset: Divide the dataset into a training set (train), a validation set (val), and a test set (test), for example, in a ratio of 8:1:1.

[0085] Set the YOLOv11 model structure and hyperparameters: YOLOv11 is an improved version of the YOLO series, with higher detection accuracy and inference speed. Its core includes a backbone network, a feature pyramid, and an output head for predicting bounding boxes and confidence. Hyperparameters need to be set before training. Among them, the batch size is set according to the GPU video memory size (such as 16, 32); the learning rate can adopt a cosine annealing or adaptive learning rate strategy (such as 0.001 - 0.01); the number of epochs is usually between 50 and 300, and is dynamically adjusted according to the data scale and convergence situation; the loss function usually uses a weighted combination of CIoU / GIoU Loss, classification loss, and confidence loss.

[0086] Model training: Through cyclic iteration (epoch), continuously perform forward propagation, backward propagation, and parameter update on the training set, and record the performance (metrics such as mAP, Precision, Recall) on the validation set. According to the best performance on the validation set, select the optimal weight best.pt as the final model file for inference.

[0087] Use the YOLOv11 algorithm to train a port target detection model. The trained port target detection model can identify ports in the image and provide the detection box position of the port (such as the diagonal coordinates of the detection box), status classification (such as idle status or occupied status), and confidence.

[0088] In step S103 of some embodiments, video object tracking is performed by analyzing the sequence of image frames in the video to locate and track the position and movement trajectory of a specific port. In this embodiment, a multi-object tracking algorithm is mainly used to identify the trajectory of the specific port in different image frames, and this trajectory can be represented by a unique tracking identifier.

[0089] Please refer to Figure 3 , in some embodiments, step S103 may include but is not limited to steps S301 to S304:

[0090] Step S301, select the detection information with a confidence level greater than the expected value according to the port detection result of the image, and determine the tracking port according to the selected detection information;

[0091] Step S302, perform correlation matching on the tracking ports of the current frame image and the previous frame image according to the detection information to obtain a matching result;

[0092] Step S303, determine whether the tracking port in the current frame image is successfully matched according to the matching result;

[0093] Step S304, when the tracking port is successfully matched, assign the tracking identifier of the port that matches in the previous frame image to the tracking port of the current frame image;

[0094] Step S305, when the tracking port fails to match, assign a new tracking identifier to the tracking port of the current frame image.

[0095] In this embodiment, the port detection result of each frame image includes the detection information of multiple ports, and the detection information includes the confidence level of the port detection box. For each frame image, select the detection information with a confidence level greater than the expected value in its port detection result, and determine the tracking port according to the detection box in the selected detection information. In this embodiment, considering that some detection boxes may not identify ports, selecting the target corresponding to the detection information with a confidence level greater than the expected value can achieve tracking of the target accurately identified as a port.

[0096] After determining multiple tracking ports in the image, the tracking ports of the current frame image and the previous frame image are associated and matched according to the detection information to obtain a matching result. Specifically, the Kalman filter can be used to predict the motion of the tracking target to obtain the predicted position of the tracking target; then, a cost matrix is constructed based on the predicted position of the tracking target in the previous frame image and the detection box position of the tracking target in the current frame image. The elements in the cost matrix represent the distance between the predicted position of the i-th tracking target in the previous frame image and the detection box position of the j-th tracking target in the current frame image; then, the tracking targets of the current frame image and the previous frame image are associated and matched according to the cost matrix to obtain a matching result. Specifically, the two tracking targets corresponding to the element values in the cost matrix that are less than the expected distance value can be marked as matched, otherwise marked as unmatched.

[0097] Judge whether the tracking ports in the current frame image are successfully matched according to the matching result. When the tracking ports are successfully matched, it is considered that the tracking ports have appeared in the previous frame image, and then the tracking identifier of the matching port in the previous frame image is assigned to the tracking ports of the current frame image, so as to track the same port in different image frames; when the tracking ports fail to match, it is considered that the tracking ports are newly captured ports, and a new tracking identifier is assigned to the tracking ports of the current frame image.

[0098] Exemplarily, after the port detection model obtains the port detection results of different image frames, the ByteTrack algorithm is combined to associate the same target in different frames of the video and generate a stable tracking identifier trackID.

[0099] The model trained based on YOLOv11 outputs a series of information such as detection boxes {x1, y1, x2, y2}, confidence conf, and class name class_name on each frame image.

[0100] Apply the ByteTrack algorithm to the port detection results of each frame image for multi-target tracking. The core of ByteTrack is to maintain the target ID across frames through data association (associating detection boxes with existing trajectories). The key steps of the ByteTrack algorithm include:

[0101] Filter the detection results: First, filter out high-confidence targets according to the confidence threshold.

[0102] Kalman filter: Predict the motion of the target to reduce jitter and loss.

[0103] Hungarian algorithm: Construct a cost matrix (Cost Matrix) between the detection boxes of the current frame and the targets that have been tracked in the previous frame (or multiple historical frames), and obtain the best association through maximum matching or minimum cost matching.

[0104] ID Maintenance: If the detection box successfully matches an existing target, its track ID remains the same; if it cannot be matched, a new track ID is generated; if an existing target fails to match the detection box for multiple frames, it is determined that the target has exited the scene or is lost.

[0105] Further, after target tracking processing, a detection and tracking list track_list for each frame of the image can be obtained, which includes the tracking identifier track_id, bounding box coordinates (x1, y1, x2, y2), status classification (class_name), confidence level (conf), etc. for each target (i.e., port) in the image.

[0106] In step S104 of some embodiments, the optical facility ports are sorted according to the port detection results and tracking identifiers of each frame of the image to obtain a video port list. The video port list includes multiple ports that appear in the port video stream, and the multiple ports are arranged in their distribution order. According to the port order, each port is assigned a port index and its corresponding detection information.

[0107] Please refer to Figure 4 In some embodiments, step S104 may include but is not limited to steps S401 to S405:

[0108] Step S401: For each port in the current frame image, determine whether the tracking identifier of the port appears in the tracking identifiers of the historical frame images.

[0109] Step S402: When the tracking identifier of the port appears in the tracking identifiers of the historical frame images, determine the order of the port in the current frame image according to the port sorting list of the historical frame images.

[0110] Step S403: When the tracking identifier of the port does not appear in the tracking identifiers of the historical frame images, determine the adjacent port in the opposite shooting direction in the current frame image according to the detection information of the port, and determine the order of the port according to the order of the adjacent port.

[0111] Step S404: Determine the port sorting list of the current frame image according to the order of each port in the current frame image.

[0112] Step S405: Determine the video port list according to the port sorting lists of each frame image in the port video stream.

[0113] Exemplarily, after obtaining the detection and tracking list (track_id, x1, y1, x2, y2, class_name, conf,...) for each frame of the image, it is necessary to perform cross-frame sorting of the detected port targets and insert new ports, as follows:

[0114] First, perform the sorting process for the first frame: Based on the center point values of the ports in the first frame, arrange them from left to right.

[0115] The set of ports Π1 in the first frame (each port contains information such as {trackID, x1, y2, x2, y2, class, …}). In the first frame of the video, sort all the detected ports to provide a consistent sorting basis for subsequent frames.

[0116] For each port π in the first frame 1，i , calculate its center point abscissa where and represent the x - coordinates of the upper - left and lower - right corners of the port respectively.

[0117] Sort from left to right according to the center point cx Record the sorted result as This step obtains the port order of the first frame, providing a unified reference for subsequent frames.

[0118] Initialize the set of known port IDs, that is, all the trackIDs that appear in the first frame form the set of known ports

[0119] Secondly, perform the port matching and new port detection processing for subsequent frames: For each frame f = 2, 3, …, T, it is necessary to associate the new ports detected in the subsequent frames with the known ports and insert the new ports into the port list. The specific steps are as follows:

[0120] For each port π in frame f f，j ∈∏ f , check whether its trackID is in the set ; if it is in , it is regarded as a known port, and its original order information is followed or updated as needed (such as position, confidence, etc.); if it is not in , it is regarded as a new port, and the new port insertion process is executed. The new port insertion process is as follows:

[0121] When the port π f，j is determined to be a new port, it needs to be inserted into the port sorting list with a unified order. The key of the algorithm is to find the "nearest port on the left" according to the mid - point abscissa of the port, so as to keep the overall order consistent with the first frame style. It can be understood that for a port video stream with a shooting direction from left to right, the reference order can be the "nearest port on the left", and for a port video stream with a shooting direction from right to left, the reference order can be the "nearest port on the right".

[0122] The new port insertion and sorting process is as follows:

[0123] S11. Calculate the abscissa of the midpoint of the new port as follows:

[0124]

[0125] S12. Search for the nearest port on the left. In the set of known ports find a port π k that satisfies:

[0126] Θ(π k ) = max{Θ(π m ): Θ(π m ) < cx f,j , π m ∈P};

[0127] where Θ(π) represents the abscissa of the midpoint of port π, that is:

[0128]

[0129] If such a π k is found, then record nearest_tid = π k .trackID. If no left port is found (i.e., no port satisfies Θ(π m ) < cx f，j , then record nearest_tid = -1, indicating that the port exists independently and needs to be inserted at the leftmost end or according to other strategies.

[0130] S13. Update the set of known port IDs,

[0131] S14. Insert the new port into the port sorted list of the f-th frame image. When maintaining the port sorted list of each frame image, when inserting a new port, determine the insertion position according to its cx f，j ; if only recording in a global structure is required, then it can be inserted according to business requirements.

[0132] Please refer to Figure 5 , in some embodiments, the optical facility port management method based on target tracking in the embodiments of the present application may further include but is not limited to steps S501 to S503:

[0133] Step S501. For each port in the current frame image, determine the adjacent ports according to the port sorted list of the current frame image;

[0134] Step S502. Calculate the position gap between the detection box position of the port and the detection box position of the adjacent port;

[0135] Step S503: Determine whether the position of the port is abnormal according to the position difference.

[0136] In this embodiment, this embodiment can also identify whether the position of the port is abnormal based on the deviation of the positions of adjacent ports, specifically as follows:

[0137] S21. During the port sorting (usually based on the X-axis center or the left boundary), each port can find the previous port (left neighbor) and the next port (right neighbor). If it is a newly added port, its correct position should be approximately between the previous and the next ports.

[0138] S22. Denote the center coordinate of the previous port on the X-axis as X prev , and the upper coordinate of the Y-axis as Y prev ; the center coordinate of the next port on the X-axis is X next , and the upper coordinate of the Y-axis is Y next .

[0139] If the X coordinate X new of the new port completely falls outside the region of (X prev , X next ), or if the upper coordinate Y new of the new port on the Y-axis completely falls outside the region of (Y prev , Y next ) or is too close to / too far from the adjacent port, it may be abnormal.

[0140] In another example, the method of dynamic threshold comparison can also be used to determine whether the position of the new port is abnormal, specifically as follows:

[0141] S31. Calculate the distance between the center of the newly added port in the X-axis direction and the centers of the previous port and the next port, that is, ΔX left = X new - X prev , ΔX right = X next - X new . If ΔX left or ΔX right is much smaller than or much larger than the expected value threshold_X, it can be regarded as a position abnormality. Among them, threshold_X = α(X next - X pre ) / 2, 0.1 <= α <= 2.

[0142] S32. Calculate the distance between the upper end of the newly added port in the Y-axis direction and the previous port and the next port, that is, ΔY left = Y new - Y prev , ΔY right= Y next -Y new 。If ΔY left or ΔY right is greater than the expected value threshold_Y, it can be regarded as a position anomaly. Among them, threshold_Y = α(Y next -Y pre ), where 1 <= α <= 2.

[0143] Please refer to Figure 6 , in some embodiments, step S405 includes but is not limited to steps S601 to S602:

[0144] Step S601, merge and deduplicate the sorted lists of each port according to the order of each frame image in the port video stream to obtain a video port list;

[0145] Step S602, add a port index to each port in the video port list in order, and associate the detection information of the port to the corresponding port in the video port list.

[0146] In this embodiment, the sorted lists of each port are merged according to the order of each frame image in the port video stream and deduplicated to obtain a video port list. Specifically, for each frame image, it is merged in the order of "appearance sequence" to ensure the consistency of all target positions and motion trends. The ports of the first frame (first_frame_ports) and the newly added ports (new_added_ports) are merged in order, and the merge result is as follows:

[0147] final_ports = first_frame_ports + new_added_ports;

[0148] By merging in the order of "appearance sequence", the order of the ports in the first frame can be kept unchanged, and the new ports are connected in the order of appearance, so as to obtain a video port list with ports arranged in order.

[0149] Add a sorting index (sort_index) to the sorted video port list to record the order of each port in the video port list. After sorting, sort_index assigns an index value from 0 to N to the ports, so that subsequent operations can refer to the sorting order.

[0150] Further, when searching for the nearest port to the left of the newly added port above, a parameter of the nearest port tracking identifier (nearest_tid) will be recorded. For the ports of the first-frame image, there is no need to search for the nearest port to its left, so the parameter nearest_tid is not recorded for the ports of the first-frame image. This embodiment takes into account that the port data formats of first_frame_ports and new_added_ports are slightly different. For ports with different formats (for example, some ports may have 12 parameters and some have 11 parameters), simple compatibility processing is required. For example, if the port format has 11 parameters, the nearest_tid is default set to -1.

[0151] Finally, a video port list containing all port information is output. Each port information includes its sorting index, target ID, category name, confidence level, coordinates, width, height, and other information.

[0152] Please refer to Figure 7 , in some embodiments, the optical facility port management method based on target tracking in the embodiments of the present application may further include but is not limited to steps S701 to S702:

[0153] Step S701: Receive a network resource comparison request, where the network resource comparison request includes a target comparison device and device port information;

[0154] Step S702: Determine the video port list of the corresponding device according to the target comparison device;

[0155] Step S703: Compare the device port information according to the video port list to obtain a comparison result, and return the comparison result.

[0156] In this embodiment, an interface for obtaining optical facility port information in the cloud network resource center can be established. This interface can receive an optical facility resource acquisition request or a network resource comparison request from the caller. If it is an optical facility resource acquisition request, the interface outputs information including facility encoding, facility ID, facility name, and detailed encoding, name, serial number, and occupancy status of each port based on the video port list. If it is a network resource comparison request, the target comparison device and its device port information indicated in the network resource comparison request are compared according to the video port list in the resource system, that is, the port status in the resource system is compared, including the occupied and idle status of the ports. The comparison result is returned in the form of an API interface text file. The API interface returns it to the interface caller, and at the same time, the data of the comparison result is stored in the network resource data sharing platform of the cloud network resource center in the form of a text file.

[0157] The optical facility port management method based on target tracking provided by the embodiments of the present application can perform real-time detection, accurate sorting of optical resource ports in the video stream, and eliminate abnormal ports that do not meet the conditions. Combining the YOLOv11 object detection and ByteTrack tracking algorithms improves the high precision and stability of object detection and tracking. In the case of port occlusion, the port information in the video is detected to improve the authenticity and effectiveness of on-site resource inspection.

[0158] According to some embodiments of the present application, please refer to Figure 10 , and describe the overall implementation process of the optical facility port management method based on target tracking in the embodiments of the present application.

[0159] First, data collection and annotation.

[0160] Obtain the image or video data of the splitter facility containing ports, and disassemble the video into frame images. Use the labelimg image annotation tool to perform data annotation on the positions and states of the optical facility ports to obtain a data set.

[0161] Second, training of the YOLOv11 object detection model. Divide the training data set based on the above data set; set the YOLOv11 hyperparameters, where the batch size is set to 32 according to the GPU video memory size; the learning rate adopts an adaptive learning rate strategy with an initial value of 0.01; the number of epochs is set to 300; train the model according to the training data set and hyperparameters. After 300 iterations, record that the performance mAP50-95 on the validation set reaches 82.9%, Precision reaches 98.3%, and Recall reaches 98.7%. According to the best performance on the validation set, select the optimal weight best.pt as the final model file for inference.

[0162] Second, use the trained object detection model to perform port object detection on the port video stream to obtain the detection results of each frame image. The detection results include the bounding box of the recognized port, the bounding box confidence, and the status classification. Based on the bounding box, the port coordinate information can be extracted.

[0163] Third, combine ByteTrack to associate the same target in different frames of the video to generate a stable trackID. According to the detection results output by the model, use the Kalman filter to predict the motion of the target, and associate the targets in the front and back frame images according to the motion prediction results, so as to obtain the tracking identifier trackID of the port target in each frame image.

[0164] Fourth, multi-frame port sorting. Sort the ports of the first frame of the video to obtain the first-frame port set Π1 (each port contains information such as {trackID, x1, y1, x2, y2, class, …}, and all trackIDs that appear in the first frame form the known port set) In subsequent frames, use the set to identify new targets (i.e., ports). Calculate the position of the new target on the X-axis, find the nearest known target based on the position of the new target, and then based on this known target, determine the order of the new target to be behind this known target. After determining the order of the new target, the new target is considered a known target and added to the known port set Then proceed with the sorting of the next newly added port.

[0165] Fifth, port position filtering. Calculate the deviation of the port from adjacent ports in the X-axis and Y-axis directions, and calculate the dynamic threshold of the deviation based on this deviation. Determine whether the deviation exceeds the dynamic threshold. If it exceeds, consider the port position abnormal, and remove the abnormal ports and retain the remaining ports.

[0166] Sixth, sort and merge the port sort lists of each frame of the image, and output the video port list. Merge the ports of the first frame (first_frame_ports) and the newly added ports of subsequent frames (new_added_ports) in order, add a sorting index (sort_index) to the sorted video port list, record the order of each port in the final list, and finally output a video port list containing all port information. Each port information contains its sorting index, target ID, class name, confidence, coordinates, width, height, etc. The video port list is as Figure 11 shown.

[0167] Sixth, application of the port list. Obtain the optical facility port information of the resource system and the detection port sorting information (i.e., the video port list) obtained by the method of this embodiment. Compare the status of the optical facility port information of the resource system and the detection port sorting information to determine the status of the optical facility ports stored in the resource system and obtain the comparison result.

[0168] According to some embodiments of the present application, the method for managing optical facility ports based on target tracking in the embodiments of the present application has at least one of the following beneficial effects:

[0169] The method of this embodiment improves the automation level of optical network device management. Traditional port detection usually relies on manual operation, which is not only inefficient but also error-prone. This method uses the YOLOv11 and ByteTrack algorithms to accurately identify and track the port status in real time, generate a stable track_id, and achieve the unique identification of each port in the entire video frame. It effectively reduces manual intervention and can effectively solve the problem of static detection ports being blocked, improving the efficiency of device management.

[0170] The method of this embodiment enables the ports to maintain a consistent order between different video frames through a multi-frame port sorting and new port insertion mechanism. The sorting of the ports detected for the first time is achieved by calculating the abscissa of the center point of the port, ensuring that the ports are in the same order from left to right and the order remains unchanged. The newly detected ports are inserted in sequence according to the order of appearance. This consistency is crucial for subsequent monitoring, data comparison, and system analysis, and helps to provide stable and reliable port monitoring results.

[0171] The method of this embodiment can optimize the processing of abnormal data. The introduced Y-axis or X-axis deviation abnormal port elimination mechanism sets a dynamic deviation threshold (threshold) to eliminate ports with position deviations exceeding the threshold. It effectively improves the robustness of target detection, reduces the interference of noise and false detections, thereby improving the accuracy of port status, and avoiding misjudgments caused by abnormal data.

[0172] The method of this embodiment compares the detected port status with the port information in the optical network resource management system. By comparing the port status in the detection results (such as occupied and idle status), it ensures that the detection results are consistent with the actual device status, thereby improving the accuracy and reliability of the resource management system. And the comparison results are stored and returned in the form of a text file, providing technical support for subsequent port status monitoring, management, and subsequent automated maintenance.

[0173] The method of this embodiment is not limited to the application of splitter ports, but can also be widely applied to the port monitoring and management of other optical network devices and communication devices. Its flexibility and efficiency enable this technical solution to be widely applied in different scenarios and requirements.

[0174] Please refer to Figure 8 , this embodiment of the application also provides an optical facility port management system based on target tracking, including:

[0175] The first module is used to obtain the port video stream of the target optical facility;

[0176] The second module is used to perform port target detection on each frame of the port video stream to obtain the detection results of each frame of the image. The detection results include multiple detection information about the ports;

[0177] A third module, configured to perform port target tracking based on the port detection results of multiple frames of images in the port video stream, and obtain a tracking identifier for the port in each frame of image;

[0178] A fourth module, configured to sort the optical facility ports according to the port detection results and the tracking identifiers of each frame of image, and obtain a video port list.

[0179] In some embodiments, the system further includes a fifth module, configured to, for each port in the current frame of image, determine adjacent ports according to the port sorting list of the current frame of image, calculate the position difference between the detection box position of the port and the detection box position of the adjacent port, and determine whether the port is in an abnormal position according to the position difference.

[0180] In some embodiments, the system further includes a sixth module, configured to receive a network resource comparison request, determine a video port list of a corresponding device according to the target comparison device, compare the device port information according to the video port list to obtain a comparison result, and return the comparison result.

[0181] It can be understood that the content in the above embodiments of the optical facility port management method based on target tracking is applicable to the embodiments of this system. The functions specifically implemented by the embodiments of this system are the same as those of the above embodiments of the optical facility port management method based on target tracking, and the beneficial effects achieved are also the same as those of the above embodiments of the optical facility port management method based on target tracking.

[0182] An embodiment of this application further provides an electronic device, including: a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for implementing connection communication between the processor and the memory. When the program is executed by the processor, it implements the above-mentioned optical facility port management method based on target tracking. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0183] Please refer to Figure 9 , Figure 9 , which schematically shows the hardware structure of an electronic device in another embodiment. The electronic device includes:

[0184] A processor 901, which can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of this application;

[0185] The memory 902 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 902 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 902, and are called by the processor 901 to execute the optical facility port management method based on target tracking in the embodiments of this application;

[0186] The input / output interface 903 is used to implement information input and output;

[0187] The communication interface 904 is used to implement communication interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or through wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0188] The bus 905 transmits information between the various components of the device (such as the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0189] Among them, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904 achieve communication connections with each other inside the device through the bus 905.

[0190] The embodiments of this application also provide a storage medium. The storage medium is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned optical facility port management method based on target tracking.

[0191] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory optionally includes a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0192] The optical facility port management method, device, electronic device, and storage medium provided by the embodiments of the present application collect the port video stream of the target optical facility, perform port target detection on each frame of the port video stream to obtain the detection result of each frame of the image, then perform port target tracking based on the port detection results of multiple frames of the image in the port video stream to obtain the tracking identifier of the port in each frame of the image, and sort the optical facility ports according to the port detection results and tracking identifiers of each frame of the image to obtain a video port list. By dynamically collecting the port video of the target optical facility, the embodiments of the present application can reduce the problem of port occlusion, and use a dynamic tracking algorithm to track the port, improving the accuracy of video port detection, thereby improving the accuracy of optical facility port management, and there is no need to set cameras in each viewing area of the optical facility device, reducing the cost of optical facility port management.

[0193] The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0194] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or combine certain steps, or different steps.

[0195] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0196] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations.

[0197] In the description of the present application and the above-mentioned accompanying drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0198] It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or its similar expression means any combination of these items, including any combination of single item (one) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0199] In several embodiments provided by the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the above-mentioned division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of systems or units can be in electrical, mechanical, or other forms.

[0200] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] In addition, in each embodiment of the present application, each functional unit can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0202] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: various media that can store programs such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0203] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings. This does not limit the scope of the rights of the embodiments of the present application. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A method for managing optical facility ports based on target tracking, characterized in that It includes the following steps: Obtain the port video stream of the target optical facility; Perform port target detection on each frame image in the port video stream to obtain the detection result of each frame image, where the detection result includes multiple pieces of detection information about the port; Perform port target tracking based on the port detection results of multiple frame images in the port video stream to obtain the tracking identifier of the port in each frame image; Sort the optical facility ports according to the port detection results and tracking identifiers of each frame image to obtain a video port list.

2. The optical facility port management method based on target tracking according to claim 1, wherein, The step of performing port target detection on each frame image in the port video stream to obtain the detection result of each frame image includes the following steps: Obtain a port target detection model trained based on the YOLO algorithm; Input each frame image in the port video stream into the port target detection model in sequence to obtain the detection result of each frame image; Among them, the detection result includes multiple pieces of detection information about the port, and the detection information includes the detection box position, status classification, and confidence level.

3. The method for managing optical facility ports based on target tracking according to claim 2, wherein, The step of performing port target tracking based on the port detection results of multiple frame images in the port video stream to obtain the tracking identifier of the port in each frame image includes the following steps: Select the detection information with a confidence level greater than the expected value according to the port detection result of the image, and determine the tracking port according to the selected detection information; Associate and match the tracking ports of the current frame image and the previous frame image according to the detection information to obtain a matching result; Judge whether the tracking port in the current frame image is successfully matched according to the matching result; When the tracking port is successfully matched, assign the tracking identifier of the port that matches in the previous frame image to the tracking port of the current frame image; When the tracking port fails to match, assign a new tracking identifier to the tracking port of the current frame image.

4. The optical facility port management method based on target tracking according to claim 3, wherein The step of sorting the optical facility ports according to the port detection results and tracking identifiers of each frame image to obtain a video port list includes the following steps: For each port in the current frame image, judge whether the tracking identifier of the port appears in the tracking identifiers of the historical frame images; When the tracking identifier of the port appears in the tracking identifiers of the historical frame images, determine the order of the port in the current frame image according to the port sorting list of the historical frame images; When the tracking identifier of the port does not appear in the tracking identifiers of the historical frame images, determine the adjacent ports in the opposite shooting direction in the current frame image according to the detection information of the port, and determine the order of the port according to the order of the adjacent ports; Determine the port sorting list of the current frame image according to the order of each port in the current frame image; Determine the video port list according to the port sorting lists of each frame image in the port video stream.

5. The method for managing an optical facility port based on target tracking according to claim 4, wherein The optical facility port management method based on target tracking further includes the following steps: For each port in the current frame image, determine the adjacent ports according to the port sorting list of the current frame image; Calculate the position gap between the detection box position of the port and the detection box positions of the adjacent ports; Determine whether the port is in an abnormal position according to the position gap.

6. The optical facility port management method based on target tracking according to claim 4, wherein Determining a video port list according to the port sorting list of each frame image in the port video stream includes the following steps: Merging each of the port sorting lists according to the order of each frame image in the port video stream and removing duplicates to obtain a video port list; Adding a port index to each port in the video port list in order and associating the detection information of the port with the corresponding port in the video port list.

7. The method for managing optical facility ports based on target tracking according to claim 6, wherein The optical facility port management method based on target tracking further includes the following steps: Receiving a network resource comparison request, where the network resource comparison request includes a target comparison device and device port information; Determining a video port list of the corresponding device according to the target comparison device; Comparing the device port information according to the video port list to obtain a comparison result and returning the comparison result.

8. An optical facility port management system based on target tracking, characterized in that Including: A first module for obtaining a port video stream of a target optical facility; A second module for performing port target detection on each frame image in the port video stream to obtain a detection result of each frame image, where the detection result includes multiple pieces of detection information about the port; A third module for performing port target tracking according to the port detection results of multiple frame images in the port video stream to obtain a tracking identifier about the port in each frame image; A fourth module for sorting the optical facility ports according to the port detection results and tracking identifiers of each frame image to obtain a video port list.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory. When the program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the steps of the method according to any one of claims 1 to 7.

Images