Optical cable cross connecting cabinet port identification method, device, equipment, medium and program product

Abstract

The invention discloses an optical cable cross connecting cabinet port identification method, device and equipment, a medium and a program product, and relates to the technical field of communication. The method comprises the following steps: collecting multiple frames of optical cable cross-connecting cabinet images; carrying out image fusion processing on the multiple frames of optical cable cross connecting cabinet images to obtain confidence of the multiple frames of optical cable cross connecting cabinet images and port states in the multiple frames of optical cable cross connecting cabinet images; and under the condition that the confidence coefficient of the optical cable cross connecting cabinet image meets a preset condition, determining a target port state of the optical cable cross connecting cabinet port based on the port states in the multiple frames of optical cable cross connecting cabinet images. By adopting the method provided by the embodiment of the invention, the overall robustness of the system can be improved, and the efficiency and accuracy of port identification of the optical cable cross connecting cabinet are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to an optical cross-connect box port identification method, device, equipment, medium and program product. Background Art

[0002] In the related art, the optical cross-connect box ports of the optical cross-connect box are responsible for the connection and distribution of optical fibers. The accurate update of the specifications and occupancy status of the optical cross-connect box ports is an important part of the production process. At present, the port status is usually identified by manually identifying them one by one on site and registering them in the background. However, this solution relies on manual labor, is time-consuming and prone to errors, resulting in low efficiency and accuracy in optical cross-connect box port identification. Summary of the invention

[0003] The present disclosure discloses a method, device, equipment, medium and program product for identifying an optical cross-connect box port. The technical solution of the present disclosure is as follows:

[0004] In a first aspect, the present disclosure discloses a method for identifying an optical cross-connect box port, comprising:

[0005] Collect multiple frames of optical cross-box images;

[0006] Performing image fusion processing on the multiple frames of optical cross-box images to obtain confidences of the multiple frames of optical cross-box images and port states in the multiple frames of optical cross-box images;

[0007] When the confidence of the optical cross-box image meets a preset condition, the target port state of the optical cross-box port is determined based on the port states in the multiple frames of optical cross-box images.

[0008] In a possible implementation, the image fusion processing includes at least one of decision-level fusion, feature-level fusion, and data-level fusion;

[0009] The performing image fusion processing on the multiple frames of optical cross-box images to obtain the confidence of the multiple frames of optical cross-box images and the port status in the multiple frames of optical cross-box images includes:

[0010] Performing decision-level fusion processing on the multiple frames of optical cross-box images to obtain a first confidence and a first port state of a port in the optical cross-box images;

[0011] When the first confidence level is less than a first threshold, performing feature-level fusion processing on the multiple frames of optical cross-box images to obtain a second confidence level and a second port state of a port in the optical cross-box image;

[0012] When the second confidence level is less than the second threshold, data level fusion processing is performed on the multiple frames of optical cross-box images to obtain a third confidence level and a third port state of the port in the optical cross-box image.

[0013] In a possible implementation, performing decision-level fusion processing on the multiple frames of optical cross-box images to obtain a first confidence and a first port state of a port in the optical cross-box images includes:

[0014] For each frame of the optical cross-box image, determine a first confidence of each frame of the optical cross-box image and a first port state of a port in each frame of the optical cross-box image;

[0015] The step of determining the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images when the confidence of the optical cross-box image meets a preset condition includes:

[0016] If the proportion of confidences greater than or equal to the first threshold in the first confidences is greater than a preset ratio, it is determined that the confidence of the optical cross-box image meets a preset condition;

[0017] When the confidence of the optical cross-box image meets a preset condition, the target port state of the optical cross-box port is determined based on the first port state of the port in each frame of the optical cross-box image by a voting method.

[0018] In a possible implementation, when the first confidence is less than a first threshold, performing feature-level fusion processing on the multiple frames of optical cross-box images to obtain a second confidence and a second port state of a port in the optical cross-box image includes:

[0019] When the first confidence level is less than a first threshold, extracting feature information of each frame of the optical cross-box image;

[0020] Performing image recognition processing on feature information of each frame of the optical cross-box image through a pulse coupled neural network model to obtain a fused image;

[0021] determining a second confidence level of the fused image, and determining a second port state of a port in the optical cross-box image based on the fused image;

[0022] The step of determining the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images when the confidence of the optical cross-box image meets a preset condition includes:

[0023] When the second confidence of the fused image is greater than or equal to the second threshold, the second port state is determined as the target port state of the optical cross-box port.

[0024] In a possible implementation, when the second confidence level is less than a second threshold, performing data-level fusion processing on the multiple frames of optical cross-box images to obtain a third confidence level and a third port state of the port in the optical cross-box image includes:

[0025] When the second confidence level is less than a second threshold, obtaining an optical cross-box port model based on the multi-frame optical cross-box image recognition by means of line number recognition;

[0026] Performing wavelet transform on the optical cross-box port model to obtain a fused image;

[0027] determining a third confidence level of the fused image, and determining the third port state based on the fused image;

[0028] The step of determining the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images when the confidence of the optical cross-box image meets a preset condition includes:

[0029] When the third confidence level is greater than or equal to the third threshold, the third port state is determined as the target port state of the optical cross-connect box port.

[0030] In a possible implementation, it further includes:

[0031] When the confidence of the optical cross-box image does not meet the preset condition, a prompt message is output; wherein the prompt message is used to instruct to re-shoot the optical cross-box image.

[0032] In a second aspect, the present disclosure discloses an optical cross-connect box port identification device, comprising:

[0033] An acquisition module, used for acquiring multiple frames of optical cross-box images;

[0034] A fusion module, used for performing image fusion processing on the multiple frames of optical cross-box images to obtain the confidence of the multiple frames of optical cross-box images and the port status in the multiple frames of optical cross-box images;

[0035] The identification module is used to determine the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images when the confidence of the optical cross-box image meets a preset condition.

[0036] In a third aspect, the present disclosure discloses an electronic device, including:

[0037] processor;

[0038] a memory for storing instructions executable by the processor;

[0039] The processor is configured to execute the instructions to implement the method described in the first aspect.

[0040] In a fourth aspect, the present disclosure discloses a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method described in the first aspect when executed by a processor.

[0041] In a fifth aspect, the present disclosure discloses a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the method described in the first aspect is implemented.

[0042] The technical solution disclosed in this disclosure brings at least the following beneficial effects:

[0043] In the technical solution disclosed in the present disclosure, multiple frames of optical cross-box images are collected; image fusion processing is performed on the multiple frames of optical cross-box images to obtain the confidence of the multiple frames of optical cross-box images and the port status in the multiple frames of optical cross-box images; when the confidence of the optical cross-box images meets the preset conditions, the target port status of the optical cross-box port is determined based on the port status in the multiple frames of optical cross-box images. In this way, the multiple frames of the optical cross-box images can be fused at multiple levels to determine the port status of the optical cross-box port, thereby effectively improving the effectiveness and accuracy of the acquisition, thereby improving the overall robustness of the system, and improving the efficiency and accuracy of optical cross-box port identification.

[0044] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0046] Figure 1 A schematic diagram of a flow chart of a method for identifying an optical cross-connect box port provided in an embodiment of the present disclosure;

[0047] Figure 2 is a schematic diagram of an implementation of a decision-level fusion process provided by an embodiment of the present disclosure;

[0048] Figure 3 is a schematic diagram of an implementation of a feature-level fusion process provided by an embodiment of the present disclosure;

[0049] Figure 4 is a schematic diagram of an ignition map provided by an embodiment of the present disclosure;

[0050] Figure 5It is a schematic diagram of an execution system of an optical cross-connect box port identification method provided by an embodiment of the present disclosure;

[0051] Figure 6 is a schematic diagram of an implementation process of a multi-frame image fusion module provided in an embodiment of the present disclosure;

[0052] Figure 7 A schematic diagram of a flow chart of a method for identifying an optical cross-connect box port provided in an embodiment of the present disclosure;

[0053] Figure 8 It is a structural schematic diagram of an optical cross-connect box port identification device provided by an embodiment of the present disclosure;

[0054] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0055] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0057] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0058] The acquisition, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations.

[0059] It should be noted that in the embodiments of the present disclosure, there may be existing solutions in the industry such as certain software, components, models, etc., which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present disclosure, but it does not mean that the applicant has or will necessarily use the solution.

[0060] In the related technology, the optical cross-connection box terminal is responsible for the connection and distribution of optical fibers in the optical cross-connection box. The accurate update of the specifications and occupancy status of the optical cross-connection box terminal is an important part of the production process. At present, the following methods are usually used to maintain the resource status: (1) Manual on-site identification and registration to the background. This solution is currently widely used, which is time-consuming and prone to manual errors; (2) Transforming the optical cross-connection box into an active optical cross-connection box and updating the resource status through optical path information transmission. This solution has a high transformation cost and requires ensuring that the power supply is always on; (3) Using passive tags such as RFID (Radio-Frequency Identification). This solution is low-cost but requires personnel to scan one by one on site; (4) Using image recognition. This solution is affected by the on-site environment and has a low accuracy rate.

[0061] Based on this, in the related technology, there is still a method of using a single frame image to identify the port status of the optical cross-connect box, specifically by taking photos to perform photo image recognition. However, due to the complex actual environment of the optical cross-connect box in the existing network, including the influence of light intensity in open-air places, and the serious obstruction of the terminal by the pigtail, when performing single frame image recognition, most of the existing network scenes cannot be recognized or the recognition accuracy is low.

[0062] Therefore, this proposal provides a method, device, equipment, medium and program product for optical cross-box port identification, which can perform multi-level image fusion of multiple frame images of the optical cross-box to determine the port status of the optical cross-box port, thereby effectively improving the effectiveness and accuracy of acquisition, and then improving the overall robustness of the system and improving the efficiency and accuracy of optical cross-box port identification.

[0063] The technical solutions provided by various embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings.

[0064] Figure 1 The present invention provides a flowchart of a method for identifying an optical cross-connect box port. The method can be applied to a server, such as a single server or a server cluster. Figure 1 As shown, the optical cross-connect box port identification method may include the following steps:

[0065] S101, collecting multiple frames of optical cross-box images.

[0066] In the embodiment of the present disclosure, when executing the optical cross-box port identification method, the optical cross-box image can be obtained first, and the optical cross-box image is a plurality of frames. Exemplarily, the plurality of frames of the optical cross-box image can be collected by video recording. It is understandable that human-computer interaction can also be used at the same time, and VR (Virtual Reality) can be used to prompt the on-site operator on the screen how to operate in order to improve the effectiveness of the collection.

[0067] S102, performing image fusion processing on multiple frames of optical cross-box images to obtain confidences of the multiple frames of optical cross-box images and port states in the multiple frames of optical cross-box images.

[0068] In an embodiment of the present disclosure, after collecting multiple frames of optical cross-box images of an optical cross-box, image fusion processing can be performed on the multiple frames of optical cross-box images. For example, image fusion can include one or more fusion processing such as decision-level fusion, feature-level fusion, and data-level fusion. After the image fusion processing, the confidence of the multiple frames of optical cross-box images and the port status in the multiple frames of optical cross-box images can be obtained. Exemplarily, the confidence can be used to indicate the degree of confidence in the port status of the identified optical cross-box image, for example, the confidence can be expressed in percentage form or decimal form; the port status can be in the form of a status value. As a specific example, in the port status recognition result, the status values ​​of 0, 1, 2, 3, and 6 can be used to indicate that the port is idle and available, where 0, 1, and 2 represent red dust caps, transparent dust caps, and white dust caps, respectively, 3 and 4 represent iron flanges without dust caps and plastic flanges without dust caps, respectively, and the status values ​​of 4 and 7 can be used to indicate that the port is idle and damaged, and the status value of 5 can be used to indicate that the port is occupied.

[0069] S103, when the confidence of the optical cross-box image meets a preset condition, determine the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images.

[0070] In an embodiment of the present disclosure, after obtaining the confidence of a multi-frame optical cross-box image and the port status in the multi-frame optical cross-box image, it can be determined whether the confidence of the optical cross-box image meets the preset conditions. If the confidence of the optical cross-box image does not meet the preset conditions, it can be considered that the recognition result is questionable and the accuracy is insufficient, and it is necessary to re-collect the multi-frame optical cross-box image for port status recognition. On the contrary, if the confidence of the optical cross-box image meets the preset conditions, it is considered that the recognition result is credible and has high accuracy, and the port status of the optical cross-box port identified in the previous step can be determined as the final port status of the optical cross-box port (that is, the target port status).

[0071] In the technical solution disclosed in the present disclosure, multiple frames of optical cross-box images are collected; image fusion processing is performed on the multiple frames of optical cross-box images to obtain the confidence of the multiple frames of optical cross-box images and the port status in the multiple frames of optical cross-box images; when the confidence of the optical cross-box images meets the preset conditions, the target port status of the optical cross-box port is determined based on the port status in the multiple frames of optical cross-box images. In this way, the multiple frames of the optical cross-box images can be fused at multiple levels to determine the port status of the optical cross-box port, thereby effectively improving the effectiveness and accuracy of the acquisition, thereby improving the overall robustness of the system, and improving the efficiency and accuracy of optical cross-box port identification.

[0072] In some possible implementations, the image fusion processing includes at least one of decision-level fusion, feature-level fusion, and data-level fusion;

[0073] Accordingly, the multi-frame optical cross-box images are subjected to image fusion processing to obtain the confidence of the multi-frame optical cross-box images and the port status in the multi-frame optical cross-box images, including:

[0074] Performing decision-level fusion processing on multiple frames of optical cross-box images to obtain a first confidence and a first port state of a port in the optical cross-box image;

[0075] When the first confidence is less than the first threshold, performing feature-level fusion processing on the multiple frames of optical cross-box images to obtain a second confidence and a second port state of the port in the optical cross-box image;

[0076] When the second confidence level is less than the second threshold, data level fusion processing is performed on the multiple frames of optical cross-box images to obtain a third confidence level and a third port state of the port in the optical cross-box image.

[0077] In the embodiments of the present disclosure, a single-level or multi-level method can be adaptively used to fuse multiple frames of images according to the quality of multiple frames of optical cross-box images. For a certain port, decision-level fusion can be used first, followed by feature-level fusion, and finally data-level fusion. From the perspective of the entire optical cross-box, different ports can adaptively use different levels for fusion decisions according to different data qualities. Exemplarily, when performing image fusion processing on multiple frames of optical cross-box images, decision-level fusion processing can be performed on the multiple frames of optical cross-box images first to obtain decision-level fusion results, including a first confidence level and a first port state. Decision-level fusion can infer multiple single-frame images in parallel, with high real-time performance. However, it requires that the inference results of single-frame images have high credibility, so the robustness is low. It is suitable for identification in the early stage of optical cross-box construction, when the on-site environment is good and there is little fiber pigtail occlusion.

[0078] Then, it can be determined whether the first confidence is greater than or equal to the preset threshold (first threshold). If the first confidence is less than the first threshold, it can be considered that the recognition result is questionable and further fusion recognition is required. At this time, the multi-frame optical cross-box image can be subjected to feature-level fusion processing to obtain the recognition result of the feature-level fusion processing, including the second confidence and the second port state. Feature-level fusion is based on the original multi-frame optical cross-box single-frame image. After feature extraction, the extracted features are judged and selected. In this way, different areas of the multi-frame image of the optical cross-box can select the clearest image by feature. Therefore, this method has moderate real-time and robustness, and is a multi-frame image fusion method that is more suitable for most existing optical cross-boxes.

[0079] Afterwards, it can be determined whether the second confidence is greater than or equal to the preset threshold (second threshold). If the second confidence is less than the second threshold, it can be considered that the recognition result is questionable and further fusion recognition is required. At this time, the multi-frame optical cross-box image can be processed at the data level to obtain the recognition result of the data-level fusion processing, including the third confidence and the third port state. Data-level fusion requires image registration of all single-frame images, and then undergoes wavelet transform-fusion-inverse wavelet transform. The processing time is long and the real-time performance is poor, but the robustness is high. It is suitable for optical cross-box terminals in harsh environments that cannot be effectively identified by the first two methods.

[0080] In a further possible implementation, performing decision-level fusion processing on multiple frames of optical cross-box images to obtain a first confidence and a first port state of a port in the optical cross-box images includes:

[0081] For each frame of the optical cross-box image, determining a first confidence of each frame of the optical cross-box image and a first port state of a port in each frame of the optical cross-box image;

[0082] When the confidence of the optical cross-box image meets a preset condition, determining the target port state of the optical cross-box port based on the port states in the multiple frames of the optical cross-box image includes:

[0083] If the proportion of confidence levels greater than or equal to the first threshold in the first confidence levels is greater than a preset ratio, it is determined that the confidence level of the optical cross-box image meets a preset condition;

[0084] When the confidence of the optical cross-box image meets a preset condition, the target port state of the optical cross-box port is determined based on the first port state of the port in each frame of the optical cross-box image by a voting method.

[0085] In an embodiment of the present disclosure, the decision-level fusion processing can be combined with the voting method to determine the target port state of the optical cross-box port. Exemplarily, for each frame of the optical cross-box image, the feature information of the single-frame image is used as input, and the single-frame image recognition reasoning is used to determine the first confidence and the first port state of each frame of the optical cross-box image, that is, each frame of the optical cross-box image corresponds to a set of first confidence and first port states. Then, the confidence ratio greater than or equal to the first threshold in the first confidence can be counted, and the confidence ratio is compared with the preset ratio to determine whether the confidence ratio is greater than the preset ratio. The preset ratio can be set to 80%, 85%, etc., and the first threshold can be set to 0.7, 0.8, etc. If the confidence ratio greater than or equal to the first threshold in the first confidence is greater than the preset ratio, the recognition result can be considered to be credible, can be used for voting, and meet the preset conditions. On the contrary, if the confidence ratio greater than or equal to the first threshold in the first confidence is less than or equal to the preset ratio, it is considered that the recognition result is doubtful and the optical cross-box image needs to be re-collected for recognition. Afterwards, the target port state of the optical cross-box port is determined based on the first port state of the port in each frame of the optical cross-box image by a voting method. Figure 2 , taking 3 frames of optical cross-box images, a preset ratio of 80%, and a first threshold of 0.7 as an example, a single-frame image inference judgment is performed for each frame of the optical cross-box image, and the confidence and judgment result (that is, the first confidence and the first port state) of each frame of the optical cross-box image are obtained, and then it is determined whether the confidence ratio greater than or equal to 0.7 is greater than or equal to 80%. If so, the voting method is entered to determine the final judgment result; otherwise, the next level of data-level fusion processing is entered. Combined with Table 1, Table 1 schematically shows two specific examples. It can be seen that Example 1 meets the preset conditions, and the judgment result obtained by voting can be used as the final recognition result; Example 2 does not meet the preset conditions, does not enter the voting link, and enters the next level of feature-level fusion processing.

[0086] Table 1

[0087]

[0088] In a further possible implementation, when the first confidence is less than the first threshold, performing feature-level fusion processing on multiple frames of optical cross-box images to obtain a second confidence and a second port state of the port in the optical cross-box image includes:

[0089] When the first confidence level is less than a first threshold, extracting feature information of each frame of the optical cross-box image;

[0090] The pulse coupled neural network model is used to perform image recognition processing on the characteristic information of each frame of the optical cross-box image to obtain a fused image;

[0091] determining a second confidence level of the fused image, and determining a second port state of the port in the optical cross-box image based on the fused image;

[0092] When the confidence of the optical cross-box image meets a preset condition, determining the target port state of the optical cross-box port based on the port states in the multiple frames of the optical cross-box image includes:

[0093] When the second confidence of the fused image is greater than or equal to the second threshold, the second port state is determined as the target port state of the optical cross-box port.

[0094] In an embodiment of the present disclosure, if the first confidence level is less than the first threshold, the feature information of each frame of the optical cross-box image can be extracted, and the feature information of each frame of the optical cross-box image is subjected to image fusion processing by a simplified pulse coupled neural network (SPCNN) to obtain a fused image. The fused image is subjected to recognition processing to obtain a second confidence level and a second port state. The second confidence level is then compared with the second threshold to determine whether the second confidence level is greater than or equal to the second threshold. If the second confidence level is greater than or equal to the second threshold, the judgment result is considered to be credible, and the second port state is determined as the target port state of the optical cross-box port.

[0095] As an example, feature-level fusion can extract feature information from the original image and use a parameter-adaptive simplified pulse coupled neural network (SPCNN) for multi-frame image recognition. Taking the single-frame image of the light intersection of frames A and B as an example of feature-level fusion, the specific implementation method is as follows: the multi-frame image can take the fused image F and the next frame image as input and iterate. Figure 3 , the implementation process of feature-level fusion includes the following processing:

[0096] Step 1: Use the grayscale pixel value of the fused image at (i, j) as the external stimulus for the SPCNN neuron to stimulate each neuron. Step 2: Combine Figure 4 After the ignition of SPCNN, the ignition map corresponding to the single-frame image (optical cross-box image) is obtained by SPCNN calculation; the (i, j) grayscale pixel value of the single-frame image of the optical cross-box is used as the external stimulus for the forced excitation of the neuron When the neuron's internal activity term U i,j Greater than or equal to the dynamic threshold θ ij When , the neuron generates a pulse, that is, 1 ignition. The more ignition times, the clearer the (i, j) area is. Step 3, determine the coefficient source of the fused image through the decision selection operator to generate the image fusion feature value. The decision selection operator uses the formula Among them, I F,ij , IA,ij , I B,ij Represent the pixel grayscale values ​​of the fused image F and the single-frame images A and B of the optical cross-box, T A,ij (n) and T B,ij (n) represents the ignition map corresponding to the single-frame images A and B of the optical cross-box after n SPCNN iterations. Since the image with more ignition times is a clearer image, the pixel image with a larger operator is selected as the pixel value corresponding to the fused image by judgment, and the effectiveness of the feature value can be improved by using multiple frames of images. Step 4, the image fusion feature value after feature-level fusion is sent to the result judgment module for judgment. If the confidence is greater than or equal to 0.7, it means that the judgment result is reliable, and the judgment result is output as the final judgment result; if the confidence is less than 0.7, it means that the judgment result is unreliable, and data-level fusion will be used for processing. In this way, feature-level fusion is based on the original single-frame image of the optical cross-box, and feature extraction is first performed, and then the extracted features are judged and selected. Therefore, different areas of the multi-frame image of the optical cross-box can select the clearest image by feature. Therefore, this method has moderate real-time and robustness, and is a multi-frame image fusion method that is more suitable for most existing optical cross-boxes.

[0097] In a further possible implementation, when the second confidence level is less than the second threshold, performing data-level fusion processing on multiple frames of optical cross-box images to obtain a third confidence level and a third port state of the port in the optical cross-box image includes:

[0098] When the second confidence level is less than the second threshold, obtaining the optical cross-box port model based on the multi-frame optical cross-box image recognition by means of line number recognition;

[0099] Perform wavelet transform on the optical cross-box port model to obtain a fused image;

[0100] determining a third confidence level of the fused image, and determining a third port state based on the fused image;

[0101] When the confidence of the optical cross-box image meets a preset condition, determining the target port state of the optical cross-box port based on the port states in the multiple frames of the optical cross-box image includes:

[0102] When the third confidence level is greater than or equal to the third threshold, the third port state is determined as the target port state of the optical cross-connect box port.

[0103] In an embodiment of the present disclosure, if the second confidence is less than the second threshold, the optical cross-box port model can be obtained based on multi-frame optical cross-box image recognition by row number recognition, for example, the input optical cross-box image can be used to identify a standard N-row 12-column optical cross-box terminal model by row number recognition. Then, the optical cross-box port model can be subjected to wavelet transformation to carry out data-level fusion recognition, to obtain a third confidence of the fused image, and to determine the third port state based on the fused image. Afterwards, the third confidence can be compared with the third threshold to determine whether the third confidence is greater than or equal to the third threshold. If the third confidence is greater than or equal to the third threshold, the recognition result is considered to be credible, and the third port state can be determined as the target port state of the optical cross-box port.

[0104] As an example, data-level fusion first identifies the input image into a standard N-row 12-column optical cross-box terminal model by row number recognition, then unifies the objects and pixels of multiple frames of images by image cutting, and finally uses the wavelet transform algorithm to carry out data-level fusion recognition. The data-level fusion of multiple frames of optical cross-box images includes several steps, such as image preprocessing, wavelet transform, image fusion, and wavelet inverse transform. Specifically, step 1: image preprocessing is to identify the input optical cross-box image into a standard N-row 12-column optical cross-box terminal model by row number recognition, including row division and column determination. Step 2: for the registration image obtained in step 1, wavelet transform is carried out to obtain low-frequency components and high-frequency components. Step 3: for optical cross-box terminals, the low-frequency component is the global feature of the image, and the average value is used for fusion. The high-frequency component is the local feature of the image. The feature is particularly obvious at the edge of the terminal, and the maximum value is used for fusion. Step 4: a higher-definition image is obtained by inverse wavelet transform, and the fused image is sent to the result judgment module for feature extraction and reasoning judgment. If the judgment confidence is greater than or equal to 0.7, the judgment result is considered reliable; if the judgment confidence is less than 0.7, the judgment result is considered unreliable. At this time, the three levels of fusion cannot obtain reliable judgment results, indicating that the terminal is severely blocked, and it is necessary to prompt the on-site personnel to tidy up the pigtails through human-computer interaction and re-collect multiple frames of images. In this way, data-level fusion requires image registration of all single-frame images, and then undergoes wavelet transform-fusion-inverse wavelet transform. The processing time is long and the real-time performance is poor, but the robustness is high. It is suitable for optical cross-box terminals in harsh environments that cannot be effectively identified by the first two methods.

[0105] In a possible implementation manner, the optical cross-connect box port identification method further includes:

[0106] When the confidence of the optical cross-box image does not meet a preset condition, a prompt message is output; wherein the prompt message is used to instruct to re-shoot the optical cross-box image.

[0107] In the embodiment of the present disclosure, if the confidence of the optical cross-box image does not meet the preset conditions, it is necessary to prompt the on-site personnel through human-computer interaction to adjust the angle, remove the blocked pigtail, etc. At this time, prompt information can be output so that the staff can re-shoot the optical cross-box image according to the prompt information to further improve the effectiveness of image acquisition and increase the confidence.

[0108] In order to make the optical cross-box port identification method provided by the embodiment of the present disclosure clearer, the optical cross-box port identification method provided by the embodiment of the present disclosure is described below with reference to specific examples.

[0109] Combination Figure 5 , the method for realizing optical cross-box port identification based on multi-frame image fusion proposed in the embodiment of the present disclosure, its execution system may include a multi-frame image acquisition module, a multi-frame image fusion module, a result judgment module and a human-computer interaction feedback module. The main implementation process is: the multi-frame image acquisition module collects multi-frame optical cross-box image information through the terminal, and sends the multi-frame optical cross-box image information to the multi-frame image fusion module. After three-layer fusion of data-level fusion, feature-level fusion and decision-level fusion, the confidence result of image recognition is output. The result judgment module classifies and identifies the ports according to the confidence, and at the same time, feedback is given to the human-computer interaction module for the ports that cannot be correctly classified. The human-computer interaction module further guides the on-site personnel to improve the effectiveness of image acquisition by adjusting the acquisition angle and removing the obstructed pigtails, and collects more multi-frame image data for fusion and recognition processing. Among them:

[0110] 1. Multi-frame image acquisition module: This module is different from the existing method of photographing and collecting images. It uses video recording to collect multiple frames of images. At the same time, with the help of the human-computer interaction module, VR is used to prompt on-site operators on the screen on how to operate in order to improve the effectiveness of the acquisition.

[0111] 2. Multi-frame image fusion module: Combination Figure 6 , this module is divided into three levels: data-level fusion, feature-level fusion, and decision-level fusion. According to the quality of image acquisition, a single-level or multi-level method can be adaptively used to fuse multiple frames of images. For a certain port, decision-level fusion is preferred, followed by feature-level fusion, and finally data-level fusion. From the perspective of the entire optical cross-box, different ports may adaptively use different levels for fusion decisions according to different data quality. If the three-level fusion cannot achieve a judgment result with a confidence level greater than or equal to 0.7, the system considers that the port cannot be identified temporarily, and the human-computer interaction module needs to be used to remind the on-site personnel to take the approach of arranging the pigtails and reshooting, and reacquire multiple frames of images to carry out fusion identification again.

[0112] Combine the following Figure 7The optical cross-connect box port identification method provided by the embodiment of the present disclosure is described with a specific example as follows:

[0113] (1) Decision-level fusion belongs to high-level fusion, and its input is the feature information of a single-frame image (optical cross-box image). The commonly used single-frame image recognition reasoning is adopted. If the confidence of the single-frame image recognition reasoning is greater than 0.7, the reasoning judgment result is considered to be credible and can be used for voting; if the confidence is less than or equal to 0.7, the reasoning result is considered to be unreliable and cannot be used for voting. Thus, multiple frames of images will obtain multiple confidences and reasoning judgment results. Whether the decision-level fusion is applicable is determined based on whether the proportion of single-frame images with a confidence greater than 0.7 is greater than 80%. If applicable, the voting method is used for judgment. If not applicable, data-level fusion is performed. The specific implementation of decision-level fusion has been described in the above method embodiment and will not be repeated here.

[0114] (2) Feature-level fusion: extract feature information from the original image, and use a parameter-adaptive simplified pulse coupled neural network (SPCNN) to perform multi-frame image recognition. The multi-frame image can use the fused image F and the next frame image as input for iteration. The specific implementation of feature-level fusion has been described in the above method embodiment and will not be repeated here.

[0115] (3) Data-level fusion: First, the input image is identified by row number recognition to obtain a standard N-row 12-column optical cross-connect box terminal model, and then the objects and pixels of multiple frames of images are unified by image cutting. Finally, the wavelet transform algorithm is used to perform data-level fusion recognition. The data-level fusion of multiple frames of optical cross-connect box images includes several steps, such as image preprocessing, wavelet transform, image fusion, and wavelet inverse transform. The specific implementation of data-level fusion has been described in the above method embodiment and will not be repeated here.

[0116] 3. Result judgment module: The discrete value of the judgment result is used as the port status value, and the confidence reflects the reliability of the result. In the port status recognition result, the output results of 0, 1, 2, 3, and 6 are that the port is idle and available, where 0, 1, and 2 represent red, transparent, and white dust caps, respectively, 3 and 4 represent iron flanges without dust caps and plastic flanges without dust caps, respectively, 4 and 7 output results that the port is idle and damaged, and 5 outputs The result is that the port is occupied. In the recognition result, if the confidence is less than 0.7, it means that the result is questionable, and it is necessary to re-shoot multiple frames of images and re-recognize through human-computer interaction, such as arranging the pigtail, to further improve the confidence. If the confidence is greater than or equal to 0.7, the result is considered credible, and the port recognition result is locked. The port with the locked result will no longer change the judgment result in the next recognition of human-computer interaction.

[0117] 4. Human-computer interaction module: For ports with a confidence level less than 0.7 in the result judgment module, it means that the result is questionable. The human-computer interaction module prompts on-site personnel to adjust the angle, remove obstructing pigtails, and other operations to further improve the effectiveness of image acquisition.

[0118] It can be seen that the embodiment of the present disclosure adopts multi-frame image fusion, which helps to improve the recognition accuracy in complex environments through multi-dimensional fusion at three levels: data-level fusion, feature-level fusion, and decision-level fusion. At the same time, an artificial interaction module with a feedback path is embedded in the system to improve the effectiveness of image acquisition, thereby further improving the recognition accuracy and speed, and improving the overall robustness of the system.

[0119] The specific implementation and technical effects of each step of this embodiment are similar to those of the above method embodiments, and will not be repeated here.

[0120] Based on the same inventive concept, the embodiment of the present disclosure also provides an optical cross-connect box port identification device. Figure 8 As shown, the optical cross-connect box port identification device 800 includes:

[0121] An acquisition module 810 is used to acquire multiple frames of optical cross-box images;

[0122] A fusion module 820 is used to perform image fusion processing on the multiple frames of optical cross-box images to obtain the confidence of the multiple frames of optical cross-box images and the port status in the multiple frames of optical cross-box images;

[0123] The identification module 830 is used to determine the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images when the confidence of the optical cross-box image meets a preset condition.

[0124] In a possible implementation, the image fusion processing includes at least one of decision-level fusion, feature-level fusion, and data-level fusion;

[0125] The fusion module 820 includes:

[0126] A first fusion unit is used to perform decision-level fusion processing on the multiple frames of optical cross-box images to obtain a first confidence and a first port state of a port in the optical cross-box image;

[0127] A second fusion unit is used to perform feature-level fusion processing on the multiple frames of optical cross-box images to obtain a second confidence and a second port state of the port in the optical cross-box image when the first confidence is less than a first threshold;

[0128] The third fusion unit is used to perform data-level fusion processing on the multiple frames of optical cross-box images when the second confidence is less than a second threshold, so as to obtain a third confidence and a third port state of the port in the optical cross-box image.

[0129] In a possible implementation manner, the first fusion unit is used to:

[0130] For each frame of the optical cross-box image, determine a first confidence of each frame of the optical cross-box image and a first port state of a port in each frame of the optical cross-box image;

[0131] The identification module 830 is used to:

[0132] If the proportion of confidences greater than or equal to the first threshold in the first confidences is greater than a preset ratio, it is determined that the confidence of the optical cross-box image meets a preset condition;

[0133] When the confidence of the optical cross-box image meets a preset condition, the target port state of the optical cross-box port is determined based on the first port state of the port in each frame of the optical cross-box image by a voting method.

[0134] In a possible implementation manner, the second fusion unit is used to:

[0135] When the first confidence level is less than a first threshold, extracting feature information of each frame of the optical cross-box image;

[0136] Performing image recognition processing on feature information of each frame of the optical cross-box image through a pulse coupled neural network model to obtain a fused image;

[0137] determining a second confidence level of the fused image, and determining a second port state of a port in the optical cross-box image based on the fused image;

[0138] The identification module 830 is used to:

[0139] When the second confidence of the fused image is greater than or equal to the second threshold, the second port state is determined as the target port state of the optical cross-box port.

[0140] In a possible implementation manner, the third fusion unit is used to:

[0141] When the second confidence level is less than a second threshold, obtaining an optical cross-box port model based on the multi-frame optical cross-box image recognition by means of line number recognition;

[0142] Performing wavelet transform on the optical cross-box port model to obtain a fused image;

[0143] determining a third confidence level of the fused image, and determining the third port state based on the fused image;

[0144] The identification module 830 is used to:

[0145] When the third confidence level is greater than or equal to the third threshold, the third port state is determined as the target port state of the optical cross-connect box port.

[0146] In a possible implementation, it further includes:

[0147] The prompt module is used to output a prompt message when the confidence of the optical cross-box image does not meet the preset condition; wherein the prompt message is used to instruct to re-shoot the optical cross-box image.

[0148] The specific implementation method and technical effect of the device provided in the embodiment of the present disclosure are similar to those of the above-mentioned method embodiment, and will not be repeated here.

[0149] According to an embodiment of the present disclosure, the present disclosure also discloses an electronic device, a computer-readable storage medium, and a computer program product.

[0150] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 900 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0151] like Fig. 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0152] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the electronic device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0153] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the optical cross-box port identification method. For example, in some embodiments, the optical cross-box port identification method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the optical cross-box port identification method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the optical cross-box port identification method in any other appropriate manner (e.g., by means of firmware).

[0154] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0155] The program code of the computer program product for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0156] In the context of the present disclosure, a computer-readable storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may be a machine-readable signal medium or a machine-readable storage medium. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0158] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0159] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0160] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0161] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for identifying an optical cross-connect box port, characterized in that: include: Collect multiple frames of optical cross-box images; Performing image fusion processing on the multiple frames of optical cross-box images to obtain confidences of the multiple frames of optical cross-box images and port states in the multiple frames of optical cross-box images; When the confidence of the optical cross-box image meets a preset condition, the target port state of the optical cross-box port is determined based on the port states in the multiple frames of optical cross-box images.

2. The optical cross-connect box port identification method according to claim 1, characterized in that: The image fusion processing includes at least one of decision-level fusion, feature-level fusion, and data-level fusion; The performing image fusion processing on the multiple frames of optical cross-box images to obtain the confidence of the multiple frames of optical cross-box images and the port status in the multiple frames of optical cross-box images includes: Performing decision-level fusion processing on the multiple frames of optical cross-box images to obtain a first confidence and a first port state of a port in the optical cross-box images; When the first confidence level is less than a first threshold, performing feature-level fusion processing on the multiple frames of optical cross-box images to obtain a second confidence level and a second port state of a port in the optical cross-box images; When the second confidence level is less than a second threshold, data level fusion processing is performed on the multiple frames of optical cross-box images to obtain a third confidence level and a third port state of the port in the optical cross-box image.

3. The optical cross-connect box port identification method according to claim 2, characterized in that: The performing decision-level fusion processing on the multiple frames of optical cross-box images to obtain a first confidence and a first port state of a port in the optical cross-box images includes: For each frame of the optical cross-box image, determine a first confidence of each frame of the optical cross-box image and a first port state of a port in each frame of the optical cross-box image; The step of determining the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images when the confidence of the optical cross-box image meets a preset condition includes: If the proportion of confidences greater than or equal to the first threshold in the first confidences is greater than a preset ratio, it is determined that the confidence of the optical cross-box image meets a preset condition; When the confidence of the optical cross-box image meets a preset condition, the target port state of the optical cross-box port is determined based on the first port state of the port in each frame of the optical cross-box image by a voting method.

4. The optical cross-connect box port identification method according to claim 2, characterized in that: When the first confidence level is less than a first threshold, performing feature-level fusion processing on the multiple frames of optical cross-box images to obtain a second confidence level and a second port state of a port in the optical cross-box images includes: When the first confidence level is less than a first threshold, extracting feature information of each frame of the optical cross-box image; Performing image recognition processing on feature information of each frame of the optical cross-box image through a pulse coupled neural network model to obtain a fused image; determining a second confidence level of the fused image, and determining a second port state of a port in the optical cross-box image based on the fused image; The step of determining the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images when the confidence of the optical cross-box image meets a preset condition includes: When the second confidence of the fused image is greater than or equal to the second threshold, the second port state is determined as the target port state of the optical cross-box port.

5. The optical cross-connect box port identification method according to claim 2, characterized in that: When the second confidence level is less than a second threshold, performing data-level fusion processing on the multiple frames of optical cross-box images to obtain a third confidence level and a third port state of the port in the optical cross-box image includes: When the second confidence level is less than a second threshold, obtaining an optical cross-box port model based on the multi-frame optical cross-box image recognition by means of line number recognition; Performing wavelet transform on the optical cross-box port model to obtain a fused image; determining a third confidence level of the fused image, and determining the third port state based on the fused image; The step of determining the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images when the confidence of the optical cross-box image meets a preset condition includes: When the third confidence level is greater than or equal to the third threshold, the third port state is determined as the target port state of the optical cross-connect box port.

6. The optical cross-connect box port identification method according to any one of claims 1 to 5, characterized in that: Also includes: When the confidence of the optical cross-box image does not meet the preset condition, a prompt message is output; wherein the prompt message is used to instruct to re-shoot the optical cross-box image.

7. An optical cross-connect box port identification device, characterized in that: include: An acquisition module, used for acquiring multiple frames of optical cross-box images; A fusion module, used for performing image fusion processing on the multiple frames of optical cross-box images to obtain the confidence of the multiple frames of optical cross-box images and the port status in the multiple frames of optical cross-box images; The identification module is used to determine the target port state of the optical cross-box port based on the port states in the multiple frames of optical cross-box images when the confidence of the optical cross-box image meets a preset condition.

8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the optical cross-connect box port identification method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the optical cross-connect box port identification method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the optical cross-connect box port identification method according to any one of claims 1 to 6 is implemented.

Images