An unsupervised optical module fault data screening and labeling method, device and medium

Abstract

The application discloses an unsupervised optical module fault data screening and labeling method and device and a medium, relates to the technical field of AI cluster training network, and comprises the following steps: constructing an optical module sample set, and performing sample density calculation on the optical module sample set to obtain sample density data; performing density threshold division on the sample density data to determine an optical module density group; performing data point labeling on the optical module density group to obtain first cluster classification data; determining an intra-cluster voting result through intra-cluster voting analysis based on the first cluster classification data; and obtaining second cluster classification data through residual sample labeling according to the intra-cluster voting result. The application solves the technical problems that the optical module fault detection cannot capture complex fault modes and the optical module fault prediction data set is difficult to label in the prior art.

Description

Technical Field

[0001] This application relates to the field of AI cluster training network technology, and in particular to an unsupervised method, device and medium for screening and labeling optical module fault data. Background Technology

[0002] As a core component for high-speed interconnection in AI cluster training networks, the stability of optical modules directly impacts the efficiency and reliability of large-scale distributed training. Although the failure rate of optical modules is sufficiently low, with failure cycles measured in years, the sheer number of optical modules in a large-scale cluster reduces the failure cycle to days or even minutes. AI cluster training at the kilocalorie level typically requires continuous operation for days or even weeks. Sudden optical module failures causing node communication interruptions will lead to training task crashes, resulting in high costs for breakpoint recovery and significant loss of computing power. To ensure training continuity, avoid resource waste, and optimize training efficiency, optical module failure prediction is necessary to provide early warnings of performance degradation and proactively replace modules before complete failure.

[0003] Traditional methods for optical module fault detection or prediction are threshold-based. These methods measure the optical module's Direct Digital Meter (DDM) data, including five indicators: transmit optical power, receive optical power, operating temperature, operating voltage, and bias current. Alarms are triggered based on the measured data and the corresponding threshold values ​​for these DDM indicators. However, this approach suffers from several problems: it only focuses on instantaneous exceedances of the indicators, ignoring trends and other information; static thresholds are unsuitable for different operating environments, are unstable, and prone to misjudgments. Furthermore, because each indicator is considered individually without a combined assessment, it fails to capture complex fault patterns. To more effectively detect and predict optical module faults, existing technologies incorporate deep learning. By training models on labeled datasets, multiple indicators can be considered together, and patterns in fault samples can be learned from the training data to achieve fault detection and prediction. However, constructing such datasets is extremely difficult, relying on expert annotation of large amounts of data—a technically demanding and labor-intensive task that limits the application of deep learning methods in optical module fault analysis. Summary of the Invention

[0004] This application provides an unsupervised method, device, and medium for filtering and labeling optical module fault data, which solves the technical problems in the prior art where optical module fault detection cannot capture complex fault modes and where it is difficult to label optical module fault prediction datasets.

[0005] In a first aspect, embodiments of this application provide an unsupervised method for screening and labeling optical module fault data. The method comprises: constructing an optical module sample set and calculating the sample density of the optical module sample set to obtain sample density data; dividing the sample density data into density thresholds to determine optical module density groups; labeling data points in the optical module density groups to obtain first cluster classification data; determining intra-cluster voting results based on the first cluster classification data through intra-cluster voting analysis; and obtaining second cluster classification data based on the intra-cluster voting results through residual sample labeling.

[0006] In one implementation of this application, the sample density of the optical module sample set is calculated to obtain sample density data. Specifically, this includes: performing data standardization processing on the optical module sample set to obtain the optical module sample set to be calculated; and performing sample density calculation on the optical module sample set to be calculated based on a preset sample density calculation method to obtain sample density data.

[0007] In one implementation of this application, the sample density data is divided into density thresholds to determine optical module density groups. Specifically, this includes: classifying the sample density data into abnormal states to determine the optical module states; wherein the optical module states include: normal optical module and abnormal optical module; when the optical module state is normal, a first optical module density group is determined based on a preset density threshold through high-density cluster analysis; when the optical module state is abnormal, an abnormal state category is obtained by determining the probability of abnormality occurrence; wherein the abnormal state categories include: high-probability abnormality and low-probability abnormality; when the optical module abnormal state category is high-probability abnormality, a second optical module density group is determined based on a density threshold through interval-like sample construction; when the optical module abnormal state category is low-probability abnormality, a third optical module density group is determined based on a density threshold through outlier noise point cluster construction; and an optical module density group is determined based on the first, second, and third optical module density groups.

[0008] In one implementation of this application, after dividing the sample density data into density thresholds to determine the optical module density groups, the method further includes: setting clustering parameters and a stopping threshold when the data consistency of the optical module density groups is lower than a preset range; obtaining noise point samples through density calculation analysis based on the clustering parameters, and annotating the noise point samples to obtain annotated noise point samples; performing data consistency detection on the annotated noise point samples to determine clustering iteration instructions; performing clustering iteration according to the clustering iteration instructions, and obtaining an optical module density group with passed data consistency when the number of clustering iterations reaches the stopping threshold; and annotating the remaining samples of the optical module density groups with passed data consistency to obtain a fourth optical module density group.

[0009] In one implementation of this application, data consistency detection is performed on the labeled noise point samples to determine the clustering iteration instructions. Specifically, this includes: performing a sample proportion analysis on the labeled noise point samples to determine the majority sample proportion of the labeled noise point samples; wherein the proportion calculation formula for the sample proportion analysis is:

[0010]

[0011] label(·) represents the labeling results, l represents all cluster classifications, and χ represents the cluster classifications. noise The clustering iteration command is set to be labeled when the proportion of most samples is greater than the preset proportion threshold; when the proportion of most samples is less than or equal to the proportion threshold, the clustering iteration command is set to be iterated.

[0012] In one implementation of this application, data point annotation is performed on the optical module density group to obtain the first cluster classification data. Specifically, this includes: extracting core points from the optical module density group to obtain core points to be annotated; and annotating the core points to be annotated with data points to obtain the first cluster classification data.

[0013] In one implementation of this application, the intra-cluster voting result is determined based on the first cluster classification data through intra-cluster voting analysis. Specifically, this includes: performing voting mechanism analysis on the first cluster classification data to determine cluster state voting data; and determining the intra-cluster voting result based on the cluster state voting data. The intra-cluster voting result includes: normal clusters and abnormal clusters.

[0014] In one implementation of this application, the second cluster classification data is obtained by labeling the remaining samples based on the intra-cluster voting results. Specifically, this includes: determining outlier noise points based on the intra-cluster voting results, and labeling the outlier noise points with remaining samples to obtain the second cluster classification data.

[0015] Secondly, embodiments of this application also provide an unsupervised optical module fault data screening and labeling device, characterized in that the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: construct an optical module sample set and calculate the sample density of the optical module sample set to obtain sample density data; divide the sample density data into density thresholds to determine optical module density groups; label data points in the optical module density groups to obtain first cluster classification data; determine the intra-cluster voting result through intra-cluster voting analysis based on the first cluster classification data; and obtain second cluster classification data through residual sample labeling based on the intra-cluster voting result.

[0016] Thirdly, embodiments of this application also provide a non-volatile computer storage medium for unsupervised screening and labeling of optical module fault data, storing computer-executable instructions, characterized in that the computer-executable instructions are configured to: construct an optical module sample set and calculate the sample density of the optical module sample set to obtain sample density data; divide the sample density data into density thresholds to determine optical module density groups; label data points in the optical module density groups to obtain first cluster classification data; determine the intra-cluster voting results through intra-cluster voting analysis based on the first cluster classification data; and obtain second cluster classification data through residual sample labeling based on the intra-cluster voting results.

[0017] This application provides an unsupervised method, device, and medium for screening and labeling optical module fault data. By adjusting clustering parameters, the data is screened in batches until the proportion of target samples within a cluster exceeds a threshold. This significantly reduces the workload of data labeling while ensuring labeling quality. It solves the technical problems of existing technologies, such as the inability to capture complex fault modes in optical module fault detection and the difficulty in labeling optical module fault prediction datasets. It realizes a deep learning-based optical module fault detection method, significantly reducing the workload of data labeling for optical module fault detection and improving the efficiency of capturing complex fault modes of optical modules. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 A flowchart illustrating an unsupervised optical module fault data filtering and annotation method provided in this application embodiment;

[0020] Figure 2 This is a schematic diagram illustrating the classification effect after sample density calculation or clustering, provided in an embodiment of this application.

[0021] Figure 3 This is a schematic diagram of cluster core point extraction provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the internal structure of an unsupervised optical module fault data screening and labeling device provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] This application provides an unsupervised method, device, and medium for screening and labeling optical module fault data. By adjusting clustering parameters, the data is screened in batches until the proportion of target samples within a cluster exceeds a threshold. This significantly reduces the workload of data labeling while ensuring labeling quality. It solves the technical problems of existing technologies, such as the inability to capture complex fault modes in optical module fault detection and the difficulty in labeling optical module fault prediction datasets. It realizes a deep learning-based optical module fault detection method, significantly reducing the workload of data labeling for optical module fault detection and improving the efficiency of capturing complex fault modes of optical modules.

[0025] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0026] Figure 1 A flowchart illustrating an unsupervised optical module fault data filtering and annotation method provided in this application embodiment. Figure 1 As shown in the figure, the unsupervised optical module fault data filtering and annotation method provided in this application embodiment specifically includes the following steps:

[0027] Step 101: Construct an optical module sample set and calculate the sample density of the optical module sample set to obtain sample density data.

[0028] For example, since the measurement values ​​of normal optical modules are stable over a long period of time, labeling all of them without screening would generate a lot of meaningless work, which would prolong the data labeling cycle and be detrimental to the development of optical module fault detection schemes based on deep learning. First, it is necessary to calculate the sample density of the optical module sample set to obtain sample density data, which serves as the background data for analysis.

[0029] Specifically, the sample density of the optical module sample set is calculated to obtain sample density data, including: performing data standardization processing on the optical module sample set to obtain the optical module sample set to be calculated; and calculating the sample density of the optical module sample set to be calculated based on a preset sample density calculation method to obtain sample density data.

[0030] In one embodiment, the optical module sample set is first subjected to data standardization processing to obtain the optical module sample set to be calculated.

[0031] Then, the number of other samples within a given range σ for each sample in the sample set of optical modules to be calculated is used as its density to obtain sample density data.

[0032] Step 102: Divide the sample density data into density thresholds to determine the optical module density groups.

[0033] For example, density thresholds are typically set before optical module analysis to distinguish different intervals and divide them into several high-density clusters and discrete noise points. Since the performance of optical modules is relatively stable over a long period, different analyses are required when the measured values ​​of the optical modules are in different states, in order to calculate or cluster the density of density intervals under different failure probabilities.

[0034] The sample density data is divided into density thresholds to determine the optical module density groups. Specifically, this includes: classifying the sample density data into abnormal states to determine the optical module status; where the optical module status includes: normal optical module and abnormal optical module; when the optical module status is normal, a first optical module density group is determined based on a preset density threshold through high-density cluster analysis; when the optical module status is abnormal, the optical module abnormal state category is obtained through anomaly probability determination; where the optical module abnormal state category includes: high-probability abnormality and low-probability abnormality; when the optical module abnormal state category is high-probability abnormality, a second optical module density group is determined based on a density threshold and through interval-like sample construction; when the optical module abnormal state category is low-probability abnormality, a third optical module density group is determined based on a density threshold and through outlier noise point cluster construction; and finally, the optical module density group is determined based on the first, second, and third optical module density groups.

[0035] In one embodiment, the sample of various indicators measured at a certain moment for an optical module or a lane of an optical module is interpreted using the following formula:

[0036] X = (x 1 ,x 2 ,…,x M (1)

[0037] Where M is the number of indicators being measured, x i Let be the measured value of the i-th indicator.

[0038] Under normal circumstances, the performance of optical modules is stable over a long period. Therefore, the measured values ​​of normal modules generally fall within the same finite range, resulting in high-density clusters within this range. The set of samples from normal optical modules can be explained by the following formula:

[0039]

[0040] in, It is a subspace of M-dimensional space, and is the boundary established by high-density points, within the interval The points within have a higher density than the threshold τ1, which can be explained by the following formula:

[0041]

[0042] f(·) is the density calculation function for the sample, bound(·) is the boundary establishment function, and τ1 is the density threshold, thereby determining the density group of the first optical module.

[0043] For faulty optical modules, their indicators differ from those of normal optical modules. Therefore, the samples will deviate from the normal range, resulting in the following two scenarios:

[0044] In the first scenario, for fault types with a high probability of occurrence, the fault will occur within the normal range. There are a large number of similar samples outside, which constitute a new high-density set, which can be explained by the following formula.

[0045]

[0046] Furthermore, multiple high-probability fault types will generate multiple high-density intervals. This leads to the determination of the second optical module density group.

[0047] In the second scenario, for fault types with a low probability of occurrence, the corresponding samples are insufficient to form a new high-density cluster, thus appearing as outlier noise points:

[0048]

[0049] in This indicates all high-density areas.

[0050] It is important to note that the process of classifying sample density data into abnormal states to determine the state of optical modules is designed for different scenarios that may occur with optical modules in practical applications. Even after the optical module density groups are obtained through clustering or density calculation, they are essentially still in an unknown state. Distinguishing between the first, second, and third optical module density groups requires subsequent annotation of core data points. That is, an optical module density group still includes the first, second, and third optical module density groups, and the explanatory formula is as described above, but the optical module density groups are not labeled.

[0051] Figure 2This diagram illustrates the classification effect after sample density calculation or clustering, as provided in an embodiment of this application. After performing density calculation or clustering on a large number of samples, several cluster measurement data χ1, χ2, ..., χ2 are obtained. c In the high-density clusters after clustering, the middle part is the cluster measurement data, and there are four groups of cluster measurement data. The density of dark points at the edge of each group of clusters is very low, which are outliers and noise points, thus determining the third optical module density group.

[0052] Furthermore, after dividing the sample density data into density thresholds to determine the optical module density groups, the method further includes: setting clustering parameters and a stopping threshold when the data consistency of the optical module density groups is lower than a preset range; obtaining noise point samples through density calculation analysis based on the clustering parameters, and annotating the noise point samples to obtain annotated noise point samples; performing data consistency detection on the annotated noise point samples to determine the clustering iteration instructions; performing clustering iteration according to the clustering iteration instructions, and obtaining the optical module density groups with passed data consistency when the number of clustering iterations reaches the stopping threshold; and annotating the remaining samples of the optical module density groups with passed data consistency to obtain the fourth optical module density group.

[0053] In one embodiment, when the threshold setting is unreasonable and the consistency of samples within a cluster is poor, the parameters are first initialized, and the clustering parameters and stopping threshold are set. By setting a large density calculation range σ, the number of noise points after clustering is reduced and they are distributed at the outermost edge. A consistency threshold is set, and iteration stops when the consistency detection result meets the threshold condition.

[0054] Then, using the set parameters, the density calculation and interval division of the samples are performed, the samples corresponding to the noise points are extracted, and the selected noise points are labeled.

[0055] Next, check if the proportion of majority class samples in the annotation results exceeds a set threshold. If so, pass the consistency check and proceed with the remaining sample annotation; otherwise, fail and perform parameter iteration. When the consistency check fails, it indicates that a large number of samples in this batch do not belong to the corresponding cluster, i.e., they are too close to the cluster edge, and the remaining samples may contain many incorrectly clustered samples. In this case, it is necessary to adjust the clustering algorithm parameters, reduce the density calculation range σ, and include more points in the noise points for consideration. After parameter adjustment, repeat the above steps for iteration.

[0056] Finally, when the consistency check passes, it indicates that a large number of samples in this batch belong to the corresponding cluster, meaning that the vast majority of samples in this noise segment are correctly clustered. At this point, it can be approximately assumed that the remaining samples with higher density belong to this cluster. The remaining samples within the cluster are then labeled according to the type of the nearest neighbor cluster, and the iterative screening process is stopped.

[0057] Furthermore, data consistency checks are performed on the labeled noise point samples to determine the clustering iteration instructions. Specifically, this includes: performing sample proportion analysis on the labeled noise point samples to determine the majority sample proportion; wherein the formula for calculating the proportion in the sample proportion analysis is:

[0058]

[0059] Where label(·) represents the labeling result, l represents all cluster classifications, and χ represents the cluster classification. noise The clustering iteration command is set to be labeled when the proportion of most samples is greater than the preset proportion threshold; when the proportion of most samples is less than or equal to the proportion threshold, the clustering iteration command is set to be iterated.

[0060] Step 103: Label the data points of the optical module density group to obtain the first cluster classification data.

[0061] Specifically, data point annotation is performed on the optical module density group to obtain the first cluster classification data, including: extracting core points from the optical module density group to obtain core points to be annotated; and annotating the core points to be annotated with data points to obtain the first cluster classification data.

[0062] For example, after determining the core points, the core points are labeled by experts, and the first cluster of classification data is determined based on the core point labeling information. The labeling information is the classification of the current cluster.

[0063] Step 104: Based on the classification data of the first cluster, determine the intra-cluster voting results through intra-cluster voting analysis.

[0064] Specifically, based on the first cluster classification data, the intra-cluster voting results are determined through intra-cluster voting analysis, including: performing voting mechanism analysis on the first cluster classification data to determine cluster status voting data; and determining the intra-cluster voting results based on the cluster status voting data; wherein the intra-cluster voting results include: normal clusters and abnormal clusters.

[0065] Figure 3 This is a schematic diagram of cluster core point extraction provided in an embodiment of this application.

[0066] In one embodiment, since the clusters in the optical module density group include both normal clusters and faulty clusters, identification is required.

[0067] By using a higher density threshold τ2>τ1, the core samples in each cluster are extracted; where the dark part is the outlier noise point and the light part is the core sample.

[0068] For any cluster χ i Its core point is represented by the following formula:

[0069] χ core,i ={X∈χ i |f(X)>τ2}(7)

[0070] Step 105: Based on the intra-cluster voting results, obtain the second cluster classification data through residual sample labeling.

[0071] For example, the remaining sample labeling involves labeling the remaining unlabeled samples in a cluster according to the cluster classification, resulting in second cluster classification data. For outlier noise points, since they contain both normal and faulty samples, data labeling is required.

[0072] Specifically, based on the intra-cluster voting results, the second cluster classification data is obtained through residual sample labeling, including: determining outlier noise points based on the intra-cluster voting results, and performing residual sample labeling on the outlier noise points to obtain the second cluster classification data.

[0073] This application significantly reduces the workload of data annotation for optical module fault detection. A density-based data annotation strategy and an iterative screening strategy for noise points avoid extensive repetitive data annotation work, requiring only the annotation of a few noise points and cluster core points. Furthermore, it supports deep learning-based optical module fault detection methods, considering the trends in metric changes. Currently, there is no public optical module fault detection dataset, and dataset construction requires significant investment from industry experts, making deep learning-based methods difficult to apply. This application supports the application of deep learning methods in optical module fault detection, and the method naturally supports learning the trends in metric changes. In different environments, the metrics of optical modules fluctuate, requiring adjustments to thresholds in traditional threshold-based methods. However, regardless of the environment, the metric distribution of normal and faulty optical modules is universal; therefore, parameters such as density thresholds and density calculation regions are universal and do not need to be adapted to the environment. When clustering the data, all collected metrics are treated as a vector for density calculation, resulting in several clusters. In this process, all metrics are considered comprehensively rather than independently, which is beneficial for capturing complex fault modes.

[0074] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide an unsupervised optical module fault data screening and labeling device, the structure of which is as follows: Figure 2 As shown.

[0075] Figure 2 This is a schematic diagram of the internal structure of an unsupervised optical module fault data filtering and labeling device provided in an embodiment of this application. Figure 2 As shown, the device includes:

[0076] At least one processor 201;

[0077] And a memory 202 that is communicatively connected to at least one processor;

[0078] The memory 202 stores instructions executable by at least one processor, which are executed by at least one processor 201 to enable at least one processor 201 to:

[0079] A sample set of optical modules is constructed, and the sample density of the sample set is calculated to obtain sample density data. The sample density data is divided into density thresholds to determine the optical module density groups. Data points are labeled in the optical module density groups to obtain the first cluster classification data. Based on the first cluster classification data, the intra-cluster voting results are determined through intra-cluster voting analysis. Based on the intra-cluster voting results, the second cluster classification data is obtained through residual sample labeling.

[0080] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium for unsupervised screening and labeling of optical module fault data, storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0081] A sample set of optical modules is constructed, and the sample density of the sample set is calculated to obtain sample density data. The sample density data is divided into density thresholds to determine the optical module density groups. Data points are labeled in the optical module density groups to obtain the first cluster classification data. Based on the first cluster classification data, the intra-cluster voting results are determined through intra-cluster voting analysis. Based on the intra-cluster voting results, the second cluster classification data is obtained through residual sample labeling.

[0082] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0083] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0084] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0088] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0089] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0090] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0091] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0092] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An unsupervised method for filtering and labeling optical module fault data, characterized in that, The method includes: Construct an optical module sample set and calculate the sample density of the optical module sample set to obtain sample density data; The sample density data is divided into density thresholds to determine the optical module density groups; Data points are labeled on the optical module density group to obtain the first cluster classification data; Based on the first cluster classification data, the intra-cluster voting results are determined through intra-cluster voting analysis; Based on the intra-cluster voting results, the second cluster classification data is obtained through residual sample labeling.

2. The unsupervised optical module fault data filtering and annotation method according to claim 1, characterized in that, The sample density of the optical module sample set is calculated to obtain sample density data, specifically including: The optical module sample set is subjected to data standardization processing to obtain the optical module sample set to be calculated; Based on a preset sample density calculation method, the sample density of the optical module sample set to be calculated is calculated to obtain the sample density data.

3. The unsupervised optical module fault data filtering and annotation method according to claim 1, characterized in that, The sample density data is divided into density thresholds to determine the optical module density groups, specifically including: The sample density data is divided into abnormal states to determine the optical module status; wherein, the optical module status includes: optical module normal and optical module abnormal. When the optical module is in normal condition, the first optical module density group is determined by high-density cluster analysis based on a preset density threshold. When the optical module is in an abnormal state, the abnormal state category of the optical module is obtained by determining the probability of the abnormality; wherein, the abnormal state category of the optical module includes: high probability abnormality and low probability abnormality. When the optical module's abnormal state category is high-probability abnormal, a second optical module density group is determined based on the density threshold and by constructing interval-like samples. When the abnormal state category of the optical module is low probability abnormal, a third optical module density group is determined based on the density threshold by constructing an outlier noise point cluster. The optical module density group is determined based on the first optical module density group, the second optical module density group, and the third optical module density group.

4. The unsupervised optical module fault data filtering and annotation method according to claim 3, characterized in that, After dividing the sample density data into density thresholds to determine the optical module density groups, the method further includes: If the data consistency of the optical module density group is lower than a preset range, set the clustering parameters and the stop threshold; Based on the clustering parameters, noise point samples are obtained through density calculation analysis, and the noise point samples are labeled to obtain labeled noise point samples. Data consistency detection is performed on the labeled noise point samples to determine the clustering iteration instructions; According to the clustering iteration instruction, clustering iteration is performed, and when the number of clustering iterations reaches the stopping threshold, a density group of optical modules that has passed data consistency is obtained. The remaining samples are labeled on the optical module density group that has passed the data consistency test to obtain the fourth optical module density group.

5. The unsupervised optical module fault data filtering and annotation method according to claim 4, characterized in that, Data consistency checks are performed on the labeled noise point samples to determine the clustering iteration instructions, specifically including: A sample proportion analysis is performed on the labeled noise point samples to determine the majority sample proportion; wherein, the proportion calculation formula for the sample proportion analysis is: label(·) represents the labeling results, l represents all cluster classifications, and χ represents the cluster classifications. noise It is a set of noise points; If the proportion of the majority of samples is greater than a preset proportion threshold, the clustering iteration instruction is set to be labelable. If the proportion of the majority of samples is less than or equal to the proportion threshold, the clustering iteration instruction is set to require iteration.

6. The unsupervised optical module fault data filtering and annotation method according to claim 1, characterized in that, Data point annotation is performed on the optical module density group to obtain the first cluster classification data, specifically including: Core points are extracted from the optical module density group to obtain the core points to be labeled; Data point labeling is performed on the core points to be labeled to obtain the first cluster of classification data.

7. The unsupervised optical module fault data filtering and annotation method according to claim 1, characterized in that, Based on the first cluster classification data, the intra-cluster voting results are determined through intra-cluster voting analysis, specifically including: The voting mechanism analysis is performed on the first cluster classification data to determine the cluster status voting data; Based on the cluster status voting data, the intra-cluster voting results are determined; wherein, the intra-cluster voting results include: normal clusters and abnormal clusters.

8. The unsupervised optical module fault data filtering and annotation method according to claim 1, characterized in that, Based on the intra-cluster voting results, the second cluster classification data is obtained through residual sample labeling, specifically including: Based on the intra-cluster voting results, outlier noise points are identified, and residual samples are labeled for these outlier noise points to obtain the second cluster classification data.

9. An unsupervised optical module fault data screening and labeling device, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Construct an optical module sample set and calculate the sample density of the optical module sample set to obtain sample density data; The sample density data is divided into density thresholds to determine the optical module density groups; Data points are labeled on the optical module density group to obtain the first cluster classification data; Based on the first cluster classification data, the intra-cluster voting results are determined through intra-cluster voting analysis; Based on the intra-cluster voting results, the second cluster classification data is obtained through residual sample labeling.

10. A non-volatile computer storage medium for unsupervised screening and labeling of optical module fault data, storing computer-executable instructions, characterized in that, The computer-executable instructions are set as follows: Construct an optical module sample set and calculate the sample density of the optical module sample set to obtain sample density data; The sample density data is divided into density thresholds to determine the optical module density groups; Data points are labeled on the optical module density group to obtain the first cluster classification data; Based on the first cluster classification data, the intra-cluster voting results are determined through intra-cluster voting analysis; Based on the intra-cluster voting results, the second cluster classification data is obtained through residual sample labeling.

Images