A fault prediction method, device, apparatus and storage medium

Abstract

The application provides a fault prediction method, device and equipment and a storage medium. The method comprises the following steps: obtaining multi-dimensional frequency point amplitudes collected by a production line microphone in a sealing state of a to-be-tested sealing foam as input features; inputting the input features into a pre-trained CNN-Capsule network model, wherein the model comprises: a feature extraction layer, which adopts a one-dimensional convolution structure in multiple groups in series to perform feature extraction on the input features and outputs a scalar feature map; a main capsule layer, which converts the scalar feature map into a multi-dimensional vector capsule; a digital capsule layer, which processes the multi-dimensional vector capsule through a dynamic routing mechanism and outputs a classification result, wherein the length of an output vector represents the existence probability of a corresponding category; and determining whether the to-be-tested sealing foam has a fault according to the output result of the digital capsule layer. Through the vector coding and dynamic routing mechanism of the capsule network, the spatial correlation features between frequency points are accurately captured, and the sealing foam fault prediction has the advantages of high recognition accuracy and strong generalization ability.

Description

Technical Field

[0001] This invention relates to the field of fault detection technology, and specifically to a fault prediction method, apparatus, device, and storage medium. Background Technology

[0002] In recent years, with the increasing sophistication of consumer electronics products, acoustic testing on production lines (such as microphone (MIC) testing) has become a crucial aspect of product quality control. In acoustic testing stations, sealing foam serves as a core sealing element, and its performance directly impacts the airtightness of the testing environment and the accuracy of the test results. However, over long-term use, sealing foam can fail due to compression deformation, surface wear, or aging, leading to abnormal signals at test frequencies, resulting in false or missed tests, and affecting product yield and production line capacity.

[0003] Currently, maintenance strategies for sealing foam mainly rely on periodic replacement or manual judgment based on experience, lacking precise fault prediction methods. Traditional maintenance models often replace foam according to fixed cycles, resulting in significant waste of consumables and failing to prevent production losses caused by sudden seal failures. Some production lines have attempted to use traditional machine learning methods (such as random forests and XGBoost) for fault classification based on frequency data, but these methods are highly dependent on data quality and the number of labeled data, have weak generalization ability, and their performance deteriorates significantly when production line conditions change slightly, making them difficult to adapt to complex production environments with multiple workstations and batches. Summary of the Invention

[0004] The purpose of this invention is to provide a fault prediction method, device, equipment, and storage medium that can solve the technical problems in the prior art, such as the lack of accurate fault prediction means for sealing foam, the weak generalization ability of traditional models, and the high migration cost.

[0005] In a first aspect, embodiments of the present invention provide a fault prediction method for testing sealed production line microphones in an acoustic workstation using sealing foam, the method comprising: The multi-dimensional frequency amplitude values ​​collected by the production line microphone under the sealing state of the foam to be tested are used as input features. The input features are fed into a pre-trained CNN-Capsule network model, the CNN-Capsule network model comprising: The feature extraction layer uses multiple sets of cascaded one-dimensional convolutional structures to extract features from the input features and outputs a scalar feature map. The main capsule layer transforms the scalar feature map into a multidimensional vector capsule; The digital capsule layer processes the multidimensional vector capsules through a dynamic routing mechanism and outputs classification results. The length of the output vector represents the probability of the existence of the corresponding category. Based on the output of the digital capsule layer, determine whether the sealing foam under test is faulty.

[0006] In one possible implementation, the training steps of the pre-trained CNN-Capsule network model include: Construct a training dataset, which includes the basic features of the multi-dimensional frequency amplitude values ​​collected by the production line microphone when the sealing foam is in the sealing state, as well as the statistical features derived from the multi-dimensional frequency amplitude values, and label them with corresponding fault labels or normal labels. Construct an initial CNN-Capsule network model, which includes the feature extraction layer, the main capsule layer, and the digital capsule layer; The training dataset is input into the initial CNN-Capsule network model, and the predicted output is calculated through forward propagation. The loss is calculated based on the predicted output and the fault label or normal label, and the model parameters are updated through backpropagation until the model converges, thus obtaining the pre-trained CNN-Capsule network model.

[0007] In one possible implementation, the feature extraction layer is composed of multiple sets of standardized convolutional structures connected in series, and each set of standardized convolutional structures includes, in sequence: a one-dimensional convolutional layer, a normalization layer, an activation layer, and a random deactivation layer.

[0008] In one possible implementation, the main capsule layer is configured to transform the scalar feature map into a vector capsule of a preset dimension to encode the variation pattern of the amplitude of each frequency point in the multi-dimensional frequency point data and the correlation between frequency points.

[0009] In one possible implementation, the digital capsule layer is configured as a binary classification output structure, outputting a normal label to represent a normal state and an output fault label to represent a fault state; the length of the output vector represents the probability of the fault existing, and the direction of the output vector encodes the attitude information of frequency point changes.

[0010] In one possible implementation, the multi-dimensional frequency amplitude is amplitude data of multiple frequency points in the microphone test; the input features also include statistical features derived from the multi-dimensional frequency amplitude, and the derived statistical features include at least the features processed by dynamic time warping (DTW), the mean feature, and the standard deviation feature.

[0011] In one possible implementation, the training step further includes model transfer training, used to transfer the model trained for the first station foam to the second station foam, the model transfer training including: In the first stage, the weights of the feature extraction layer and the main capsule layer are frozen, and only the parameters of the digital capsule layer and the classifier are updated. In the second stage, all network layers are unfrozen, and all network parameters are globally fine-tuned with a learning rate lower than that in the first stage.

[0012] Secondly, embodiments of the present invention provide a fault prediction device for testing the sealing of production line microphones in acoustic workstations using sealing foam, the device comprising: The acquisition module is used to acquire the multi-dimensional frequency amplitude values ​​collected by the production line microphone in the sealing state of the sealing foam under test as input features; A prediction module is used to input the input features into a pre-trained CNN-Capsule network model, the CNN-Capsule network model comprising: The feature extraction layer uses multiple sets of cascaded one-dimensional convolutional structures to extract features from the input features and outputs a scalar feature map. The main capsule layer transforms the scalar feature map into a multidimensional vector capsule; The digital capsule layer processes the multidimensional vector capsules through a dynamic routing mechanism and outputs classification results. The length of the output vector represents the probability of the existence of the corresponding category. Based on the output of the digital capsule layer, determine whether the sealing foam under test is faulty.

[0013] Thirdly, the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when running the computer program, performs to implement the method as described in the first aspect.

[0014] Fourthly, the present invention provides a computer-readable storage medium having computer-readable instructions stored thereon, which can be executed by a processor to implement the method described in the first aspect.

[0015] Compared to existing technologies, the fault prediction method provided by this invention obtains the multi-dimensional frequency amplitude values ​​collected by the production line microphone in the blocked state of the sealing foam under test as input features. These input features are then input into a pre-trained CNN-Capsule network model, which includes: a feature extraction layer that uses multiple sets of cascaded one-dimensional convolutional structures to extract features from the input features and outputs a scalar feature map; a main capsule layer that transforms the scalar feature map into multi-dimensional vector capsules; and a digital capsule layer that processes the multi-dimensional vector capsules through a dynamic routing mechanism and outputs classification results, where the length of the output vector represents the probability of the corresponding category. The output results of the digital capsule layer are used to determine whether the sealing foam under test is faulty. This invention, through the vector encoding and dynamic routing mechanism of the capsule network, accurately captures the spatial correlation features between frequency points, and has the advantages of high recognition accuracy, strong generalization ability, and cross-workstation transferability in sealing foam fault prediction. Attached Figure Description

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a fault prediction method provided by an embodiment of the present invention is shown; Figure 2 This diagram illustrates the overall network structure of the CNN-Capsule network model provided in this embodiment of the invention. Figure 3 The confusion matrix of the CNN-Capsule network model provided in this embodiment of the invention on the MIC1 workstation test set is shown. Figure 4 The confusion matrix of the CNN-Capsule network model provided in this embodiment of the invention after migrating from the MIC1 station to the MIC4 station is shown. Figure 5 A schematic diagram of the structure of a fault prediction device provided in an embodiment of the present invention is shown; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0017] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0018] It should be noted that, unless otherwise stated, the technical or scientific terms used in this invention should have the ordinary meaning as understood by one of ordinary skill in the art.

[0019] Furthermore, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to those processes, methods, products, or apparatuses.

[0020] Please refer to Figure 1 , Figure 1 This is a flowchart of the fault prediction method provided in this embodiment of the invention. The sealing foam is used to seal the acoustic testing station in the microphone testing of the production line, and specifically includes the following steps: S101. Obtain the multi-dimensional frequency amplitude values ​​collected by the production line microphone under the sealing state of the sealing foam to be tested as input features; S102. Input the input features into the pre-trained CNN-Capsule network model, and determine whether the sealing foam to be tested has a fault based on the output of the model.

[0021] Existing deep learning solutions mostly use general convolutional neural networks to process frequency point data, but the pure CNN structure has obvious defects when processing one-dimensional frequency point signals: on the one hand, the pooling operation of CNN will lose the spatial correlation information and local phase change features between frequency points; on the other hand, the general network structure cannot specifically encode the core acoustic characteristics of low-frequency signal anomalies when foam sealing fails.

[0022] To address the aforementioned issues, this invention employs a CNN-Capsule network model, which includes a feature extraction layer, a main capsule layer, and a digital capsule layer. The feature extraction layer uses multiple cascaded one-dimensional convolutional structures to extract features from the input, outputting a scalar feature map. The main capsule layer transforms the scalar feature map into multi-dimensional vector capsules. The digital capsule layer processes these multi-dimensional vector capsules through a dynamic routing mechanism, outputting classification results, where the length of the output vector represents the probability of the corresponding category. Based on the output of the digital capsule layer, it can be determined whether the tested sealing foam is faulty.

[0023] The fault prediction method provided in this invention lies in its use of a customized CNN-Capsule hybrid network architecture to intelligently diagnose the sealing status of the sealing foam in the microphone (MIC) testing station on the production line. The invention will be described in detail below with specific application scenarios and experimental data.

[0024] This invention can be applied to the acoustic testing stage of the microphone (MIC) in the audio testing station of eyeglasses products, specifically to test the sound sealing and sound pickup performance of the eyeglass microphone. In this testing station, sealing foam is used to seal the microphone testing path, ensuring the testing environment is free from external noise interference. The production line microphone testing system collects amplitude data (i.e., a 74-dimensional frequency amplitude vector) at 74 frequency points in each testing cycle, serving as the basic signal reflecting the sealing status of the foam.

[0025] Based on the actual measured data, it can be seen that the amplitude curves of the sealing foam at 74 frequency points show significant differences between normal and faulty states (such as sealing failure, deformation and aging), especially in the mid-to-low frequency range (frequency point index 10~40), where the amplitude changes are significant. Therefore, this invention uses multi-dimensional frequency point amplitudes as model input and automatically extracts discriminative features through a deep learning network to achieve accurate fault identification.

[0026] refer to Figure 2 The CNN-Capsule network model used in this embodiment of the invention is specifically composed of the following connected parts: 1. Input data The model receives basic amplitude features from 74 frequency points measured after the MIC1 workstation was blocked. It also integrates derived features processed by Dynamic Time Warping (DTW), the mean features of each frequency point's amplitude, and the standard deviation features to form the input feature vector. In one specific embodiment, the input dimension is 74 basic amplitudes plus three types of derived statistical features (DTW distance, mean, and standard deviation), totaling 77 dimensions. In another embodiment, when the production line only provides raw frequency data and cannot calculate derived features, the input layer can directly receive the 74 basic amplitudes and learn the statistical patterns between frequency points through the network's feature extraction capabilities.

[0027] 2. Feature Extraction Layer The feature extraction layer uses three sets of normalized convolutional structures connected in series. Each set of normalized convolutional structures includes, in sequence: a one-dimensional convolutional layer (Conv1d), a batch normalization layer (BatchNorm1d), a ReLU activation layer, and a random deactivation layer (Dropout).

[0028] In one specific embodiment, such as Figure 2 As shown, the first group of Conv1d has 1 input channel, 128 output channels, a kernel size of 5, a stride of 1, and padding of 2; the second group of Conv1d has 128 input channels, 256 output channels, and a kernel size of 3; the third group of Conv1d has 256 input channels, 512 output channels, a kernel size of 3, and a dilation rate of 2. Each group is followed by BatchNorm1d for normalization, ReLU for non-linear activation, and a Dropout ratio of 0.3 to prevent overfitting.

[0029] In another embodiment, when the input data contains multi-dimensional derived features (such as 77 dimensions), the number of input channels of the first group of Conv1d is adjusted accordingly to match the number of dimensions to ensure data dimensionality matching. This feature extraction layer outputs a 256-dimensional scalar feature map.

[0030] 3. Main capsule layer The main capsule layer receives the scalar feature map output from the feature extraction layer and transforms the scalar features into multi-dimensional vector capsules through convolutional capsule operations. In one specific embodiment, the main capsule layer transforms the scalar feature map into 8-dimensional vector capsules, with each capsule corresponding to the feature encoding of a local region at a frequency point. The 8-dimensional vectors fully encode the variation pattern of the frequency point amplitude and the spatial correlation between adjacent frequency points.

[0031] In another embodiment, when dealing with foam testing scenarios with large data fluctuations, the vector dimension of the main capsule layer can be configured to 16 dimensions to accommodate more complex frequency change patterns in a higher-dimensional representation space. The main capsule layer outputs a series of bottom capsules, where the modulus of each capsule represents the probability of the existence of the local feature, and the direction represents the pose information of the feature (such as the trend direction of frequency changes).

[0032] 4. Digital Capsule Layer The digital capsule layer uses a dynamic routing mechanism to iteratively weight and aggregate the lower-level capsules output by the main capsule layer, establishing connection weights between the lower-level capsules and the higher-level classification capsules. In one specific embodiment, the dynamic routing iterations are set to three times, updating the routing coefficients in each iteration until the contribution distribution from the lower-level capsules to the higher-level capsules stabilizes. The digital capsule layer is customized as a binary classification output structure, containing two higher-level capsules, corresponding to the normal and fault categories, respectively. The length of the output vector of each higher-level capsule represents the existence probability of that category, and the direction of the output vector encodes global pose information of frequency changes.

[0033] In another embodiment, if the production line needs to identify different types of failures (such as minor wear, severe deformation, surface contamination, etc.), the digital capsule layer can be expanded into a multi-classification structure (such as 3 or 5 categories), with each category corresponding to a high-level capsule.

[0034] The training process of the CNN-Capsule network model described above in this invention is as follows: First, a training dataset was constructed. 6869 samples were collected from the production line (including test data for normal and faulty foam at various lifecycle stages). Each sample included 74 frequency amplitude points and their corresponding DTW derived features, mean, and standard deviation, with manual labeling for faulty / normal. The dataset was then divided into training, validation, and test sets at 60%, 20%, and 20% respectively.

[0035] Next, an initial CNN-Capsule network model is constructed using the above-described structure. The training set is input into the model, and the predicted output is calculated through forward propagation. Margin Loss is used as the loss function (to adapt to the vector output characteristics of the capsule network), and backpropagation is performed using the Adam optimizer. The initial learning rate is set to 0.001, and the batch size is set to 32. Iterative training continues until the model's F1 score on the validation set no longer improves.

[0036] In one specific embodiment, an early stopping mechanism (patience=10) and a learning rate decay strategy are also introduced during the training process (if the loss does not decrease after every 5 iterations, it is decayed to 0.5 times the current value) to improve convergence stability.

[0037] The validation results on the test set (approximately 1374 samples) are as follows: Figure 3 The confusion matrix is ​​shown. The CNN-Capsule model significantly outperforms traditional ML models such as XGBoost and Random Forest (RF) as well as pure CNN models in terms of the number of correctly identified faulty samples.

[0038] Specifically, as shown in Table 1 below, the model performance indicators for the MIC1 station are as follows: the F1 score of the CNN-Capsule model reaches 0.8974, which is the best compared to the best traditional model; the fault recall rate reaches 0.8974, which is also the best compared to the best traditional model.

[0039] Table 1: Model Indicator Results for MIC1 Workstation

[0040] AUC stands for Area Under ROC Curve, which is a comprehensive ranking performance metric for binary classification tasks.

[0041] In addition, the data distribution of foam in different test stations (such as MIC1 to MIC4) is offset due to differences in installation position and stress state. Traditional model migration requires the collection of a large amount of data and retraining, which results in high iteration costs.

[0042] For foam located at different positions within the same project (e.g., migrating from MIC1 to MIC4), this invention designs a two-stage migration strategy. MIC4 only has 178 samples (1:1 ratio of positive to negative samples, 8 stations, 74-dimensional features), which is insufficient to support training from scratch.

[0043] The specific transfer training steps are as follows: Phase 1: Freeze all weights of the feature extraction layer (3 sets of Conv1d structures) and the main capsule layer, and update the digital capsule layer and classifier parameters only with a learning rate of 0.001. Train for 10 iterations to enable the high-level classifier to quickly adapt to the data distribution of the target domain.

[0044] Phase 2: Unfreeze all network layers and fine-tune all network parameters globally with a low learning rate (0.0001) for 20 epochs until convergence.

[0045] After the model transfer training is completed, the validation results on the test set are as follows: Figure 4 The confusion matrix is ​​shown. The CNN-Capsule model significantly outperforms traditional ML models such as XGBoost and Random Forest (RF) as well as pure CNN models in terms of the number of correctly identified faulty samples.

[0046] Specifically, as shown in Table 2 below, the model metrics for the MIC4 workstation show that the CNN-Capsule model achieved an F1 score of 1, which is the best compared to the optimal traditional model; the fault recall rate also reached 1, which is also the best compared to the optimal traditional model. This demonstrates a significant improvement compared to training a pure CNN model from scratch, and it eliminates the need to rebuild feature engineering and model structure, greatly reducing iteration costs.

[0047] Table 2: Model Indicator Results for MIC4 Workstation

[0048] In another embodiment, when it is necessary to migrate a model from one product line (such as an audio testing station for glasses) to another product line (such as a microphone testing station for headphones), since the acoustic test frequency ranges of the two products are different (e.g., headphones have 128 frequency points), a resampling adaptation layer needs to be added before the feature extraction layer to linearly interpolate the 128-dimensional frequency points to 74 dimensions. Then, the complete CNN-Capsule network structure is reused, and a two-stage transfer training is performed to achieve rapid reuse of the model across product lines.

[0049] In the actual production line inference and deployment phase, the fault prediction process for the sealing foam in this embodiment of the invention is as follows: 1. Obtain amplitude data of 74 frequency points collected by the production line MIC when the foam to be tested is in the blocking state, and simultaneously calculate derived statistical features such as DTW distance, mean and standard deviation online, and splice them to form the input feature vector.

[0050] 2. Input the input feature vector into the pre-trained CNN-Capsule network model. The scalar feature map is extracted by the three sets of Conv1d structures in the feature extraction layer. The main capsule layer transforms the scalar feature map into an 8-dimensional vector capsule. The digital capsule layer outputs the normal / fault binary classification result through dynamic routing.

[0051] 3. Based on the vector lengths of the two high-level capsules output by the digital capsule layer, if the modulus of the fault category capsule is greater than that of the normal category capsule, the foam under test is determined to be faulty, an alarm is triggered and a replacement is prompted; otherwise, it is determined to be normal and can continue to be used.

[0052] In one specific embodiment, the output vector length difference (faulty modulus minus normal modulus) is also used to characterize the severity of the fault: when the difference is between 0.2 and 0.5, it is judged as a minor fault, which can continue to be used but requires enhanced monitoring; when the difference is greater than 0.5, it is judged as a serious fault, which requires immediate replacement, thereby providing a more granular decision basis for production line maintenance.

[0053] This invention employs a customized CNN-Capsule network architecture, using multi-dimensional frequency point amplitudes and derived statistical features as inputs. It leverages the vector representation and dynamic routing mechanism of the capsule network to accurately encode the spatial correlation and changing posture between frequency points, achieving superior recognition accuracy compared to traditional ML and general CNNs in foam plugging fault prediction. Furthermore, through a two-stage migration strategy of freezing and fine-tuning, it effectively solves the problem of cross-workstation model reuse, reducing the iterative cost of intelligent upgrades to production lines. This invention possesses strong industrial application and promotion value.

[0054] In the above embodiments, a fault prediction method is provided. Correspondingly, the present invention also provides a fault prediction device. The fault prediction device provided in the embodiments of the present invention can implement the above-described fault prediction method. The fault prediction device can be implemented by software, hardware, or a combination of both. For example, the fault prediction device may include integrated or separate functional modules or units to perform the corresponding steps in the above methods.

[0055] Please refer to Figure 5 As shown, the fault prediction device 10 provided by the present invention includes: The acquisition module 101 is used to acquire the multi-dimensional frequency amplitude values ​​collected by the production line microphone when the sealing foam under test is in the sealing state as input features; Prediction module 102 is used to input the input features into a pre-trained CNN-Capsule network model, the CNN-Capsule network model comprising: The feature extraction layer uses multiple sets of cascaded one-dimensional convolutional structures to extract features from the input features and outputs a scalar feature map. The main capsule layer transforms the scalar feature map into a multidimensional vector capsule; The digital capsule layer processes the multidimensional vector capsules through a dynamic routing mechanism and outputs classification results. The length of the output vector represents the probability of the existence of the corresponding category. Based on the output of the digital capsule layer, determine whether the sealing foam under test is faulty.

[0056] In one possible implementation, the apparatus further includes a model training module for training a CNN-Capsule network model, wherein the training steps of the CNN-Capsule network model include: Construct a training dataset, which includes the basic features of the multi-dimensional frequency amplitude values ​​collected by the production line microphone when the sealing foam is in the sealing state, as well as the statistical features derived from the multi-dimensional frequency amplitude values, and label them with corresponding fault labels or normal labels. Construct an initial CNN-Capsule network model, which includes the feature extraction layer, the main capsule layer, and the digital capsule layer; The training dataset is input into the initial CNN-Capsule network model, and the predicted output is calculated through forward propagation. The loss is calculated based on the predicted output and the fault label or normal label, and the model parameters are updated through backpropagation until the model converges, thus obtaining the pre-trained CNN-Capsule network model.

[0057] In one possible implementation, the feature extraction layer is composed of multiple sets of standardized convolutional structures connected in series, and each set of standardized convolutional structures includes, in sequence: a one-dimensional convolutional layer, a normalization layer, an activation layer, and a random deactivation layer.

[0058] In one possible implementation, the main capsule layer is configured to transform the scalar feature map into a vector capsule of a preset dimension to encode the variation pattern of the amplitude of each frequency point in the multi-dimensional frequency point data and the correlation between frequency points.

[0059] In one possible implementation, the digital capsule layer is configured as a binary classification output structure, outputting a normal label to represent a normal state and an output fault label to represent a fault state; the length of the output vector represents the probability of the fault existing, and the direction of the output vector encodes the attitude information of frequency point changes.

[0060] In one possible implementation, the multi-dimensional frequency amplitude is amplitude data of multiple frequency points in the microphone test; the input features also include statistical features derived from the multi-dimensional frequency amplitude, and the derived statistical features include at least the features processed by dynamic time warping (DTW), the mean feature, and the standard deviation feature.

[0061] In one possible implementation, the model training module is further used for model transfer training, which transfers the model trained on foam at the first workstation to foam at the second workstation. The model transfer training includes: In the first stage, the weights of the feature extraction layer and the main capsule layer are frozen, and only the parameters of the digital capsule layer and the classifier are updated. In the second stage, all network layers are unfrozen, and all network parameters are globally fine-tuned with a learning rate lower than that in the first stage.

[0062] The fault prediction device and the fault prediction method provided in the embodiments of the present invention are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0063] The present invention also provides an electronic device corresponding to the method provided in the foregoing embodiments. The electronic device may be a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to perform the above-described fault prediction method.

[0064] Please refer to Figure 6 This illustrates a schematic diagram of an electronic device provided by some embodiments of the present invention. For example... Figure 6 As shown, the electronic device 20 includes: a processor 200, a memory 201, a bus 202, and a communication interface 203. The processor 200, the communication interface 203, and the memory 201 are connected via the bus 202. The memory 201 stores a computer program that can run on the processor 200. When the processor 200 runs the computer program, it executes the fault prediction method provided by any of the foregoing embodiments of the present invention.

[0065] The memory 201 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 203 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0066] Bus 202 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs. After receiving an execution instruction, the processor 200 executes the program. The fault prediction method disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 200, or implemented by the processor 200.

[0067] The processor 200 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 200 or by instructions in software form. The processor 200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 201. The processor 200 reads the information in memory 201 and, in conjunction with its hardware, completes the steps of the above method.

[0068] The electronic device provided in this embodiment of the invention and the fault prediction method provided in this embodiment of the invention are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0069] The present invention also provides a computer-readable storage medium corresponding to the fault prediction method provided in the foregoing embodiments, wherein a computer program (i.e., a program product) is stored thereon, and the computer program, when run by a processor, executes the fault prediction method provided in any of the foregoing embodiments.

[0070] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0071] The computer-readable storage medium provided in the above embodiments of the present invention and the fault prediction method provided in the embodiments of the present invention are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A fault prediction method, characterized in that, The method for testing sealed production line microphones in acoustic workstations using sealing foam includes: The multi-dimensional frequency amplitude values ​​collected by the production line microphone under the sealing state of the foam to be tested are used as input features. The input features are fed into a pre-trained CNN-Capsule network model, the CNN-Capsule network model comprising: The feature extraction layer uses multiple sets of cascaded one-dimensional convolutional structures to extract features from the input features and outputs a scalar feature map. The main capsule layer transforms the scalar feature map into a multidimensional vector capsule; The digital capsule layer processes the multidimensional vector capsules through a dynamic routing mechanism and outputs classification results. The length of the output vector represents the probability of the existence of the corresponding category. Based on the output of the digital capsule layer, determine whether the sealing foam under test is faulty.

2. The method according to claim 1, characterized in that, The training steps for the pre-trained CNN-Capsule network model include: Construct a training dataset, which includes the basic features of the multi-dimensional frequency amplitude values ​​collected by the production line microphone when the sealing foam is in the sealing state, as well as the statistical features derived from the multi-dimensional frequency amplitude values, and label them with corresponding fault labels or normal labels. Construct an initial CNN-Capsule network model, which includes the feature extraction layer, the main capsule layer, and the digital capsule layer; The training dataset is input into the initial CNN-Capsule network model, and the predicted output is calculated through forward propagation. The loss is calculated based on the predicted output and the fault label or normal label, and the model parameters are updated through backpropagation until the model converges, thus obtaining the pre-trained CNN-Capsule network model.

3. The method according to claim 1 or 2, characterized in that, The feature extraction layer is composed of multiple sets of standardized convolutional structures connected in series. Each set of standardized convolutional structures includes, in sequence: a one-dimensional convolutional layer, a normalization layer, an activation layer, and a random deactivation layer.

4. The method according to claim 1 or 2, characterized in that, The main capsule layer is configured to transform the scalar feature map into a vector capsule of a preset dimension to encode the variation pattern of the amplitude of each frequency point in the multi-dimensional frequency point data and the correlation between frequency points.

5. The method according to claim 1, characterized in that, The digital capsule layer is configured as a binary classification output structure, with a normal label representing a normal state and a fault label representing a fault state; the length of the output vector represents the probability of the fault existing, and the direction of the output vector encodes the attitude information of the frequency point change.

6. The method according to claim 1 or 2, characterized in that, The multi-dimensional frequency amplitude is the amplitude data of multiple frequency points in the microphone test; the input feature also includes statistical features derived from the multi-dimensional frequency amplitude, and the derived statistical features include at least the features after dynamic time warping (DTW), the mean feature, and the standard deviation feature.

7. The method according to claim 2, characterized in that, The training step further includes model transfer training, used to transfer the model trained on foam at the first workstation to foam at the second workstation. The model transfer training includes: In the first stage, the weights of the feature extraction layer and the main capsule layer are frozen, and only the parameters of the digital capsule layer and the classifier are updated. In the second stage, all network layers are unfrozen, and all network parameters are globally fine-tuned with a learning rate lower than that in the first stage.

8. A fault prediction device, characterized in that, The apparatus is used for testing sealed production line microphones in acoustic workstations using sealing foam. The acquisition module is used to acquire the multi-dimensional frequency amplitude values ​​collected by the production line microphone in the sealing state of the sealing foam under test as input features; A prediction module is used to input the input features into a pre-trained CNN-Capsule network model, the CNN-Capsule network model comprising: The feature extraction layer uses multiple sets of cascaded one-dimensional convolutional structures to extract features from the input features and outputs a scalar feature map. The main capsule layer transforms the scalar feature map into a multidimensional vector capsule; The digital capsule layer processes the multidimensional vector capsules through a dynamic routing mechanism and outputs classification results. The length of the output vector represents the probability of the existence of the corresponding category. Based on the output of the digital capsule layer, determine whether the sealing foam under test is faulty.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that can be executed by a processor to implement the method as described in any one of claims 1 to 7.

Images