Method for detecting redundant deep information of sealed electronic equipment

Abstract

The invention discloses a sealed electronic equipment redundancy deep information detection method, and belongs to the technical field of sealed electronic equipment redundancy detection. The objective of the invention is to solve the problem that useful information may be discarded due to active neglect of ground noise signals in existing sealed electronic equipment redundancy detection based on pulse signals. The method comprises the following steps: aiming at to-be-detected redundant signals obtained by N channels, taking a group of redundant signals corresponding to the N channels as a redundant signal group; from the perspective of signals, features are extracted from each section of redundancy signals in the redundancy signal group, the features comprise time domain features and / or frequency domain features, N feature vectors are constructed, and the N feature vectors are converted into a fusion feature vector which is called to-be-detected data; and classifying the to-be-detected data by adopting a classifier to obtain a positioning identification result of the redundant object. On the basis, a material identification result of the redundant object can be obtained from a pulse view angle. The method is used for detecting the redundant substances.

Description

Technical Field

[0001] The invention belongs to the technical field of redundant object detection of sealed electronic equipment, and in particular relates to a method for detecting deep information of redundant objects. Background Art

[0002] Sealed electronic equipment is an important part of aerospace systems. During its production process, copper wires, solder particles and other waste materials may be left inside. Sealed electronic equipment usually works in overweight, weightlessness and severe impact environments, and waste materials are activated to move randomly, posing a major threat to the reliable operation of sealed electronic equipment. Therefore, it is very important to detect waste materials before sealed electronic equipment leaves the factory for service. The particle impact noise detection (PIND) method is a widely used waste material detection method, and its basic principle is as follows. By applying mechanical excitation to the sealed electronic equipment to be tested, the waste materials that may exist inside it are in a random motion state, thereby generating a collision signal or a sliding signal, which is called a waste material signal. The sound sensor is placed on the surface of the sealed electronic equipment to be tested to capture the possible signals. The presence of waste materials is determined by judging whether the waste material signal is captured. Generally, scholars regard the judgment of the presence or absence of waste materials as superficial information of waste material detection.

[0003] In fact, the redundant signal can reflect the properties of the redundant object, such as the material and location of the redundant object. There are differences in signal characteristics between the redundant signals generated by redundant objects with different properties. Machine learning can quantify the differences in signal characteristics between signals with different properties, convert them into differences in distribution characteristics between data with different labels, build data sets and train classifiers with good performance to achieve data classification, and then complete the identification of signals with different properties. Referring to this idea, scholars have successively carried out research on the identification and positioning of redundant materials based on machine learning, trying to obtain the material and location of the redundant objects based on the redundant signals. These properties are regarded as deep information for redundant object detection.

[0004] After being activated, the superfluous matter is in a state of random motion. Every time it contacts the inner wall or internal components of the sealed electronic device (including collision and sliding), it releases energy. The sound sensor captures this energy, which generates a pulse during the signal acquisition process. Every time the superfluous matter contacts the inner wall or internal components, it will be bounced off and suspended in the air until it contacts again. In this process, there is almost no energy released, only negligible weak energy released by media such as air, and the sound sensor also captures weak energy, which generates a section of background noise during the signal acquisition process. In this way, for the superfluous matter signal, its interior is composed of multiple pulses and multiple sections of background noise. In a narrow sense, the pulse is the energy released by the superfluous matter, which is related to the superfluous matter, and the background noise is the energy released by other media when the superfluous matter is suspended, which has nothing to do with the superfluous matter. In this way, the aforementioned quantification of the difference in signal characteristics between signals of different attributes is further quantified as the difference in signal characteristics between pulses of different attributes, and the background noise becomes interference. Based on this, in the early stage, scholars usually used pulse extraction methods to extract pulses from redundant signals, and carried out feature engineering or spectrogram technology with pulses as research objects, converting pulses into feature vectors or images, thereby constructing data sets or image sets.

[0005] However, background noise is not necessarily interference. For the superfluous signal, the multiple pulses and multiple background noises inside it reflect the random motion state of the superfluous. For example, the duration of the pulse reflects the duration of the contact between the superfluous and the inner wall or internal components, and the duration of the background noise reflects the duration of the superfluous in the suspended state. Therefore, in a broad sense, background noise is also related to superfluous, but no relevant scholars have considered the problem of signal recognition from this perspective, and no scholars have conducted relevant research. Summary of the invention

[0006] The present invention aims to solve the problem that the existing detection of redundant objects in sealed electronic equipment based on pulse signals actively ignores background noise signals and may lose useful information.

[0007] A method for detecting deep information of redundant objects in sealed electronic equipment, comprising:

[0008] For the undesired object signals to be tested obtained by N channels, a group of undesired object signals corresponding to the N channels is taken as a undesired object signal group;

[0009] From the signal perspective, features are extracted from each segment of the redundant object signal in the redundant object signal group, and the features include time domain features and / or frequency domain features. N feature vectors are constructed, and the N feature vectors are converted into a fused feature vector, which is called the data to be tested. A classifier is used to classify the data to be tested to obtain the positioning and recognition results of the redundant objects.

[0010] Preferably, the classifier used in the process of classifying the data to be tested using a classifier is XGBoost.

[0011] Furthermore, the method for detecting deep information of excess matter in a sealed electronic device further comprises the following steps:

[0012] From the pulse perspective, a pulse extraction algorithm is used to extract pulses from each segment of the redundant object signal group, and then a pulse matching algorithm is used to match the N segments of the redundant object signal group to obtain a pulse group; each pulse in the pulse group is converted into a spectrogram to obtain N spectrograms, and the N spectrograms are converted into a fused spectrogram, called a spectrogram for material identification; multiple pulse groups in the redundant object signal group obtain multiple spectrograms for material identification; a deep learning model is used to obtain predicted labels for multiple spectrograms for material identification; a majority vote is performed on multiple predicted labels to obtain a label that receives the most votes, and the material identification result of the redundant object is obtained.

[0013] Preferably, the deep learning model used in the process of obtaining predicted labels of multiple spectrograms to be tested using a deep learning model is a VGG19 network model.

[0014] Furthermore, the pulse extraction algorithm used in the process of respectively extracting pulses from each section of the redundant signal in the redundant signal group is a two-level variable multi-threshold pulse extraction algorithm.

[0015] Furthermore, the pulse matching algorithm used in the process of matching the N segments of the redundant signal in the redundant signal group by using the pulse matching algorithm is a threshold determination search pulse matching algorithm.

[0016] Furthermore, the process of converting N spectrograms into a fused spectrogram includes:

[0017] For N spectrograms, the values ​​of the three color channels R, G, and B at the same pixel are obtained sequentially, which are N red values, N green values, and N blue values ​​in total; the N values ​​of the three color channels at the same pixel are respectively smoothed and filtered to obtain a processed red value, a green value, and a blue value; the same processing is performed on all pixels in the N spectrograms, and then they are fused to obtain a fused spectrogram.

[0018] A method for detecting deep information of redundant objects in sealed electronic equipment, comprising:

[0019] For the undesired object signals to be tested obtained by N channels, a group of undesired object signals corresponding to the N channels is taken as a undesired object signal group;

[0020] From the signal perspective, each redundant object signal in the redundant object signal group is converted into a spectrogram to obtain N spectrograms, which are then converted into a fused spectrogram, called a spectrogram for positioning and recognition. A deep learning model is used to obtain the label of the spectrogram for positioning and recognition, that is, the positioning and recognition result of the redundant object is obtained.

[0021] Furthermore, the method for detecting deep information of excess matter in a sealed electronic device further comprises the following steps:

[0022] From the pulse perspective, a pulse extraction algorithm is used to extract pulses from each segment of the redundant object signal group, and then a pulse matching algorithm is used to match the N segments of the redundant object signal group to obtain a pulse group; each pulse in the pulse group is converted into a spectrogram to obtain N spectrograms, and the N spectrograms are converted into a fused spectrogram, called a spectrogram for material identification; multiple pulse groups in the redundant object signal group obtain multiple spectrograms for material identification; a deep learning model is used to obtain predicted labels for multiple spectrograms for material identification; a majority vote is performed on multiple predicted labels to obtain a label that receives the most votes, and the material identification result of the redundant object is obtained.

[0023] Furthermore, the process of converting N spectrograms into a fused spectrogram includes:

[0024] For N spectrograms, the values ​​of the three color channels R, G, and B at the same pixel are obtained sequentially, which are N red values, N green values, and N blue values ​​in total; the N values ​​of the three color channels at the same pixel are respectively smoothed and filtered to obtain a processed red value, a green value, and a blue value; the same processing is performed on all pixels in the N spectrograms, and then they are fused to obtain a fused spectrogram.

[0025] The present invention has the beneficial effects:

[0026] The present invention is the first to conduct research on the method of detecting deep information of superfluous objects using superfluous object signals as the research object. It can not only provide new ideas and methods for existing superfluous object detection, but also break through the traditional way of using the same features or signals for detection in positioning and material detection. The present invention adopts a task differentiation processing method based on the support characteristics of the signal characteristics for the task, especially determining the best viewing angle and corresponding data processing process suitable for superfluous object positioning, which can improve the classification performance of existing superfluous object positioning research. On this basis, the present invention also proposes to determine the best viewing angle and corresponding data processing process suitable for superfluous object material identification, so as to improve the classification performance of existing superfluous object material identification research. The feasibility, practicality and superiority of the present invention have been fully verified through experiments in real application scenarios, ensuring the reliability of superfluous object deep information detection of sealed electronic equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flowchart for implementing a specific implementation method;

[0028] Figure 2 For sealed electronic equipment (overall and internal) view;

[0029] Figure 3 An image of a device for obtaining a process of a redundant signal group;

[0030] Figure 4 It is a schematic diagram of a two-stage variable multi-threshold pulse extraction algorithm;

[0031] Figure 5 Schematic diagram of pulse matching algorithm for threshold determination search;

[0032] Figure 6 is the average classification accuracy achieved by the classifier on the signal-material dataset and the pulse-material dataset;

[0033] Figure 7 is the average classification accuracy of the neural network on the signal-texture image set and the pulse-texture image set;

[0034] Figure 8 is the average classification accuracy achieved by the classifier on the signal-position dataset and the pulse-position dataset;

[0035] Fig. 9 is the average classification accuracy achieved by the neural network on the signal-position image set and the pulse-position image set;

[0036] Fig.10 is the average classification accuracy achieved by the classifier on the signal-material validation dataset and the pulse-material validation dataset;

[0037] Fig.11 is the average classification accuracy of the neural network on the signal-material verification image set and the pulse-material verification image set;

[0038] Fig.12 is the average classification accuracy achieved by the classifier on the signal-position validation dataset and the pulse-position validation dataset;

[0039] Fig.13 is the average classification accuracy achieved by the neural network on the signal-position validation image set and the pulse-position validation image set. DETAILED DESCRIPTION

[0040] Based on the problems pointed out in the background technology, the present invention takes the entire superfluous signal as the research object for the first time, analyzes the quantitative effect of the difference in signal characteristics between superfluous signals generated by superfluous with different attributes, and compares it with the quantitative effect of the difference in signal characteristics between pulses in superfluous signals generated by superfluous with different attributes, thereby proposing a method for detecting deep information of superfluous in sealed electronic equipment from the perspective of signal and pulse. First, taking the superfluous signal as the research object, feature engineering and spectrogram technology are directly carried out on the signal, and the signal is converted into a feature vector and a spectrogram respectively, thereby establishing a data set and a picture set and training a classifier and a neural network respectively suitable for superfluous material identification and positioning. Secondly, the superfluous signal is processed to obtain the pulses therein, and feature engineering and spectrogram technology are carried out with the pulses as the research object, and the pulses are converted into feature vectors and spectrograms, thereby establishing a data set and a picture set and training a classifier and a neural network respectively suitable for superfluous material identification and positioning. Finally, the classification performance of the classifier and the neural network trained with the superfluous material recognition and positioning as the research objects are compared to determine the best research object, that is, the best viewing angle, and thus determine the best solution for superfluous material recognition and positioning in the deep information detection method. The present invention is described in detail below in conjunction with specific embodiments. Specific implementation method one:

[0042] For the sealed electronic device to be tested, find a sealed electronic device of the same model without any waste as the experimental object, and place multiple sound sensors at different positions on its surface. Prepare waste samples of different materials based on the latest waste detection information in real application scenarios. Place waste samples of different materials at different positions inside the sealed electronic device in turn, and collect a large number of waste signal groups marked with different materials and positions.

[0043] On the one hand, the multiple segments of redundant signals in each redundant signal group are directly converted into multiple feature vectors and multiple spectrograms, which are called signal-feature vectors and signal-spectrograms, respectively, thereby constructing fused feature vectors and fused spectrograms, which are called signal-fusion feature vectors and signal-fusion spectrograms. By adding labels of material and position respectively, signal-material fusion feature vectors, signal-position fusion feature vectors, signal-material fusion spectrograms and signal-position fusion spectrograms are obtained, thereby establishing signal-material data sets, signal-position data sets, signal-material picture sets and signal-position picture sets from the signal perspective. On this basis, multiple representative classifiers with good performance are trained on the signal-material data sets and signal-position data sets, respectively, and two optimal classifiers from the signal perspective are obtained by comparison, which are called optimal signal-material classifiers and optimal signal-position classifiers. Similarly, multiple representative neural networks with good performance are trained on the signal-material picture set and signal-position picture set, respectively, and two optimal neural networks from the signal perspective are obtained by comparison, which are called optimal signal-material neural network and optimal signal-position neural network.

[0044] On the other hand, a two-level variable multi-threshold pulse extraction algorithm is first used to process each segment of the redundant signal in each redundant signal group to extract the pulses therein. Then, a threshold judgment search pulse matching algorithm is used to process the multiple segments of the redundant signal in each redundant signal group to match the pulse groups therein. Then, the multiple pulses in each pulse group are converted into multiple feature vectors and multiple spectrograms, which are respectively called pulse-feature vectors and pulse-spectrograms, thereby constructing fused feature vectors and fused spectrograms, which are called pulse-fusion feature vectors and pulse-fusion spectrograms. By adding labels of material and position respectively, pulse-material fusion feature vectors, pulse-position fusion feature vectors, pulse-material fusion spectrograms and pulse-position fusion spectrograms are obtained, thereby establishing pulse-material data sets, pulse-position data sets, pulse-material picture sets and pulse-position picture sets. On this basis, multiple representative classifiers with good performance are trained on the pulse-material data sets and pulse-position data sets, respectively, and the two optimal classifiers under the pulse perspective are compared, which are called the optimal pulse-material classifier and the optimal pulse-position classifier. Similarly, multiple representative neural networks with good performance are trained on the pulse-material image set and the pulse-position image set respectively, and the two optimal neural networks from the pulse perspective are compared, which are called the optimal pulse-material neural network and the optimal pulse-position neural network.

[0045] Finally, by comparing the optimal signal-material classifier, optimal signal-material neural network, optimal pulse-material classifier and optimal pulse-material neural network under the signal and pulse perspective, the most outstanding classifier or neural network and the corresponding signal or pulse perspective are obtained, thereby determining the optimal perspective for the identification of the material of the redundant object, thereby determining the optimal feature vector or spectrogram construction process, the optimal classifier or optimal neural network that can be used, which is called the optimal solution for the identification of the material of the redundant object. Similarly, by comparing the optimal signal-position classifier, optimal signal-position neural network, optimal pulse-position classifier and optimal pulse-position neural network under the signal and pulse perspective, the most outstanding classifier or neural network and the corresponding signal or pulse perspective are obtained, thereby determining the optimal solution for the positioning of the redundant object. Figure 1 The implementation process of the method proposed in the present invention is given.

[0046] In this way, for the sealed electronic device to be tested, a group of unwanted object signals can be collected, which is called the unwanted object signal group to be tested. On this basis, the corresponding feature vector or spectrogram construction process is used for processing according to the best viewing angle for the identification and positioning of the unwanted object material, and a fused feature vector or a fused spectrogram is obtained, which is called the feature vector to be tested or the spectrogram to be tested. In this way, the corresponding optimal classifier or the optimal neural network is used to predict the feature vector to be tested or the spectrogram to be tested, and the classification result of the feature vector to be tested or the spectrogram to be tested is obtained, thereby realizing the identification and positioning of the unwanted object material.

[0047] 1. Signal group and pulse group:

[0048] 1.1. Preparation:

[0049] In the present invention, a core component of a certain type of aerospace equipment is selected as a sealed electronic device, and four sound sensors are placed at different positions on its surface to cover the entire detection range, such as Figure 2 The sealed electronic device is divided into seven independent sealed spaces, which are not connected to each other. Figure 2 These are actually seven locations where excess material may be located inside a sealed electronic device.

[0050] According to the latest information on excess material detection in real application scenarios, inspectors often detect five types of excess materials from the sealed electronic equipment, including four metal materials and one non-metallic material, namely: aluminum, tin, iron, copper and silicone. Among them, the weight of the four metal materials is between 0.8mg and 3.0mg, and the weight of silicone is about 10mg. Referring to the principle of vibration sound generation of the PIND method, under the same conditions, the greater the weight of the excess material, the clearer the excess material signal generated and the higher the quality. Therefore, for extreme conditions, the above five materials with the smallest weight are selected to make excess material samples for subsequent experiments. Other scholars who make heavier excess material samples will obtain better experimental results than the present invention. Among them, the weight of aluminum, tin and copper is 1.0mg, the weight of iron is 0.8mg, and the weight of silicone is 10mg.

[0051] 1.2. Get the signal group:

[0052] Find and Figure 2 The sealed electronic device of the same model but without any extra objects is used as the experimental object. The shell of the sealed electronic device is opened, and the extra objects of a certain material are placed in a closed space, and the sealed electronic device is resealed. Referring to the PIND method, the sealed electronic device after packaging is fixed to the vibration table, and the vibration table is controlled to apply mechanical excitation to the sealed electronic device, such as Figure 3 As shown. Referring to the Chinese GJB65B standard, the mechanical excitation applied by the vibration table is set as shown in Table 1. Special equipment is used to process and save the redundant signals captured by the four sound sensors. Four sections of redundant signals with a duration of five seconds are processed and saved each time to establish a redundant signal group. Multiple redundant signal groups can be obtained by processing and saving multiple times. In this way, the redundant samples of five materials are placed in seven confined spaces in turn. After multiple processing and saving, 100 redundant signal groups representing the redundant samples of each material placed in each confined space are obtained, totaling 100×5×7=3500 redundant signal groups. Label the material and location to each redundant signal group.

[0053] Table 1 Mechanical excitation specified in GJB65B

[0054]

[0055] 1.3. Pulse extraction:

[0056] The two-level variable multi-threshold pulse extraction algorithm is used to process the four segments of the redundant signal group and extract the pulses. First, it calculates the average energy and average zero-crossing rate of the redundant signal and sets them as the basic threshold. Then, referring to the 3σ criterion of the normal distribution, the peak detection threshold is set to E p =E a +3σe , set the endpoint energy threshold to E be_1 =E a -σ E , set the endpoint zero crossing rate threshold to Z be_1 =Z a -σ Z Secondly, it divides the redundant signal into frames and calculates the energy and zero-crossing rate of each frame signal. Thirdly, it starts from the first frame signal and compares the energy of each frame signal with the peak detection threshold E p , determine the highest point of the pulse and obtain the peak time t p Then, starting from the peak moment, it compares the energy of each frame signal with the endpoint energy threshold E be_1 , and then compare the zero-crossing rate of each frame signal with the endpoint zero-crossing rate threshold Z forward and backward respectively be_1 , the first starting point and the first ending point of the pulse are determined by combining the two rounds of comparison results, and the first starting time t is obtained. b_1 With the first end time t e_1 Then, it resets the endpoint energy threshold to E be_2 =E a +σ E , set the endpoint zero crossing rate threshold to Z be_2 =Z a +σ Z Through the same comparison and analysis steps, the second starting point and the second ending point of the pulse are determined, and the second starting time t is obtained. b_2 and the second end time t e_2 Finally, the duration between the first start time and the first end time, and the duration between the second start time and the second end time is calculated, the continuous multi-pulse interference is analyzed, and the final start time and the final end time of the pulse are determined, thereby completing the pulse extraction. Figure 4 A schematic diagram of a two-stage variable multi-threshold pulse extraction algorithm is given. The two-stage variable multi-threshold pulse extraction algorithm can be implemented by referring to the prior art.

[0057] 1.4. Pulse matching:

[0058] On the basis of using the two-level variable multi-threshold pulse extraction algorithm to process the four segments of the 3500 redundant signal groups separately, the threshold judgment search pulse matching algorithm is used to simultaneously process the four segments of the redundant signals in the redundant group to obtain the pulse group. First, it obtains the starting time of the pulse with the largest amplitude in the four segments of the redundant signal respectively, thereby determining the redundant signal captured by the sound sensor closest to and farthest from the redundant object, and calculates the difference in the starting time of the pulse with the largest amplitude in these two segments of the redundant signal, which is set as the starting time threshold T. Secondly, it sets the redundant signal captured by the sound sensor farthest from the redundant object as the reference signal, and obtains the starting time t of the pulse therein. i (i=1,2,…,n). Then, it sets the starting time threshold interval of the pulse in the reference signal to [t i -R,t i ](i=1,2,…,n). Finally, it searches for the starting time between [t i -T,t i ](i=2,3,…,n), and perform pulse matching. If three pulses can be found, match them with P i A pulse group is formed, thus completing pulse matching. For more detailed steps, please refer to the published papers. Figure 5 A schematic diagram of a threshold determination search pulse matching algorithm is given, and the threshold determination search pulse matching algorithm can be implemented by referring to the prior art.

[0059] 1.5. Pulse group:

[0060] The two-level variable multi-threshold pulse extraction algorithm and the threshold judgment search pulse matching algorithm were used to process the 3500 redundant signal groups, and a total of 264435 pulse groups were obtained. Material and position labels were added to each pulse group. According to the material and position labels, the distribution of the 3500 redundant signal groups and the 264435 pulse groups was statistically analyzed, as shown in Table 2 and Table 3, respectively.

[0061] Table 2 Distribution of labels by material

[0062]

[0063] Table 3 Distribution of labels by position

[0064]

[0065] 2. Datasets and image sets:

[0066] 2.1. Feature Engineering:

[0067] In the previous research on the identification and positioning of the material of the superfluous object, 11 features with good performance were selected from the time domain and frequency domain, and the values ​​of these features were calculated on the pulse to construct the feature vector, and their feasibility and applicability were fully analyzed and verified. In the present invention, the values ​​of these 11 features were also calculated on 3500 superfluous object signal groups and 264435 pulse groups, that is, 14000 superfluous object signals and 1057740 pulses, respectively, to obtain 14000 feature vectors from the signal perspective and 1057740 feature vectors from the pulse perspective. The description of these 11 time domain and frequency domain features is shown in Table 4.

[0068] Table 4 11 time domain and frequency domain features

[0069]

[0070] 2.2. Spectrogram Technology:

[0071] As mentioned above, both the redundant signal and the pulse can be regarded as signals of different durations. Therefore, for 3500 redundant signal groups and 264435 pulse groups, that is, 14000 redundant signal segments and 1057740 pulses, the following spectrogram construction process is used to convert them into 14000 spectrograms from the signal perspective and 1057740 spectrograms from the pulse perspective, respectively.

[0072] First, a high-pass filter is used to pre-emphasize the signal, and its principle is shown in formula (1).

[0073] S[n]=x[n]-μ×x[n-1] (1)

[0074] Wherein, x[n] represents a sampling point sequence in a redundant signal or a pulse signal, μ is a pre-emphasis coefficient, and μ=0.97 is usually set.

[0075] Secondly, the pre-emphasized signal is windowed to make it continuous. The principle is shown in formula (2).

[0076] S w [n] = S[n] × w[n] (2)

[0077] Where S[n] represents the sampling point sequence in the pre-emphasized signal, and w[n] is the window function of the Hamming window. The calculation formula of the Hamming window is as follows.

[0078]

[0079] Among them, S w [n] represents the sequence of sampling points in the windowed signal, and N represents the total number of points in the signal.

[0080] Finally, the windowed signal is subjected to a fast Fourier transform, and the power spectrum is passed through a bandpass filter to obtain the spectrogram corresponding to the signal.

[0081] 2.3. Data fusion and image fusion:

[0082] As mentioned above, whether it is four segments of redundant signals in the same redundant signal group or four pulses in the pulse group, they all need to be considered simultaneously, rather than considered separately as independent individuals. Therefore, whether it is four segments of redundant signals or four pulses, the four transformed feature vectors or four spectrograms need to construct a fused feature vector or a fused spectrogram.

[0083] A fusion feature vector construction method of multi-channel feature coding is used to construct the feature vector. Specifically, whether the feature vector is calculated on the redundant signal or the pulse, it is constructed by arranging the values ​​of the 11 features shown in Table 4 calculated on the signal or pulse in a row. The order of the values ​​is consistent with the sequence number of the features shown in Table 4. In this way, for four feature vectors, the mean of the four values ​​of the order one is calculated in sequence, the mean of the four values ​​of the order two is calculated, until the mean of the four values ​​of the order eleven is calculated, eleven new values ​​are obtained, and they are arranged in a row in the same order to obtain a preliminary fusion feature vector. At the same time, four sound sensors are represented respectively using four-bit encoding. Each bit of encoding can take a value of "1", "2", "3" and "4", representing the order in which the four segments of redundant signals in the current redundant signal group arrive at the four sound sensors, or representing the order in which the redundant signals belonging to the four pulses in the current pulse group arrive at the four sound sensors. For example, the four-bit encoding of "1 3 4 2" means that the redundant signal arrives at sound sensor No. 1 first and arrives at sound sensor No. 3 last. Thus, each redundant signal group and each pulse group has a four-bit code, but the four-bit codes of multiple pulse groups belonging to the same redundant signal group are the same. The four-bit code is added as four values ​​to the back of the eleven new values, rearranged in a row, and a 15-dimensional fusion feature vector is obtained.

[0084] In this way, 3500 redundant signal groups eventually construct 3500 fused feature vectors, called signal-fused feature vectors, and 264435 pulse groups eventually construct 264435 fused feature vectors, called pulse-fused feature vectors. According to the material and position labels of the redundant signal group and pulse group, the material and position labels are added to the signal-fused feature vector and the pulse-fused feature vector, respectively, to establish four data sets for redundant material recognition and positioning, called signal-material data set, signal-position data set, pulse-material data set and pulse-position data set, and their descriptions are shown in Table 5 and Table 6, respectively.

[0085] A multi-channel integrated spectrogram construction method is used to construct a spectrogram. Specifically, for four spectrograms, the values ​​of the three color channels of red (R), green (G), and blue (B) at the same pixel point are sequentially obtained, with a total of four red values, four green values, and four blue values. The four values ​​of the three color channels at the same pixel point are respectively smoothed and filtered to obtain a processed red value, a green value, and a blue value. All the pixels in the four spectrograms are processed to obtain a fused spectrogram. In this way, 3500 redundant signal groups finally construct 3500 fused spectrograms, called signal-fused spectrograms, and 264435 pulse groups finally construct 264435 fused spectrograms, called pulse-fused spectrograms, thereby establishing four picture sets for redundant material identification and positioning, called signal-material picture set, signal-position picture set, pulse-material picture set, and pulse-position picture set, and their descriptions are shown in Tables 5 and 6, respectively.

[0086] Table 5 Description of the dataset and image set used for redundant material recognition

[0087]

[0088]

[0089] Table 6 Description of the datasets and image sets used for redundant object localization

[0090]

[0091] 3. Model training:

[0092] 3.1. Identification of redundant materials:

[0093] Classifiers in machine learning are mainly divided into three types, one is a classifier based on mathematical theory, including k-nearest neighbor (kNN) and support vector machine (SVM), one is a classifier based on tree structure or tree-like structure, including random forest and XGBoost, and the last one is a neural network used for data classification in deep learning, including back propagation neural network (BPNN) and one-dimensional convolutional neural network (1D-CNN). In previous studies, multiple classifiers were trained on data sets used for redundant material identification and positioning to verify the feasibility, applicability and superiority of the above six classifiers. Therefore, in the present invention, ten-fold cross validation was performed on the signal-material data set and the pulse-material data set, and the above six classifiers were trained on the training set, and their classification accuracy on the test set was statistically analyzed, and their average classification accuracy was calculated, such as Figure 6 shown.

[0094] from Figure 6It can be seen that all six classifiers achieve higher average classification accuracy on the pulse-material dataset, and their average classification accuracy is significantly better than that on the signal-material dataset. This shows that for redundant material recognition, the quality of the dataset constructed by pulses is higher, and the pulse perspective is the best perspective. Among them, XGBoost and random forest achieve the most significant average classification accuracy on the signal-material dataset and the pulse-material dataset, which is significantly better than the average classification accuracy achieved by the other four classifiers. It is worth noting that both the signal-material dataset and the pulse-material dataset contain data with five labels, and the ordinary SVM is used to solve the binary classification problem. Therefore, the radial basis kernel function (RBF) is introduced into the SVM to construct the RBF-SVM. RBF-SVM also achieves a relatively significant average classification accuracy on the signal-material dataset and the pulse-material dataset, indicating that as a classic classifier based on mathematical theory, it still plays an important role at present. The classification performance of kNN is affected by the data distribution of different features in the dataset, so it achieves a general average classification accuracy. BPNN is prone to fall into the trap of local optimality in the initialization stage, and usually cannot obtain the global optimal result, so it achieves the lowest average classification accuracy. Compared with BPNN, 1D-CNN further improves its classification performance by combining optimization algorithms to find the optimal weights and biases, so it also achieves good average classification accuracy.

[0095] The more popular and well-performing neural networks used for image classification in deep learning include Alexnet, VGG19, Resnet-18, Resnet-50, MobileNet and EfficientNet. Among them, Alexnet and VGG19 are both deep convolutional neural networks, Resnet-18 and Resnet-50 are residual neural networks, MobileNet and EfficientNet are lightweight neural networks, and they are all widely representative. Therefore, in the present invention, a 6-2-2 type ten-fold cross-validation method is customized. Specifically, the signal-material image set and the pulse-material image set are randomly divided into a training set, a validation set and a test set in a ratio of 6:2:2, and the above six neural networks are trained on the training set, and the hyperparameters are adjusted on the validation set, and their classification accuracy on the test set is counted. By repeating this process ten times, the ten classification accuracies obtained by the above six neural networks are obtained, and the average classification accuracy they achieved is calculated, such as Figure 7 shown.

[0096] from Figure 7It can be seen that the six neural networks also achieved higher average classification accuracy on the pulse-texture picture set, and their average classification accuracy was significantly better than that on the signal-texture picture set. This once again shows that for redundant material recognition, the picture set constructed by pulses has higher quality, and the pulse perspective is the best perspective. Among them, VGG19 achieved the highest average classification accuracy on both the signal-texture picture set and the pulse-texture picture set, and Alexnet, Resnet-18 and Resnet-50 achieved significant average classification accuracy on both the signal-texture picture set and the pulse-texture picture set. This reflects the significant advantages of deep convolutional neural networks and residual neural networks for image classification. Unfortunately, although MobileNet and EfficientNet are currently well-performing lightweight neural networks, they achieved the lowest average classification accuracy on the signal-texture picture set and the pulse-texture picture set. This may be because the number of pictures in the signal-texture picture set and the pulse-texture picture set is large, and MobileNet and EfficientNet cannot fully learn all the features of the pictures due to the model mechanism setting, and their original design intention is to reduce the number of parameters and the amount of calculation, which makes their expression ability relatively weak.

[0097] Therefore, the above-mentioned several almost representative classifiers and neural networks all achieved higher average classification accuracy on the dataset and picture set constructed by pulses than on the dataset and picture set constructed by superfluous signals, and the advantages are obvious. This fully shows that for superfluous material recognition, the quality of the dataset and picture set constructed by pulses is higher, and the pulse perspective is the best perspective. Superfluous signals are composed of multiple pulses and multiple segments of background noise. Pulses are the energy released when superfluous objects contact the inner wall or internal components of sealed electronic equipment. Superfluous objects are the acting party, so pulses can feedback the properties of superfluous objects. Background noise is the energy released by other media when superfluous objects are in a suspended state, which has nothing to do with superfluous objects, so background noise cannot feedback the properties of superfluous objects. In this way, because the superfluous signal is a mixed state of pulses and background noise, it can only feedback limited superfluous properties. Especially when the proportion of pulses is low, the superfluous signal can only feedback a small amount of superfluous properties. In contrast, pulses can directly feedback abundant superfluous properties. This explains why pulses are the best perspective for superfluous material recognition.

[0098] In addition, although the redundant signal can only feedback limited redundant properties, the classifier has achieved a not-too-bad average classification accuracy on the dataset constructed by the redundant signal because of the artificial extraction of good time domain and frequency domain features. Also, because the neural network is equipped with a feature extractor, it can extract key feature information from the spectrogram converted from the redundant signal, so the neural network has also achieved a satisfactory average classification accuracy on the picture set. However, a careful analysis shows that the average classification accuracy achieved by the neural network on the pulse-texture picture set is slightly higher than that achieved by the classifier on the pulse-texture dataset. In particular, if BPNN, 1D-CNN and two lightweight neural networks are not considered, and only the four classic classifiers and four neural networks are compared, the performance advantage of the neural network can be more prominent. More importantly, if the average classification accuracy of the neural network on the signal-texture picture set is compared with the average classification accuracy of the classifier on the signal-texture dataset, it can be found that the performance advantage of the neural network is significant. This shows that compared with the fixed time domain and frequency domain features extracted manually, the feature extractor of the neural network can extract more critical feature information from the spectrogram. In particular, fixed time and frequency domain features are more than adequate for describing simple pulses, but are inadequate for describing complex redundant signals. In contrast, the feature extractor can easily extract significant feature information, whether it is a spectrogram converted from a simple pulse or a spectrogram converted from a complex redundant signal. This also explains why the average classification accuracy of the neural network on the pulse-material dataset is smaller than that on the signal-material dataset, and why the neural network achieves a higher average classification accuracy than the classifier. This shows that converting pulses into spectrograms and training neural networks is better than converting pulses into feature vectors and training classifiers.

[0099] In general, for the identification of redundant materials, the pulse perspective is the best perspective, and converting the pulse into a spectrogram is a better construction process, and the classification performance of the neural network trained in this way is more superior. In addition, VGG19 is the optimal neural network worth considering.

[0100] 3.2、Location of redundant objects:

[0101] Similarly, ten-fold cross validation is performed on the signal-position dataset and the pulse-position dataset, and the classification accuracy achieved is counted, and the average classification accuracy is calculated, such as Figure 8 shown.

[0102] from Figure 8It can be seen that the six classifiers all achieved relatively higher average classification accuracy on the signal-position dataset, which is somewhat different from the average classification accuracy they achieved on the pulse-position dataset. This shows that for redundant object positioning, the quality of the dataset constructed by redundant object signals is higher, and the signal perspective is the best perspective. Similarly, XGBoost and random forest achieved the most significant average classification accuracy on the signal-position dataset and the pulse-position dataset, BPNN achieved the lowest average classification accuracy, and the other three classifiers achieved medium average classification accuracy. This is consistent with the trend of the advantages and disadvantages of the classification performance achieved by different classifiers on the signal-material dataset and the pulse-material dataset.

[0103] Similarly, for the signal-position image set and the pulse-position image set, with the help of a customized 6-2-2 ten-fold cross-validation method, they are divided into training set, validation set and test set in a ratio of 6:2:2. Six neural networks are trained on the training set, and the hyperparameters are adjusted on the validation set. The classification accuracy obtained by them on the test set is counted, and the average classification accuracy obtained by the six neural networks is calculated, as shown in Fig. 9 shown.

[0104] from Fig. 9 It can be seen that the six neural networks all achieved relatively higher average classification accuracy on the signal-position picture set, which is somewhat different from the average classification accuracy they achieved on the pulse-position picture set. This shows that for redundant object positioning, the picture set constructed by redundant object signals has higher quality, and the signal perspective is the best perspective. Among them, the two deep convolutional neural networks and the two residual neural networks still achieved more significant average classification accuracy on the signal-position picture set and the pulse-position picture set, and the two lightweight neural networks still achieved the lowest average classification accuracy. This is also consistent with the trend of the advantages and disadvantages of the classification performance achieved by different neural networks on the signal-position picture set and the pulse-position picture set.

[0105] Therefore, the above-mentioned multiple representative classifiers and neural networks all achieve higher average classification accuracy on the data set and picture set constructed by the superfluous signal than on the data set and picture set constructed by the pulse, and the advantage is obvious. This fully demonstrates that for the positioning of superfluous objects, the data set and picture set constructed by the superfluous signal have higher quality, and the signal perspective is the best perspective. The pulse feedbacks the duration of the contact between the superfluous object and the inner wall or internal components of the sealed electronic device, and the background noise feedbacks the duration of the superfluous object in the suspended state. In this way, by analyzing the pulses and background noise that appear in the superfluous signal in time sequence, the detailed motion trajectory of the superfluous object over a period of time can be inferred. The pulse or background noise that appears in the superfluous signal can infer the motion state of the superfluous object at a certain time node over a period of time. This illustrates the importance of comprehensively considering multiple pulses and multiple segments of background noise in the superfluous signal. As for independent pulses, they are all orphans, and they do not know the superfluous signal to which they belong, so it is impossible to infer the motion state of the superfluous object at a certain time node over a period of time based on them. In the acoustic emission source positioning, the motion trajectory of the acoustic emission source fed back in the acoustic emission signal is the phase, which, together with the energy, can be used as important parameters to infer the position of the acoustic emission source. Therefore, the superfluous signal can feed back the phase of the superfluous, while the pulse cannot feed back the phase of the superfluous. More importantly, the energy of the superfluous signal can feed back the distance between the acoustic sensor that captures it and the superfluous. However, the energy of an independent pulse cannot provide any information about the distance between the acoustic sensor and the superfluous. This explains why the superfluous signal is the best perspective for superfluous positioning.

[0106] In addition, the average classification accuracy of the classifier on the signal-position dataset is similar to that of the neural network on the signal-position picture set, with no significant difference. As mentioned above, only the redundant signal composed of pulses and background noise can feedback the phase and energy of the redundant object. Compared with the identification of the material of the redundant object, the background noise at this time is not only not an interference factor, but can play a positive role. Therefore, the time domain and frequency domain features with good performance can effectively feedback the phase and energy of the redundant object, which is an important basis for realizing the location of the redundant object. But at the same time, the feature extractor of the neural network can also extract key feature information from the spectrogram transformed by the complex redundant signal, and these feature information can also effectively feedback the phase and energy of the redundant object. This once again reflects the advanced nature of the feature extractor. In contrast, XGBoost achieved the highest average classification accuracy on the signal-position dataset, which is higher than the average classification accuracy of the six neural networks on the signal-position picture set. This is consistent with the conclusions obtained in the previous study on the location of redundant objects, that is, the classification accuracy of the classifier on the dataset is higher than that of the neural network on the picture set. In addition, the average classification accuracy of 1D-CNN on the signal-position dataset is impressive, close to the average classification accuracy of two deep convolutional neural networks and two residual neural networks on the signal-position image set. This shows the unlimited potential of convolutional neural networks for signal-position datasets. The redundant signal is essentially a time series, which is composed of sampling points arranged at equal intervals. In step 2.3, the four feature vectors constructed by the four segments of redundant signals of the same redundant signal group are converted into a fused feature vector using the fusion feature vector construction method of multi-channel feature encoding. If the four feature vectors are directly retained to form a tensor of the form of 4×11, and then the deep convolutional neural network and the residual neural network are applied, there may be surprising phenomena. This shows that converting the redundant signal into a feature vector and training the classifier is better than converting the redundant signal into a spectrogram and training the neural network.

[0107] In summary, for redundant object localization, the signal perspective is the best perspective, and converting redundant object signals into feature vectors is a better construction process, and the classification performance of the classifier trained in this way is more superior. In addition, XGBoost is the optimal classifier worth considering.

[0108] It should be noted that the research focus of the present invention is to explore the method of identifying and locating the material of the superfluous object, that is, to compare the superiority of the method of identifying and locating the material of the superfluous object with the superfluous object signal or pulse as the research object, rather than to build a high-performance classifier suitable for identifying and locating the material of the superfluous object. It is hoped that by comparing the classification accuracy obtained by the classifier or neural network, the quality of the data set or picture set constructed by the superfluous object signal or pulse can be compared, and then the best viewing angle suitable for identifying and locating the material of the superfluous object can be obtained. Therefore, the classifier and the neural network are only an evaluation tool, and it is hoped that more comprehensive and fair evaluation results can be given by using as many representative classifiers and neural networks of different categories as possible.

[0109] Therefore, in fact, the method for detecting deep information of unwanted objects in sealed electronic devices based on the signal perspective has broken through the conventional cognition of the prior art and provided excellent detection effects for the positioning of unwanted objects. When combined with the identification of unwanted material based on the pulse perspective, the present invention is a comprehensive solution that can achieve both position identification and material identification, and it is also a solution that can ensure that both aspects of detection have excellent effects. At this time, the present invention is a method for detecting deep information of unwanted objects in sealed electronic devices under the signal and pulse perspectives (dual perspectives). The dual perspective will break the traditional signal processing habits and single perspective cognition, especially breaking the single pulse perspective. The unwanted detection provides a more comprehensive detection idea and improves the comprehensive detection effect. Specifically, the method for detecting deep information of unwanted objects in sealed electronic devices under the dual perspective is as follows:

[0110] For the sealed electronic device to be tested, a signal group of the excess object to be tested is collected to obtain the material and position of the excess object. The specific process includes the following steps:

[0111] Step 1: From the signal perspective, the values ​​of the 11 time domain and frequency domain features shown in Table 4 are calculated on each segment of the residual signal group to be tested, and four feature vectors are constructed. The fused feature vector construction method of multi-channel feature coding is used to transform the four feature vectors into a fused feature vector, which is called the test data.

[0112] Step 2: Use XGBoost to give the predicted labels of the data to be tested. Refer to Table 6 to obtain the location of the redundant objects, thereby completing the redundant object positioning.

[0113] Step 3: From the pulse perspective, use a two-level variable multi-threshold pulse extraction algorithm to process each segment of the redundant signal in the detected redundant signal group and extract the pulses therein. Use a threshold judgment search pulse matching algorithm to process the four segments of the redundant signal in the detected redundant signal group and match the pulse group therein.

[0114] Step 4: Convert each pulse in the pulse group into a spectrogram to obtain four spectrograms. Use the multi-channel integrated spectrogram construction method to convert the four spectrograms into a fused spectrogram, which is called the spectrogram to be tested. Multiple pulse groups in the redundant signal group obtain multiple spectrograms to be tested.

[0115] Step 5: Use VGG19 to give predicted labels for multiple spectrograms to be tested. Perform majority voting on multiple predicted labels to obtain the label with the most votes. Refer to Table 5 to obtain the material of the redundant object, thereby completing the identification of the redundant object material.

[0116] The following process is used to verify and analyze the generalization performance:

[0117] Similarly, the sealed electronic device in step 1.2 is used as the experimental object, and the six kinds of redundant materials in step 1.1 are placed in seven confined spaces in turn. Through the same experimental setting and processing process, 20 redundant signal groups representing the redundant samples of each material placed in each confined space are obtained, totaling 20×5×7=700 redundant signal groups. Labels of material and position are added to each redundant signal group. The 700 redundant signal groups are processed in turn using the two-level variable multi-threshold pulse extraction algorithm and the threshold judgment search pulse matching algorithm to obtain 55419 pulse groups. On this basis, through the same processing process as step 2, four verification data sets are established, namely the signal-material verification data set, the signal-position verification data set, the pulse-material verification data set and the pulse-position verification data set, and four verification picture sets are established, namely the signal-material verification picture set, the signal-position verification picture set, the pulse-material verification picture set and the pulse-position verification picture set. Table 7 and Table 8 give detailed descriptions of the verification data set and verification picture set, respectively.

[0118] Table 7 Description of the validation dataset and validation image set for redundant material recognition

[0119]

[0120]

[0121] Table 8 Description of the validation dataset and validation image set for redundant object positioning

[0122]

[0123] Each validation dataset and each validation picture set are randomly divided into ten data subsets and ten picture subsets respectively. On this basis, the six classifiers trained in step 3.1 are tested on the ten data subsets of the signal-material validation dataset and the pulse-material validation dataset respectively, and the six neural networks trained in step 3.1 are tested on the ten picture subsets of the signal-material validation picture set and the pulse-material validation picture set respectively. The average classification accuracy of the classifiers and neural networks on the validation dataset and validation picture set is calculated as follows: Fig.10 and Fig.11 Similarly, the six classifiers trained in step 3.2 are tested on ten data subsets of the signal-position verification dataset and the pulse-position verification dataset, and the six neural networks trained in step 3.2 are tested on ten image subsets of the signal-position verification picture set and the pulse-position verification picture set, and the average classification accuracy of the classifiers and neural networks on the verification dataset and the verification picture set is calculated, as shown in Fig.12 and Fig.13 shown.

[0124] from Fig.10 and Fig.11 It can be seen that the classifiers all achieved higher average classification accuracy on the pulse-material verification dataset, and the neural network all achieved higher average classification accuracy on the pulse-material verification picture set. In other words, both the classifier and the neural network achieved higher average classification accuracy on the verification dataset and verification picture set constructed by the pulse group. Moreover, among the classifiers, XGBoost also achieved the highest average classification accuracy of 96.15% on the pulse-material verification dataset. Among the neural networks, VGG19 also achieved the highest average classification accuracy of 97.22% on the pulse-material verification picture set. In contrast, VGG19 still achieved an average classification accuracy higher than XGBoost. This is consistent with the conclusions obtained in step 3.1. This once again fully demonstrates that for the recognition of redundant materials, the pulse perspective is the best perspective, and converting pulses into spectrograms is a better construction process. The classification performance of the neural network trained in this way is more superior, and VGG19 is the optimal neural network worthy of consideration.

[0125] from Fig.12 and Fig.13It can be seen that the classifiers all achieved higher average classification accuracy on the signal-position verification dataset, and the neural network all achieved higher average classification accuracy on the signal-position verification picture set. In other words, both the classifier and the neural network achieved higher average classification accuracy on the verification dataset and verification picture set constructed by the redundant object signal group. Moreover, among the classifiers, XGBoost also achieved the highest average classification accuracy of 95.50% on the signal-position verification dataset. Among the neural networks, VGG19 also achieved the highest average classification accuracy of 94.57% on the signal-position verification picture set. In contrast, XGBoost still achieved a higher average classification accuracy than VGG19. This is consistent with the conclusions obtained in step 3.2. This once again fully demonstrates that for redundant object positioning, the signal perspective is the best perspective, and converting redundant object signals into feature vectors is a better construction process. The classification performance of the classifier trained in this way is more superior, and XGBoost is the optimal classifier worthy of consideration.

[0126] It is worth noting that in previous studies, for the identification of redundant materials, pulses were used as the research object, and the pulses were converted into spectrograms to construct a picture set, and the optimal classifier was finally determined to be VGG19. This is consistent with the conclusions drawn by the present invention. In previous studies, for the positioning of redundant objects, pulses were also used as the research object, and the pulses were converted into feature vectors to construct a data set, and the optimal classifier was finally determined to be XGBoost. This is inconsistent with the conclusions drawn by the present invention. Fig.12 It can be seen that the average classification accuracy obtained by the previous study is 95.09%, and the average classification accuracy obtained by the method proposed in this invention is 95.50%. In comparison, the average classification accuracy obtained by the method proposed in this invention is higher. The same conclusion can also be drawn from Figure 8 It can be concluded.

[0127] Therefore, the feasibility and stability of the method proposed in the present invention have been fully verified, and its superiority over previous studies has been fully demonstrated.

[0128] The present invention is used to conduct actual application tests: in a real application scenario, a batch of core components on a certain type of aerospace equipment are tested for excess materials, and it is determined that seven of them have excess materials, which are called the seven sealed electronic devices to be tested. The method proposed in the present invention is used to obtain in-depth information about the excess materials of the seven sealed electronic devices to be tested, including materials and positions, which are called predicted materials and predicted positions. In addition, professionals are invited to open the sealed electronic equipment to be tested to check and confirm the actual materials and actual positions of the excess materials. The above detection and inspection results are shown in Table 9. It should be noted that the material and position labels are given in the table, and the specific material and position can be determined by referring to Tables 5 and 6.

[0129] Table 9 Results of deep information detection of redundant objects in the tested sealed electronic equipment

[0130]

[0131]

[0132] As can be seen from the table, the deep information of the excess materials of the seven sealed electronic devices to be tested was detected correctly, and the practicality of the method proposed in the present invention was fully demonstrated. It is worth noting that the weight of the excess materials detected from the seven sealed electronic devices to be tested was measured, among which the weight of the metal materials was between 1.8mg and 2.6mg, and the weight of the silica gel was 12.2mg, which were all higher than the weight of the excess material sample in step 1.1. This illustrates the correctness of selecting the minimum weight of metal and non-metallic excess materials, which provides a guarantee for the smooth application test in the real application scenario.

[0133] In this invention, for the first time, the redundant object signal is used as the research object to carry out the research on the identification and positioning of the redundant object material, and thus a method for detecting the deep information of the redundant objects in the sealed electronic equipment from the perspective of signals and pulses is proposed. In the real application scenario, sealed electronic equipment is found as the experimental object, and the samples of the redundant objects of different materials are made, thereby collecting a large number of redundant object signal groups with marked materials and positions, and gradually carrying out the research based on them.

[0134] Firstly, the redundant signal is taken as the research object, and feature engineering and spectrogram technology are carried out on the redundant signal group. With the help of the fusion feature vector construction method of multi-channel feature coding and the multi-channel integrated spectrogram construction method, the feature vector and the fusion spectrogram are fused, thereby establishing a data set and a picture set for redundant material recognition and positioning from the signal perspective, thereby training the classifier and neural network for redundant material recognition and positioning from the signal perspective. Secondly, the pulse is taken as the research object, and the pulse group is obtained from the redundant signal group with the help of the two-level variable multi-threshold pulse extraction algorithm and the threshold judgment search pulse matching algorithm. Similarly, the data set and picture set for redundant material recognition and positioning from the pulse perspective are established from the pulse group, thereby training the classifier and neural network for redundant material recognition and positioning from the pulse perspective. Finally, the classification performance of representative classifiers and neural networks for redundant material recognition and positioning from the signal and pulse perspectives is comprehensively analyzed, and the following conclusions are drawn. The pulse perspective is the best perspective for identifying the material of the superfluous material. Converting the pulse into a spectrogram is a better construction process, and VGG19 is the optimal neural network to consider. The signal perspective is the best perspective for locating superfluous objects. Converting the superfluous object signal into a feature vector is a better construction process, and XGBoost is the optimal classifier to consider. The tests in real application scenarios fully verified the feasibility, practicality and superiority of the method proposed in the present invention. The deep information detection results of superfluous objects obtained on multiple sealed electronic devices to be tested are accurate and reliable.

[0135] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for detecting deep information of redundant objects in sealed electronic equipment, characterized in that: include: For the undesired object signals to be tested obtained by N channels, a group of undesired object signals corresponding to the N channels is taken as a undesired object signal group; From the signal perspective, features are extracted from each segment of the redundant object signal in the redundant object signal group, and the features include time domain features and / or frequency domain features. N feature vectors are constructed, and the N feature vectors are converted into a fused feature vector, which is called the data to be tested. A classifier is used to classify the data to be tested to obtain the positioning and recognition results of the redundant objects.

2. The method for detecting deep information of redundant objects in sealed electronic equipment according to claim 1, characterized in that: The classifier used in the process of classifying the data to be tested is XGBoost.

3. The method for detecting deep information of excess matter in sealed electronic equipment according to claim 1 or 2, characterized in that: The method further comprises the following steps: From the pulse perspective, a pulse extraction algorithm is used to extract pulses from each segment of the redundant object signal group, and then a pulse matching algorithm is used to match the N segments of the redundant object signal group to obtain a pulse group; each pulse in the pulse group is converted into a spectrogram to obtain N spectrograms, and the N spectrograms are converted into a fused spectrogram, called a spectrogram for material identification; multiple pulse groups in the redundant object signal group obtain multiple spectrograms for material identification; a deep learning model is used to obtain predicted labels for multiple spectrograms for material identification; a majority vote is performed on multiple predicted labels to obtain a label that receives the most votes, and the material identification result of the redundant object is obtained.

4. The method for detecting deep information of redundant objects in sealed electronic equipment according to claim 3, characterized in that: The deep learning model used in the process of obtaining the predicted labels of multiple spectrograms to be tested is the VGG19 network model.

5. The method for detecting deep information of redundant objects in sealed electronic equipment according to claim 3, characterized in that: The pulse extraction algorithm is used to extract pulses from each segment of the redundant signal in the redundant signal group. The pulse extraction algorithm used is a two-level variable multi-threshold pulse extraction algorithm.

6. The method for detecting deep information of redundant objects in sealed electronic equipment according to claim 3, characterized in that: The pulse matching algorithm used in the process of matching the N segments of the redundant signal in the redundant signal group by using the pulse matching algorithm is a threshold determination search pulse matching algorithm.

7. The method for detecting deep information of redundant objects in sealed electronic equipment according to claim 3, characterized in that: The process of converting N spectrograms into a fused spectrogram includes: For N spectrograms, the values ​​of the three color channels R, G, and B at the same pixel are obtained sequentially, which are N red values, N green values, and N blue values ​​in total; the N values ​​of the three color channels at the same pixel are respectively smoothed and filtered to obtain a processed red value, a green value, and a blue value; the same processing is performed on all pixels in the N spectrograms, and then they are fused to obtain a fused spectrogram.

8. A method for detecting deep information of excess material in sealed electronic equipment, characterized in that: include: For the undesired object signals to be tested obtained by N channels, a group of undesired object signals corresponding to the N channels is taken as a undesired object signal group; From the signal perspective, each redundant object signal in the redundant object signal group is converted into a spectrogram to obtain N spectrograms, which are then converted into a fused spectrogram, called a spectrogram for positioning and recognition. A deep learning model is used to obtain the label of the spectrogram for positioning and recognition, that is, the positioning and recognition result of the redundant object is obtained.

9. The method for detecting deep information of redundant objects in sealed electronic equipment according to claim 8, characterized in that: The method further comprises the following steps: From the pulse perspective, a pulse extraction algorithm is used to extract pulses from each segment of the redundant object signal group, and then a pulse matching algorithm is used to match the N segments of the redundant object signal group to obtain a pulse group; each pulse in the pulse group is converted into a spectrogram to obtain N spectrograms, and the N spectrograms are converted into a fused spectrogram, called a spectrogram for material identification; multiple pulse groups in the redundant object signal group obtain multiple spectrograms for material identification; a deep learning model is used to obtain predicted labels for multiple spectrograms for material identification; a majority vote is performed on multiple predicted labels to obtain a label that receives the most votes, and the material identification result of the redundant object is obtained.

10. The method for detecting deep information of redundant objects in sealed electronic equipment according to claim 8 or 9, characterized in that: The process of converting N spectrograms into a fused spectrogram includes: For N spectrograms, the values ​​of the three color channels R, G, and B at the same pixel are obtained sequentially, which are N red values, N green values, and N blue values ​​in total; the N values ​​of the three color channels at the same pixel are respectively smoothed and filtered to obtain a processed red value, a green value, and a blue value; the same processing is performed on all pixels in the N spectrograms, and then they are fused to obtain a fused spectrogram.

Images