Method and system for detecting abnormal plugging of wire harness

Abstract

The invention relates to the field of industrial detection, in particular to a wire harness plugging abnormity detection method and system, and the method comprises the steps: carrying out the Mel-spectrum transformation of a sound signal to obtain a frequency signal, extracting a Mel-spectrum feature map of the frequency signal, carrying out the transposition and remodeling of the Mel-spectrum feature map, and carrying out the detection of the plugging abnormity of a wire harness. The method comprises the following steps: obtaining a first serialized feature, obtaining a serialized feature, carrying out accumulation processing on the serialized feature to obtain a second serialized feature, carrying out classification detection on the second serialized feature to obtain an anomaly detection result, and the anomaly detection result comprises a model category, a prediction probability and an anomaly mark. The system comprises a sound input device, a Mel spectrum converter and a sound recognition model. Compared with visual detection, the cost is lower, and the defect that visual detection depends too much on the working environment can be overcome. The Mel spectrum converter and the sound recognition model are adjusted and optimized, so that the method has a better detection effect on the wiring harness plugging sound.

Description

Technical Field

[0001] The present application relates to the field of industrial detection, and in particular to a method and system for detecting abnormal wiring harness plugging and unplugging. Background Art

[0002] In recent years, visual inspection technology has made significant progress in the field of industrial inspection technology. Due to its advantages of high precision and high speed, visual inspection technology has been widely used in various fields of manufacturing, such as appearance defect detection, dimension measurement and color recognition. When inspecting the wiring harness plug-in and unplug process, the most widely used method is also visual inspection. However, this method has obvious defects: Industrial site environments are prone to factors such as light, dust, workpiece reflection, and vibration, which can reduce the accuracy of visual inspection; Visual inspection requires high-resolution cameras and complex image processing systems, which are expensive to develop and maintain. Mechanical devices are expensive to install and maintain, and require manual calibration and replacement on a regular basis. Visual inspection has high requirements on the installation position and angle of the camera and optical sensor. There must be no obstruction, and the process of wiring harness plugging and unplugging is difficult to be captured by the camera.

[0003] Therefore, the visual inspection method is not completely applicable to the detection of abnormal wiring harness plugging and unplugging. Summary of the invention

[0004] The technical problem to be solved by this application is that it is difficult to detect abnormal wiring harness plugging and unplugging.

[0005] In a first aspect, the present application provides a method for detecting abnormal wiring harness plugging and unplugging, comprising the following steps: Acquire the sound signal during the wiring harness plugging and unplugging process; Performing Mel spectrum transformation on the sound signal to obtain a frequency signal; Extracting a Mel-spectrogram feature map of the frequency signal; Transposing and reshaping the Mel spectrum feature map to obtain serialized features; Accumulating the serialized features using a long short-term memory network to obtain a second serialized feature; The second serialization feature is classified and detected to obtain an abnormality detection result, wherein the abnormality detection result includes a predicted category of the second serialization feature of the wiring harness plugging and unplugging process as normal or abnormal, a predicted probability, and an abnormality mark.

[0006] Specifically, performing Mel spectrum transformation on the sound signal to obtain a frequency signal comprises the following steps: Adjust processing parameters to obtain processed parameters; Denoising the sound signal to obtain an enhanced signal, and performing a Mel spectrum transform on the enhanced signal according to the processed parameters to obtain a first frequency signal; Performing logarithmic amplitude transformation on the first frequency signal to obtain the frequency signal; The adjustment processing parameters include: Adjust the original lowest frequency parameter and the original highest frequency parameter; Shorten the original analysis window step size; Adjust the high resolution parameters.

[0007] Specifically, the step of transposing and reshaping the Mel spectrum feature map includes: Rearranging and flattening the Mel frequency spectrum feature map to obtain an adapted feature; The adapted features are transposed and reshaped to obtain the serialized features, which are tensors in the format of "batch × number of time frames × feature dimension".

[0008] Specifically, the method for accumulating the serialized features includes: Context information of the serialization feature is accumulated to obtain the second serialization feature.

[0009] Specifically, the method for classifying and detecting the second serialization feature includes: The second serialization feature is mapped to a category space to obtain a classification feature, and a classification function is used to calculate a probability distribution of the classification feature in the category space to obtain the anomaly detection result.

[0010] In a second aspect, the present application provides a wiring harness plug-in abnormality detection system, which is operated using the detection method as described above, and includes: A sound input device, used for collecting sound signals; A mel spectrum converter, used for receiving the sound signal collected by the sound input device, and performing a mel spectrum conversion on the sound signal to obtain a frequency signal; The sound recognition model includes convolutional pooling layers, reshape transposition layers, long short-term memory network layers, and classification detection layers; The convolutional pooling layer is used to extract the Mel spectrum feature map of the frequency signal; The reshaping transposition layer is used to transpose and reshape the Mel spectrum feature map to obtain serialized features; The long, short and memory network layer is used to process the serialization feature to obtain a second serialization feature; The classification detection layer is used to perform classification detection on the second serialization feature to obtain an anomaly detection result, and the anomaly detection result includes a model category, a prediction probability and an anomaly label.

[0011] Specifically, the Mel spectrum converter is provided with a denoising module, an adjustment module and a logarithmic amplitude conversion module, the denoising module is used to denoise the sound signal; the adjustment module is used to increase the original lowest frequency parameter and the original highest frequency parameter of the Mel spectrum converter; shorten the original analysis window step of the Mel spectrum converter; increase the original frequency resolution parameter of the Mel spectrum converter; The logarithmic amplitude conversion module is used to perform logarithmic amplitude conversion on the first frequency signal obtained by the Mel spectrum converter to obtain the frequency signal.

[0012] Specifically, the sound recognition model is further provided with a discard layer, and the discard layer is arranged before the classification detection layer to suppress overfitting of the classification detection layer.

[0013] As a preferred solution, the sound input device is arranged in a soundproof room.

[0014] As a preferred solution, the inner top wall of the soundproof room is provided with a crossbeam, a sound collecting cover is slidably mounted on the crossbeam, and the sound input device is installed in the sound collecting cover.

[0015] This application has the following technical effects: A detection method based on sound recognition is proposed to detect abnormal wiring harness plugging and unplugging. Compared with visual detection, sound recognition has lower installation and training costs, lower later maintenance costs, and can make up for the defect that visual detection is too dependent on the working environment. It can still perform high-precision abnormal wiring harness plugging and unplugging detection in dusty, poor light conditions, and obstructed environments. In addition, the present application adjusts and optimizes the Mel spectrum converter and the sound recognition model according to the characteristics of the wiring harness plugging and unplugging sound, so that the detection method and detection system of the present application have better detection effect for the wiring harness plugging and unplugging sound. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-restrictive manner, and the same or corresponding numbers represent the same or corresponding parts.

[0017] Figure 1 It is a flow chart of a method for detecting abnormal wiring harness plugging and unplugging according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0019] It should be understood that when the terms "first", "second", etc. are used in the claims, specification and drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0020] When detecting abnormalities in wiring harness plugging and unplugging at industrial sites, if visual detection methods are used, it will be found that there will be interference from many environmental factors, resulting in unsatisfactory visual detection results. In order to solve this problem, the inventors of the present application have observed that when the wiring harness is plugged and unplugged, the docking interface of the wiring harness will physically vibrate, thereby generating a short and high-frequency sound. Such a sound can be well captured and distinguished from other noises even in the presence of noise at the industrial site. Based on this, the inventors of the present application collected samples of wiring harness plugging and unplugging sounds, and trained a sound recognition model that can recognize wiring harness plugging and unplugging sounds based on these samples. Subsequently, a method and system for detecting abnormalities in wiring harness plugging and unplugging was further proposed, and an embodiment was given.

[0021] In this embodiment, a soundproof room is set up around the workstation where the wiring harness is plugged in and out. The soundproof room is composed of soundproof panels spliced ​​together and installed on a frame, which can be easily adjusted according to the size and shape of the workstation. A sound collecting cover is set on the crossbeam of the frame. The sound collecting cover is designed to be bowl-shaped to enhance the sound collection effect, and a sound input device such as a microphone is installed in the sound collecting cover through a telescopic rod. The microphone can slide on the crossbeam following the sound collecting cover, and constantly adjust its position to obtain the best sound collection effect. At the same time, such a setting is also conducive to further adjusting the position of the microphone when the evaluation index of the detection result of the sound recognition model is unqualified.

[0022] The detection method of abnormal wiring harness plugging and unplugging is as follows: Figure 1 The steps shown are performed, the steps comprising: S1. Acquire the sound signal during the wiring harness plugging and unplugging process.

[0023] S2. Perform a Mel spectrum transformation on the sound signal to obtain a frequency signal. Specifically, when performing the Mel spectrum transformation, appropriately set the minimum frequency parameter (fmin) and the maximum frequency parameter (fmax), and focus the feature extraction on the region of interest (ROI) corresponding to the high frequency band. Normally, fmin is set to 0Hz and fmax is set to half of the sampling rate. However, in this embodiment, because the sound of wiring harness plugging and unplugging is short and high-frequency, in order to shield the influence of low-frequency noise, fmin is increased to more than 100Hz. At the same time, in order to focus on the high frequency band, fmax is also appropriately increased.

[0024] In addition, since the sound of the wiring harness plugging and unplugging is short and changes very quickly, the adjustments that need to be made include shortening the step size (hop_length) of the analysis window appropriately, so that the Mel spectrum transform has a higher resolution on the time axis, so as to more sensitively track high-frequency acoustic events that change rapidly, and selecting a higher resolution parameter (n_fft) to further optimize the feature mapping, so that the originally weak high-frequency subtle features of the wiring harness plugging and unplugging sound can be highlighted under the logarithmic scale conversion, thus laying a solid foundation for the subsequent model to identify highly dynamic and highly complex wiring harness plugging and unplugging sounds. The specific principle is, according to the formula: in, x (n) is the input signal, w (n) is the window function, R is the hop length, m is the time frame, w This application uses a shorter step length (hop_length = 32) combined with a higher number of Mel frequencies (n_mels = 512), so the time frame can be increased m The number of time domain resolution is improved.

[0025] After the adjustment is completed, the denoised sound signal is transformed into a Mel spectrum, and the first frequency signal obtained by the transformation is transformed into a logarithmic amplitude transform (log-scale) to obtain a frequency signal, so that the weak characteristics of the high-frequency part of the wiring harness plugging and unplugging sound are effectively highlighted on a relative scale, thereby achieving effective amplification of the high-frequency characteristics and ensuring the complete presentation of the overall audio frequency domain information.

[0026] S3. Extract the Mel spectrum feature map of the frequency signal. Specifically, the frequency signal is input into the convolutional pooling layer, which consists of two convolutional layers, each of which is provided with a ReLU activation function and a pooling layer. These layers are used to extract high-level time-frequency features and reduce the dimension of the feature map.

[0027] S4. Transpose and reshape the Mel spectrum feature map to obtain serialized features. Specifically, it includes: rearranging and flattening the Mel spectrum feature map to form a shape suitable for the input to obtain adaptive features; Transpose and reshape the adaptive features to obtain serialized features, which are tensors in the format of "batch × number of time frames × feature dimension". For example, the initial input of the sound signal is 0.8 seconds and the sampling rate is 44100Hz, so the size of the input tensor is [n,35280], where n is the batch size; the output size after the Mel spectrum transformation of the sound signal is [n,1,512,1103], where 512 represents the number of Mel frequency bands and 1103 represents the number of time frames; after the Mel spectrum feature map is extracted, the size of the feature map changes from [n,1,512,1103] to [n,64,128,275]; after the Mel spectrum feature map is transposed and reshaped, the size of the feature map becomes [n,275,1024], where 275 represents the number of time frames and 1024 is the hidden state dimension of LSTM. After the above size transformation, the audio signal can be input into LSTM.

[0028] S5. Accumulate the serialized features to obtain a second serialized feature. Specifically, it includes: accumulating the context information of the serialized features to obtain the second serialized feature. The serialized features are input into the long short-term memory network layer (LSTM), and the context information of the serialized features is accumulated through the input gate, forget gate, output gate and memory unit inside the LSTM, so that the detection method of this embodiment is capable of coping with the temporal changes of short-term audio events.

[0029] S6. Classify and detect the second serialization feature to obtain an abnormality detection result, which includes the predicted category of the second serialization feature of the wiring harness plugging and unplugging process as normal or abnormal, the predicted probability and the abnormality mark. Specifically, it includes: mapping the second serialization feature to the category space to obtain the classification feature, and using the classification function such as the softmax function to calculate the probability distribution of the classification feature in the category space to obtain the abnormality detection result. When an abnormal mark appears, an alarm signal can be sent to notify the operator to handle the abnormality.

[0030] The operation of the detection method of this embodiment requires the use of a corresponding detection system, including: a sound input device for collecting sound signals; A mel spectrum converter, used for receiving a sound signal collected by a sound input device, and performing a mel spectrum conversion on the sound signal to obtain a frequency signal; The sound recognition model includes convolutional pooling layers, reshape transposition layers, long short-term memory network layers, and classification detection layers; The convolutional pooling layer is used to extract the Mel spectrum feature map of the frequency signal; The reshape transposition layer is used to transpose and reshape the Mel spectrum feature map to obtain serialized features; The long, short and memory network layers are used to process the serialized features to obtain the second serialized features; The classification detection layer is used to perform classification detection on the second serialized feature to obtain anomaly detection results, which include model categories, prediction probabilities, and anomaly labels.

[0031] Specifically, the mel spectrum converter is provided with an adjustment module and a logarithmic amplitude conversion module, the adjustment module is used to increase the fmin parameter and fmax parameter of the mel spectrum converter; shorten the analysis window step of the mel spectrum converter; increase the frequency resolution parameter of the mel spectrum converter; The logarithmic amplitude conversion module is used to perform logarithmic amplitude conversion on the first frequency signal obtained by the Mel spectrum converter to obtain a frequency signal.

[0032] Specifically, the sound recognition model is further provided with a dropout layer (Dropout), and the dropout layer (Dropout) is provided before the classification detection layer to suppress overfitting of the classification detection layer.

[0033] Although this specification has shown and described a plurality of embodiments of the present application, it is obvious to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will think of many changes, modifications and alternatives without departing from the thought and spirit of the present application. It should be understood that in the process of practicing the present application, various alternatives to the embodiments of the present application described herein may be adopted.

[0034] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for detecting abnormal wiring harness plugging and unplugging, characterized in that: The following steps are involved: Acquire the sound signal during the wiring harness plugging and unplugging process; Performing Mel spectrum transformation on the sound signal to obtain a frequency signal; Extracting a Mel-spectrogram feature map of the frequency signal; Transposing and reshaping the Mel spectrum feature map to obtain serialized features; Accumulating the serialized features using a long short-term memory network to obtain a second serialized feature; Performing classification detection on the second serialization feature to obtain an abnormality detection result, wherein the abnormality detection result includes a predicted category of the second serialization feature of the wiring harness plugging and unplugging process as normal or abnormal, a predicted probability, and an abnormality mark; Performing Mel spectrum transformation on the sound signal to obtain a frequency signal comprises the following steps: Adjust processing parameters to obtain processed parameters; Denoising the sound signal to obtain an enhanced signal, and performing a Mel spectrum transform on the enhanced signal according to the processed parameters to obtain a first frequency signal; Performing logarithmic amplitude transformation on the first frequency signal to obtain the frequency signal; The adjustment processing parameters include: Adjust the original lowest frequency parameter and the original highest frequency parameter; Shorten the original analysis window step size; Adjust the high resolution parameters.

2. The detection method according to claim 1, characterized in that: The step of transposing and reshaping the Mel spectrum feature map comprises: Rearranging and flattening the Mel frequency spectrum feature map to obtain an adapted feature; The adapted features are transposed and reshaped to obtain the serialized features, which are tensors in the format of "batch × number of time frames × feature dimension".

3. The detection method according to claim 1, characterized in that: The method for accumulating the serialized features comprises: Context information of the serialization feature is accumulated to obtain the second serialization feature.

4. The detection method according to claim 1, characterized in that: The method for classifying and detecting the second serialization feature comprises: The second serialization feature is mapped to a category space to obtain a classification feature, and a classification function is used to calculate a probability distribution of the classification feature in the category space to obtain the anomaly detection result.

5. A wiring harness plug-in abnormality detection system, operated using the detection method according to any one of claims 1 to 4, characterized in that: include: A sound input device, used to collect sound signals during the wiring harness plugging and unplugging process; A mel spectrum converter, used for receiving the sound signal collected by the sound input device, and performing a mel spectrum conversion on the sound signal to obtain a frequency signal; The sound recognition model includes convolutional pooling layers, reshape transposition layers, long short-term memory network layers, and classification detection layers; The convolutional pooling layer is used to extract the Mel spectrum feature map of the frequency signal; The reshaping transposition layer is used to transpose and reshape the Mel spectrum feature map to obtain serialized features; The long, short and memory network layer is used to accumulate the serialization features to obtain a second serialization feature; The classification detection layer is used to perform classification detection on the second serialization feature to obtain an anomaly detection result, wherein the anomaly detection result includes a model category, a prediction probability, and an anomaly label; The Mel spectrum converter is provided with a denoising module, an adjustment module and a logarithmic amplitude conversion module. The denoising module is used to denoise the sound signal; the adjustment module is used to increase the original lowest frequency parameter and the original highest frequency parameter of the Mel spectrum converter; shorten the original analysis window step of the Mel spectrum converter; and increase the original frequency resolution parameter of the Mel spectrum converter; The logarithmic amplitude conversion module is used to perform logarithmic amplitude conversion on the first frequency signal obtained by the Mel spectrum converter to obtain the frequency signal.

6. The detection system according to claim 5, characterized in that: The sound recognition model is also provided with a discard layer, which is arranged before the classification detection layer and is used to suppress overfitting of the classification detection layer.

7. The detection system according to claim 5, characterized in that: The sound input device is arranged in a soundproof room.

8. The detection system according to claim 7, characterized in that: The inner top wall of the soundproof room is provided with a crossbeam, a sound collecting cover is slidably mounted on the crossbeam, and the sound input device is installed in the sound collecting cover.

Images