Method, device and equipment for detecting 3D printing equipment failure and storage medium

Abstract

Embodiments of the present application disclose a 3D printing equipment fault detection method, device, equipment and storage medium; the detection method comprises: acquiring a voiceprint image corresponding to audio data collected in real time in the running process of a target 3D printing equipment; inputting the voiceprint image into a trained neural network model to determine the fault type in the running process of the target 3D printing equipment.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of additive manufacturing (commonly known as 3D printing), in particular to a 3D printing equipment fault detection method, device, equipment and storage medium. BACKGROUND

[0002] In the process of laser forming of parts by using three-dimensional (3-Dimension, 3D) printing technology, various mechanical equipment and electrical equipment in the 3D printing equipment need to be operated in cooperation to ensure the good operation of the 3D printing equipment and to manufacture qualified parts. However, in the process of manufacturing parts by using the 3D printing equipment, the friction force, impact force or unbalanced force between the parts in the long-time cooperation process of a large number of mechanical equipment and electrical equipment can cause irregular vibration between the metal plates, bearings, gears and housings of the 3D printing equipment, and further cause impact noise, excitation noise, friction noise, structure noise, gear noise, bearing noise and other types of noise. In the 3D printing equipment, the noise emitted by the gear transmission box, the galvanometer light output, the motor and the fan is the typical mechanical noise. The characteristics (such as sound level, frequency characteristics and time characteristics) of the mechanical noise are related to the excitation characteristics, the speed of the surface vibration of the 3D printing equipment and the inherent vibration mode factors of the boundary conditions.

[0003] On the other hand, due to the large number of 3D printing equipment models and the irregular vibration of each 3D printing equipment, the noises generated are not completely the same, and the influence of the same kind of noise on different models of 3D printing equipment is also different, such as the vibration amplitude caused by the noise also affects the forming quality of the parts. The ordinary noise sensor on the market cannot cope with the variable noise environment of the 3D printing equipment, and the noise parameters suitable for a 3D printing equipment cannot be well extended to other 3D printing equipment, so it cannot monitor all running 3D printing equipment in batches. However, in the 3D printing production process, the stable operation of the 3D printing equipment is crucial to the production line. In order to ensure the stability of the 3D printing equipment, a large number of personnel need to be invested in daily maintenance, even so, unplanned downtime occurs from time to time, which may cause damage to the 3D printing equipment or even lead to the shutdown of the production line. SUMMARY

[0004] Therefore, the embodiment of the present application aims to provide a 3D printing equipment fault detection method, device, equipment and storage medium, which can analyze the running state of the 3D printing equipment in real time and ensure the forming quality of the 3D printed parts.

[0005] The technical scheme of the embodiment of the present application is as follows:

[0006] In a first aspect, embodiments of the present application provide a 3D printing equipment fault detection method, the detection method comprising:

[0007] For the audio data collected in real time in the running process of the target 3D printing equipment, an acoustic fingerprint image corresponding to the audio data is acquired;

[0008] The acoustic fingerprint image is input into a trained neural network model to determine the fault type in the running process of the target 3D printing equipment.

[0009] In a second aspect, embodiments of the present application provide a 3D printing equipment fault detection device, the detection device comprising a first acquisition part and a determination part; wherein,

[0010] The first acquisition part is configured to acquire an acoustic fingerprint image corresponding to audio data collected in real time in the running process of the target 3D printing equipment;

[0011] The output part is configured to input the acoustic fingerprint image into a trained neural network model to determine the fault type in the running process of the target 3D printing equipment.

[0012] In a third aspect, embodiments of the present application provide a 3D printing equipment fault detection device, the detection device comprising an audio data collector, a memory and a processor; wherein,

[0013] The audio data collector is used to collect audio data in the running process of the target 3D printing equipment in real time, and to collect multiple sample audio data in the running process of the target 3D printing equipment;

[0014] The memory is used to store a computer program capable of running on the processor;

[0015] The processor is used to execute the steps of the 3D printing equipment fault detection method of the first aspect when the computer program is running.

[0016] In a fourth aspect, embodiments of the present application provide a storage medium, the storage medium storing a 3D printing equipment fault detection method program, the 3D printing equipment fault detection method program being executed by at least one processor to implement the steps of the 3D printing equipment fault detection method of the first aspect.

[0017] The embodiment of the application provides a 3D printing equipment fault detection method, device and equipment and a storage medium; first, audio data in a running process of a target 3D printing equipment is collected in real time, and a corresponding voiceprint image is acquired; then the voiceprint image is input into a trained neural network model to determine a fault type in the running process of the target 3D printing equipment. The detection method can analyze the running state of the 3D printing equipment in real time, can intelligently diagnose possible running problems of the 3D printing equipment, can monitor information such as abnormal noise and abnormal amplitude in real time, can discover hidden dangers in time, can make maintenance arrangements in advance, can reserve raw materials, can coordinate downtime, can avoid losses caused by unplanned downtime, and further realizes digitalization, intelligentization, scientification and systematization of management of the 3D printing equipment. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 An SLM equipment structure schematic diagram provided by the embodiment of the application;

[0019] Figure 2 A 3D printing equipment fault detection method flowchart provided by the embodiment of the application;

[0020] Figure 3 An initial sub-neural network model structure schematic diagram provided by the embodiment of the application; Figure 4 A process schematic diagram of training an initial sub-neural network model based on a sample voiceprint image of one scale provided by the embodiment of the application;

[0021] Figure 5 An initial neural network model structure schematic diagram provided by the embodiment of the application;

[0022] Figure 6 Another initial neural network model structure schematic diagram provided by the embodiment of the application;

[0023] Figure 7 Still another initial neural network model structure schematic diagram provided by the embodiment of the application;

[0024] Figure 8 A frequency spectrum image schematic diagram when a scraper card part occurs in the 3D printing equipment provided by the embodiment of the application;

[0025] Figure 9 A frequency spectrum image schematic diagram when a scraper card occurs in the 3D printing equipment provided by the embodiment of the application;

[0026] Figure 10 A frequency spectrum image schematic diagram when a fan abnormal rotation sound occurs in the 3D printing equipment provided by the embodiment of the application;

[0027] Figure 11This is a schematic diagram of the spectrum of a 3D printing device when the powder-feeding shaft jams, as provided in an embodiment of the present invention.

[0028] Figure 12 This is a schematic diagram of the spectrum of a circuit breaker tripping due to electrical component failure in a 3D printing device, provided in an embodiment of the present invention.

[0029] Figure 13 A schematic diagram of the spectrum image of an abnormal light output from a galvanometer in a 3D printing device provided in an embodiment of the present invention;

[0030] Figure 14 A schematic diagram of the spectrum of abnormal motor noise in a 3D printing device provided in an embodiment of the present invention;

[0031] Figure 15 A schematic diagram of a 3D printing equipment fault detection device provided in an embodiment of the present invention;

[0032] Figure 16 A schematic diagram of another 3D printing equipment fault detection device provided in an embodiment of the present invention;

[0033] Figure 17 This is a schematic diagram of a 3D printing equipment fault detection device provided in an embodiment of the present invention. Detailed Implementation

[0034] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0035] Before describing the embodiments of the present invention in detail, it should be noted that, with Figure 1 The Selective Laser Melting (SLM) device 1 shown is used as an example to illustrate the malfunction problem of the 3D printing equipment involved in the embodiments of the present invention. However, the problem of the 3D printing equipment malfunction detection method involved in the embodiments of the present invention is not limited to SLM device 1, and can also be applied to other 3D printing equipment, such as Laser Solid Forming (LSF) equipment.

[0036] like Figure 1As shown, the SLM device 1 mainly comprises a forming chamber 101, a laser generator 102, a substrate 103 arranged in the forming chamber 101, a powder feeder 104, a horizontal scraper 105 and a lifting device 106; wherein the substrate 103 is used for carrying the formed part; the powder feeder 104 is used for storing metal powder or non-metal powder; the horizontal scraper 105 is used for evenly laying the metal powder or non-metal powder on the substrate 103; and the lifting device 106 is used for lifting and lowering the substrate 103 and the powder feeder 104. It can be understood that a window 107 is arranged above the forming chamber 101, so that the laser beam emitted by the laser generator 102 can pass through the window 107 to irradiate on the metal powder or non-metal powder laid on the substrate 103 to melt the metal powder or non-metal powder.

[0037] Specifically, the SLM technology based on the above-mentioned SLM device 1 is a new generation of part processing technology, and its principle is to use metal powder or non-metal powder to completely melt under the heat action of the laser beam, and to form a new technology by solidifying after cooling. This technology can quickly, directly and accurately convert design ideas into physical models with certain functions. Specifically, first, the CAD three-dimensional model of the part is sliced by using 3D printing slicing software to obtain the layered scanning data of the part, and the data is imported into the SLM device 1; second, the horizontal scraper 105 evenly lays a thin layer of metal powder or non-metal powder on the substrate 103, and the laser generator 102 emits a high-energy laser beam through the window 107 to selectively melt the metal powder or non-metal powder on the substrate 103 according to the data information of the current layer of the three-dimensional model, and the laser forms the shape of the current layer of the part; then the substrate 103 is lowered by a certain distance, which can be understood as the thickness of the next layer of laser-formed parts. The horizontal scraper 105 lays another layer of metal powder on the current layer of the formed part, and the laser generator 102 emits a high-energy laser beam through the window 107 to selectively melt according to the data information of the next layer of the model, and the melted layer is automatically bonded with the formed part. This cycle is repeated until the entire part is completed.

[0038] It should be noted that the SLM device 1 may have various failure problems during operation due to the cooperation between the internal mechanical devices and electrical devices of the SLM device 1 to hinder the normal operation of the device. Common failure types can include: abnormal sound of motor, abnormal rotating sound of fan, scraper stuck parts, scraper stuck, powder falling shaft stuck, galvanometer light abnormality, electrical component damage trip. However, the current detection method cannot accurately and timely detect the failure type and failure position in the SLM device 1, so that the SLM device 1 can only be non-planned downtime to find the component position and reason of the failure, which will affect the production efficiency of the production line. On the other hand, if the failure problem cannot be detected and solved in time, it will also cause damage to the SLM device 1 during long-term operation, which will affect the service life of the SLM device 1 and the forming quality of the parts.

[0039] It can be understood that the above failure problems cause different noises in the SLM device 1 during operation, and the features corresponding to the different noises can identify the failure type and failure position in the SLM device 1. Therefore, based on the above description, referring to Figure 2 , it shows that the embodiment of the present application provides a detection method for 3D printing equipment failure, which specifically comprises:

[0040] S201, for the audio data collected in real time in the running process of the target 3D printing equipment, acquiring the voiceprint image corresponding to the audio data;

[0041] S202, inputting the voiceprint image into the trained neural network model to determine the failure type in the running process of the target 3D printing equipment.

[0042] It should be noted that, as Figure 1 shown, the audio data collector 108 is arranged on the inner wall of the forming chamber 101 in the embodiment of the present application, which is used to collect the audio data generated in the running process of the SLM device 1 and the audio data around the SLM device 1 in the running process of the SLM device 1. It can be understood that the audio data collector 108 can also be arranged on the outer wall of the forming chamber 101, and the arrangement position and number of the audio data collector 108 are not limited in the embodiment of the present application. Specifically, the audio data collector 108 can be a high-sensitivity pickup.

[0043] For Figure 2In the technical solution shown, first, audio data in the running process of the target 3D printing equipment is collected in real time, and a corresponding voiceprint image is obtained; then the voiceprint image is input into a trained neural network model to determine the fault type in the running process of the target 3D printing equipment. This detection method can analyze the running state of the 3D printing equipment in real time, intelligently diagnose possible running problems of the 3D printing equipment, monitor abnormal noise, abnormal amplitude and other information in real time, discover hidden dangers in a timely manner, make maintenance arrangements in advance, reserve raw materials, coordinate downtime, avoid losses caused by unplanned downtime, and further realize digital, intelligent, scientific and systematic management of 3D printing equipment.

[0044] As some possible implementations, the voiceprint image corresponding to the audio data collected in real time in the running process of the target 3D printing equipment is obtained, including:

[0045] Real-time collection of original audio data in a preset time length in the running process of the target 3D printing equipment;

[0046] The collected original audio data is denoised to obtain the final collected audio data;

[0047] Based on the denoised audio data, the audio spectrum of the audio data is obtained; wherein the audio spectrum is used to represent the voiceprint image.

[0048] It should be understood that, since the 3D printing equipment is in a variable noise environment, the collected original audio data may contain some noise irrelevant to fault detection, such as surrounding human voice, forklift sound and device alarm siren sound, etc. Therefore, the original audio data collected by the audio data collector 108 needs to be denoised. Specifically, the collected original audio data can be subjected to speech enhancement processing to extract as pure audio data as possible from the original audio data containing irrelevant noise. In the embodiment of the present application, the speech enhancement processing method can be a least mean square (LMS) adaptive filter, an adaptive notch filter, and a basic spectral subtraction method. At the same time, the collected original audio data can be subjected to speech separation processing to separate the required audio data from the irrelevant noise. In the embodiment of the present application, the speech separation processing method can be based on independent component analysis method, based on computational auditory scene analysis method, and based on spatial filtering method. In the embodiment of the present application, the collected original audio data can be subjected to speech denoising processing by using a recursive average noise algorithm to remove the interference of irrelevant noise in the collected original audio data. In the embodiment of the present application, the method of speech enhancement, speech separation and speech denoising of the original audio data is not specifically limited.

[0049] It should be noted that when obtaining the audio spectrum from the denoised audio data, either the Fast Fourier Transform (FFT) algorithm or the Mel-Frequency Spectrum (MFC) algorithm can be used, and the embodiments of the present invention do not specifically limit this.

[0050] As some possible implementations, before acquiring the audio data during the real-time operation of the target 3D printing equipment, the detection method further includes:

[0051] Multiple sample audio data were collected during the operation of the target 3D printing equipment, and corresponding multiple sample voiceprint images were obtained respectively;

[0052] Construct N initial sub-neural network models; where N≥2;

[0053] The N initial sub-neural network models are trained based on the multiple sample voiceprint images at different scales to obtain the corresponding N trained sub-neural network models; wherein, an initial sub-neural network model is trained based on sample voiceprint images at one scale.

[0054] An initial neural network model is constructed based on the N trained sub-neural network models;

[0055] The initial neural network models are trained using multiple maximum-scale sample voiceprint images used to train the N initial neural network models to obtain the trained neural network models.

[0056] Understandably, in this embodiment of the invention, the collected sample audio data also needs to be denoised beforehand. The specific denoising method is consistent with the method described in the foregoing technical solution, and will not be repeated here. Similarly, the method for obtaining sample voiceprint images is the same as the method for obtaining voiceprint images described above, and will not be repeated here. It should be noted that in this embodiment of the invention, multiple sample audio data can be collected in real time using the audio data acquisition device 108, or multiple sample audio data can be selected from a pre-established sample audio database to train the initial neural network model.

[0057] like Figure 3 As shown, it exemplarily illustrates any initial sub-neural network model constructed by... Figure 3 As can be seen, the initial sub-neural network model described above includes a sub-feature extraction network, a sub-decision output network, and a sub-result output network.

[0058] In this embodiment of the invention, the training of three sample voiceprint images at three scales—320×320, 228×228, and 128×128—is described in detail below as an example.Figure 3 The process of the initial sub-neural network model is shown in the middle, wherein in the present example, the sub-feature extraction network of the initial sub-neural network model can be a convolutional neural network (CNN), which contains multiple convolutional layers, multiple pooling layers, and a nonlinear layer (such as a ReLU layer), so that the features of the same dimension can be stacked by adjusting the size of the convolution kernel; the sub-decision output network can be a fully connected neural network (FCN), which contains multiple fully connected layers and a nonlinear layer (such as a ReLU layer).

[0059] Preferably, as some possible implementations, before the N initial sub-neural network models are trained based on the sample voiceprint images of different scales, the detection method further comprises:

[0060] When the sub-feature extraction network is a convolutional neural network and the sub-decision output network is a fully connected neural network:

[0061] determining a termination condition of the training;

[0062] determining the input layer dimension number, the output layer dimension number, the convolutional layer number, the convolution kernel number of the convolutional layer, and the pooling layer number of the convolutional neural network; wherein each layer of the convolutional layer is connected by residual connection after convolution processing;

[0063] determining the fully connected layer number of the fully connected neural network, and using a softmax loss function in the last layer of the fully connected layer to output a classification result.

[0064] Exemplarily, the termination condition of the training comprises:

[0065] the initial sub-neural network model converges via the training or the number of training reaches a preset value.

[0066] It should be noted that the input of the softmax function is the result x T w obtained from K different linear functions, and the probability of the sample vector x belonging to the jth class is:

[0067]

[0068] wherein w represents the weight matrix of the sample vector x; and k represents the number of sample classifications.

[0069] Therefore, the softmax function can be considered as normalizing a vector to highlight the largest vector and suppress other vectors that are much smaller than the largest vector.

[0070] Specifically, such as Figure 4 As shown, this diagram illustrates the detailed process of training an initial sub-neural network model based on a 128×128 scale sample voiceprint image. The input layer has a dimension of 128×128, the output layer has a dimension of 7, and there are 3 convolutional layers. Specifically, the first convolutional layer has 24 kernels, the second convolutional layer has 48 kernels, the third convolutional layer has 48 kernels, there are 2 pooling layers, and there are 2 fully connected layers. The specific training process is as follows:

[0071] S401. After processing the 128×128 scale sample voiceprint image through 24 5×5 convolution kernels, a 24-dimensional voiceprint image feature of 124×124×24 is obtained.

[0072] S402. Max pooling is performed on the voiceprint image features obtained in step S401 using a 4×2 convolution kernel to obtain compressed 31×62×24 24-dimensional voiceprint image features.

[0073] S403. Use 48 5×5 convolution kernels again to convolve the voiceprint image features obtained in step S402 to obtain 48-dimensional voiceprint image features of 27×58×48.

[0074] S404. The voiceprint image features obtained in step S403 are subjected to max pooling again using a 4×2 convolution kernel to obtain compressed 6×9×48 48-dimensional voiceprint image features.

[0075] S405. Use 48 5×5 convolution kernels again to convolve the voiceprint image features obtained in step S404 to obtain 48-dimensional voiceprint image features of 2×25×48.

[0076] S406. Expand the 2×25×48 48-dimensional voiceprint image features obtained in step 405 to obtain 1×2400-dimensional voiceprint image features.

[0077] S407. After performing two layers of fully connected processing on the 1×2400-dimensional voiceprint image features obtained in step S406, the output of the last layer of the fully connected layer is 7-dimensional.

[0078] It should be noted that in step S406, the 48-dimensional voiceprint image features of 2x25x48 are unfolded into 1x2400-dimensional voiceprint image features to facilitate full connection processing in the full connection layer. In step S407, the output result of the last layer of the full connection layer is a 7-dimensional representation, indicating that 7 classification results are obtained from the audio data during the operation of the target 3D printing device.

[0079] It should be noted that, since Figure 3 The initial sub-neural network model shown in the convolutional neural network is provided with multiple convolutional layers and pooling layers, and residual connections are made for different convolutional layers, so that deep features can be obtained while shallow features are obtained, so that voiceprint image features under multiple noises can be obtained, and the fine features of the noise are avoided. Loss due to environmental noise.

[0080] It can be understood that after the initial sub-neural network model is trained based on a sample voiceprint image of 128x128 scale, the next sample voiceprint image of 128x128 scale can be randomly selected from the entire sample voiceprint image to train the initial sub-neural network model until the final trained sub-neural network model meets the training termination condition. When one of the initial sub-neural network models is trained, the input layer dimension, the output layer dimension, the convolutional layer number, the convolutional kernel number, the pooling layer number and the full connection layer number in the convolutional neural network are adjusted, and the remaining two scale (320x320, 228x228) sample voiceprint images are trained according to the same training method. Two initial sub-neural network models are trained.

[0081] It can be understood that for an initial sub-neural network model, the parameters of the sub-feature extraction network of the trained sub-neural network model based on a scale of sample voiceprint image remain unchanged. That is, after training an initial sub-neural network model based on a scale of sample voiceprint image, a set of sub-feature extraction network parameters can be saved, and after training the three initial sub-neural network models corresponding to the three scales of sample voiceprint images, three sets of feature extraction network parameters can be obtained.

[0082] Preferably, as some possible implementations, the initial neural network model comprises:

[0083] a feature extraction network formed by the sub-feature extraction networks of the N trained sub-neural network models in parallel;

[0084] a decision output network for determining the classification result corresponding to the sample voiceprint image based on the sample voiceprint image features extracted by the feature extraction network.

[0085] The result output network determines the fault type corresponding to the sample voiceprint image based on the classification result.

[0086] like Figure 5 As shown, it illustrates an initial neural network model containing three trained sub-neural network models. Specifically, after the three initial sub-neural network models corresponding to the three scales are successfully trained, the decision output network of each trained sub-neural network model is removed, and only the corresponding sub-feature extraction networks are retained and connected in parallel to form the feature extraction network in the initial neural network model. That is, in the specific process of detecting faults in the target 3D printing equipment, after each sub-feature extraction network extracts different voiceprint image features, the voiceprint image features extracted by different sub-feature extraction networks are spliced ​​together to obtain the voiceprint image features during the operation of the target 3D printing equipment. Specifically, the voiceprint image feature splicing method can be to add the voiceprint image features extracted from the same dimension, or to splice the corresponding positions. Voiceprint image features are added pointwise; simultaneously, a decision output network is established in the initial neural network model, specifically, the decision output network can be a fully connected neural network; furthermore, the result output network of the initial neural network model can be any sub-result output network from the various trained sub-neural network models, or it can be a newly established result output network; then, the initial neural network model is trained using multiple maximum-scale (320×320) sample voiceprint images used when training the aforementioned initial sub-neural network models. Understandably, training the initial neural network model is actually training to obtain the parameters corresponding to the decision output network, while the parameters of the feature extraction network remain unchanged during the initial neural network model training process. Once the initial neural network model is successfully trained, the parameters of the decision output network established in the trained neural network model are also determined, thus forming the trained neural network model in this embodiment of the invention.

[0087] It should be noted that, in this embodiment of the invention, the provided neural network model is a multi-scale network model, specifically capable of extracting voiceprint image features at multiple scales and superimposing the extracted voiceprint image features of the same dimension. It should also be noted that the aforementioned multi-scale refers to the inclusion of multiple sub-feature extraction networks within the feature extraction network of the neural network model; the number of sub-feature extraction networks is not specifically limited in this embodiment of the invention.

[0088] On the other hand, it should be noted that in order to avoid the voiceprint image features being too localized, when there are tiny voiceprint image features on the obtained spectrum image and the localization is tiny, it can be understood that after the voiceprint image features are extracted from the spectrum image, it is difficult to obtain the above-mentioned tiny voiceprint image features, and therefore the spectrum image can be divided into multiple parts, such as four parts, and the division method is not limited, which can be horizontal division or vertical division. Then the corresponding sub-voiceprint images are obtained from the multiple sub-spectrum images, and input into the trained neural network model to output the classification results corresponding to each sub-voiceprint image, and when the fault type corresponding to a sub-voiceprint image is determined, it is considered that the above-mentioned complete spectrum image also corresponds to the fault type, which can facilitate the acquisition of local tiny voiceprint image features and avoid the voiceprint image features being too localized to a certain extent.

[0089] In addition, as some possible implementations, as shown in Figure 6 The feature extraction network further includes a scale adjustment network before the feature extraction network, and the scale adjustment network is used to adjust the scale of the input voiceprint image to the scale suitable for the feature extraction network of each trained sub-neural network model. For example, when the initial neural network model is trained successfully by using multiple sample voiceprint images with the maximum scale of 320x320, and the scale of the voiceprint image obtained by the collected audio data is 128x128 in the specific implementation process, that is, the scale of the voiceprint image is not the maximum scale used to train the initial neural network model. The maximum scale (320x320) in the trained neural network model can be scaled to 128x128 by the scale adjustment network, and then the voiceprint image features corresponding to the voiceprint image are extracted by the feature extraction network. Of course, it can be understood that the processing method of the scale adjustment network for the voiceprint image is not limited to scaling, and the maximum scale (320x320) in the trained neural network model can be cropped to 128x128 by the scale adjustment network, and then the voiceprint image feature extraction operation is performed.

[0090] Meanwhile, as some possible implementations, as shown in Figure 7As shown, the feature extraction network is provided with a Long Short Term Memory network (LSTM) after the feature extraction network, and the LSTM is used to process the voiceprint image feature sequence according to the past historical voiceprint image features, so that the current voiceprint image feature change trend is smoother. The LSTM contains a forgetting gate, an input gate and an output gate, so as to filter out unimportant voiceprint image features in the fault detection process, output important voiceprint image features, and prevent false alarms caused by interference information.

[0091] Through the trained neural network model provided in the embodiment of the present application, a plurality of voiceprint image features corresponding to the audio data can be extracted, and the deep features and the shallow features are fused to extract subtle voiceprint image features under multiple noises, so as to improve the accuracy and comprehensiveness of the voiceprint image feature extraction.

[0092] As some possible implementations, the input of the voiceprint image into the trained neural network model to determine the fault type in the running process of the 3D printing device includes:

[0093] The fault type with the maximum probability value in the classification result output by the decision output network of the trained neural network model is determined as the fault type in the running process of the target 3D printing device.

[0094] For example, the classification result output by the neural network model includes 7 types of motor abnormal sound, fan abnormal rotating sound, scraper card part, scraper card knife, powder falling shaft jamming, galvanometer light abnormality, and electrical element damage tripping. Referring to Figure 8 to Figure 14 which shows the spectrum images corresponding to the 7 types of fault types, wherein, Figure 8 is the spectrum image corresponding to the scraper card part; Figure 9 is the spectrum image corresponding to the scraper card knife; Figure 10 is the spectrum image corresponding to the fan abnormal rotating sound; Figure 11 is the spectrum image corresponding to the powder falling shaft jamming; Figure 12 is the spectrum image corresponding to the electrical element damage tripping; Figure 13 is the spectrum image corresponding to the galvanometer light abnormality; Figure 14 is the spectrum image corresponding to the motor abnormal sound. As shown in Figure 9 The audio data of the scraper card knife is a sharp card knife feature on the spectrum image; as shown in Figure 11 The audio data of the powder falling shaft jamming is reflected on the spectrum image, and a clear climbing and downhill feature can be observed; as shown in Figure 13As shown, the audio data of the galvanometer light-emitting abnormality is reflected on the spectrum image, and a wide stripe feature with light and dark alternation can be observed; but the spectrum images corresponding to the other several fault types cannot observe obvious and easily distinguishable features, so the detection method in the embodiment of the application can quickly and accurately identify the fault type in the 3D printing equipment, so as to facilitate the process personnel to timely adjust the process.

[0095] It should be understood that the softmax function is used in the last full connection layer in the neural network model, which can normalize the output value to a probability value, limited between 0 and 1, and the probability of each voiceprint image belonging to each fault type is just 1. Different classification results correspond to different probability values, and the classification result with the maximum probability value is used to represent the fault type in the current running process of the target 3D printing equipment. For example, the motor abnormal sound classification probability is 60%, the fan abnormal rotation sound classification probability is 10%, the scraper card part classification probability is 5%, the scraper card classification probability is 5%, the powder falling shaft stuck classification probability is 10%, the galvanometer light-emitting abnormality classification probability is 5%, and the electrical element damage tripping classification probability is 5%. It can be determined that the classification probability of the motor abnormal sound classification in the classification result of the input voiceprint image is the largest, so the motor abnormal sound can be determined as the fault problem existing in the running process of the 3D printing equipment at this time.

[0096] Based on the above description, the classification result can be obtained based on the audio data collected in the running process of the 3D printing equipment, so as to determine the possible fault problem of the target 3D printing equipment, and based on this, the possible problem can be adaptively adjusted. For example, taking the fan abnormal vibration as an example, when the 3D printing equipment is working normally, the fan speed is stable and the vibration is stable. When a sudden air leakage occurs at a certain part of the 3D printing equipment, the 3D printing equipment will increase the speed of the fan, and the fan will have an abnormal vibration at the moment. After the neural network model detects the abnormal vibration, it will give a prompt of the fan abnormal vibration to the software control interface, providing ideas for the plant workers to troubleshoot the problem. It can be understood that the detection of other faults related to the 3D printing equipment is similar to the fan abnormal vibration fault.

[0097] Of course, in the specific implementation process, the 3D printing equipment can not be adjusted automatically when a fault problem occurs, but the fault reason can be displayed on the screen to remind the process personnel to decide whether to perform subsequent processing. Of course, the interface can also be provided with a corresponding option, and the process personnel can start collecting the audio data in the running process of the 3D printing equipment to monitor the 3D printing equipment after selecting the option.

[0098] Based on the same inventive concept of the foregoing technical solutions, the detection device 150 for detecting the fault of the 3D printing equipment provided in the embodiment of the application is like Figure 15As shown, the detection apparatus comprises a first acquisition part 1501 and a determination part 1502; wherein,

[0099] The first acquisition part 1501 is configured to acquire a voiceprint image corresponding to audio data collected in real time during running of a target 3D printing device;

[0100] The determination part 1502 is configured to input the voiceprint image into a trained neural network model to determine a fault type during running of the target 3D printing device.

[0101] Exemplarily, the first acquisition part 1501 is configured to:

[0102] acquire original audio data of a preset time length during running of the target 3D printing device in real time;

[0103] perform denoising processing on the acquired original audio data to obtain final collected audio data;

[0104] acquire an audio spectrum of the audio data based on the audio data after denoising processing; wherein the audio spectrum is used to represent the voiceprint image.

[0105] Exemplarily, the determination part 1502 is configured to:

[0106] determine a fault type with the largest probability value in the classification result output by the decision output network of the trained neural network model as the fault type during running of the target 3D printing device.

[0107] Exemplarily, referring to Figure 16 , the apparatus 150 further comprises a second acquisition part 1601, a first construction part 1602, a third acquisition part 1603, a second construction part 1604 and a fourth acquisition part 1605; wherein,

[0108] The second acquisition part 1601 is configured to collect multiple sample audio data during running of the target 3D printing device and acquire corresponding multiple sample voiceprint images respectively;

[0109] The first construction part 1602 is configured to construct N initial sub-neural network models; wherein N≥2;

[0110] The third acquisition part 1603 is configured to train the N initial sub-neural network models based on multiple sample voiceprint images of different scales to obtain corresponding N trained sub-neural network models; wherein one initial sub-neural network model is trained based on sample voiceprint images of one scale;

[0111] The second constructing part 1604 is configured to construct an initial neural network model based on the N trained sub neural network models;

[0112] The fourth obtaining part 1605 is configured to train the initial neural network model by using the maximum scale sample voiceprint images used for training the N initial neural network models to obtain the trained neural network model.

[0113] Exemplarily, the second constructing part 1604 is configured to:

[0114] The feature extraction network is formed by the sub feature extraction networks of the N trained sub neural network models in parallel;

[0115] The decision output network determines the classification result corresponding to the sample voiceprint image based on the sample voiceprint image features extracted by the feature extraction network;

[0116] The result output network determines the fault type corresponding to the sample voiceprint image based on the classification result.

[0117] Exemplarily, the second constructing part 1604 is configured to:

[0118] The feature extraction network further comprises a scale adjustment network, which is used to adjust the scale of the input voiceprint image to the scale suitable for the sub feature extraction network of the N trained sub neural network models.

[0119] Exemplarily, the second constructing part 1604 is further configured to:

[0120] The feature extraction network is provided with a long short-term memory network (LSTM) after the feature extraction network, and the long short-term memory network (LSTM) comprises a forget gate, an input gate and an output gate.

[0121] It can be understood that in the embodiment, the "part" can be a part of circuit, a part of processor, a part of program or software, etc., and of course can also be a unit, and can also be a module or non-modular.

[0122] In addition, each component in the embodiment can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function module.

[0123] The integrated unit, if implemented in the form of a software function module and not sold or used as an independent product, can be stored in a computer readable storage medium based on such understanding. The technical solutions of the embodiments essentially or the parts that contribute to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiments. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0124] Therefore, the embodiments provide a computer storage medium storing a 3D printing equipment fault detection method program. The 3D printing equipment fault detection method program is executed by at least one processor to implement the steps of the 3D printing equipment fault detection method described in the above technical solutions.

[0125] According to the 3D printing equipment fault detection device 150 and the computer storage medium described above, referring to Figure 17 , a specific hardware structure of a 3D printing equipment fault detection device 170 capable of implementing the 3D printing equipment fault detection device 150 is shown. The 3D printing equipment fault detection device 170 includes an audio data collector 108, a memory 1701, and a processor 1702. Each component is coupled together through a bus system 1703. It can be understood that the bus system 1703 is used to realize the connection and communication between the components. The bus system 1703 includes a data bus, a power bus, a control bus, and a status signal bus. However, for the purpose of clarity, all buses are marked as the bus system 1703 in Figure 17 . Among them,

[0126] The audio data collector 108 is configured to collect audio data in real time during the operation of the target 3D printing equipment, and collect a plurality of sample audio data during the operation of the target 3D printing equipment.

[0127] The memory 1701 is configured to store a computer program capable of running on the processor.

[0128] The processor 1702 is configured to execute the steps of the 3D printing equipment fault detection method described in the foregoing technical solutions when the computer program is running.

[0129] It is to be appreciated that the memory 1701 in embodiments of the application can be volatile, nonvolatile, or a combination of both. The non-volatile memory can be, for example, read only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory. The volatile memory can be, for example, random access memory (RAM), which acts as external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The memory 1701 of the system and method described herein are intended to include, without being limited to, these and any other suitable types of memory.

[0130] The processor 1702 can be an integrated circuit chip on which signal processing capabilities are implemented. In implementation, the steps of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 1702. The processor 1702 described above can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 1701, and the processor 1702 reads the information in the memory 1601, and combines the hardware to complete the steps of the above method.

[0131] It can be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described in the present application, or a combination thereof.

[0132] For software implementation, the techniques described herein can be implemented by modules (for example, procedures, functions, and so on) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0133] Specifically, the processor 1702 is further configured to execute the steps of the method for detecting the failure of the 3D printing device in the foregoing technical solutions when the computer program is executed, which will not be repeated here.

[0134] It should be noted that the technical solutions disclosed in the embodiments of the present application can be combined arbitrarily without conflict.

[0135] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of detecting a failure of a 3D printing device, characterized by, The detection method comprises: For the audio data collected in real time during the running process of the target 3D printing equipment, the voiceprint image corresponding to the audio data is obtained; The voiceprint image is input into the trained neural network model to determine the fault type in the running process of the target 3D printing equipment; Wherein, the neural network model is trained by the following steps: Collecting multiple sample audio data in the running process of the target 3D printing equipment, and respectively obtaining corresponding multiple sample voiceprint images; Constructing N initial sub neural network models; Wherein, N≥2; Training the N initial sub neural network models based on different scales of the multiple sample voiceprint images to obtain corresponding N trained sub neural network models; Wherein, training one initial sub neural network model based on one scale of sample voiceprint image; Based on the N trained sub neural network models, an initial neural network model is constructed; Using the multiple maximum scale sample voiceprint images used to train the N initial sub neural network models, the initial neural network model is trained to obtain the trained neural network model.

2. The detection method according to claim 1, characterized in that, The audio data collected in real time during the running process of the target 3D printing equipment, the voiceprint image corresponding to the audio data is obtained, comprising: Real-time acquisition of original audio data in the running process of the target 3D printing equipment for a predetermined time length; The collected original audio data is denoised to obtain the final collected audio data; Based on the denoised audio data, the audio spectrum of the audio data is obtained; Wherein, the audio spectrum is used to represent the voiceprint image.

3. The method of claim 1, wherein The initial neural network model comprises: The feature extraction network is formed by the parallel connection of the sub feature extraction networks of each trained sub neural network model in the N trained sub neural network models; The decision output network determines the classification result corresponding to the sample voiceprint image based on the sample voiceprint image features extracted by the feature extraction network; The result output network determines the fault type corresponding to the sample voiceprint image based on the classification result.

4. The detection method according to claim 3, characterized in that, The feature extraction network further comprises a scale adjustment network, which is used to adjust the scale of the input voiceprint image to the scale suitable for the sub feature extraction network of each trained sub neural network model.

5. The detection method according to claim 3, characterized in that, The feature extraction network is provided with a long short-term memory network LSTM, which contains a forgetting gate, an input gate and an output gate.

6. The detection method according to claim 3, characterized in that, The voiceprint image is input into the trained neural network model to determine the fault type in the running process of the target 3D printing equipment, comprising: The fault type with the maximum probability value in the classification result output by the decision output network of the trained neural network model is determined as the fault type in the running process of the target 3D printing equipment.

7. A device for detecting a failure of a 3D printing apparatus, characterized by, The detection device comprises a first acquisition part, a second acquisition part, a first construction part, a third acquisition part, a second construction part, a fourth acquisition part and a determination part; wherein, The first acquisition part is configured to acquire a voiceprint image corresponding to audio data of a target 3D printing device in real time. The second acquisition part is configured to acquire a plurality of sample voiceprint images corresponding to a plurality of sample audio data of the target 3D printing device. The first construction part is configured to construct N initial sub-neural network models, where N≥2. The third acquisition part is configured to train the N initial sub-neural network models based on a plurality of sample voiceprint images of different scales to obtain N trained sub-neural network models, where one initial sub-neural network model is trained based on sample voiceprint images of one scale. The second construction part is configured to construct an initial neural network model based on the N trained sub-neural network models. The fourth acquisition part is configured to train the initial neural network model based on a plurality of sample voiceprint images of the largest scale used to train the N initial sub-neural network models to obtain a trained neural network model. The determination part is configured to input the voiceprint image into the trained neural network model to determine a fault type of the target 3D printing device.

8. A 3D printing device failure detection device, characterized by, The detection device comprises an audio data collector, a memory and a processor, wherein The audio data collector is configured to collect audio data of a target 3D printing device in real time and a plurality of sample audio data of the target 3D printing device. The memory is configured to store a computer program capable of running on the processor. The processor is configured to execute the steps of the 3D printing device fault detection method of any one of claims 1 to 6 when running the computer program.

9. A storage medium, characterized by The storage medium stores a 3D printing device fault detection method program, and the 3D printing device fault detection method program is executed by at least one processor to implement the steps of the 3D printing device fault detection method of any one of claims 1 to 6.

Images