System and method for diagnosing a device and method of diagnosing components of a device

Abstract

A method and system for diagnosing a device, and a method for diagnosing a component of a device, the method for diagnosing a device can include recording a run of a device to create an audio file, extracting features from the audio file, inputting the extracted features into a machine learning model, and determining a score indicative of the run of the device with the machine learning model. The system can include an audio sensor for recording audio of a run of a device and generating an audio file, and one or more processors. The one or more processors extract features from the audio file, input the extracted features into a machine learning model, and determine a score indicative of the run of the device with the machine learning model.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. non-provisional application No. 17 / 523,297, filed November 10, 2021, and U.S. provisional application No. 63 / 122,312, filed December 7, 2020, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] The disclosed subject matter described herein relates to systems and methods for diagnostic devices, as well as methods for components of diagnostic devices. Background Technology

[0004] Equipment (such as vehicle components) can be diagnosticated to detect parts that may be damaged or in a failure mode. Diagnostics can vary depending on who performs them, potentially leading to inaccurate results. Diagnostics may not consider previous diagnoses, making it difficult to determine if the current diagnosis is correct. If equipment is incorrectly diagnosed as operating as expected, a failure in the equipment (such as a locomotive) could cause it to break down. Conversely, if a component is inspected and incorrectly diagnosed as not operating as expected—for example, being damaged, defective, or malfunctioning—unnecessary replacement of that component can lead to equipment downtime and increased maintenance costs. Summary of the Invention

[0005] According to one example or aspect, a method may include recording the operation of a device to create an audio file and extracting features from the audio file. The method may include feeding the extracted features into a machine learning model and using the machine learning model to determine a score indicative of the device's operation.

[0006] According to one example or aspect, a system may include an audio sensor to record audio of device operation and generate an audio file, and one or more processors. The one or more processors extract features from the audio file and input the extracted features into a machine learning model. The one or more processors use the machine learning model to determine a score indicative of device operation.

[0007] According to one example or aspect, a method may include using a recording device to record the operation of a component to create a file, and extracting features from the file to create an extraction file. The method may include feeding the extraction file into a machine learning model, and using the machine learning model to determine a score indicative of component operation based at least in part on the extraction file. Attached Figure Description

[0008] The subject matter of the invention can be understood by reading the following description of non-limiting embodiments and referring to the accompanying drawings, in which:

[0009] Figure 1 A system and method for a diagnostic device according to one embodiment are illustrated schematically;

[0010] Figure 2 A system and method for a diagnostic device according to one embodiment are illustrated schematically;

[0011] Figure 3 A system and method for a diagnostic device according to one embodiment are illustrated schematically;

[0012] Figure 4 A method according to one embodiment is illustrated schematically;

[0013] Figure 5 A method according to one embodiment is illustrated schematically. Detailed Implementation

[0014] The embodiments of the subject matter described herein relate to a device capable of diagnosing equipment, such as equipment associated with vehicle systems, power generation systems, and building equipment. Failure to diagnose equipment not operating as expected can lead to equipment malfunctions, resulting in costly repairs, and the equipment may also be unusable during repairs. Incorrectly diagnosing equipment operating as expected as not operating as expected can lead to equipment downtime and unnecessary repairs.

[0015] Currently, diagnostic devices using sound rely on operators familiar with the equipment's operation. These operators can identify unwanted operation by listening to the equipment's sounds and making judgments based on experience. Diagnosing devices in this way may not detect unwanted operation early enough to prevent equipment malfunction and can lead to inconsistent results based on different operators' experiences. The embodiments described herein provide a method for diagnosing devices using a machine learning model based on audio data and image data generated from the audio data. This model is trained to distinguish between desired and unwanted operation and continues to learn and improve its ability to differentiate between desired and unwanted operation.

[0016] While one or more embodiments are described in connection with rail vehicle systems, not all embodiments are limited to rail vehicle systems. Furthermore, the embodiments described herein extend to a variety of vehicle systems. Suitable vehicle systems may include rail vehicles, automobiles, trucks (with or without trailers), buses, ships, aircraft, mining vehicles, agricultural vehicles, and off-highway vehicles. Suitable vehicle systems described herein may consist of a single vehicle. In other embodiments, vehicle systems may include multiple vehicles moving in a coordinated manner. Suitable vehicle systems may be rail vehicle systems that travel on tracks or vehicle systems that travel on roads or paths. Regarding multi-vehicle systems, vehicles may be mechanically coupled to each other (e.g., via couplers) or they may be virtually or logically coupled rather than mechanically coupled. For example, vehicles may be communicatively coupled rather than mechanically when they communicate with each other to coordinate their relative movement so that they travel together (e.g., as a convoy, platoon, swarm, fleet, etc.).

[0017] Suitable examples of devices or components may include those undergoing periodic diagnostics. In one embodiment, the component may be a component of an engine or vehicle system. For example, the device may be a high-pressure fuel pump for a locomotive engine. In another example, the component may be an electric motor. Rotating devices are generally suitable for diagnostics using the methods of the present invention.

[0018] refer to Figure 1Device 10 can be diagnosed by recording device 12. According to one embodiment, the recording device can be a mobile handheld device. The recording device can be a smartphone, tablet, personal digital assistant (PDA), or laptop. The recording device includes an audio acquisition device, such as a microphone, accelerometer, or probe, which acquires audio indicating the operation of the device or part of the device to be diagnosed and stores the audio as a raw audio file 14. According to one example, the raw audio file can be a WAV format file. The WAV file may include uncontained and uncompressed audio data. The recording device can be connected to a sensor or probe or an external microphone, for example, via a USB connection. The sensor or probe or microphone can be placed near and / or in contact with the device part to generate the raw audio file. The recording device can transmit the audio file to one or more processors, which can execute instructions stored in memory to make determinations and evaluations about the device or part of the device using a machine learning model. For example, the determination may involve whether the device or part of the device is operating in a desired or undesirable mode. Regarding proximity, specific parameters can be referenced to select the distance. In one embodiment, the microphone may be within a few inches of a part of the device or a component of the device.

[0019] Suitable audio file types can include both lossy and lossless file types. Examples of audio file types include .wav, .mp3, .wma, .aac, .ogg, .midi, .aif, .aifc, .aiff, .au, and .ea. The file type can be selected, at least in part, based on compression ratio, compression algorithm, and other application-specific parameters.

[0020] According to one example, the device is a high-pressure fuel pump of a vehicle. The original audio file can be generated while the vehicle engine is running under idle (i.e., no-load) conditions. The recording device can be placed adjacent to the high-pressure fuel pump, and the recording device or audio acquisition device can be moved between different locations (e.g., from a first recording position 16 to a second recording position 18, to a third recording position 20, and so on). Although the example shown illustrates recording at three positions, alternatively, recording can occur at fewer positions (e.g., a single position or two positions) or more than three positions. Figure 1As shown, the recording locations extend from the top to the bottom of the device. When the recording device or audio acquisition device is moved from a first recording location to a second recording location and then to a third recording location, it can hover over each recording location for a period of time. According to one example, the operation of the high-pressure fuel pump can be recorded for a period of time, such as 30 seconds, one minute, or another duration. The recording device or audio acquisition device can be used to output two or more audio files. For example, the recording device or audio acquisition device can output a first audio file of a first fuel pump on a first side of the vehicle and can acquire a second audio file of a second fuel pump on the opposite second side of the vehicle.

[0021] refer to Figure 2 A method 24 for a diagnostic device according to one embodiment includes extracting 26 features from a raw audio file. Extractable features include, but are not limited to, zero-crossing rate, spectral centroid, spectral bandwidth, root mean square error (RMSE), chromatic STFT (short-time Fourier transform), or spectral roll-off, one or more of these. The extracted features are input to input layer 30 of a machine learning model 28. One or more processors of the recording device may transform or convert the raw audio file into image data 22, such as a mel spectrogram. The audio data of the audio file can be transformed into image data of the mel spectrogram using, for example, a fast Fourier transform (FFT) with a window function having a defined window size. Analysis can be performed by sampling the audio file a defined number of times between consecutive windows using a defined hop size. The FFT for each window can be computed to transform from the time domain to the frequency domain. A mel scale can be generated by dividing the entire spectrum into frequencies at a defined number of uniform intervals. The spectrogram can then be generated by decomposing the amplitude of the signal into its components for each window, these components corresponding to frequencies in the mel scale. As an example, a Mel spectrogram can be a 23-feature set.

[0022] Mel spectrograms, along with the extracted features, can be fed into the input layer. As an example, the machine learning model includes an input layer 30, a hidden layer 32, and an output layer 36. Figure 2 As shown, according to one example, the hidden layer is a single layer and the output layer is a single output neuron. According to other examples, the hidden layer may include multiple hidden layers and / or the output layer may include multiple neurons. In one embodiment, an audio file can be visualized as a wavefunction. Fourier transform can be used to extract information from audio files, video files, or both. In other embodiments, other transform algorithms may be employed. Suitable transform models may include Laplace transform, wavelet transform, and Kramers-Kronig transform.

[0023] The machine learning model's algorithm applies input bias 34 to the inputs of the hidden layers and guides them through an activation function as output. According to one embodiment, the algorithm applies output bias 38 to the output provided as a measure of abnormal device operation. In one embodiment, the machine learning model is a supervised machine learning model. The machine learning model may be provided with labeled training data. The training data may include audio files and image data of devices operating in desired modes and audio files and image data of devices operating in undesired modes. The training data may come from similar devices, such as other high-pressure fuel pumps. The training data may come from one or more previous diagnostics of the same device. For example, the machine learning model may include previous data on high-pressure fuel pumps and determine the number of times the high-pressure fuel pump has been diagnosed previously. The machine learning model may include previous data on previous diagnostics. The machine learning model may determine that a piece of equipment or part of equipment that has been diagnosed a specific number of times and is consistently identified as operating in desired modes is more likely to operate in undesired modes. The machine learning model may also determine, based on the input data, that the device being diagnosed is older than other devices that have been diagnosed, thereby determining that a device component has deteriorated over time.

[0024] As one example, a machine learning model can refer to its own results while recording the device's operation to provide more accurate decisions. (Refer to again) Figure 1 When a recording device is moved from one recording position to another, the results of one or more previous recording positions can be used as concurrent reference points at the next recording position. When a recording device or audio acquisition device, for example, moves from one cylinder to another in an engine, or from one cylinder to another in a pump, or from one pump to another in the case of multiple pumps, the machine learning model's algorithm can refer to previous device component behavior and evaluations, and can adjust thresholds while recording the device being diagnosed. The machine learning model can adjust previous evaluations of the device and its components after completing an evaluation of the entire device.

[0025] The machine learning model can be stored in the memory of the recording device and executed by one or more processors. The memory of the recording device can store training data and input data from previous diagnostics performed by the recording device or from other recording devices. The input data of the machine learning model is labeled and structured, and through the operation of hidden layers, the machine learning model detects patterns in the input data and detects any anomalies in the patterns. According to one embodiment, the output 40 of the machine learning model is a single-value score on a series of single-value scores indicative of device operation.

[0026] refer to Figure 3 The process 42 for diagnosing a diagnostic device, as shown in the example, includes determining a standard score. The original audio file is processed into a normalized audio file 44, and features are extracted from the normalized audio file. According to one example, the extracted features correspond to a combination of... Figure 2 The features described. As in the example, the extracted audio features are provided as a feature set in comma-separated value format.

[0027] The extracted features are input into machine learning model 46. According to one embodiment, the machine learning model corresponds to combining... Figure 2 The machine learning model is described. The output of the machine learning model is fed into the loss function 48. The loss function determines the maximum likelihood of normal data. The loss function has a scoring component 50 that assigns scores to the data based on a Gaussian distribution.

[0028] The scores assigned by the loss function are standardized scores. Normal data will be assigned standardized scores within a range. According to one embodiment, the range is between -3 and 3. Standardized scores outside the range represent outlier data. Standardized scores within a first numerical range can indicate that the device is operating in a desired mode, and standardized scores within a second numerical range can indicate that the device is operating in an undesirable mode. According to one embodiment, standardized scores within the second range can be higher than those within the first range. The error derivative 52 determined by the loss function is determined and provided to the machine learning model. The machine learning model learns to fit the data and predicts scores, assigning minimum values ​​to normal data and statistically higher values ​​to outlier data.

[0029] Reference Figure 4 Method 400 may include a step 410 of recording device operation to create an audio file and a step 420 of extracting features from the audio file. The method further includes a step 430 of inputting the extracted features into a machine learning model and a step 440 of using the machine learning model to determine a score indicative of device operation.

[0030] refer to Figure 5 Method 500 may include step 510 of recording the operation of a component using a recording device to create a file and step 520 of extracting features from the file to create an extraction file. The method may include step 530 of inputting the extraction file into a machine learning model and step 540 of using the machine learning model to determine, at least in part, a score indicative of component operation based on the extraction file.

[0031] One method may include recording device operation to create an audio file and extracting features from the audio file. The method may also include feeding the extracted features into a machine learning model and using the machine learning model to determine a score indicative of device operation.

[0032] The method may include generating at least one of the following: a suggestion to modify equipment operation, a request to maintain the equipment, and a notification of a future date for scheduling subsequent recording and that the equipment score falls within a defined acceptable operating parameter range.

[0033] The machine learning model may include training data, which includes a first audio file that runs in a desired device mode and a second audio file that runs in an unwanted device mode. The method may include adding the audio files to the training data.

[0034] The method may include normalizing the data in the audio file before extracting features.

[0035] The method may include inputting scores into a loss function, the loss function being based on a Gaussian distribution of features extracted and input into a machine learning model.

[0036] The score can be a standard score. A standard score in a first range can indicate that the device is operating in a desired mode, and a standard score in a second range can indicate that the device is operating in an undesirable mode.

[0037] Features extracted from audio files may include one or more of the following: zero-crossing rate, spectral centroid, root mean square error, and spectral roll-off.

[0038] The method may include generating image data corresponding to an audio file, simultaneously inputting the generated image data and the recorded audio file into a machine learning model, and determining a score based at least in part on the generated image data.

[0039] A system may include an audio sensor for recording audio of device operation and generating audio files, and one or more processors. The one or more processors extract features from the audio files, input the extracted features into a machine learning model, and use the machine learning model to determine a score indicating device operation.

[0040] The one or more processors can use the score to generate at least one of the following: a suggestion to modify device operation, a request to maintain the device, and a future date for scheduling subsequent recording, as well as a notification that the device score falls within a defined acceptable operating parameter.

[0041] The machine learning model may include training data, which includes a first audio file running in the device's desired mode and a second audio file running in the device's undesired mode. The one or more processors may add the audio files to the training data.

[0042] The one or more processors can normalize the data in the audio file before extracting features.

[0043] The one or more processors can input scores into a loss function. The loss function can be based on a Gaussian distribution of features extracted and input into the machine learning model.

[0044] The score can be a standard score. A standard score in a first range indicates that the device is operating in a desired mode, and a standard score in a second range indicates that the device is operating in an undesirable mode.

[0045] Features extracted from audio files may include one or more of the following: zero-crossing rate, spectral centroid, root mean square error, and spectral roll-off.

[0046] The extracted features can be video features. The one or more processors can generate image data based at least in part on the extracted features from the audio file, input the generated image data into a machine learning model, and determine a score based at least in part on the generated image data.

[0047] Audio files and image data can be fed into machine learning models simultaneously.

[0048] One method may include using a recording device to record the operation of a component to create a file, and extracting features from the file to create an extraction file. The method may include feeding the extraction file into a machine learning model, and using the machine learning model to determine a score indicative of component operation based at least in part on the extraction file.

[0049] The extracted files may include audio files, video files, or audio-visual files. The machine learning model may include training data, which includes a first reference file running in the component's desired mode and a second reference file running in the component's undesired mode. The method may include adding the extracted files to the training data as either the desired mode or the undesired mode.

[0050] The component may be at least one of a fuel pump, a water pump, a crankshaft, and a piston.

[0051] In one embodiment, the processor can determine more tiered data about the component. That is, not whether it is operating in a desired or undesirable state, but to what extent it operates in that state. The score can be tiered, corresponding to the component's expected remaining lifespan. This information can then be used to schedule maintenance, repair, or replacement for future dates prior to a calculated failure date. The calculated failure date may contain errors. In one example, the magnitude of the error can be determined based on the component's criticality and the impact of its failure. In one embodiment, the information can be used to modify the component's operation. For example, if the component is used in a less demanding work cycle, its lifespan may be longer than if it were used at maximum capacity.

[0052] In one embodiment, one or more processors or systems described herein can deploy a local data collection system and can use machine learning to achieve inference-based learning outcomes. The one or more processors can learn and make decisions from a set of data (including data provided by various sensors) by making data-driven predictions and adjusting based on the dataset. In embodiments, machine learning can involve performing multiple machine learning tasks, such as supervised learning, unsupervised learning, and reinforcement learning, through a machine learning system. Supervised learning can include presenting a set of example inputs and a desired output to the machine learning system. Unsupervised learning can include learning algorithms that construct their inputs through methods such as pattern detection and / or feature learning. Reinforcement learning can include machine learning systems that perform in dynamic environments and provide feedback on correct and incorrect decisions. In examples, machine learning can include multiple other tasks based on the output of the machine learning system. In examples, the tasks can be machine learning problems such as classification, regression, clustering, density estimation, dimensionality reduction, anomaly detection, etc. In examples, machine learning can include a variety of mathematical and statistical techniques. In the examples, many types of machine learning algorithms may include decision tree-based learning, association rule learning, deep learning, artificial neural networks, genetic learning algorithms, inductive logic programming, support vector machines (SVM), Bayesian networks, reinforcement learning, representation learning, rule-based machine learning, sparse dictionary learning, similarity and metric learning, learning classifier systems (LCS), logistic regression, random forests, K-Means, gradient boosting, K-Nearest Neighbors (KNN), prior algorithms, etc. In the embodiments, certain machine learning algorithms may be used (e.g., for solving constrained and unconstrained optimization problems that can be based on natural selection). In one example, the algorithm can be used to solve a mixed-integer programming problem, where some components are restricted to integer values. Algorithms and machine learning techniques and systems can be used in computational intelligent systems, computer vision, natural language processing (NLP), recommender systems, reinforcement learning, building graphical models, etc. In one example, machine learning can be used for determination, computation, comparison, and behavior analysis, etc.

[0053] In one embodiment, one or more processors may include a policy engine that can apply one or more policies. These policies may be at least partially based on the characteristics of a given device or environmental item. Regarding the control policy, the neural network may receive inputs of numerous environmental and task-related parameters. These parameters may include, for example, operational inputs about the operating device, data from various sensors, position and / or location data, etc. The neural network can be trained to generate outputs based on these inputs, the outputs representing actions or sequences of actions that the device or system should take to achieve an operational objective. During operation in one embodiment, the action can be determined by processing the inputs through the parameters of the neural network to generate values ​​at the output nodes specifying the desired action. This action can be translated into a signal that causes the vehicle to move. This can be achieved through backpropagation, a feedforward process, closed-loop feedback, or open-loop feedback. Alternatively, the machine learning system of the controller can use evolutionary strategy techniques to adjust various parameters of the artificial neural network instead of backpropagation. The controller may use a neural network architecture with functions that may not always be solvable using backpropagation, such as non-convex functions. In one embodiment, the neural network has a set of parameters representing the weights of its node connections. Multiple copies of the network are generated, and then the parameters are adjusted differently to complete the simulation. Once the outputs from various models are obtained, their performance can be evaluated using a defined success metric. The optimal model is selected, and the vehicle controller executes the program to achieve the desired input data that reflects the best predicted outcome scenario. Furthermore, the success metric can be a combination of optimization results, which can be weighed relative to each other.

[0054] As used herein, the terms “processor” and “computer,” as well as related terms such as “processing device,” “computing device,” and “controller,” may be limited to those integrated circuits referred to in the art as computers, but rather refer to microcontrollers, microcomputers, programmable logic controllers (PLCs), field-programmable gate arrays, application-specific integrated circuits (ASICs), and other programmable circuits. Suitable memory may include, for example, computer-readable media. Computer-readable media may be, for example, random access memory (RAM), computer-readable non-volatile media, such as flash memory. The term “non-transitory computer-readable media” represents a computer-based tangible device for short-term and long-term storage of information, such as computer-readable instructions, data structures, program modules and submodules, or other data in any device. Therefore, the methods described herein may be encoded as executable instructions contained in tangible, non-transitory, computer-readable media, including but not limited to storage devices and / or memory devices. Such instructions, when executed by a processor, cause the processor to perform at least a portion of the methods described herein. Therefore, the term includes tangible computer-readable media, including but not limited to non-transitory computer storage devices, including but not limited to volatile and non-volatile media, as well as removable and non-removable media, such as firmware, physical and virtual storage, CD-ROMs, DVDs and other digital resources, such as networks or the Internet.

[0055] When any or all of the terms “comprise”, “comprises”, “comprised” or “comprising” are used in this specification (including the claims), they shall be construed as explicitly indicating the presence of the stated feature, integer, step or component, but not excluding the presence of one or more other features, integers, steps or components.

[0056] Unless the context explicitly specifies otherwise, the singular forms “a,” “an,” and “the” include plural references. “Optional” or “optionally” means that an event or situation subsequently described may or may not occur, and the description may include both the possibility that the event occurs and the possibility that it does not. The approximate language used herein throughout the specification and claims is intended to modify any quantitative expression that allows for variation without altering the fundamental function that it may be associated with. Therefore, values ​​modified by one or more terms, such as “approximately,” “substantially,” and “approximately,” may not be limited to the specified precise value. In at least some cases, approximate language may correspond to the precision of the instrument used to measure the value. Scope limitations herein and throughout the specification and claims may be combined and / or interchanged, and unless the context or language indicates otherwise, such scopes may be identified and include all subscopes contained herein.

[0057] This written description uses examples to disclose embodiments, including best practices, and to enable those skilled in the art to practice the embodiments, including making and using any device or system and performing any included methods. The claims define the patentable scope of this disclosure and include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are indistinguishable from the literal language of the claims, or if they include equivalent structural elements that are not substantially different from the literal language of the claims.

Claims

1. A method for using a diagnostic device, comprising: Record the operation of the device to create an audio file, wherein the audio file includes audio signals at multiple different locations within the device that are related to the operation of the device; Extract features from the audio file; The extracted features are then input into a machine learning model; and The step of determining a score indicating the operation of the device using the machine learning model further includes: the machine learning model adjusting the score at a first position among the plurality of different positions based on audio signals recorded at a second position among the plurality of different positions.

2. The method of claim 1, further comprising generating at least one of the following based on the score instructing the device to operate: Recommendations for modifying the operation of the aforementioned equipment; Requests for maintenance of the device; and Notification of future dates for subsequent record-keeping arrangements and notification that the device's score falls within determined acceptable operating parameters.

3. The method according to claim 1, wherein, The machine learning model includes training data, which includes a first audio file running in the device's desired mode and a second audio file running in the device's undesirable mode. The method also includes adding the audio file to the training data.

4. The method of claim 1, further comprising normalizing the data of the audio file before extracting the features.

5. The method of claim 1, further comprising inputting the score into a loss function, the loss function being based on a Gaussian distribution of the features extracted and input into the machine learning model.

6. The method according to claim 1, wherein, The scores are standard scores, with standard scores in a first range indicating that the device is operating in the desired mode and standard scores in a second range indicating that the device is operating in the undesired mode.

7. The method according to claim 1, wherein, The features extracted from the audio file include one or more of the following: Zero crossing rate, spectral centroid, root mean square error, and spectral roll-off.

8. The method according to claim 1, further comprising: Generate image data corresponding to the audio file; The generated image data and the recorded audio file are simultaneously input into the machine learning model, and The score is determined at least in part based on the generated image data.

9. A system for a diagnostic device, comprising: An audio sensor for recording audio of device operation and generating audio files, wherein the audio files include audio signals at multiple different locations within the device that are related to the operation of the device; and One or more processors, for: Extract features from the audio file; The extracted features are then input into a machine learning model; and The step of determining a score indicating the operation of the device using the machine learning model further includes: the machine learning model adjusting the score at a first position among the plurality of different positions based on audio signals recorded at a second position among the plurality of different positions.

10. The system according to claim 9, wherein, The one or more processors generate at least one of the following based on the score instructing the device to operate: Recommendations for modifying the operation of the aforementioned equipment; Requests for maintenance of the device; and Notification of future dates for subsequent record-keeping arrangements and notification that the device's score falls within determined acceptable operating parameters.

11. The system according to claim 9, wherein, The machine learning model includes training data, which includes a first audio file running in the device's desired mode and a second audio file running in the device's undesirable mode. The one or more processors are also used to add the audio file to the training data.

12. The system according to claim 9, wherein, The one or more processors are also used to normalize the data of the audio file before extracting the features.

13. The system according to claim 9, wherein, The one or more processors are also used for: The score is input into a loss function, which is based on a Gaussian distribution of the features extracted and input into the machine learning model.

14. The system according to claim 9, wherein, The scores are standard scores, with standard scores in a first range indicating that the device is operating in the desired mode and standard scores in a second range indicating that the device is operating in the undesired mode.

15. The system according to claim 9, wherein, The features extracted from the audio file include at least one of the following: Zero crossing rate, spectral centroid, root mean square error, and spectral roll-off.

16. The system according to claim 9, wherein, The extracted features are video features, and the one or more processors are further configured to: Image data is generated at least in part based on the extracted features of the audio file; The generated image data is input into the machine learning model; and The score is determined at least in part based on the generated image data.

17. The system according to claim 16, wherein, The audio file and the image data are simultaneously input into the machine learning model.

18. A method for diagnosing components of a device, comprising: A recording device is used to record the operation of the components to create a file, wherein the file includes signals at multiple different locations in the device that are related to the operation of the device; Extract features from the file to create an extraction file; The extracted file is then input into the machine learning model; and The step of determining a score indicative of the component's operation using the machine learning model, at least in part, based on the extracted file, further includes: the machine learning model adjusting the score at a first position among the plurality of different positions based on signals recorded at a second position among the plurality of different positions.

19. The method according to claim 18, wherein, The extracted files include audio files, video files, or audio-visual files; the machine learning model includes training data, which includes a first reference file running in the component's desired mode and a second reference file running in the component's undesired mode. The method further includes adding the extracted file to the training data as either the desired running mode or the undesired running mode.

20. The method according to claim 18 or 19, wherein, The component is at least one of the following: Fuel pump, water pump, crankshaft, and piston.

Images