Fault handling method, device, information processing device, storage medium and server

By receiving characteristic sound data of home appliances, using preset models to identify fault types and predict probability, the problem of inaccurate failure analysis is solved, efficient fault diagnosis and resolution is achieved, and user experience is improved.

CN112633492BInactive Publication Date: 2024-11-12FOSHAN SHUNDE MIDEA ELECTRICAL HEATING APPLIANCES MFG CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN201910951474.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-10-08
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional home appliance fault analysis methods rely on fault reports presented by the user interface, resulting in inaccurate fault diagnosis and manufacturers are unable to accurately locate problems immediately and affect user experience.

Method used

By receiving characteristic sound data sent by home appliances, using preset diagnostic models and fault prediction models, identifying fault types and predicting fault probability, generating solutions, including training of neural networks and logistic regression models, using spectrograms, short-term Sound characteristics such as amplitude zero-crossing rate, short-term average energy and Mel frequency cepspectral coefficient.

Benefits of technology

Improves the efficiency of fault diagnosis and resolution, accurately identify fault types and provide specific solutions, and improves user satisfaction with the equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112633492B_ABST
    Figure CN112633492B_ABST
Patent Text Reader

Abstract

The present invention discloses a fault handling method, comprising: receiving at least one characteristic sound data sent by a household appliance; obtaining a preset diagnostic model, using the diagnostic model to identify the at least one characteristic sound data, determining a first target fault type and a solution corresponding to the first target fault type when the household appliance fails; and / or obtaining a preset fault prediction model, using the fault prediction model to identify the at least one characteristic sound data, predicting a second target fault type when an abnormality exists in the household appliance. The present invention also discloses a fault handling device, an information processing device, a storage medium and a server.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to artificial intelligence technology, and in particular to a fault handling method, device, information processing device, computer-readable storage medium and server. Background Art

[0002] Traditional fault analysis for home appliances generally involves fault diagnosis and guessing based on the fault report and fault content presented by the user interface (UI) of the user terminal. However, users do not understand fault diagnosis methods and can only complain to the manufacturer when encountering a fault during use, and the fault description may be inaccurate; the manufacturer cannot view the equipment in real time, cannot accurately locate the problem, and cannot troubleshoot, which affects the user experience. Summary of the invention

[0003] To solve the existing technical problems, the embodiments of the present invention provide a fault handling method, device, information processing device, computer-readable storage medium and server.

[0004] The technical solution of the present invention is achieved in this way:

[0005] An embodiment of the present invention provides a fault handling method, the method comprising:

[0006] receiving at least one characteristic sound data sent by the home appliance;

[0007] Obtain a preset diagnostic model, use the diagnostic model to identify the at least one characteristic sound data, and when determining that a fault occurs in the household appliance, determine a first target fault type and a solution corresponding to the first target fault type; and / or obtain a preset fault prediction model, use the fault prediction model to identify the at least one characteristic sound data, and when predicting that an abnormality exists in the household appliance, predict a second target fault type.

[0008] In the above solution, after predicting the second target fault type, the method further includes:

[0009] The probability of occurrence of the second target fault is predicted based on the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data.

[0010] In the above solution, the method further includes: generating the diagnostic model; the generating the diagnostic model includes:

[0011] Acquire a first training sound data set including at least one first training sound data when a fault occurs and a first fault type corresponding to each first training sound data; the first fault type represents the cause of the fault of the faulty component in the household appliance;

[0012] Classifying the at least one first training sound data to obtain a first sound data set for at least one first fault type; the first sound data set includes at least one first training sound data;

[0013] A preset neural network is obtained, and the neural network is trained according to the first sound data set for at least one first fault type, and the trained neural network is obtained as the diagnostic model.

[0014] In the above solution, the step of training the neural network according to the first sound data set for at least one first fault type includes:

[0015] Determine the sound features of each first training sound data in the first sound data set; the sound features include at least one of the following: a spectrogram, a short-time amplitude zero-crossing rate, a short-time average energy, and a Mel Frequency Cepstrum Coefficient (MFCC);

[0016] The neural network is trained according to the sound features of each first training sound data in the first sound data set for at least one first fault type.

[0017] In the above solution, the determining the first target fault type and the solution corresponding to the first target fault type includes:

[0018] Obtain a preset correspondence between a fault type and a solution; query the correspondence according to the determined first target fault type, and determine a solution corresponding to the first target fault type.

[0019] In the above solution, the method further comprises: generating the fault prediction model;

[0020] The generating of the fault prediction model comprises:

[0021] Acquire a second training sound data set; the second training sound data set includes: at least one second training sound data in a preset time period before and after the fault occurs, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of the fault of the faulty component in the household appliance;

[0022] Classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, wherein the second sound data set includes the at least one second training sound data;

[0023] A preset logistic regression model is obtained, and the logistic regression model is trained according to the second sound data set for at least one second fault type, and the trained logistic regression model is obtained as the fault prediction model.

[0024] The embodiment of the present invention provides a fault processing device, the device comprising: a first processing module and a second processing module; wherein:

[0025] The first processing module is used to receive at least one characteristic sound data sent by the household appliance;

[0026] The second processing module is used to obtain a preset diagnostic model, use the diagnostic model to identify the at least one characteristic sound data, and when a fault occurs in the household appliance, determine a first target fault type and a solution corresponding to the first target fault type; and / or obtain a preset fault prediction model, use the fault prediction model to identify the at least one characteristic sound data, and when an abnormality occurs in the household appliance, predict a second target fault type.

[0027] In the above scheme, the second processing module is also used to predict the probability of the second target fault occurring based on the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data.

[0028] In the above solution, the device further comprises: a first preprocessing module, used to generate the diagnosis model;

[0029] The first preprocessing module is used to obtain a first training sound data set including at least one first training sound data when a fault occurs and a first fault type corresponding to each first training sound data; the first fault type represents the cause of the fault of the faulty component in the household appliance;

[0030] Classifying the at least one first training sound data to obtain a first sound data set for at least one first fault type; the first sound data set includes at least one first training sound data;

[0031] A preset neural network is obtained, and the neural network is trained according to the first sound data set for at least one first fault type, and the trained neural network is obtained as the diagnostic model.

[0032] In the above scheme, the first preprocessing module is used to determine the sound features of each first training sound data in the first sound data set; the sound features include at least one of the following: spectrogram, short-time amplitude zero-crossing rate, short-time average energy and MFCC; the neural network is trained according to the sound features of each first training sound data in the first sound data set for at least one first fault type.

[0033] In the above solution, the second processing module is used to obtain a correspondence between a preset fault type and a solution; query the correspondence according to the determined first target fault type, and determine a solution corresponding to the first target fault type.

[0034] In the above solution, the device includes: a second preprocessing module, used to generate the fault prediction model;

[0035] The second preprocessing module is used to obtain a second training sound data set; the second training sound data set includes: at least one second training sound data in a preset time period before and after the fault occurs, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of the fault of the faulty component in the household appliance;

[0036] Classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, wherein the second sound data set includes the at least one second training sound data;

[0037] A preset logistic regression model is obtained, and the logistic regression model is trained according to the second sound data set for at least one second fault type, and the trained logistic regression model is obtained as the fault prediction model.

[0038] An embodiment of the present invention provides an information processing device, the device comprising: a processor and a memory for storing a computer program that can be run on the processor;

[0039] Wherein, when the processor is used to run the computer program, it executes the steps of any of the above fault handling methods.

[0040] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above fault handling methods are implemented.

[0041] An embodiment of the present invention further provides a server, comprising a processor and a memory storing an executable program run by the processor, wherein the processor executes the steps of any of the above fault handling methods when running the executable program.

[0042] The fault handling method, device, information processing device, computer-readable storage medium and server provided in the embodiments of the present invention receive at least one characteristic sound data sent by a household appliance; obtain a preset diagnostic model, use the diagnostic model to identify the at least one characteristic sound data, determine a first target fault type and a solution corresponding to the first target fault type when a fault occurs in the household appliance; and / or obtain a preset fault prediction model, use the fault prediction model to identify the at least one characteristic sound data, predict a second target fault type when an abnormality occurs in the household appliance. In the solution of the embodiments of the present invention, by analyzing the sounds emitted by each component in the household appliance when a fault occurs, the fault type is diagnosed and a specific solution is provided, thereby improving the efficiency of fault diagnosis and problem solving, thereby improving user satisfaction with the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a fault handling method provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the structure of a household appliance provided by an embodiment of the present invention;

[0045] Figure 3 A flowchart of another fault handling method provided by an embodiment of the present invention;

[0046] Figure 4 A flowchart of another fault handling method provided by an embodiment of the present invention;

[0047] Figure 5 A schematic diagram of the structure of a fault handling device provided by an embodiment of the present invention;

[0048] Figure 6 A schematic diagram of the structure of another information processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In various embodiments of the present invention, at least one characteristic sound data sent by a household appliance is received; a preset diagnostic model is obtained, and the diagnostic model is used to identify the at least one characteristic sound data, and when a fault occurs in the household appliance, a first target fault type and a solution corresponding to the first target fault type are determined; and / or a preset fault prediction model is obtained, and the fault prediction model is used to identify the at least one characteristic sound data, and when an abnormality is predicted in the household appliance, a second target fault type is predicted.

[0050] The present invention will be further described in detail below in conjunction with the embodiments.

[0051] Figure 1A flowchart of a fault handling method provided by an embodiment of the present invention; the method can be applied to a server, and the server can communicate with various household appliances, including cooking equipment, cleaning equipment, etc.; Figure 1 As shown, the method includes:

[0052] Step 101: Receive at least one characteristic sound data sent by a household appliance.

[0053] Here, the characteristic sound data includes: the sound emitted by a device of a household appliance when it is in operation. The at least one characteristic sound data characterizes the sound emitted by a device of at least one household appliance when it is in operation. The household appliance includes: at least one device to be monitored that needs to be monitored for faults, and at least one sound collector is provided for each device to be monitored; the sound collector may include: a microphone, a recorder, a microphone, and other devices with a sound collection function.

[0054] The household appliance further comprises: a central controller for sending the sound data collected by the at least one sound collector to a server. The sound data carries the identity of the household appliance and / or the identity of the monitored device. The household appliance has a communication module, which can send and receive data with the server through a network.

[0055] After receiving the sound data, the server stores the sound data and the identity identifier of the home appliance in correspondence.

[0056] The household appliances include cooking equipment and non-cooking equipment. The cooking equipment may include rice cookers, induction cookers, etc.; the non-cooking equipment may include televisions, refrigerators, sweeping robots, etc.

[0057] Step 102: obtain a preset diagnostic model, use the diagnostic model to identify the at least one characteristic sound data, and when determining that a fault occurs in the household appliance, determine a first target fault type and a solution corresponding to the first target fault type; and / or, obtain a preset fault prediction model, use the fault prediction model to identify the at least one characteristic sound data, and when predicting that an abnormality exists in the household appliance, predict a second target fault type.

[0058] Specifically, the using the diagnostic model to identify the at least one characteristic sound data, determining a first target fault type and a solution corresponding to the first target fault type when determining that the household appliance fails, includes:

[0059] The server uses the diagnostic model to identify the at least one characteristic sound data to determine whether the household appliance has a fault. When it is determined that the household appliance has a fault, the server determines a first target fault type and a solution corresponding to the first target fault type. When it is determined that the household appliance has not a fault, the household appliance can be directly judged to be normal without performing any operation. Alternatively, a preset fault prediction model can be further obtained, and the fault prediction model can be used to identify the at least one characteristic sound data, and when it is predicted that the household appliance has an abnormality, a second target fault type is predicted.

[0060] Here, the first target fault type represents the faulty component currently faulty in the household appliance and the reason for the fault. That is, the first target fault type includes at least one of the following information: the faulty component and the reason for the fault of the component.

[0061] Specifically, the method further includes: generating the diagnosis model.

[0062] Here, generating a diagnostic model includes:

[0063] Acquire a first training sound data set; here, the first training sound data set may specifically include: at least one first training sound data when a fault occurs, and a first fault type corresponding to each first training sound data; the first fault type indicates the cause of the fault of the faulty component in the household appliance;

[0064] Classifying the at least one first training sound data to obtain a first sound data set for at least one first fault type; the first sound data set includes at least one first training sound data;

[0065] A preset neural network is obtained, and the neural network is trained according to the first sound data set for at least one first fault type, and the trained neural network is obtained as the diagnostic model.

[0066] Before training the neural network according to the first sound data set for at least one first fault type, the method may further include:

[0067] Formatting is performed on each first training sound data in the first sound data set to obtain sound data that can be used for neural network training.

[0068] Here, the sound data used for neural network training includes: sound features; the sound features include at least one of the following: spectrogram, short-time amplitude zero-crossing rate, short-time average energy and Mel Frequency Cepstrum Coefficient (MFCC).

[0069] The sound features may also include one or any combination of the following: wavelet packet decomposition coefficients, fundamental subband energy, amplitude or power, neighboring band feature vectors, Linear Prediction Coding Cepstrum Coefficients (LPCC)

[0070] Specifically, the training of the neural network according to the first sound data set for at least one first fault type includes:

[0071] Determine the sound features of each first training sound data in the first sound data set; the sound features include at least one of the following: a spectrogram, a short-time amplitude zero-crossing rate, a short-time average energy, and MFCC;

[0072] The neural network is trained according to the sound features of each first training sound data in the first sound data set for at least one first fault type.

[0073] Specifically, determining the first target fault type and a solution corresponding to the first target fault type includes:

[0074] Obtain a preset correspondence between a fault type and a solution; query the correspondence according to the determined first target fault type, and determine a solution corresponding to the first target fault type.

[0075] Here, the first target fault type obtained and its corresponding solution are used to be sent to the home appliance and / or a mobile terminal associated with the home appliance, and provided to the user to solve the fault.

[0076] Specifically, the solutions include: solutions that can solve the fault through software upgrades and solutions that cannot solve the fault through software upgrades.

[0077] In this embodiment, the method further includes: when it is determined that the solution is a solution that can solve the fault through software upgrade, sending the first target fault type and its corresponding solution to the home appliance and to a mobile terminal associated with the home appliance to remind the user to upgrade the software of the home appliance to solve the fault. When it is determined that the solution is a solution that cannot solve the fault through software upgrade, sending the first target fault type and its corresponding solution to the mobile terminal associated with the home appliance to remind the user to send the home appliance to a maintenance point for maintenance.

[0078] It should be noted that the mobile terminal may be installed with an application, which is a program developed by a developer and provided to users for download and installation; the application connects to the server and displays various services provided by the server to the user. The server is developed and maintained by the developer of the home appliance.

[0079] In this embodiment, considering that the household appliance may be abnormal (such as an impending failure) after running for a period of time, a fault prediction model is also provided in this embodiment to predict whether the household appliance is abnormal (i.e., whether a failure may occur; for example, the household appliance may not have a failure at present, but there may be a hidden danger of failure. At this time, the sound emitted by some components of the household appliance may be mixed with noise, so the failure can be predicted based on the sound data). Therefore, in this embodiment, after the server determines that the household appliance has been running for a period of time, it can obtain a preset fault prediction model, use the fault prediction model to identify the at least one characteristic sound data, and predict the second target fault type when predicting that the household appliance is abnormal.

[0080] It should be noted that, in this embodiment, the obtaining of a preset fault prediction model and the use of the fault prediction model to identify the at least one characteristic sound data may also be performed immediately after determining that the household appliance has not failed. Specifically, the method may further include: when determining that the household appliance has not failed, further obtaining a preset fault prediction model, using the fault prediction model to identify the at least one characteristic sound data, predicting whether the household appliance has an abnormality, and when predicting that the household appliance has an abnormality, predicting a second target fault type of the household appliance.

[0081] Here, the second target fault type is a predicted faulty component of the household appliance that may fail, and the reason for the failure. That is, the second target fault type includes at least one of the following information: a component that may fail, and the reason for the failure of the component.

[0082] Specifically, after predicting the second target fault type, the method may further include:

[0083] The probability of occurrence of the second target fault type is predicted based on the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data.

[0084] Here, predicting the probability of the second target fault occurring according to the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data specifically includes:

[0085] Determining, according to the second target fault type, sound data corresponding to the device that may fail from the received at least one characteristic sound data;

[0086] The sound data corresponding to the second target fault type in the fault prediction model is matched with the sound data corresponding to the device that may fail, so as to determine the probability of the second target fault occurring.

[0087] Here, after predicting the second target fault type of the household appliance, the method may further include:

[0088] Sending the predicted second target fault type to a mobile terminal associated with the household appliance to notify the user; and / or,

[0089] The second target fault type and the information of the mobile terminal associated with the home appliance are sent to the developer of the home appliance, and the developer's maintenance personnel contact the user to solve the fault.

[0090] The information of the mobile terminal associated with the home appliance may include: the corresponding communication number of the mobile terminal (such as the user's mobile phone number), and the login account of the application installed in the mobile terminal. The application is a program developed by the developer and provided to the user for download and installation, and the application is connected to the server.

[0091] It should be noted that the failure of household appliances may be discontinuous, for example, the failure may occur only once after being used five times. In this case, the first target failure type may not be identified using the above-mentioned diagnostic model. Therefore, a fault prediction model is provided to predict the possible failure of the household appliance, that is, to determine the second target failure type.

[0092] Here, after predicting the second target fault type of the household appliance, the method may further include: obtaining a correspondence between a preset fault type and a solution; querying the correspondence according to the determined second target fault type, and determining a solution corresponding to the second target fault type.

[0093] Correspondingly, when it is determined that the solution corresponding to the second target fault type is a solution that can solve the fault through software upgrade, when the predicted second target fault type is sent to the mobile terminal associated with the home appliance, the solution corresponding to the second fault type can be sent to the home appliance.

[0094] Specifically, the method further includes: generating the fault prediction model.

[0095] Here, generating a fault prediction model includes:

[0096] Acquire a second training sound data set; the second training sound data set includes: at least one second training sound data in a preset time period before and after the fault occurs, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of the fault of the faulty component in the household appliance;

[0097] Classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, wherein the second sound data set includes the at least one second training sound data;

[0098] A preset logistic regression model is obtained, and the logistic regression model is trained according to the second sound data set for at least one second fault type, and the trained logistic regression model is obtained as the fault prediction model.

[0099] Here, specifically, a linear logistic regression prediction algorithm or a nonlinear logistic regression prediction method may be used to train the logistic regression model according to the second sound data set for at least one second fault type.

[0100] Specifically, the training of the logistic regression model according to the second sound data set for at least one second fault type includes:

[0101] Determine the sound features of each second training sound data in the second sound data set; the sound features include at least one of the following: spectrogram, short-time amplitude zero-crossing rate, short-time average energy and MFCC;

[0102] The neural network is trained according to the sound features of each second training sound data in the second sound data set for at least one second fault type.

[0103] Here, the logistic regression algorithm may adopt a linear regression prediction method or a nonlinear regression prediction method.

[0104] Before training the logistic regression model according to the second sound data set for at least one second fault type, the method may further include:

[0105] Formatting is performed on each second training sound data in the second sound data set to obtain second training sound data in a target format.

[0106] Accordingly, the logistic regression model is trained according to the second training sound data in the target format.

[0107] Figure 2 A schematic diagram of the structure of a household appliance provided by an embodiment of the present invention; Figure 2As shown, the household appliance may be a cooking device, and the cooking device may include a central controller and at least one sound collector.

[0108] The sound collector may include: microphones 1 to n (eg Figure 2 , the 1st microphone, the 2nd microphone, the Kth microphone, the nth microphone shown in ).

[0109] Each sound collector is arranged near a device that is prone to failure and can generate noise, so as to collect the sound of the device.

[0110] Figure 3 A flowchart of another fault handling method provided by an embodiment of the present invention; Figure 3 As shown, the method is applied to the server, and the method includes:

[0111] Step 301: Obtain a first training sound data set.

[0112] The first training sound data set includes: at least one first training sound data when a fault occurs, and a first fault type corresponding to each first training sound data; the first fault type represents the cause of the fault of the faulty component in the household appliance.

[0113] Step 302: preprocess the first training sound data set.

[0114] Here, the preprocessing may include: classification and formatting. Specifically, the preprocessing includes: classifying the at least one set of first training sound data to obtain a first sound data set for at least one first fault type, the first sound data set including at least one first training sound data; formatting each first training sound data to obtain a format that can be used for sound data analysis.

[0115] Here, the classification of the at least one training sound data includes: determining the first fault type corresponding to each training sound data; the first fault type refers to the fault cause of a certain device failure; and classifying according to the first fault type to obtain a first sound data set for at least one first fault type.

[0116] Step 303: Perform classification training based on a deep neural network according to the first training sound data set to obtain a diagnosis model.

[0117] Specifically, the step 303 includes: obtaining a preset neural network, training the neural network according to the first sound data set for at least one first fault type, and obtaining the trained neural network as the diagnostic model.

[0118] The diagnostic model is used to determine the failed device and the cause of the failure of the device based on the sound data, so as to obtain the fault type.

[0119] The operation of step 303 can be specifically referred to Figure 1 The generation of the diagnostic model described in the method shown will not be repeated here.

[0120] Step 304: obtain the sound data sent by the household appliance to be identified, use the diagnosis model to perform fault identification, and obtain a diagnosis result.

[0121] Specifically, the step 304 includes: receiving the sound data sent by the household appliance; the sound data may carry the identification (ID) of the corresponding device;

[0122] The diagnostic model identifies the sound data to obtain a diagnostic result; the diagnostic result includes: a first target fault type and a solution corresponding to the first target fault type.

[0123] The obtained diagnosis result can be sent to the household appliance and / or a mobile terminal associated with the household appliance to inform the user of the specific fault type and solution.

[0124] Using the above method, the server receives a fault diagnosis request, which includes the sound data emitted by the device when a fault occurs, and performs fault diagnosis based on the sound data; the server can locate the fault type through the diagnosis model when a fault occurs, specifically including: a certain device, the cause of the fault, etc., and at the same time determine whether the cause of the fault is a software problem or a hardware problem. When it is determined that the cause of the fault is a software problem, the problem is solved by upgrading the software.

[0125] Figure 4 A flowchart of another fault handling method provided by an embodiment of the present invention; Figure 4 As shown, the method comprises:

[0126] Step 401: Obtain a second training sound data set.

[0127] Here, the second training sound data set includes: at least one second training sound data within a preset time period before and during a fault occurrence, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of the fault of a faulty component in the household appliance.

[0128] Step 402: pre-process the second training sound data set.

[0129] Here, step 402 includes: classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, the second sound data set including at least one second training sound data; and formatting the second training sound data to obtain a format that can be used for sound data analysis.

[0130] Here, the preprocessing of the at least one second training sound data includes: determining the second fault type corresponding to each second training sound data, classifying the at least one second training sound data according to the second fault type, and obtaining a second sound data set for at least one second fault type, wherein the second sound data set includes at least one second training sound data.

[0131] Step 403: Train a logistic regression model according to the second sound data set for at least one second fault type to obtain a fault prediction model.

[0132] Specifically, the step 403 includes: obtaining a preset logistic regression model, using a logistic regression algorithm to train the logistic regression model according to the second sound data set for at least one second fault type, and obtaining the trained logistic regression model as the fault prediction model.

[0133] The fault prediction model is used to predict the device that may fail and the reason why the device may fail based on the sound data. The logistic regression algorithm can use a linear regression prediction method or a nonlinear regression prediction method.

[0134] The operation of step 403 can be specifically referred to Figure 1 The generation of the fault prediction model described in the method shown will not be repeated here.

[0135] Step 404: obtain the sound data sent by the household appliance to be identified, use the fault prediction model to perform fault identification, and obtain a prediction result.

[0136] Specifically, the step 404 includes: receiving the sound data sent by the household appliance, wherein the sound data may carry the ID of the device corresponding to the sound data;

[0137] The fault prediction model identifies the sound data to obtain a prediction result; the prediction result includes at least one of the following: a second target fault type, a probability of occurrence of the second target fault, and a solution corresponding to the second fault type.

[0138] The prediction result may be sent to the household appliance and / or a mobile terminal associated with the household appliance to inform the user of the specific fault type and solution.

[0139] In this embodiment, the method is that the server side continuously receives sound data reported by the household appliances. After a certain amount of sound data is collected, the sound data within a preset time period can be obtained, and the sound data is pre-processed and then input into a fault prediction model; through a machine learning logistic regression algorithm, a prediction is made based on the sound data whether the household appliance is operating normally or the state that will occur in the appliance. If it is predicted that the appliance will fail, the user or the server side, or even the developer, is reminded to perform equipment maintenance. The model can be used to locate the type of fault, specifically a certain component and its cause, and at the same time determine whether the cause of the fault is a software problem or a hardware problem. When it is determined that the cause of the fault is a software problem, the problem is solved by upgrading the software.

[0140] The embodiment of the present invention also provides a method, Figure 3 and Figure 4 The method shown is combined, that is, after receiving the sound data sent by the household appliance; obtaining a preset diagnostic model, identifying the at least one characteristic sound data according to the diagnostic model, determining that the household appliance fails, determining the first target fault type and the solution corresponding to the first target fault type; and / or, obtaining a preset fault prediction model, using the fault prediction model to identify the at least one characteristic sound data, predicting that the household appliance has an abnormality, and predicting the second target fault type. Here, the use of the fault prediction model to identify the at least one characteristic sound data can occur immediately when it is determined that the household appliance has not failed; or it can be an autonomous fault prediction after the household appliance has been running for a period of time, that is, after determining that the household appliance has been running for a period of time, obtaining a preset fault prediction model, identifying the at least one characteristic sound data according to the fault prediction model, and predicting the second target fault type of the household appliance; it is not limited here. Its specific steps belong to the same concept as the above-mentioned fault handling method embodiment, and its specific implementation process is detailed in Figure 1 , Figure 3 , and / or Figure 4 The method embodiments shown are not described in detail here.

[0141] Figure 5 A schematic diagram of a fault handling device provided by an embodiment of the present invention; Figure 5 As shown, the device includes: the first processing module 501 and the second processing module 502.

[0142] The first processing module 501 is used to receive at least one characteristic sound data sent by the home appliance;

[0143] The second processing module 502 is used to obtain a preset diagnostic model, use the diagnostic model to identify the at least one characteristic sound data, and when a fault occurs in the household appliance, determine a first target fault type and a solution corresponding to the first target fault type; and / or obtain a preset fault prediction model, use the fault prediction model to identify the at least one characteristic sound data, and when an abnormality occurs in the household appliance, predict a second target fault type.

[0144] Specifically, the second processing module 502 is further used to predict the probability of occurrence of the second target fault according to the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data.

[0145] Specifically, the device further includes: a first preprocessing module, used to generate the diagnosis model.

[0146] The first preprocessing module is used to obtain a first training sound data set including at least one first training sound data when a fault occurs and a first fault type corresponding to each first training sound data; the first fault type represents the cause of the fault of the faulty component in the household appliance;

[0147] Classifying the at least one first training sound data to obtain a first sound data set for at least one first fault type; the first sound data set includes at least one first training sound data;

[0148] A preset neural network is obtained, and the neural network is trained according to the first sound data set for at least one first fault type, and the trained neural network is obtained as the diagnostic model.

[0149] The first preprocessing module is specifically used to determine the sound features of each first training sound data in the first sound data set; the sound features include at least one of the following: spectrogram, short-time amplitude zero-crossing rate, short-time average energy and MFCC; according to the sound features of each first training sound data in the first sound data set for at least one first fault type, the neural network is trained.

[0150] Specifically, the second processing module 502 is used to obtain a preset correspondence relationship between a fault type and a solution; query the correspondence relationship according to the determined first target fault type, and determine a solution corresponding to the first target fault type.

[0151] Specifically, the device includes: a second preprocessing module, used to generate the fault prediction model.

[0152] The second preprocessing module is used to obtain a second training sound data set; the second training sound data set includes: at least one second training sound data in a preset time period before and after the fault occurs, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of the fault of the faulty component in the household appliance;

[0153] Classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, wherein the second sound data set includes the at least one second training sound data;

[0154] A preset logistic regression model is obtained, and the logistic regression model is trained according to the second sound data set for at least one second fault type, and the trained logistic regression model is obtained as the fault prediction model.

[0155] It should be noted that: the fault handling device provided in the above embodiment only uses the division of the above program modules as an example when performing fault handling. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the fault handling device provided in the above embodiment and the fault handling method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0156] To implement the method of the embodiment of the present invention, the embodiment of the present invention provides an information processing device, such as Figure 6 As shown, the device 60 includes: a processor 601 and a memory 602 for storing a computer program that can be run on the processor; wherein, when the processor 601 is used to run the computer program, it executes: receiving at least one characteristic sound data sent by a household appliance; obtaining a preset diagnostic model, using the diagnostic model to identify the at least one characteristic sound data, and when determining that the household appliance fails, determining a first target fault type and a solution corresponding to the first target fault type; and / or obtaining a preset fault prediction model, using the fault prediction model to identify the at least one characteristic sound data, and when predicting that the household appliance has an abnormality, predicting a second target fault type.

[0157] In one embodiment, when the processor 601 is used to run the computer program, the processor 601 executes: predicting the probability of the second target fault occurring based on the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data.

[0158] In one embodiment, the processor 601 is used to execute the following when running the computer program: obtaining a first training sound data set including at least one first training sound data when a fault occurs and a first fault type corresponding to each first training sound data; the first fault type characterizes the cause of the fault of a faulty component in the household appliance; classifying the at least one first training sound data to obtain a first sound data set for at least one first fault type; the first sound data set includes at least one first training sound data; obtaining a preset neural network, training the neural network according to the first sound data set for at least one first fault type, and obtaining the trained neural network as the diagnostic model.

[0159] In one embodiment, when the processor 601 is used to run the computer program, the following steps are performed: determining the sound features of each first training sound data in the first sound data set; the sound features include at least one of the following: a spectrogram, a short-time amplitude zero-crossing rate, a short-time average energy, and a MFCC; and training the neural network according to the sound features of each first training sound data in the first sound data set for at least one first fault type.

[0160] In one embodiment, when the processor 601 is used to run the computer program, the processor 601 executes: obtaining a correspondence between a preset fault type and a solution; querying the correspondence according to the determined first target fault type, and determining a solution corresponding to the first target fault type.

[0161] In one embodiment, when the processor 601 is used to run the computer program, the following is performed: obtaining a second training sound data set; the second training sound data set includes: at least one second training sound data within a preset time period before and during a fault, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of a fault in a faulty component in a household appliance; classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, the second sound data set including at least one second training sound data; obtaining a preset logistic regression model, training the logistic regression model based on the second sound data set for at least one second fault type, and obtaining the trained logistic regression model as the fault prediction model.

[0162] It should be noted that the information processing device and the fault handling method embodiment provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0163] Of course, in practical applications, Figure 6As shown, the device 60 may also include: at least one network interface 603. The various components in the information processing device 60 are coupled together via a bus system 604. It is understood that the bus system 604 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 6 In the embodiment of the present invention, various buses are marked as bus system 604. The number of the processor 601 can be at least one. The network interface 603 is used for wired or wireless communication between the information processing device 60 and other devices. The memory 602 in the embodiment of the present invention is used to store various types of data to support the operation of the device 60.

[0164] The method disclosed in the above embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 601 or the instruction in the form of software. The above processor 601 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 601 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 602. The processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0165] In an exemplary embodiment, the information processing device 60 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0166] Specifically, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program performs the following steps: receiving at least one characteristic sound data sent by a household appliance; obtaining a preset diagnostic model, using the diagnostic model to identify the at least one characteristic sound data, and when determining that a fault occurs in the household appliance, determining a first target fault type and a solution corresponding to the first target fault type; and / or obtaining a preset fault prediction model, using the fault prediction model to identify the at least one characteristic sound data, and when predicting that an abnormality exists in the household appliance, predicting a second target fault type.

[0167] In one embodiment, when the computer program is executed by a processor, the computer program performs: predicting the probability of occurrence of a second target fault based on the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data.

[0168] In one embodiment, when the computer program is executed by a processor, the following is performed: obtaining a first training sound data set including at least one first training sound data when a fault occurs and a first fault type corresponding to each first training sound data; the first fault type characterizes the cause of the fault of a faulty component in a household appliance; classifying the at least one first training sound data to obtain a first sound data set for at least one first fault type; the first sound data set includes at least one first training sound data; obtaining a preset neural network, training the neural network according to the first sound data set for at least one first fault type, and obtaining the trained neural network as the diagnostic model.

[0169] In one embodiment, when the computer program is executed by the processor, the computer program performs the following steps: determining the sound features of each first training sound data in the first sound data set; the sound features include at least one of the following: a spectrogram, a short-time amplitude zero-crossing rate, a short-time average energy, and a MFCC; and training the neural network according to the sound features of each first training sound data in the first sound data set for at least one first fault type.

[0170] In one embodiment, when the computer program is executed by a processor, the computer program performs: obtaining a correspondence between a preset fault type and a solution; and querying the correspondence according to the determined first target fault type to determine a solution corresponding to the first target fault type.

[0171] In one embodiment, when the computer program is executed by a processor, the following is performed: obtaining a second training sound data set; the second training sound data set includes: at least one second training sound data within a preset time period before and during a fault, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of a fault in a faulty component in a household appliance; classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, the second sound data set including at least one second training sound data; obtaining a preset logistic regression model, training the logistic regression model based on the second sound data set for at least one second fault type, and obtaining the trained logistic regression model as the fault prediction model.

[0172] An embodiment of the present invention also provides a server, which includes a processor and a memory storing an executable program run by the processor; when the processor runs the executable program, it performs: receiving at least one characteristic sound data sent by a household appliance; obtaining a preset diagnostic model, using the diagnostic model to identify the at least one characteristic sound data, and when determining that a fault occurs in the household appliance, determining a first target fault type and a solution corresponding to the first target fault type; and / or obtaining a preset fault prediction model, using the fault prediction model to identify the at least one characteristic sound data, and when predicting that an abnormality exists in the household appliance, predicting a second target fault type.

[0173] The processor is further configured to, when running the computer program, execute: predicting the probability of occurrence of a second target fault based on the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data.

[0174] The processor is also used to execute the following when running the computer program: obtaining a first training sound data set including at least one first training sound data when a fault occurs and a first fault type corresponding to each first training sound data; the first fault type characterizes the cause of the fault of a faulty component in the household appliance; classifying the at least one first training sound data to obtain a first sound data set for at least one first fault type; the first sound data set includes at least one first training sound data; obtaining a preset neural network, training the neural network according to the first sound data set for at least one first fault type, and obtaining the trained neural network as the diagnostic model.

[0175] The processor is also used to execute the following when running the computer program: determining the sound features of each first training sound data in the first sound data set; the sound features include at least one of the following: spectrogram, short-time amplitude zero-crossing rate, short-time average energy and MFCC; training the neural network according to the sound features of each first training sound data in the first sound data set for at least one first fault type.

[0176] The processor is further configured to execute, when running the computer program: obtaining a correspondence between a preset fault type and a solution; and querying the correspondence according to the determined first target fault type to determine a solution corresponding to the first target fault type.

[0177] The processor is also used to execute the following when running the computer program: obtaining a second training sound data set; the second training sound data set includes: at least one second training sound data within a preset time period before and during the occurrence of a fault, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of a fault in a faulty component in a household appliance; classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, the second sound data set including at least one second training sound data; obtaining a preset logistic regression model, training the logistic regression model based on the second sound data set for at least one second fault type, and obtaining the trained logistic regression model as the fault prediction model.

[0178] In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0179] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0180] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0181] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0182] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0183] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A fault handling method, characterized in that: The method comprises: Receive at least one characteristic sound data sent by a household appliance; the at least one characteristic sound data represents the sound emitted by the household appliance during operation, the household appliance includes at least one monitored device that needs to be monitored for faults, and each of the monitored devices is provided with at least one sound collector; each sound collector is arranged at an attachment of a device that is prone to faults and can generate noise sources, so as to collect the sound of the device; Obtain a preset diagnostic model, and use the diagnostic model to identify at least one characteristic sound data, wherein the at least one characteristic sound data carries the identity of the household appliance and / or the identity of the monitored device; when it is determined that the household appliance fails, determine a first target fault type and a solution corresponding to the first target fault type; and / or, obtain a preset fault prediction model, and use the fault prediction model to identify the at least one characteristic sound data, and when it is predicted that the household appliance has an abnormality, predict a second target fault type.

2. The method according to claim 1, characterized in that After predicting the second target fault type, the method further includes: The probability of occurrence of the second target fault is predicted based on the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data.

3. The method according to claim 1, characterized in that: The method further includes: generating the diagnostic model; the generating the diagnostic model includes: Acquire a first training sound data set including at least one first training sound data when a fault occurs and a first fault type corresponding to each first training sound data; the first fault type represents the cause of the fault of the faulty component in the household appliance; Classifying the at least one first training sound data to obtain a first sound data set for at least one first fault type; the first sound data set includes at least one first training sound data; A preset neural network is obtained, and the neural network is trained according to the first sound data set for at least one first fault type, and the trained neural network is obtained as the diagnostic model.

4. The method according to claim 3, characterized in that The step of training the neural network according to the first sound data set for at least one first fault type comprises: Determine the sound features of each first training sound data in the first sound data set; the sound features include at least one of the following: a spectrogram, a short-time amplitude zero-crossing rate, a short-time average energy, and a Mel-frequency cepstral coefficient MFCC; The neural network is trained according to the sound features of each first training sound data in the first sound data set for at least one first fault type.

5. The method according to claim 1, characterized in that The determining a first target fault type and a solution corresponding to the first target fault type includes: Obtain a preset correspondence between a fault type and a solution; query the correspondence according to the determined first target fault type, and determine a solution corresponding to the first target fault type.

6. The method according to claim 2, characterized in that The method further comprises: generating the fault prediction model; The generating of the fault prediction model comprises: Acquire a second training sound data set; the second training sound data set includes: at least one second training sound data in a preset time period before and after the fault occurs, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of the fault of the faulty component in the household appliance; Classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, wherein the second sound data set includes the at least one second training sound data; A preset logistic regression model is obtained, and the logistic regression model is trained according to the second sound data set for at least one second fault type, and the trained logistic regression model is obtained as the fault prediction model.

7. A fault handling device, characterized in that: The device comprises: a first processing module and a second processing module; wherein, The first processing module is used to receive at least one characteristic sound data sent by the household appliance; the at least one characteristic sound data represents the sound emitted by the household appliance during operation, the household appliance includes at least one monitored device that needs to be monitored for faults, and each of the monitored devices is provided with at least one sound collector; each sound collector is arranged at an attachment of a device that is prone to faults and can generate noise sources, so as to collect the sound of the device; The second processing module is used to obtain a preset diagnostic model, use the diagnostic model to identify the at least one characteristic sound data, and the at least one characteristic sound data carries the identity of the household appliance and / or the identity of the monitored device; when it is determined that the household appliance has a fault, determine a first target fault type and a solution corresponding to the first target fault type; and / or, obtain a preset fault prediction model, use the fault prediction model to identify the at least one characteristic sound data, and when it is predicted that the household appliance has an abnormality, predict a second target fault type.

8. The device according to claim 7, characterized in that The second processing module is further used to predict the probability of occurrence of a second target fault based on the sound data corresponding to the second target fault type in the fault prediction model and the at least one characteristic sound data.

9. The device according to claim 7, characterized in that The device further comprises: a first preprocessing module, configured to generate the diagnostic model; The first preprocessing module is used to obtain a first training sound data set including at least one first training sound data when a fault occurs and a first fault type corresponding to each first training sound data; the first fault type represents the cause of the fault of the faulty component in the household appliance; Classifying the at least one first training sound data to obtain a first sound data set for at least one first fault type; the first sound data set includes at least one first training sound data; A preset neural network is obtained, and the neural network is trained according to the first sound data set for at least one first fault type, and the trained neural network is obtained as the diagnostic model.

10. The device according to claim 9, characterized in that The first preprocessing module is used to determine the sound features of each first training sound data in the first sound data set; the sound features include at least one of the following: spectrogram, short-time amplitude zero-crossing rate, short-time average energy and Mel-frequency cepstral coefficient MFCC; the neural network is trained according to the sound features of each first training sound data in the first sound data set for at least one first fault type.

11. The device according to claim 7, characterized in that The second processing module is used to obtain a preset correspondence between a fault type and a solution; query the correspondence according to the determined first target fault type, and determine a solution corresponding to the first target fault type.

12. The device according to claim 8, characterized in that The device comprises: a second preprocessing module, configured to generate the fault prediction model; The second preprocessing module is used to obtain a second training sound data set; the second training sound data set includes: at least one second training sound data in a preset time period before and after the fault occurs, and a second fault type corresponding to each second training sound data; the second fault type characterizes and predicts the cause of the fault of the faulty component in the household appliance; Classifying the at least one second training sound data to obtain a second sound data set for at least one second fault type, wherein the second sound data set includes the at least one second training sound data; A preset logistic regression model is obtained, and the logistic regression model is trained according to the second sound data set for at least one second fault type, and the trained logistic regression model is obtained as the fault prediction model.

13. An information processing device, characterized in that: The apparatus comprises: a processor and a memory for storing a computer program capable of running on the processor; Wherein, when the processor is used to run the computer program, it executes the steps of any one of the methods of claims 1 to 6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

15. A server, comprising a processor and a memory storing an executable program run by the processor, characterized in that: When the processor runs the executable program, the steps of the method according to any one of claims 1 to 6 are performed.

Citation Information

Cited By

  • Small household appliance fault work order parameter generation method and system based on image-text feature fusion

    CN123032842A