Household appliance fault diagnosis method, terminal device and server

Abstract

The embodiment of the invention discloses a household appliance fault diagnosis method, terminal equipment and a server. The terminal equipment execution method comprises the following steps: acquiring fault description data input by a user; sending a first query instruction to a server, wherein the first query instruction is used for instructing the server to query target knowledge from a knowledge base according to the fault description data; the knowledge base is generated according to a fault tree structured file, the fault tree structured file comprises at least one fault tree associated with the target household appliance, the fault tree comprises a plurality of nodes, and the nodes are configured with a fault representation image, a fault log, a repair mode and a fault phenomenon corresponding to one fault type; generating prompt word information according to the fault description data and the target knowledge and sending the prompt word information to a server; displaying corresponding response content on the man-machine conversation page according to response information sent by the server; the response information comprises the fault type and the recommended repair mode of the target household appliance. Therefore, the fault diagnosis accuracy and the fault repair efficiency of the household appliance can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of fault diagnosis, and in particular to a method for diagnosing faults of household appliances, a terminal device and a server. Background Art

[0002] The home scene includes a variety of home appliances, such as smart TVs, smart air conditioners, smart refrigerators, etc. Home appliances may have problems during use, such as software failures, hardware failures, etc. By troubleshooting and repairing the failures, the normal use of home appliances can be ensured.

[0003] The brand of home appliances provides after-sales customer service for users, and users can contact after-sales customer service to report faults. Users can explain the fault symptoms of home appliances to after-sales customer service, so that after-sales customer service can initially screen the cause of the fault, and then after-sales customer service will submit a repair work order to the maintenance personnel. The maintenance personnel will repair the home appliances at the time agreed with the user based on the user's contact information, address and cause of the fault indicated in the repair work order.

[0004] Due to factors such as users being unable to accurately describe equipment failures and after-sales customer service lacking professionalism and experience, the cause of the failure reported in the repair work order may not match the actual cause of the home appliance failure. For example, the repair work order indicates a software failure, but the home appliance actually has a hardware failure. In this way, a single equipment failure may be accompanied by multiple customer service communications or even multiple on-site repairs, resulting in low efficiency in home appliance failure diagnosis and repair, affecting users' use of home appliances and increasing the brand's after-sales costs. Summary of the invention

[0005] Some embodiments of the present application provide a method for diagnosing faults in home appliances, a terminal device, and a server to improve the efficiency of fault diagnosis and repair of home appliances and reduce the after-sales costs of home appliance brands.

[0006] In a first aspect, some embodiments of the present application provide a terminal device, including:

[0007] Display, used for displaying the human-computer dialogue page;

[0008] A communicator, used for communicating with a server;

[0009] The controller is configured as:

[0010] Acquire fault description data input by the user based on the human-machine dialogue page, wherein the fault description data is used to describe the fault details of the target home appliance;

[0011] Generate a first query instruction, and send the first query instruction to a server through a communicator; the first query instruction is used to instruct the server to query target knowledge from a knowledge base according to the fault description data; wherein the knowledge base is generated by the server according to a fault tree structured file, the fault tree structured file includes at least one fault tree associated with the target household appliance, the fault tree includes a plurality of nodes, and the nodes are configured with a fault manifestation image, a fault log, a repair method, and a fault phenomenon corresponding to a fault type;

[0012] receiving target knowledge sent by the server through the communicator, generating prompt word information according to the fault description data and the target knowledge, and sending the prompt word information to the server through the communicator;

[0013] Receive the response information sent by the server through the communicator, and control the display to display the response content corresponding to the response information on the human-computer dialogue page; wherein, the response information is output by the server after analyzing the prompt word information using a language processing model, and the response information includes the fault type and recommended repair method of the target home appliance.

[0014] The technical solution provided by the first aspect embodiment has the following beneficial effects: the user can conduct a human-computer dialogue with the customer service of the device brand, and enter fault description data on the human-computer dialogue page to describe the fault details of the target home appliance, providing a basis for the server to analyze the cause of the fault; the server configures at least one fault tree for the home appliance, and a node of the fault tree corresponds to a fault type, that is, the fault tree can characterize the fault classification, and the node is configured with a fault manifestation image, a fault log, a repair method and a fault phenomenon, etc., so that the fault tree can provide knowledge support for equipment fault diagnosis, so that the subsequent language processing model can accurately identify the cause of the fault and provide a feasible fault repair method; the end After the terminal device sends the first query instruction, the server can query the knowledge base according to the fault description data, and find similar or matching fault performance images, fault phenomena and repair methods from the knowledge base according to the fault details described by the user, so as to serve as reference knowledge for model decision-making; the terminal generates prompt word information based on the target knowledge and fault description data, and the language processing model on the server side analyzes the prompt word and outputs reliable response information, so that customer service can provide more accurate fault causes and recommended repair methods, helping users to solve equipment failures by themselves, thereby improving the fault diagnosis accuracy and fault repair efficiency of home appliances, reducing after-sales costs, and improving the quality of human-computer dialogue.

[0015] In some embodiments, before generating the first query instruction, the controller is further configured to: send a self-test instruction to the target home appliance through a server, the self-test instruction is used to instruct the target home appliance to execute a self-test program and upload the self-test data to the server; the self-test data includes a fault code and a self-test operation log; receive the self-test data sent by the server through a communicator, and splice the self-test data and the fault description data into the first data; generate a second query instruction, and send the second query instruction to the server through the communicator; the second query instruction is used to instruct the server to query the target knowledge from the knowledge base according to the first data. The technical solution provided by this embodiment has the following beneficial effects: for home appliances with self-test capabilities, the terminal can trigger the self-test of the target home appliance through the server, so that the target home appliance can provide self-test data, so that the terminal device expands the self-test data as a reference for fault diagnosis and analysis, so that the target knowledge queried by the knowledge base has a higher matching and more accurate target knowledge, so that the fault description data, self-test data and target knowledge input by the user are integrated to enhance the decision-making and analysis basis of the language processing model, thereby giving a more accurate cause of the fault and a recommended repair method.

[0016] In some embodiments, the fault description data is multimodal data including at least two of the fault description text, device fault image, device fault video, and fault description audio. The technical solution provided by this embodiment has the following beneficial effects: the human-computer dialogue page supports users to input fault description data through text, voice, video, audio, etc., so that the fault description data can have multimodal characteristics, realize multi-angle diagnosis and analysis of device faults, improve the accuracy of fault diagnosis, and enhance the human-computer interaction experience.

[0017] In a second aspect, some embodiments of the present application further provide a server, including:

[0018] A communicator, used for communication connection with a terminal device;

[0019] The controller is configured as:

[0020] receiving, through the communicator, a first query instruction sent by a terminal device, wherein the first query instruction includes fault description data input by a user based on a human-computer dialogue page displayed by the terminal device, wherein the fault description data is used to describe the fault details of the target household appliance;

[0021] In response to the first query instruction, query the target knowledge in the knowledge base according to the fault description data, and send the target knowledge to the terminal device through the communicator; wherein the knowledge base is generated by the server according to the fault tree structured file, the fault tree structured file includes at least one fault tree associated with the target household appliance, the fault tree includes a plurality of nodes, and the nodes are configured with a fault manifestation image, a fault log, a repair method and a fault phenomenon corresponding to a fault type;

[0022] receiving, through a communicator, prompt word information generated and sent by a terminal device according to the fault description data and the target knowledge, analyzing the prompt word information using a language processing model and outputting response information, the response information including the fault type and recommended repair method of the target home appliance;

[0023] The response information is sent to the terminal device through the communicator.

[0024] The technical solution provided by the above second aspect embodiment has the following beneficial effects: the user can conduct a human-computer dialogue with the customer service of the device brand, and enter the fault description data on the human-computer dialogue page to describe the fault details of the target home appliance, and provide a basis for the server to analyze the cause of the fault; the server configures at least one fault tree for the home appliance, and a node of the fault tree corresponds to a fault type, that is, the fault tree can characterize the fault classification, and the node is configured with a fault manifestation image, a fault log, a repair method and a fault phenomenon, etc., so that the fault tree can provide knowledge support for equipment fault diagnosis, so that the subsequent language processing model can accurately identify the cause of the fault and provide a feasible fault repair method; finally After the terminal device sends the first query instruction, the server can query the knowledge base according to the fault description data, and find similar or matching fault performance images, fault phenomena and repair methods from the knowledge base according to the fault details described by the user, so as to serve as reference knowledge for model decision-making; the terminal generates prompt word information based on the target knowledge and fault description data, and the language processing model on the server side analyzes the prompt word and outputs reliable response information, so that customer service can provide more accurate fault causes and recommended repair methods, helping users to solve equipment failures by themselves, thereby improving the fault diagnosis accuracy and fault repair efficiency of home appliances, reducing after-sales costs, and improving the quality of human-computer dialogue.

[0025] In some embodiments, the communicator is also used to communicate with the target home appliance, and before receiving the first query instruction sent by the terminal device through the communicator, the controller is also configured to: receive the self-test data uploaded by the target home appliance through the communicator, the self-test data including the fault code and the self-test operation log; send the self-test data to the terminal device through the communicator; receive the second query instruction sent by the terminal device through the communicator, the second query instruction including the fault description data and the first data after the self-test data is spliced; in response to the second query instruction, query the target knowledge in the knowledge base according to the first data. The technical solution provided by this embodiment has the following beneficial effects: for home appliances with self-test capabilities, the terminal can trigger the self-test of the target home appliance through the server, so that the target home appliance can provide self-test data, so that the terminal device expands the self-test data as a reference for fault diagnosis and analysis, so that the target knowledge queried by the knowledge base has a higher matching and more accurate target knowledge, so that the fault description data, self-test data and target knowledge input by the user are integrated to enhance the decision-making and analysis basis of the language processing model, thereby giving a more accurate fault cause and recommended repair method.

[0026] In some embodiments, the knowledge base includes a text database and a vector database, and the controller is further configured to: write the text data in the fault tree structured file into the text database and create an index according to the node entry rule, and complete the text index entry; wherein the node entry rule is to take the node of the fault tree as a unit, and perform entry processing on the text data and non-text data configured in the node; use the language processing model to encode and align the non-text data in the fault tree structured file, obtain a first semantic vector of a preset dimension, and write the first semantic vector into the vector database according to the node entry rule to complete the vector storage entry. The technical solution provided by this embodiment has the following beneficial effects: since the user can input the fault description data through different input methods (such as text, voice, video recording, etc.), the fault description data may have multimodal attributes, that is, including text data and non-text data, so that the knowledge base can construct a text database and a vector database respectively, wherein the text database is for knowledge query of text data, and the vector database is for knowledge query of non-text data, thereby being compatible with multimodal knowledge base query; the implementation of the node entry rule can ensure that when the equipment fault type only involves one node, the knowledge in the library can be quickly and accurately located and queried.

[0027] In some embodiments, the knowledge base includes a text database and a vector database, and the controller is further configured to: write the text data in the fault tree structured file into the text database and create an index according to the path entry rule to complete the text index entry; wherein the path entry rule is based on the path of the fault tree, and the text data and non-text data configured in at least two nodes contained in the path are entered into the database; the language processing model is used to encode and align the non-text data in the fault tree structured file to obtain a first semantic vector of a preset dimension, and according to the path entry rule, the first semantic vector is written into the vector database to complete the vector storage entry. The technical solution provided by this embodiment has the following beneficial effects: since the user can input fault description data through different input methods (such as text, voice, video recording, etc.), the fault description data may have multimodal attributes, that is, including text data and non-text data, so that the knowledge base can construct a text database and a vector database respectively, wherein the text database is for knowledge query of text data, and the vector database is for knowledge query of non-text data, thereby being compatible with multimodal knowledge base queries; implementing path entry rules can enhance the anti-interference ability of the knowledge base in complex environments (such as incomplete log input, unclear user fault description, etc.), and realize text retrieval and vector retrieval of any entire path, so as to provide comprehensive analysis for the language processing model and improve the robustness of the system.

[0028] In some embodiments, the controller searches for target knowledge in the knowledge base according to the fault description data, and is configured to: perform similarity calculation on the text data in the fault description data and the text data in the text database to obtain a first preset amount of text knowledge; use the language processing model to encode and align the non-text data in the fault description data to obtain a second semantic vector of a preset dimension; perform similarity calculation on the second semantic vector and the vector data in the vector database to obtain a second preset amount of non-text knowledge. The technical solution provided by this embodiment has the following beneficial effects: further defining how to use the text database and the vector database for knowledge query, wherein the text query is to perform similarity matching on the text in the fault description data (such as the text content described by the user) to find out more relevant text knowledge; when querying vectors, since it is necessary to vectorize the non-text data, similarity matching is performed with the vector in the database to find out relevant non-text knowledge (such as fault manifestation images, guidance images of recommended repair methods, etc.), thereby being compatible with multi-modal knowledge base queries, providing a comprehensive decision-making basis from multiple angles for the model, and improving the accuracy and efficiency of fault diagnosis.

[0029] In some embodiments, the controller uses a language processing model to analyze the prompt word information and output response information, and is configured to: identify the data type contained in the prompt word information; if the prompt word information includes video data, split the video data into images and audio by frame, so that the prompt word information includes at least one type of data among text type, image type and audio type; encode the text data in the prompt word information to obtain a semantic vector; encode the audio data in the prompt word information to obtain an audio vector, and align the audio vector to the same dimension as the semantic vector; encode the image data in the prompt word information to obtain an image vector, and align the image vector to the same dimension as the semantic vector; splice the semantic vector, the aligned audio vector and the image vector into a first vector, process the first vector using a large language model, and then output the response information. The technical solution provided by this embodiment has the following beneficial effects: for different types of data filled in the prompt word information, such as image data, text data, audio data, video data, etc., where video data is usually audio and video, the video can be split into image and audio, so that the prompt word contains at least one type of data of image, text and audio. By vectorizing different data, it is convenient for large language model operation and analysis, so as to be compatible with the processing of multimodal data, realize comprehensive decision-making and analysis of fault causes from multiple angles, and recommend feasible fault repair methods, thereby improving the reliability of model output, improving the efficiency and accuracy of fault diagnosis and repair, and helping to reduce after-sales costs and improve the quality of human-computer dialogue between users and customer service.

[0030] In some embodiments, before sending the response information to the terminal device through the communicator, the controller is also configured to: obtain the fault confidence calculated by the language processing model, the fault confidence is used to measure the proximity between the target fault predicted by the language processing model and the actual fault of the target home appliance; if the fault confidence is greater than a preset threshold, query the automatic repair capability of the target home appliance for the target fault; if the target home appliance has the automatic repair capability for the target fault, send an automatic repair instruction to the target home appliance through the communicator, the automatic repair instruction is used to instruct the target home appliance to execute the repair program associated with the target fault. The technical solution provided by this embodiment has the following beneficial effects: the language model can predict the target fault generated by the target home appliance and calculate the confidence of the occurrence of the target fault. If the fault confidence is greater than the preset threshold, it indicates that the accuracy of the model fault prediction is high, and if the target home appliance has the ability to automatically repair the target fault, the server controls the target home appliance to automatically repair the target fault without the need for manual repair by the user or manual on-site repair, thereby improving the fault repair efficiency and reducing after-sales costs.

[0031] In some embodiments, before sending an automatic repair instruction to the target home appliance through the communicator, the controller is configured to: send a permission request to the terminal device through the communicator, and the permission request is used to instruct the terminal device to ask the user whether to agree that the target home appliance automatically repairs the target fault; if an indication of agreement to repair is received from the terminal device through the communicator, the automatic repair instruction is sent to the target home appliance through the communicator; if an indication of refusal to repair is received from the terminal device through the communicator, the automatic repair instruction is not sent to the target home appliance through the communicator. The technical solution provided by this embodiment has the following beneficial effects: in some cases, the user may not want the target home appliance to automatically repair the fault, for example, when the cause of the fault is inaccurate, an erroneous repair may occur, or the user expects to manually repair the fault to learn and accumulate experience, or the user prefers manual on-site repair. Therefore, before issuing an automatic repair instruction, the server can send a permission request to the terminal device to obtain the automatic repair permission for the target home appliance. If the user refuses to authorize, the server will not send the automatic repair instruction. If the user agrees to authorize the server, the automatic repair instruction will be sent to the target home appliance, thereby avoiding the target home appliance from repairing the target fault without permission, thereby taking into account the safety of home appliances and user preferences and needs.

[0032] In a third aspect, some embodiments of the present application provide a method for diagnosing a fault of a household appliance, comprising:

[0033] The terminal device obtains the fault description data input by the user based on the human-computer dialogue page, wherein the fault description data is used to describe the fault details of the target home appliance;

[0034] The terminal device generates a first query instruction and sends the first query instruction to the server; the first query instruction is used to instruct the server to query the target knowledge from the knowledge base according to the fault description data; wherein the knowledge base is generated by the server according to a fault tree structured file, the fault tree structured file includes at least one fault tree associated with the target home appliance, the fault tree includes a plurality of nodes, and the nodes are configured with a fault manifestation image, a fault log, a repair method and a fault phenomenon corresponding to a fault type;

[0035] The terminal device receives the target knowledge sent by the server, generates prompt word information according to the fault description data and the target knowledge, and sends the prompt word information to the server;

[0036] The terminal device receives the response information sent by the server, and displays the response content corresponding to the response information on the human-computer dialogue page; wherein, the response information is output by the server after analyzing the prompt word information using a language processing model, and the response information includes the fault type and recommended repair method of the target home appliance.

[0037] The technical solution provided by the third aspect embodiment has the following beneficial effects: the user can conduct a human-computer dialogue with the customer service of the device brand, and enter the fault description data on the human-computer dialogue page to describe the fault details of the target home appliance, and provide a basis for the server to analyze the cause of the fault; the server configures at least one fault tree for the home appliance, and a node of the fault tree corresponds to a fault type, that is, the fault tree can characterize the fault classification, and the node is configured with a fault manifestation image, a fault log, a repair method and a fault phenomenon, etc., so that the fault tree can provide knowledge support for equipment fault diagnosis, so that the subsequent language processing model can accurately identify the cause of the fault and provide a feasible fault repair method; the end After the terminal device sends the first query instruction, the server can query the knowledge base according to the fault description data, and find similar or matching fault performance images, fault phenomena and repair methods from the knowledge base according to the fault details described by the user, so as to serve as reference knowledge for model decision-making; the terminal generates prompt word information based on the target knowledge and fault description data, and the language processing model on the server side analyzes the prompt word and outputs reliable response information, so that customer service can provide more accurate fault causes and recommended repair methods, helping users to solve equipment failures by themselves, thereby improving the fault diagnosis accuracy and fault repair efficiency of home appliances, reducing after-sales costs, and improving the quality of human-computer dialogue.

[0038] In a fourth aspect, some embodiments of the present application provide a method for diagnosing a fault of a household appliance, comprising:

[0039] The server receives a first query instruction sent by the terminal device, wherein the first query instruction includes fault description data input by the user based on a human-computer dialogue page displayed by the terminal device, and the fault description data is used to describe the fault details of the target home appliance;

[0040] The server, in response to the first query instruction, queries the target knowledge in the knowledge base according to the fault description data, and sends the target knowledge to the terminal device; wherein the knowledge base is generated by the server according to a fault tree structured file, the fault tree structured file includes at least one fault tree associated with the target household appliance, the fault tree includes a plurality of nodes, and the nodes are configured with a fault manifestation image, a fault log, a repair method, and a fault phenomenon corresponding to a fault type;

[0041] The server receives prompt word information generated and sent by the terminal device according to the fault description data and the target knowledge, analyzes the prompt word information using a language processing model and outputs response information, wherein the response information includes the fault type and recommended repair method of the target home appliance;

[0042] The server sends the response information to the terminal device.

[0043] The technical solution provided by the fourth aspect embodiment has the following beneficial effects: the user can conduct a human-computer dialogue with the customer service of the device brand, and enter the fault description data on the human-computer dialogue page to describe the fault details of the target home appliance, and provide a basis for the server to analyze the cause of the fault; the server configures at least one fault tree for the home appliance, and a node of the fault tree corresponds to a fault type, that is, the fault tree can characterize the fault classification, and the node is configured with a fault manifestation image, a fault log, a repair method and a fault phenomenon, etc., so that the fault tree can provide knowledge support for equipment fault diagnosis, so that the subsequent language processing model can accurately identify the cause of the fault and provide a feasible fault repair method; the end After the terminal device sends the first query instruction, the server can query the knowledge base according to the fault description data, and find similar or matching fault performance images, fault phenomena and repair methods from the knowledge base according to the fault details described by the user, so as to serve as reference knowledge for model decision-making; the terminal generates prompt word information based on the target knowledge and fault description data, and the language processing model on the server side analyzes the prompt word and outputs reliable response information, so that customer service can provide more accurate fault causes and recommended repair methods, helping users to solve equipment failures by themselves, thereby improving the fault diagnosis accuracy and fault repair efficiency of home appliances, reducing after-sales costs, and improving the quality of human-computer dialogue. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate some embodiments of the present application or technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 Schematic diagram of household appliance fault diagnosis scenarios provided in some embodiments of the present application Figure 1 ;

[0046] Figure 2 A schematic diagram of a service center page displayed on a terminal device provided in some embodiments of the present application;

[0047] Figure 3 A schematic diagram of a maintenance reservation page displayed on a terminal device provided in some embodiments of the present application;

[0048] Figure 4 Schematic diagram of a human-computer dialogue page displayed on a terminal device provided in some embodiments of the present application Figure 1 ;

[0049] Figure 5 Schematic diagram of household appliance fault diagnosis scenarios provided in some embodiments of the present application Figure 2 ;

[0050] Figure 6 Online fault diagnosis logic architecture provided for some embodiments of the present application Figure 1 ;

[0051] Figure 7 Schematic diagram of a human-computer dialogue page displayed on a terminal device provided in some embodiments of the present application Figure 2 ;

[0052] Figure 8 A schematic diagram of a processing framework of a language processing model provided in some embodiments of the present application;

[0053] Fig. 9 Schematic diagram of a human-computer dialogue page displayed on a terminal device provided in some embodiments of the present application Figure 3 ;

[0054] Fig.10 Online fault diagnosis logic architecture provided for some embodiments of the present application Figure 2 ;

[0055] Fig.11 A schematic diagram of a device self-check page displayed by a terminal device provided in some embodiments of the present application;

[0056] Fig.12 Online fault diagnosis logic architecture provided for some embodiments of the present application Figure 3 ;

[0057] Fig.13 A schematic diagram of a structural configuration of a fault tree provided in some embodiments of the present application;

[0058] Fig.14 Online fault diagnosis logic architecture provided for some embodiments of the present application Figure 4 ;

[0059] Fig.15 Schematic diagram of a terminal device displaying a human-computer dialogue page provided in some embodiments of the present application Figure 4 ;

[0060] Fig.16 Schematic diagram of a terminal device displaying a human-computer dialogue page provided in some embodiments of the present application Figure 5 ;

[0061] Fig.17 Schematic diagram of a terminal device displaying a human-computer dialogue page provided in some embodiments of the present application Figure 6 ;

[0062] Fig.18 A timing interaction diagram for household appliance fault diagnosis provided in some embodiments of the present application. DETAILED DESCRIPTION

[0063] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0064] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0065] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0066] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0067] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0068] Figure 1 Schematic diagram of household appliance fault diagnosis scenarios provided in some embodiments of the present application Figure 1 .like Figure 1 As shown, the usage scenario involves at least one household appliance 100 equipped and used in the home. The category of the household appliance 100 is not limited, for example, including: smart TV, smart air conditioner, smart refrigerator, smart washing machine, smart water heater, smart kitchen appliance, etc.

[0069] like Figure 1As shown, the use scenario may also involve a terminal device 200. The terminal device 200 is an electronic product used by the user, such as a smart phone, a tablet computer, a smart wearable device, a computer device, etc. The terminal device 200 may be installed with apps such as a browser and a smart home to enable the user to bind and control home appliances in the smart home, and to perform pre-sales communication, after-sales communication, and other functions through the customer service portal provided by the App.

[0070] like Figure 1 As shown, the usage scenario may also involve a server 300. The server 300 may be connected to the terminal device 200 through a local area network (LAN), a wireless local area network (WLAN) or other networks, so that the server 300 may receive a service request sent by the smart home app of the terminal device 200, and feedback service data to the smart home app, thereby realizing the service functions of the smart home app.

[0071] Figure 2 A schematic diagram of a service center page displayed on a terminal device provided in some embodiments of the present application. Figure 2 (a) of the present invention, taking a smart home app (such as "Smart Home") as an example, a user can click on the application icon of the smart home app to start the operation of the smart home app. After the smart home app is started, refer to Figure 2 (b), the terminal device 200 can display the application homepage of the smart home App, and the application homepage includes a service button 21. In response to the user clicking the service button 21, the terminal device 200 Figure 2 (c), controlling the display to display the service center page 22.

[0072] In some embodiments, continue to refer to Figure 2 (c), the service center page 22 may include a maintenance reservation button 22a. If a household appliance A (such as a smart TV) fails, the user may click the maintenance reservation button 22a.

[0073] Figure 3 Schematic diagram of a maintenance reservation page displayed by a terminal device provided in some embodiments of the present application. In response to the user clicking the maintenance reservation button 22a, the terminal device 200 displays the maintenance reservation page. Figure 3 , the control display shows the maintenance reservation page 30.

[0074] In some embodiments, continue to refer to Figure 3 The maintenance reservation page 30 includes a button 31 for selecting orders from the mall and a button 32 for manually selecting. The button 31 for selecting orders from the mall allows the user to select the home appliance A to be repaired from the historical purchase orders of the brand mall. The button 32 for manually selecting is used to allow the user to manually select the category and model of the home appliance A to be repaired.

[0075] In some embodiments, continue to refer to Figure 3 The maintenance reservation page 30 also includes a visit time setting control 33. The visit time setting control 33 is used to allow the user to set the time (referred to as visit time) when the brand party is expected to visit the faulty device for repair. The visit time may include a date and a time period of the day (e.g., 09:00-11:00).

[0076] In some embodiments, continue to refer to Figure 3 The maintenance reservation page 30 also includes a service address setting control 34. The service address setting control 34 is used to allow the user to set the address where the faulty device is located.

[0077] In some embodiments, continue to refer to Figure 3 The maintenance reservation page 30 further includes a remark box 35. The remark box 35 is used for the user to input the fault description information of the home appliance A, for example, describing the fault symptoms of the home appliance A. The fault description information may be optional content, but not mandatory content.

[0078] In some embodiments, continue to refer to Figure 3 The maintenance reservation page 30 also includes a submit button 36. After the user selects the product to be repaired, fills in the on-site time, service address and other information, the user can click the submit button 36. In response to the user clicking the submit button 36, the terminal device 200 sends a maintenance reservation request to the server 300. In response to the maintenance reservation request, the server 300 generates a maintenance work order and dispatches the maintenance work order to a maintenance personnel. In this way, the maintenance personnel can repair the faulty equipment at the time and address reserved by the user.

[0079] In some embodiments, continue to refer to Figure 2 (c), the service center page 22 may also include a telephone customer service button 22b. In response to the user clicking the telephone customer service button 22b, the terminal device 200 may control the display to display information such as a customer service hotline number and customer service hours. In this way, the user may call the customer service hotline number and report the faulty device by telephone.

[0080] In some embodiments, continue to refer to Figure 2 (c), the service center page 22 may also include an online customer service button 22c.

[0081] Figure 4 Schematic diagram of an online human-computer dialogue page displayed by a terminal device provided in some embodiments of the present application Figure 1 The terminal device 200 responds to the user clicking the online customer service button 22c, see Figure 4 , the display can be controlled to display the human-computer dialogue page 40.

[0082] In some embodiments, see Figure 4 The human-machine dialogue page 40 is a page for the user to conduct a human-machine dialogue with the online customer service (or manual customer service). The online customer service (or manual customer service) solves the problem for the user through the question-and-answer mode. For example, the online customer service asks "I am your intelligent customer service xx, how can I help you?", the user answers "My TV is broken, I want a warranty", the online customer service further asks a guiding question "Can you describe the fault performance of the TV product", the user answers "The TV is black screen", the online customer service asks "OK, please enter the time and address for the door-to-door maintenance appointment", the user enters "The door-to-door time is..., the address is...". In this way, the online customer service on the server 300 side can generate a maintenance work order and dispatch the maintenance work order to a maintenance personnel.

[0083] Through the above examples, users can make an appointment with the brand to repair the faulty equipment by making an appointment for repair, calling customer service or online customer service. Since users cannot accurately describe the fault situation, they can only describe the superficial fault symptoms of the home appliance, such as "black screen of TV", "no sound of TV", "air conditioner does not cool", etc. Customer service may lack professionalism and experience, resulting in the brand being unable to accurately analyze and locate the cause of the equipment failure. In this way, a device failure may be accompanied by multiple customer service communications, or even multiple on-site repairs, resulting in low efficiency in fault diagnosis and repair of home appliances, affecting users' use of home appliances, and increasing the brand's after-sales costs. From both the user's perspective and the brand's perspective, it is expected to improve the accuracy, efficiency and intelligence of fault diagnosis and repair of home appliances.

[0084] Figure 5 Schematic diagram of household appliance fault diagnosis scenarios provided in some embodiments of the present application Figure 2 . See Figure 5 , the interaction scenario involves a home appliance 100, a terminal device 200 and a server 300.

[0085] In some embodiments, the home appliance 100 may be configured with a first communicator, which is used to enable the home appliance 100 to communicate with the server 300 through a local area network (LAN), a wireless local area network (WLAN) or other networks. The terminal device 200 may be configured with a second communicator, which is used to enable the terminal device 200 to communicate with the server 300 through a local area network (LAN), a wireless local area network (WLAN) or other networks. In this way, the server 300 is used as a communication medium to achieve network communication between the home appliance 100 and the terminal device 200.

[0086] In some embodiments, if at least one of the home appliance 100 and the terminal device 200 is not connected to the network, the home appliance 100 and the terminal device 200 can perform end-to-end interaction through other communication protocols. Exemplarily, the first communicator includes a first Bluetooth transmission protocol module, and the second communicator includes a second Bluetooth transmission protocol module, so that the home appliance 100 can establish a Bluetooth connection with the terminal device 200.

[0087] In some embodiments, the application layer of the terminal device 200 may run multiple applications. The applications include but are not limited to: system applications, browsers, fault diagnosis applications, etc. Among them, the fault diagnosis application may be configured with at least one of the following functions: a multi-round dialogue function between the user and the customer service, a home appliance fault diagnosis function, a fault repair guidance function, and other functions.

[0088] In some embodiments, the smart home app can be configured as a fault diagnosis application. Figure 2 For example, a smart home app can provide an online customer service function. Through at least one round of dialogue between the user and the online customer service, the smart home app can analyze the cause of the failure of the home appliance based on the fault description data entered by the user, and provide the user with guidance and suggestions for repairing the fault. In this way, the user can repair the faulty device by himself according to the guidance and suggestions given by the customer service, improve the efficiency of fault diagnosis and repair, reduce the number of customer service communications and door-to-door repairs, and reduce after-sales costs.

[0089] In some embodiments, the terminal device 200 may also be configured with a display. The display is used to display a user interface, including an application page when the fault diagnosis application (or smart home App) is running.

[0090] In some embodiments, the terminal device 200 may also be configured with a controller. The controller may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor. The controller controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller may implement the home appliance fault diagnosis scheme shown in some embodiments of the present application by running relevant applications (such as fault diagnosis applications, smart home apps), and communicating and interacting with the home appliance 100 and the server 300.

[0091] Figure 6 Online fault diagnosis logic architecture provided for some embodiments of the present application Figure 1 .

[0092] In some embodiments, see Figure 6The terminal device 200 includes a front-end display layer, which can display relevant content through a display. The front-end display layer is used to display the application page of the fault diagnosis application (such as a browser, smart home app, etc.) when it is running through the display, and the application page includes a human-computer dialogue page between the user and the online customer service.

[0093] Figure 7 Schematic diagram of a human-computer dialogue page displayed on a terminal device provided in some embodiments of the present application Figure 2 . See Figure 7 The user can input fault description data through the human-machine dialogue page 40. The fault description data is used to describe the fault details of the target home appliance.

[0094] In some embodiments, see Figure 7 (a), the human-machine dialogue page 40 includes a text input box 41. The text input box 41 is used to allow the user to input fault description information in text format, such as Figure 7 (b) adds text content input and sent by the user to the human-computer dialogue page 40, and an example of the text content is "My TV model xx has a black screen."

[0095] In some embodiments, see Figure 7 (a), the human-computer dialogue page 40 includes an expansion button 42, and the terminal device 200 controls the display to display at least one of a photo button 43, a shooting button 44, a voice input button 45 and an upload document button 46 in response to the user clicking the expansion button 42, thereby enabling the user to input fault description data in a non-text format.

[0096] In some embodiments, see Figure 7 (a), in response to the user clicking the expand button 42, the terminal device 200 controls the display to display the photo button 43, which is used to allow the user to upload a local image. When a target home appliance fails, the user can take a device failure image and store the device failure image locally. In response to the user clicking the photo button 43, the terminal device 200 can control the display to display a preview list, which includes preview images (or thumbnails) of images included in the local album (or local gallery). After the user selects and sends the previously taken device failure image, as shown in FIG. Figure 7 As shown in (b), the human-machine dialogue page 40 adds a preview image (or thumbnail) of the device fault image, for example, presenting an image effect of a black TV screen. The user can click on the preview image (or thumbnail) of the device fault image to view the enlarged image or original image of the device fault image.

[0097] In some embodiments, see Figure 7(a), the terminal device 200 controls the display to display the shooting button 44 in response to the user clicking the expansion button 42, and the shooting button 44 is used to allow the user to instantly shoot the device failure image or device failure video. The terminal device 200 can control the display to display the shooting preview page in response to the user clicking the shooting button 44. The user can input and send the device failure image or device failure video by clicking or long pressing the shooting button in the shooting preview page, so that the preview image of the device failure image or device failure video can be added to the human-computer dialogue page 40. Among them, a play button can be set in the preview image of the device failure video, and the terminal device 200 controls the display to play the device failure video in response to the user clicking the play button.

[0098] In some embodiments, see Figure 7 (a), in response to the user clicking the expand button 42, the terminal device 200 controls the display to display the voice input button 45, and the voice input button 45 is used to allow the user to input the device fault description audio. In response to the user clicking the voice input button 45, the terminal device 200 can control the display to display the voice control so that the user can input the device fault description audio by long pressing the voice control. The terminal device 200 can also be configured with a sound collector (such as a microphone), and in response to the user long pressing the voice control, the sound collector is controlled to collect the user's voice, and in response to the user releasing the voice control, the sound collector is controlled to stop collecting the user's voice, thereby obtaining the device fault description audio.

[0099] In some embodiments, after the user inputs and sends the device fault description audio, such as Figure 7 As shown in (b), the human-computer dialogue page 40 can add an audio playback control corresponding to the device fault description audio, and the audio playback control displays the audio duration (e.g., 20″). The terminal device 200 is also configured with an audio output device (e.g., a speaker or other external amplifier, such as a stereo, etc.), and in response to the user clicking the audio playback control, the audio output device is controlled to play the device fault description audio.

[0100] In some embodiments, see Figure 7(a), in response to the user clicking the expand button 42, the terminal device 200 controls the display to display the upload document button 46, which is used to allow the user to input a device fault description document, and the document categories include but are not limited to Word, PDF, TXT, etc. In response to the user clicking the upload document button 46, the terminal device 200 can control the display to display a local file list. The user can select and send the device fault description document from the local file list, so that a preview image of the device fault description document can be added to the human-computer dialogue page 40. The user can open and view the device fault description document by clicking the preview image of the device fault description document.

[0101] In some embodiments, the user can input fault description data in different input modes by clicking at least one of the text input box 41, the photo button 43, the shooting button 44, the voice input button 45, and the upload document button 46. That is, the fault description data includes at least one of the following: fault description information (plain text), equipment fault image, equipment fault video, equipment fault description audio, and equipment fault description document. In this way, the fault description data input by the user based on the human-computer dialogue page 40 can have multimodal characteristics. "Multimodal" means that the fault description data has multiple media types (or media formats), including but not limited to: text, documents, images, videos, and audio, etc. In this way, using fault description data with multimodal attributes, multi-angle diagnosis and analysis of equipment faults can be achieved, the accuracy of fault diagnosis can be improved, and the human-computer interaction experience can be enhanced.

[0102] In some embodiments, see Figure 6 The terminal device 200 includes an interaction layer, which is run and managed by a controller. The front-end display layer can pass the fault description data input by the user to the interaction layer, and the interaction layer performs fault diagnosis and fault repair guidance based on the fault description data based on communication interaction with the server 300.

[0103] In some embodiments, after receiving the fault description data transmitted by the front-end presentation layer, the interaction layer may pre-process the fault description data.

[0104] In some embodiments, the preprocessing of the fault description data by the interaction layer includes: processing the text data contained in the fault description data, for example, text cleaning, special symbol processing, traditional Chinese character conversion, pinyin error correction, etc.

[0105] In some embodiments, the preprocessing of the fault description data by the interaction layer also includes: parsing the non-text data contained in the fault description data. For the device failure image, the image format is retained. For the device failure description document and the device failure description audio, the document and the audio need to be converted into text. For the device failure video, the video frame and the audio can be separated, wherein the video frame retains the image format and the audio needs to be converted into text. The video frames and audio in the same device failure video have a mapping association, for example, the 1st to 10th frames of the device failure video are mapped to the 1st audio segment.

[0106] In some embodiments, see Figure 6 The server 300 may pre-build a knowledge base. The knowledge base is used to store fault diagnosis knowledge related to home appliances in the smart home. The fault diagnosis knowledge includes but is not limited to: fault classification, fault manifestation, fault image, fault repair method, etc.

[0107] In some embodiments, see Figure 6 The interaction layer of the terminal device 200 can call the knowledge base constructed by the server 300, and retrieve the fault diagnosis knowledge (hereinafter referred to as: target knowledge) similar to the fault description data from the knowledge base according to the preprocessed fault description data.

[0108] In some embodiments, the server 300 may pre-train a language processing model. The language processing model may have the general functions of an LLM (Large Language Model), such as reasoning and outputting machine response text during human-machine dialogue. The language processing model also has the ability to process multimodal data, and can encode and align text, images, and audio, so that multimodal data is converted and spliced ​​into a unified semantic vector of the same dimension, which facilitates LLM to reason and calculate the semantic vector, and finally output the machine's response information for the fault description data input by the user, which may include the cause of the fault of the target home appliance and the recommended repair method, etc.

[0109] In some embodiments, the knowledge base query includes two situations: situation one is that the target knowledge is found in the knowledge base, and situation two is that the target knowledge is not found in the knowledge base.

[0110] In some embodiments, for the above scenario 1, the interaction layer can fill the fault description data and target knowledge into the prompt word (prompt) according to the preset prompt word template. For the above scenario 2, the interaction layer can only fill the fault description data into the prompt word (prompt) according to the prompt word template.

[0111] Among them, the prompt can be a question, an instruction, a dialogue or other forms of content input. The prompt can be understood as an expression form of a data stream, which includes raw data of different types (such as text, images, audio, video, documents, etc.). The prompt can be used as the input and guide of the language processing model, so that the language processing model can decide the output response information based on the context and semantic information provided by the prompt. The prompt template is used to specify the filling and generation strategy of the prompt, and the prompt template can be set according to the language processing model and the user instruction requirement configuration.

[0112] In some embodiments, the interaction layer of the terminal device 200 can call the language processing model trained by the server 300 and input the prompt word (prompt) into the language processing model.

[0113] Figure 8 This is a schematic diagram of the processing framework of the language processing model provided in some embodiments of the present application. Figure 8 , the language processing model receives the prompt, parses the prompt, identifies the data type involved in the prompt, splits the video data into video frames and audio, and converts the document data into text, so that the data filled in the prompt is parsed into three types: text, image, and audio. In some embodiments, see Figure 8 ,The language processing model can use an image encoder (such as CLIP or SigLIP) to encode the image data obtained after parsing the prompt word, and vectorize the image data to obtain an image vector.

[0114] In some embodiments, see Figure 8 ,The language processing model can use an audio encoder to encode the audio data obtained after parsing the prompt word, and vectorize the audio data to obtain an audio vector.

[0115] In some embodiments, see Figure 8 , the language processing model can use embedding encoding to convert text data into semantic vectors after parsing the prompt words. Embedding encoding is a technology that maps high-dimensional data to low-dimensional space. It is usually used in natural language processing and machine learning to reduce the dimension of data and retain key information. Embedding encoding is optimized through neural network training, which can express the correlation between data and improve computing efficiency.

[0116] In some embodiments, see Figure 8The language processing model can align the image vector and the audio vector according to the alignment rule, so as to map the image vector and the audio vector to the same dimension as the semantic vector. The alignment rule includes but is not limited to: linear transformation, nonlinear transformation, resampling transformation, etc.

[0117] In some embodiments, see Figure 8 The language processing model can concatenate the semantic vector, the aligned image vector and the audio vector to obtain a unified vector corresponding to the prompt word (hereinafter referred to as the first vector). The language processing model inputs the first vector to the LLM, which analyzes and infers the first vector and finally outputs the machine's response information for the fault description data input by the user, and feeds the response information back to the interaction layer of the terminal device 200.

[0118] In some embodiments, see Figure 6 The interactive layer of the terminal device 200 can determine the response content displayed by the front-end display layer on the human-machine dialogue page according to the response information output by the language processing model. For example, the response content may include text information for indicating the cause of the fault, step-by-step instructions and reference images for guiding the user to repair the fault, etc. The interactive layer can encapsulate the response content and pass the encapsulation result to the front-end display layer. The front-end display layer parses the encapsulation result and controls the display to display the corresponding response content on the customer service side (machine side) in the human-machine dialogue page 40, thereby completing a round of dialogue between the user and the customer service.

[0119] Fig. 9 Schematic diagram of a human-computer dialogue page displayed on a terminal device provided in some embodiments of the present application Figure 3 . See Fig. 9 In response to the fault description data input by the user, the front-end display layer displays the response content on the human-machine dialogue page 40 including the fault cause and fault repair guidance of the target home appliance.

[0120] In some embodiments, see Fig. 9 The fault repair guide can include a detailed process of repairing the fault (such as step one, step two and step three), and can also attach operating instructions and reference pictures to the steps. In this way, through the combination of pictures and texts, users can better understand how to repair, thereby guiding users to complete the fault repair by themselves, improving the efficiency of equipment fault diagnosis and repair, which is conducive to reducing the number of customer service communications and door-to-door repairs, and saving after-sales costs.

[0121] In some embodiments, the home appliance may have a self-checking capability. The home appliance may be configured with a self-checking module, which is used to execute a self-checking program. The self-checking program is configured to: troubleshoot the home appliance according to a self-checking item catalog preset in the home appliance, and obtain self-checking data.

[0122] In some embodiments, the self-check item catalog includes: fault classification of household electrical appliances, and each fault type is mapped to at least one check item.

[0123] In some embodiments, the self-test data includes, but is not limited to, fault codes, self-test operation logs, etc. Among them, the fault code is used to uniquely indicate the fault type, and different fault types correspond to different fault codes. The self-test operation log is a log generated by the self-test module when executing the self-test program. The self-test operation log can indicate the fault status of the inspection items included in the self-test item directory, and the fault status is used to indicate whether the fault corresponding to the inspection item has occurred.

[0124] Fig.10 Online fault diagnosis logic architecture provided for some embodiments of the present application Figure 2 See also Fig.10 , the user can input a self-test instruction on the home appliance side. The home appliance responds to the self-test instruction and controls the self-test module to execute the self-test program and obtain self-test data.

[0125] In some embodiments, the home appliance can be configured with M (M>1) local buttons. The user can enter the self-test instruction by performing a preset end-side combination key operation on N (N≤M) of the local buttons, where M and N are positive integers. For example, a smart TV can be configured with a power button, a volume + button, a volume - button, a channel + button, a channel - button, etc. An example of a preset end-side combination key operation is pressing the power button and the channel + button at the same time, so that the user can press the power button and the channel + button at the same time to enter the self-test instruction on the home appliance side.

[0126] In some embodiments, if the home appliance supports a voice recognition function, the user can input a self-test instruction by voice.

[0127] In some embodiments, if the home appliance supports a gesture recognition function, the user can input a self-check instruction by making a preset gesture action. The way in which the user inputs the self-check instruction on the home appliance side is not limited to the above embodiments.

[0128] In some embodiments, see Fig.10 , the server can configure the self-test service. After the self-test module completes the self-test procedure, the home appliance can upload the self-test data to the self-test service of the server, and the self-test service sends the self-test data to the interaction layer of the terminal device.

[0129] In some embodiments, see Fig.10 , the user can also input a self-test instruction on the terminal device side. After receiving the self-test instruction, the terminal device can send the self-test instruction to the self-test service of the server, and the self-test service forwards the self-test instruction to the home appliance to control the self-test module of the home appliance to execute the self-test program. Fig.11A schematic diagram of a device self-checking page displayed on a terminal device provided in some embodiments of the present application. Fig.11 (a), the service center page 22 may also include a device self-check button 22d. The terminal device responds to the user clicking the device self-check button 22d, such as Fig.11 As shown in (b) or (c), the display is controlled to display the device self-test page.

[0130] In some embodiments, see Fig.11 (b) The device self-test page may include a button for selecting an order from the mall and a button for selecting a manual order, so that the user can select the target home appliance device for self-test in different ways.

[0131] In some embodiments, the terminal device may store a device self-check capability list, which is used to record whether different types (or models) of home appliances have self-check capabilities. After the user selects a target home appliance through the device self-check page, the terminal device may query whether the target home appliance has self-check capabilities according to the device self-check capability list.

[0132] In some embodiments, Fig.11 As shown in (b), for example, if the target home appliance selected by the user is a smart washing machine, the terminal device queries the device self-checking capability list and determines that the smart washing machine does not have the self-checking capability, then controls the display to display the first prompt message 110 on the device self-checking page. The first prompt message 110 is used to prompt the user that the target home appliance selected by the user does not have the self-checking capability. Fig.11 The first prompt message 110 in (b) is exemplified as "The smart washing machine you selected does not support self-test."

[0133] In some embodiments, Fig.11 As shown in (c), for example, if the target home appliance selected by the user is a smart TV, the terminal device queries the device self-test capability list and determines that the smart TV has the self-test capability, then controls the display to display a start self-test button 120 on the device self-test page, and the start self-test button 120 is used to allow the user to input a self-test instruction on the terminal device side. In response to the user clicking the start self-test button 120, the terminal device sends the self-test instruction to the self-test service of the server, and the self-test service forwards the self-test instruction to the target home appliance to control the self-test module of the target home appliance to execute the self-test program.

[0134] In some embodiments, for a scenario where at least one of the terminal device and the home appliance is not connected to the network, the terminal device and the home appliance can perform an end-to-end target communication connection (e.g., Bluetooth connection), and perform data (or instructions) interaction between the devices based on the target communication connection, thereby providing the home appliance with an upload link for self-test data when the network is disconnected. In this way, the terminal device can send the self-test instruction to the target home appliance based on the target communication connection, and the home appliance can send the self-test data to the terminal device.

[0135] In some embodiments, if the user does not input a self-test instruction on either the terminal device or the home appliance, Figure 6 The fault diagnosis logic architecture of the example is processed and the knowledge base is queried only based on the preprocessed fault description data.

[0136] In some embodiments, if the user enters a self-test command on the terminal device side or the home appliance side, see Fig.10 The interactive layer of the terminal device receives the self-test data, including but not limited to the self-test fault code and the self-test operation log. The interactive layer of the terminal device can splice the self-test data and the pre-processed fault description data, and perform a knowledge base query based on the spliced ​​data (referred to as: first data).

[0137] In some embodiments, the interaction layer can perform structured splicing on the self-test data and the pre-processed fault description data. For example, the interaction layer can perform data splicing according to the structure of {fault description information (text): xxx, device fault image: xxx, device fault description audio: xxx, device fault video: xxx, self-test fault code: xxx, self-test operation log: xxx} to obtain the first data.

[0138] Fig.12 Online fault diagnosis logic architecture provided for some embodiments of the present application Figure 3 See also Fig.12 The server may include a database management layer. The database management layer is used to construct a knowledge base according to the fault tree structured file, and the knowledge base includes: an ES database (text search database) and a VS database (vector search database).

[0139] In some embodiments, see Fig.12 The server may further include a service layer. The service layer is configured with a self-checking service and a language processing model based on multimodal characteristics, and is also configured with a knowledge base query service. The knowledge base query service provides the interactive layer of the terminal device with a function of performing a knowledge base query based on the first data (or fault description data).

[0140] In some embodiments, see Fig.12, the knowledge base query service is used to search for K1 query results similar to the text data in the first data from the ES database (abbreviated as: TOP_K1 results recalled by ES), and to search for K2 query results similar to the non-text data in the first data from the VS database (abbreviated as: TOP_K2 results recalled by VS). Among them, K1 represents the first preset number, which is used to limit the number of query results of text knowledge in the ES database (i.e., the ES recall number); K2 represents the second preset number, which is used to limit the number of query results of non-text knowledge in the VS database (i.e., the VS recall number). K1 and K2 can be equal or unequal.

[0141] In some embodiments, the fault tree structured file may include device fault tree configuration files configured by the server for various home appliances in a smart home. The device fault tree configuration file maps only one type of home appliance, and the device fault tree configuration file configures at least one fault tree, which is presented as a tree structure with distributed parent-child nodes.

[0142] In some embodiments, the fault tree structured file can be configured into different structured formats such as json, csv, yaml, etc.

[0143] Fig.13 A schematic diagram of a fault tree structure configuration provided by some embodiments of the present application. Fig.13 The fault tree includes a parent node at the top, which is the main event of the fault tree and is used to characterize a major fault category of home appliances, such as display fault, audio output fault, etc. The parent node includes at least one child node, which is a sub-event (or branch event) of the fault tree and is a minor fault category covered under a major fault category.

[0144] In some embodiments, the parent node can be configured with a general fault that cannot specifically identify the cause of the fault. Taking the black screen fault as an example, there may be many reasons for the display black screen, such as cable connection problems (the power cord is not connected or the connection is loose), no signal source access, backlight failure, motherboard failure, etc. Therefore, the black screen fault can be configured as a large fault category corresponding to the parent node of the fault tree.

[0145] In some embodiments, taking home appliance A as an example, see Fig.13 , the fault tree includes a parent node (black screen fault), the parent node includes child node 1 (branch 1) and child node 2 (branch 2), wherein child node 1 includes child node 1.1 (branch 1.1) and child node 1.2 (branch 1.2), and a total of 5 nodes.

[0146] In some embodiments, see Fig.13 Any node in the fault tree can be configured with but not limited to the following attributes:

[0147] (1) At least one fault manifestation image. The fault manifestation image is used to present the symptoms and appearances corresponding to the fault type at the node.

[0148] (2) Fault log: The fault log is the key log information corresponding to the fault type at the node.

[0149] (3) Fault phenomenon description information, which is a description of the fault type at the node.

[0150] (4) Maintenance method, which is the recommended repair method corresponding to the fault type at the node. The maintenance method may include operating instructions and guidance pictures for the maintenance steps.

[0151] In some embodiments, the attribute content of any node configuration in the fault tree can be collected through different ways.

[0152] In some embodiments, the fault tree can be configured by implementing fault engineering simulation. Testers can obtain fault logs, phenomena, and performance images and provide recommended maintenance methods by simulating different types of faults.

[0153] In some embodiments, the data basis for configuring the fault tree is obtained by collecting historical conversation data between the user and after-sales customer service online and collecting real fault and maintenance records of the user's equipment through door-to-door maintenance.

[0154] In some embodiments, the database management layer can parse the fault tree structured file, and the parsing process includes: obtaining a fault tree configuration file that maps different household appliance information (including but not limited to device categories, device models, etc.), reading the fault tree contained in the fault tree configuration file, reading the parent-child node distribution of the fault tree and the attribute content of each node configuration, thereby obtaining text data (such as fault logs, fault phenomena, etc.) and non-text data (such as fault manifestation images, guidance images of maintenance steps, etc.) in the fault tree structured file.

[0155] In some embodiments, the database management layer can write the text data in the fault tree structured file into the ES database and create an index to complete the text index storage. The ES database has the characteristics of strong real-time performance and can efficiently retrieve and recall large-scale text data (ES recall). Compared with the semantic similarity retrieval after vectorization, ES recall provides text retrieval based on literal similarity.

[0156] In some embodiments, the database management layer can call the language processing model to embed the non-text data in the fault tree structured file, align it into a semantic vector of uniform dimension, and write the converted semantic vector into the VS database to complete the vector storage. In this way, the VS database can be used for retrieval and query of non-text knowledge (VS recall). The database management layer can use faiss or milvus services to build and deploy the VS database.

[0157] In some embodiments, the storage method of text and vector includes node storage. Node storage refers to the storage of text data and non-text data configured for individual nodes using nodes in the fault tree as units. Fig.13 For example, the database management layer can independently store the parent node, child node 1, child node 2, child node 1.1, and child node 1.2 in the ES database and the VS database. In this way, when the equipment fault type involves only one node, the knowledge in the database can be quickly and accurately located and queried.

[0158] In some embodiments, the storage method of text and vector also includes path storage. Path storage refers to taking the path in the fault tree as a unit, and storing the text data and non-text data configured in all nodes (the number of nodes is greater than 1) contained in the path. Fig.13 The fault tree of the example includes the following 6 paths: ① Black screen fault-branch 1, ② Black screen fault-branch 2, ③ Black screen fault-branch 1-branch 1.1, ④ Black screen fault-branch 1-branch 1.2, ⑤ Branch 1-branch 1.1, ⑥ Branch 1-branch 1.2. Path storage can enhance the anti-interference ability of the knowledge base in complex environments (such as incomplete log input, unclear user fault description, etc.), and realize ES recall and VS recall of any entire path for comprehensive analysis by the language processing model, thereby improving the robustness of the system.

[0159] In some embodiments, the interaction layer can generate a first query instruction based on the first data spliced ​​from the fault description data and the self-test data, and send the first query instruction to the server. In response to the first query instruction, the service layer of the server controls the knowledge base query service to execute ES recall and VS recall.

[0160] In some embodiments, see Fig.12, the knowledge base query service executes ES recall, including: obtaining the text data contained in the first data (referred to as: first text data), calculating and retrieving the similarity between the first text data (such as fault description information, self-test fault code, self-test operation log, etc.) and the text in the ES database, and outputting the TOP_K1 result of ES recall. The TOP_K1 result of ES recall is the text knowledge obtained from the query. The text similarity algorithm is not limited. For example, a similarity algorithm based on TF-IDF (Term Frequency-Inverse Document Frequency) can be used. For Figure 6 In the architecture, if the target home appliance does not execute the self-test procedure (i.e., there is no self-test data scenario), the knowledge base query service can perform ES recall based only on the text data in the preprocessed fault description data.

[0161] In some embodiments, see Fig.12 , the knowledge base query service executes VS recall, including: obtaining the non-text data contained in the first data (referred to as: the first non-text data), calling the language processing model to embed the first non-text data, and aligning it into a semantic vector on a unified dimension (referred to as: the first semantic vector); calculating the similarity between the first semantic vector and the in-library vector of the VS database, and outputting the TOP_K2 result of VS recall. The TOP_K2 result of VS recall is the non-text knowledge obtained from the query. The vector similarity algorithm is not limited. For example, similarity algorithms based on L1 norm (Manhattan distance), L2 norm (Euclidean distance), etc. can be used. For Figure 6 In the architecture, if the target home appliance does not execute the self-test procedure (i.e., there is no self-test data scenario), the knowledge base query service can perform VS recall based only on the non-text data in the pre-processed fault description data.

[0162] In some embodiments, the knowledge base query service can send the query result (ie, target knowledge) of the knowledge base to the interaction layer of the terminal device. The query result can include: at least one of the TOP_K1 result recalled by ES and the TOP_K2 result recalled by VS.

[0163] In some embodiments, see Fig.12 If the knowledge base query service does not obtain the query result, that is, the results of ES recall and VS recall are blank (no target knowledge), the service layer can fill in the first preset prompt word template according to the fault description data and self-test data to obtain the first prompt word, and input the first prompt word into the language processing model.

[0164] In some embodiments, see Fig.12If the knowledge base query service can obtain the query results, that is, at least one of the TOP_K1 results recalled by ES and the TOP_K2 results recalled by VS (with target knowledge), the service layer can fill in the second preset prompt word template according to the fault description data, self-test data and the query results of the knowledge base, obtain the second prompt word, and input the second prompt word into the language processing model.

[0165] By parsing the fault tree structured file, text knowledge is indexed and stored, and non-text knowledge is vectorized and stored in the database, and the ES database and VS database are constructed, so as to build a knowledge base architecture based on the fault tree system and with multimodal attributes. Through ES recall and VS recall, reliable knowledge support is provided for the language processing model. The knowledge base can quickly iterate and expand the knowledge content, which is conducive to reducing the hallucination problem of the language processing model, improving the accuracy of home appliance fault diagnosis and human-computer dialogue, and improving the efficiency of equipment fault diagnosis. Based on vertical domain knowledge and high-quality human-computer dialogue mechanism, the online fault diagnosis logic architecture is enhanced, which is conducive to the promotion of prior diagnosis of home appliance faults, the continuous accumulation of prior diagnosis experience, and the improvement of customer service communication skills.

[0166] Fig.14 Online fault diagnosis logic architecture provided for some embodiments of the present application Figure 4 See also Fig.14 The home appliance can be configured with a repair module, which is used to repair a first fault generated by the home appliance, and the first fault is a fault type that can be repaired by the repair module. For example, the repair module has the ability to automatically repair fault a, so when the home appliance generates fault a, the repair module can be controlled to automatically execute a repair program associated with fault a.

[0167] In some embodiments, see Fig.14 , the service layer of the server can also be configured with an automatic fault repair service. The automatic fault repair service can store a list of automatic repair capabilities of home appliances, which is used to record the correspondence between the fault type of home appliances and the automatic repair capabilities. For example, the automatic repair capability corresponding to fault a is "Y", which can indicate that the repair module of the home appliance has the ability to repair fault a. For another example, the automatic repair capability corresponding to fault b is "N", which can indicate that the repair module of the home appliance does not have the ability to repair fault b.

[0168] In some embodiments, see Fig.14, the language processing model can calculate the confidence level corresponding to the target fault type of the target home appliance (referred to as: fault confidence level). The fault confidence level is used to measure the closeness (or matching degree) between the target fault type predicted by the language processing model and the actual fault type of the target home appliance. The higher the fault confidence level, the closer the target fault type predicted by the model is to the actual fault type of the target home appliance, and the more reliable the prediction result of the measurement model is.

[0169] In some embodiments, see Fig.14 , the language processing model can pass the fault confidence and the predicted target fault type to the fault automatic repair service. The fault automatic repair service compares the fault confidence and the preset threshold. If the fault confidence is greater than the preset threshold, the automatic repair capability list of the target home appliance (referred to as the first automatic repair capability list) is obtained. The fault automatic repair service queries whether the target home appliance has the ability to automatically repair the target fault based on the first automatic repair capability list. The target fault is the fault corresponding to the target fault type predicted by the language processing model.

[0170] In some embodiments, see Fig.14 If the target home appliance has the ability to automatically repair the target fault, the fault automatic repair service can send a permission request to the terminal device, and the permission request is used to obtain permission from the user to allow the target home appliance to automatically repair the target fault.

[0171] In some embodiments, see Fig.14 After receiving the permission request, the interactive layer of the terminal device passes the permission request to the front-end display layer. According to the permission request, the front-end display layer can display the cause of the fault (target fault) through the human-computer dialogue page and ask the user whether he agrees that the target home appliance automatically repairs the target fault.

[0172] Fig.15 Schematic diagram of a terminal device displaying a human-computer dialogue page provided in some embodiments of the present application Figure 4 See also Fig.15 According to the permission request, the front-end display layer displays a permission window 47 on the customer service side of the human-machine dialogue page through the display. The permission window 47 prompts that the target home appliance (such as a TV) has a target fault and asks the user whether to agree that the target home appliance automatically repairs the target fault.

[0173] In some embodiments, see Fig.15 The permission window 47 further includes an approval button 47a and a rejection button 47b. The terminal device 200 responds to the user clicking the approval button 47a. Fig.14, the interactive layer may send a repair approval indication message to the fault automatic repair service of the server 300. The fault automatic repair service receives the repair approval indication message and sends an automatic repair instruction to the target home appliance. The target home appliance responds to the automatic repair instruction and controls the repair module to execute the repair program corresponding to the target fault.

[0174] In some embodiments, see Fig.14 When the repair module completes the repair of the target fault, the target home appliance sends a repair completion message to the fault automatic repair service of the server 300. The fault automatic repair service forwards the received repair completion message to the terminal device 200. After receiving the repair completion message, the interaction layer of the terminal device 200 forwards the repair completion message to the front-end display layer.

[0175] Fig.16 Schematic diagram of a terminal device displaying a human-computer dialogue page provided in some embodiments of the present application Figure 5 The front-end display layer receives the repair completion message, see Fig.16 , a repair prompt message 48 is displayed on the customer service side of the human-machine dialogue page 40 through a display, and the repair prompt message 48 is used to prompt that the target household appliance has repaired the target fault. For example, Fig.16 The repaired prompt information 48 of the example is presented as “The target fault of the TV has been repaired, please continue watching TV”.

[0176] In some embodiments, in response to the user clicking the reject button 47b, the terminal device 200 may send a rejection repair instruction message to the fault automatic repair service of the server 300 through the interaction layer. The fault automatic repair service receives the rejection repair instruction message and does not send an automatic repair instruction to the target home appliance. That is, the target home appliance does not control the repair module to execute the repair program for the target fault without obtaining the repair permission granted by the user.

[0177] Fig.17 Schematic diagram of a terminal device displaying a human-computer dialogue page provided in some embodiments of the present application Figure 6 The front-end display layer responds to the user clicking the reject button 47b, see Fig.17 , a fault repair prompt window 49 is displayed on the customer service side of the human-machine dialogue page 40 through a display, and the fault repair prompt window 49 provides the user with feasible fault repair methods. The fault repair methods include but are not limited to at least one of the following methods: self-repair by the user according to the guidance of the step flow (picture and text), remote repair, appointment for door-to-door repair (can be made by phone, customer service or the appointment repair button 22a on the service center page 22), multiple rounds of dialogue, etc.

[0178] In some embodiments, see Fig.17The fault repair prompt window 49 includes a first remote repair button 49a. The first remote repair button 49a is associated with the target home appliance device for online diagnosis. In this way, the user can quickly trigger the remote repair of the target home appliance device by clicking the first remote repair button 49a.

[0179] In some embodiments, see Fig.11 The service center page 22 may also include a second remote repair button 22e, which is a shared remote repair entry for multiple home appliances. The user can click the second remote repair button 22e to select a target home appliance to be repaired, and trigger remote fault diagnosis and repair of the target home appliance.

[0180] In some embodiments, the service layer of the server 300 may also be configured with a remote repair service. The terminal device 200 responds to the remote repair trigger operation of the target home appliance input by the user, and sends a remote repair request to the server 300. In response to the remote repair request, the remote repair service sends a remote repair instruction to the remote computer, and the remote repair instruction may include the fault information and device information of the target home appliance, and the device information includes but is not limited to the IP address, identification code, check code (or verification code), etc. of the target home appliance.

[0181] In some embodiments, the remote computer can establish a remote communication connection with the target home appliance according to the device information in response to the remote repair instruction, thereby enabling the maintenance personnel to remotely control the target home appliance, and the remote control includes fault testing, diagnosis and repair of the target home appliance. Remote repair can repair some software faults, but cannot repair hardware faults (for example, it is impossible to remotely replace the device hardware).

[0182] In some embodiments, the user can have multiple rounds of dialogue with the online customer service, and through multiple rounds of dialogue, more details and details related to the equipment failure can be input. In this way, the language processing model can output response information based on the context of multiple rounds of dialogue, so as to more accurately analyze the cause of the failure and give feasible repair measures, thereby improving the accuracy of the human-computer dialogue between the user and the online customer service.

[0183] In some embodiments, see Fig.12 The database management layer of the server also maintains a historical dialogue library, which is used to store historical dialogues between users and online customer service. The interactive layer of the terminal device 200 can send the question and answer content of each round of dialogue to the server 300. The server 300 can store the question and answer content of each round of dialogue in the historical dialogue library according to a preset storage structure (such as Mysql), so as to automatically store the online fault diagnosis data of the home appliance and realize the iterative update of the historical dialogue library.

[0184] In some embodiments, the server 300 can use the data in the historical dialogue library as training data to iteratively optimize the language processing model to enhance the performance of the language processing model, so as to enable the server 300 to continuously accumulate and enrich online fault diagnosis experience based on real equipment fault online diagnosis events, and improve the online fault diagnosis accuracy and dialogue interaction capabilities.

[0185] Fig.18 A timing interaction diagram for household appliance fault diagnosis provided in some embodiments of the present application. Fig.18 If the user triggers the quality inspection of the target home appliance on the terminal device side (or on the target home appliance side), the front-end display layer can send a self-inspection instruction to the self-inspection service, and the self-inspection service forwards the self-inspection instruction to the target home appliance. The target home appliance responds to the self-inspection instruction, controls the self-inspection module to execute the self-inspection program, and uploads the self-inspection data to the self-inspection service when the self-inspection is completed.

[0186] In some embodiments, see Fig.18 ,The user can also enter fault description data on the human-computer dialogue page with the customer service on the terminal device side, the front-end display layer sends the fault description data to the interaction layer, and the interaction layer pre-processes the fault description data.

[0187] In some embodiments, the order of the user inputting the fault description data and triggering the target home appliance to self-check is not limited. For example, the user may first trigger the target home appliance to self-check, and then input the fault description data; or the user may first input the fault description data and then trigger the target home appliance to self-check.

[0188] In some embodiments, see Fig.18 The interaction layer can perform structured splicing on the self-test data and the pre-processed fault description data to obtain first data, generate a first query instruction according to the first data, and send the first query instruction to the knowledge base query service.

[0189] In some embodiments, see Fig.18 In response to the first query instruction, the knowledge base query service can query the knowledge base for target knowledge (i.e., query results) similar to the first data, and send the query results to the interaction layer. Fig.14 The knowledge base query service can perform ES recall on the text in the first data according to the ES database, and perform VS recall on the non-text (such as images) in the first data according to the VS database, so as to obtain the query result.

[0190] In some embodiments, see Fig.18 ,The interaction layer fills in prompt words according to the fault description data, ,self-test data and query results (the target knowledge of the query), ,generates processing instructions according to the prompt words, and sends the processing instructions to the ,language processing model.

[0191] In some embodiments, see Fig.18 , the language processing model responds to processing instructions, see Figure 8 The language processing model sends the response information to the interaction layer. The interaction layer encapsulates the response content of the customer service side based on the response information. The response content includes: the cause of the fault, fault repair guidance (such as operating instructions and reference images of the repair steps, etc.).

[0192] In some embodiments, see Fig.18 , the interaction layer sends the encapsulation result to the front-end display layer, the front-end display layer parses the encapsulation result, and displays the response content on the customer service side of the human-computer dialogue page through the display, thereby guiding the user to repair the target fault caused by the target home appliance.

[0193] Fig.18 In the example timing interaction diagram, the specific implementation methods of each end side can refer to the above description of the online fault diagnosis logic architecture, which will not be repeated here. It should also be noted that the timing interaction diagram is only an example. When the online fault diagnosis logic architecture is different, the corresponding timing interaction can be generated based on the architecture.

[0194] Some embodiments of the present application also provide a computer storage medium, which can store a program. When the computer storage medium is configured in a terminal device, a server or a home appliance, the program can include the software program steps configured in the terminal device, server or home appliance in the above embodiments when it is executed. Among them, the computer storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0196] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the content of this disclosure, so that those skilled in the art can better use the embodiments.

Claims

1. A terminal device, characterized in that: include: Display, used for displaying the human-computer dialogue page; A communicator, used for communicating with a server; The controller is configured as: Acquire fault description data input by the user based on the human-machine dialogue page, wherein the fault description data is used to describe the fault details of the target home appliance; Generate a first query instruction, and send the first query instruction to a server through a communicator; the first query instruction is used to instruct the server to query target knowledge from a knowledge base according to the fault description data; wherein the knowledge base is generated by the server according to a fault tree structured file, the fault tree structured file includes at least one fault tree associated with the target household appliance, the fault tree includes a plurality of nodes, and the nodes are configured with a fault manifestation image, a fault log, a repair method, and a fault phenomenon corresponding to a fault type; receiving target knowledge sent by the server through the communicator, generating prompt word information according to the fault description data and the target knowledge, and sending the prompt word information to the server through the communicator; Receive the response information sent by the server through the communicator, and control the display to display the response content corresponding to the response information on the human-computer dialogue page; wherein, the response information is output by the server after analyzing the prompt word information using a language processing model, and the response information includes the fault type and recommended repair method of the target home appliance.

2. The terminal device according to claim 1, characterized in that: Before generating the first query instruction, the controller is further configured to: Sending a self-test instruction to the target home appliance through the server, wherein the self-test instruction is used to instruct the target home appliance to execute a self-test program and upload self-test data to the server; the self-test data includes a fault code and a self-test operation log; Receiving the self-test data sent by the server through the communicator, and splicing the self-test data and the fault description data into first data; A second query instruction is generated and sent to the server through the communicator; the second query instruction is used to instruct the server to query the target knowledge from the knowledge base according to the first data.

3. The terminal device according to claim 1 or 2, characterized in that: The fault description data is multimodal data including at least two of fault description text, equipment fault image, equipment fault video and fault description audio.

4. A server, characterized in that: include: A communicator, used for communication connection with a terminal device; The controller is configured as: receiving, through the communicator, a first query instruction sent by a terminal device, wherein the first query instruction includes fault description data input by a user based on a human-computer dialogue page displayed by the terminal device, wherein the fault description data is used to describe the fault details of the target household appliance; In response to the first query instruction, query the target knowledge in the knowledge base according to the fault description data, and send the target knowledge to the terminal device through the communicator; wherein the knowledge base is generated by the server according to the fault tree structured file, the fault tree structured file includes at least one fault tree associated with the target household appliance, the fault tree includes a plurality of nodes, and the nodes are configured with a fault manifestation image, a fault log, a repair method and a fault phenomenon corresponding to a fault type; receiving, through a communicator, prompt word information generated and sent by a terminal device according to the fault description data and the target knowledge, analyzing the prompt word information using a language processing model and outputting response information, the response information including the fault type and recommended repair method of the target home appliance; The response information is sent to the terminal device through the communicator.

5. The server according to claim 4, characterized in that: The communicator is also used to communicate with the target household appliance. Before receiving the first query instruction sent by the terminal device through the communicator, the controller is further configured to: Receiving self-test data uploaded by the target household appliance through the communicator, the self-test data including fault codes and self-test operation logs; Sending the self-test data to a terminal device via a communicator; receiving, through the communicator, a second query instruction sent by a terminal device, wherein the second query instruction includes first data obtained by splicing the fault description data and the self-test data; In response to the second query instruction, the target knowledge is queried in the knowledge base according to the first data.

6. The server according to claim 4, characterized in that: The knowledge base includes a text database and a vector database, and the controller is further configured to: According to the node storage rule, the text data in the fault tree structured file is written into the text database and an index is created to complete the text index storage; wherein the node storage rule is to take the node of the fault tree as a unit and perform storage processing on the text data and non-text data configured on the node; The language processing model is used to encode and align the non-text data in the fault tree structured file to obtain a first semantic vector of a preset dimension. The first semantic vector is written into a vector database according to the node storage rule to complete the vector storage.

7. The server according to claim 4, characterized in that: The knowledge base includes a text database and a vector database, and the controller is further configured to: According to the path storage rule, the text data in the fault tree structured file is written into the text database and an index is created to complete the text index storage; wherein the path storage rule is based on the path of the fault tree as a unit, and the text data and non-text data configured by at least two nodes contained in the path are stored in the database; The language processing model is used to encode and align the non-text data in the fault tree structured file to obtain a first semantic vector of a preset dimension, and the first semantic vector is written into a vector database according to the path storage rule to complete the vector storage.

8. The server according to claim 6 or 7, characterized in that: The controller searches for target knowledge in a knowledge base according to the fault description data, and is configured to: Calculating similarity between text data in the fault description data and text data in the text database to obtain a first preset amount of text knowledge; Using the language processing model to encode and align non-text data in the fault description data to obtain a second semantic vector of a preset dimension; The second semantic vector is subjected to similarity calculation with the vector data in the vector database to obtain a second preset amount of non-text knowledge.

9. The server according to claim 4, characterized in that: The controller uses a language processing model to analyze the prompt word information and outputs response information, and is configured as follows: Identify the data type contained in the prompt word information; If the prompt word information includes video data, split the video data into images and audios by frame, so that the prompt word information includes at least one type of data among text type, image type and audio type; Encoding the text data in the prompt word information to obtain a semantic vector; Encoding the audio data in the prompt word information to obtain an audio vector, and aligning the audio vector to the same dimension as the semantic vector; Encoding the image data in the prompt word information to obtain an image vector, and aligning the image vector to the same dimension as the semantic vector; The semantic vector, the aligned audio vector and the image vector are concatenated into a first vector, and the response information is output after the first vector is processed by a large language model.

10. The server according to claim 4, characterized in that: Before sending the response information to the terminal device through the communicator, the controller is further configured to: Acquire the fault confidence calculated by the language processing model, where the fault confidence is used to measure the degree of proximity between the target fault predicted by the language processing model and the actual fault of the target household appliance; If the fault confidence is greater than a preset threshold, querying the automatic repair capability of the target household appliance for the target fault; If the target home appliance has the ability to automatically repair the target fault, an automatic repair instruction is sent to the target home appliance via the communicator, and the automatic repair instruction is used to instruct the target home appliance to execute a repair program associated with the target fault.

11. The server according to claim 10, characterized in that: Before sending the automatic repair instruction to the target home appliance through the communicator, the controller is configured to: Sending a permission request to the terminal device through the communicator, wherein the permission request is used to instruct the terminal device to ask the user whether to agree that the target household appliance automatically repairs the target fault; If the repair approval indication information sent by the terminal device is received through the communicator, the automatic repair instruction is sent to the target home appliance through the communicator; If the repair rejection indication information sent by the terminal device is received through the communicator, the automatic repair instruction is not sent to the target home appliance through the communicator.

12. A method for diagnosing a fault of a household appliance, characterized in that: include: The terminal device obtains the fault description data input by the user based on the human-computer dialogue page, wherein the fault description data is used to describe the fault details of the target home appliance; The terminal device generates a first query instruction and sends the first query instruction to the server; the first query instruction is used to instruct the server to query the target knowledge from the knowledge base according to the fault description data; wherein the knowledge base is generated by the server according to a fault tree structured file, the fault tree structured file includes at least one fault tree associated with the target home appliance, the fault tree includes a plurality of nodes, and the nodes are configured with a fault manifestation image, a fault log, a repair method and a fault phenomenon corresponding to a fault type; The terminal device receives the target knowledge sent by the server, generates prompt word information according to the fault description data and the target knowledge, and sends the prompt word information to the server; The terminal device receives the response information sent by the server, and displays the response content corresponding to the response information on the human-computer dialogue page; wherein, the response information is output by the server after analyzing the prompt word information using a language processing model, and the response information includes the fault type and recommended repair method of the target home appliance.

13. A method for diagnosing a fault of a household appliance, characterized in that: include: The server receives a first query instruction sent by the terminal device, wherein the first query instruction includes fault description data input by the user based on a human-computer dialogue page displayed by the terminal device, and the fault description data is used to describe the fault details of the target home appliance; The server, in response to the first query instruction, queries the target knowledge in the knowledge base according to the fault description data, and sends the target knowledge to the terminal device; wherein the knowledge base is generated by the server according to a fault tree structured file, the fault tree structured file includes at least one fault tree associated with the target household appliance, the fault tree includes a plurality of nodes, and the nodes are configured with a fault manifestation image, a fault log, a repair method, and a fault phenomenon corresponding to a fault type; The server receives prompt word information generated and sent by the terminal device according to the fault description data and the target knowledge, analyzes the prompt word information using a language processing model and outputs response information, wherein the response information includes the fault type and recommended repair method of the target home appliance; The server sends the response information to the terminal device.

Images