A method and system for fault diagnosis and dynamic matching of intelligent devices

By integrating multimodal data processing and knowledge graphs, structured diagnostic reports are generated, and suitable engineers are selected. This solves the problems of inaccurate diagnosis and resource mismatch in after-sales service of smart devices, and improves diagnostic accuracy and service efficiency.

CN123070643APending Publication Date: 2026-11-10GUANGZHOU PINGYUN CRAFTSMAN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611578542.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-13
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

In existing after-sales services for smart devices, multimodal data is not systematically utilized, leading to inaccurate diagnoses, engineers bringing the wrong or missing spare parts, and the simplistic dispatching logic causing resource mismatches, making it difficult to meet users' needs for diagnostic accuracy and first-time repair rate.

Method used

By receiving multimodal fault data, feature extraction and fusion retrieval are performed using a multimodal large model and equipment-fault knowledge graph to generate a structured diagnostic report. Based on the user's location, an engineer set is filtered, overloaded engineers are removed, and a comprehensive score is calculated to determine the dispatch target.

Benefits of technology

This improved the accuracy of fault diagnosis and the precision of dispatching, enhanced overall service efficiency, reduced inaccurate spare parts preparation and resource waste, and increased the first-time repair rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123070643A_ABST
    Figure CN123070643A_ABST
Patent Text Reader

Abstract

This application discloses a method and system for fault diagnosis and dynamic matching of intelligent devices, relating to the field of pattern matching technology. The method includes: receiving multimodal fault data and device identification information uploaded by a client; extracting features from the multimodal fault data based on a pre-trained multimodal large model to obtain multimodal fault features, and performing fusion retrieval and reasoning based on the device identification information and a pre-built device-fault knowledge graph to generate a structured fault diagnosis report; filtering a set of candidate engineers based on user location information, and removing candidate engineers whose real-time load exceeds a preset load threshold; calculating a comprehensive score based on the skill matching degree, reliability score, and load penalty of each removed candidate engineer, determining the candidate engineer with the highest comprehensive score as the dispatch target, and sending the structured fault diagnosis report to the dispatch target's terminal. This solution improves the accuracy of fault diagnosis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the technical field of pattern matching, and in particular to a fault diagnosis and dynamic matching method and system for intelligent devices. Background Technology

[0002] With the rapid development of IoT and AI technologies, the types and complexity of smart devices continue to increase, and users are demanding higher and higher after-sales service response efficiency and first-time repair rates. After-sales service has become a key link in the entire product lifecycle management.

[0003] Currently, mainstream smart device after-sales service systems generally adopt a linear process of "user report - manual assessment - dispatch - on-site repair". In existing systems, users can usually only provide vague text or voice descriptions such as "the device cannot start" or "there is an abnormal noise", and the backend relies on manual or simple rule bases for judgment; the dispatching process is mostly based on single strategies such as nearest or load balancing, and the matching degree between the engineer's skills and expertise and the fault type is not fully considered; at the same time, a large amount of multimodal data such as on-site photos, audio and video recordings uploaded by users when reporting repairs is not systematically collected and utilized.

[0004] However, in the aforementioned existing technical solutions, the ambiguous single-modal information makes it difficult for the backend to accurately locate the root cause of the fault, resulting in low diagnostic accuracy. Inaccurate diagnosis directly leads to engineers bringing the wrong or missing spare parts, requiring a second on-site visit and severely reducing the first-time repair rate. The single-dimensional dispatch logic cannot achieve accurate matching of skills and faults, and ignores the real-time load status of engineers, resulting in resource mismatch and efficiency loss. In addition, the rich unstructured data has not been transformed into structured diagnostic evidence, resulting in a waste of data resources. This makes it difficult for the existing after-sales service system to meet the growing user demands in terms of diagnostic accuracy, dispatch rationality, and overall service efficiency.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide a method and system for fault diagnosis and dynamic matching of intelligent devices, which aims to solve the technical problems that existing technologies cannot comprehensively utilize multimodal data to achieve accurate fault diagnosis and dynamic matching, and the diagnostic accuracy and first-time repair rate are difficult to meet user needs.

[0007] To achieve the above objectives, this application proposes a fault diagnosis and dynamic matching method for intelligent devices, the method comprising: Receive multimodal fault data and device identification information uploaded by the client; Based on a pre-trained multimodal large model, feature extraction is performed on the multimodal fault data to obtain multimodal fault features. These features are then combined with the equipment identification information and a pre-built equipment-fault knowledge graph for fusion retrieval and reasoning to generate a structured fault diagnosis report. A candidate engineer set is obtained by filtering based on user location information, and candidate engineers whose real-time load of service resource profile exceeds a preset load threshold are removed from the candidate engineer set. Based on the skill matching degree, reliability score, and load penalty of each candidate engineer after elimination, a comprehensive score is calculated. The candidate engineer with the highest comprehensive score is determined as the dispatch target, and the structured fault diagnosis report is sent to the terminal of the dispatch target.

[0008] In one embodiment, the step of receiving multimodal fault data and device identification information uploaded by the client includes: In response to a maintenance request initiated by a user through a client, multimodal fault data is collected, including at least one of the following: text description of the fault, voice description, equipment images, and operation logs. Obtain the device identification information, and then encapsulate the multimodal fault data and the device identification information and upload them to the service platform.

[0009] In one embodiment, the step of extracting features from the multimodal fault data based on a pre-trained multimodal large model to obtain multimodal fault features includes: The speech description of the multimodal fault data is subjected to speech recognition and semantic parsing to extract the fault semantic feature vector; Target detection and image classification are performed on the equipment images of the multimodal fault data to identify the fault visual feature vector; The operation logs of the multimodal fault data are analyzed for abnormal events, and fault numerical feature vectors are extracted. The semantic feature vector, visual feature vector, and numerical feature vector are mapped to a unified embedding space for cross-modal feature fusion to generate a multimodal fault feature vector.

[0010] In one embodiment, the step of combining the device identification information and a pre-built device-fault knowledge graph for fusion retrieval and reasoning to generate a structured fault diagnosis report includes: The root node for the device model in the knowledge graph is determined based on the device identification information. Using the multimodal fault feature vector as the query condition, starting from the root node of the equipment model, a path retrieval is performed in the knowledge graph to match and obtain the corresponding set of fault phenomenon nodes; Starting from the set of fault phenomenon nodes, reasoning and traversal are performed along the predefined association edges in the knowledge graph to obtain the fault cause nodes and the corresponding maintenance solution nodes. Based on the maintenance plan nodes, the maintenance step sequence, the required spare parts list, and the standard operating parameters are extracted, and a structured fault diagnosis report is generated according to the preset data structure.

[0011] In one embodiment, the step of filtering the candidate engineer set based on user location information includes: Obtain the geographic coordinates corresponding to the user's location information; The service coverage area is defined with the geographic coordinates as the center and the preset service distance as the radius; The query service platform identifies engineers who are currently available for service and whose service location is within the service coverage area, generating the candidate engineer set.

[0012] In one embodiment, the step of removing candidate engineers whose real-time load on the service resource profile exceeds a preset load threshold from the candidate engineer set includes: The current number of tasks in progress for each engineer in the candidate engineer set is collected in real time through the load monitoring thread; The corresponding load limit threshold is dynamically configured based on each engineer's seniority level; Remove engineers whose current number of tasks in progress exceeds the load limit threshold from the candidate engineer set.

[0013] In one embodiment, the reliability rating specifically includes: Based on the engineer profile database, the historical service quality data of each candidate engineer is read. The historical service quality data includes the historical first-time repair rate, historical user rating, and historical on-time arrival rate. Based on a preset time decay function, the historical one-time repair rate, historical user rating and historical on-time arrival rate are respectively time-weighted to generate time-corrected ratings for each dimension. The timeliness-corrected scores of each dimension are weighted and aggregated to generate a reliability score.

[0014] In one embodiment, the step of calculating a comprehensive score based on the skill matching degree, reliability score, and workload penalty item of each candidate engineer after elimination includes: Analyze the fault complexity level in the structured fault diagnosis report; Based on the fault complexity level, the fusion weight coefficients of the skill matching degree, reliability score and load penalty item are dynamically determined. The skill matching degree, reliability score, and load penalty item are weighted and summed according to the fusion weight coefficient to obtain a comprehensive score.

[0015] In one embodiment, the step of sending the structured fault diagnosis report to the terminal of the dispatch object includes: In response to the on-site arrival signal sent by the dispatching object terminal, the corresponding digital twin model is loaded based on the device identification information; Based on the fault location information in the structured fault diagnosis report, a fault identifier is generated on the digital twin model; The disassembly and assembly instructions of the fault identifier and the maintenance step sequence in the structured fault diagnosis report are overlaid and displayed on the real-time scanning screen of the device on the dispatching terminal using augmented reality.

[0016] Furthermore, to achieve the above objectives, this application also proposes a fault diagnosis and dynamic matching system for intelligent devices, the fault diagnosis and dynamic matching system for intelligent devices comprising: The data acquisition module is used to receive multimodal fault data and device identification information uploaded by the client; The fault diagnosis module is used to extract features from the multimodal fault data based on a pre-trained multimodal large model to obtain multimodal fault features, and to perform fusion retrieval and reasoning by combining the equipment identification information and the pre-built equipment-fault knowledge graph to generate a structured fault diagnosis report. The load matching module is used to filter and obtain a set of candidate engineers based on user location information, and remove candidate engineers whose real-time load of service resource profile exceeds a preset load threshold from the set of candidate engineers. The dispatching module is used to calculate a comprehensive score based on the skill matching degree, reliability score, and load penalty item of each candidate engineer after elimination, determine the candidate engineer with the highest comprehensive score as the dispatching object, and send the structured fault diagnosis report to the terminal of the dispatching object.

[0017] This application proposes a method and system for fault diagnosis and dynamic matching of intelligent devices. The method includes: receiving multimodal fault data and device identification information uploaded by the client; extracting features from the multimodal fault data based on a pre-trained multimodal large model to obtain multimodal fault features, and performing fusion retrieval and reasoning based on the device identification information and a pre-built device-fault knowledge graph to generate a structured fault diagnosis report; filtering a set of candidate engineers based on user location information, and removing candidate engineers whose real-time load of service resource profiles exceeds a preset load threshold; calculating a comprehensive score based on the skill matching degree, reliability score, and load penalty of each candidate engineer after removal, determining the candidate engineer with the highest comprehensive score as the dispatch object, and sending the structured fault diagnosis report to the terminal of the dispatch object. This solution receives multimodal fault data and combines it with device identification information. It then uses a multimodal large model and knowledge graph to perform fusion retrieval and reasoning to generate a structured diagnostic report. Simultaneously, it performs load pruning on candidate engineers and calculates a comprehensive score based on skill matching, reliability, and load penalty to determine the dispatch target. This solves the problems of inaccurate diagnosis caused by relying on unimodal fuzzy descriptions and mismatch between personnel and positions caused by a single dispatch logic in existing technologies. It achieves a comprehensive improvement in diagnostic accuracy, dispatch precision, and overall service efficiency. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of a fault diagnosis and dynamic matching method for intelligent devices according to this application. Figure 2 This is a flowchart illustrating a second embodiment of a fault diagnosis and dynamic matching method for intelligent devices according to this application. Figure 3 This is a flowchart illustrating a third embodiment of a fault diagnosis and dynamic matching method for intelligent devices according to this application. Figure 4 This is a flowchart illustrating Embodiment 4 of a fault diagnosis and dynamic matching method for intelligent devices according to this application. Figure 5 This is a flowchart illustrating Embodiment 5 of the fault diagnosis and dynamic matching method for intelligent devices according to this application. Figure 6 This is a schematic diagram of the module structure of a fault diagnosis and dynamic matching system for an intelligent device according to an embodiment of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] Because existing technologies cannot comprehensively utilize multimodal data to achieve accurate fault diagnosis and dynamic matching, the diagnostic accuracy and first-time repair rate are difficult to meet user needs.

[0025] This application provides a solution that receives multimodal fault data and combines it with device identification information. It then uses a multimodal large model and knowledge graph to perform fusion retrieval and reasoning to generate a structured diagnostic report. Simultaneously, it performs load pruning on candidate engineers and calculates a comprehensive score based on skill matching, reliability score, and load penalty to determine the dispatch target. This solves the problems of inaccurate diagnosis caused by relying on unimodal fuzzy descriptions and mismatch between personnel and positions caused by a single dispatch logic in existing technologies. It achieves a comprehensive improvement in diagnostic accuracy, dispatch precision, and overall service efficiency.

[0026] Based on this, embodiments of this application provide a method for fault diagnosis and dynamic matching of intelligent devices, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the fault diagnosis and dynamic matching method for intelligent devices according to this application.

[0027] In this embodiment, the fault diagnosis and dynamic matching method for the intelligent device includes steps S10 to S40: Step S10: Receive multimodal fault data and device identification information uploaded by the client.

[0028] It should be noted that in this embodiment, the client refers to the intelligent terminal device used by the user, including but not limited to a smartphone, a tablet computer, a personal computer, or a dedicated handheld terminal matched with an intelligent device. An application program or a web client corresponding to the after-sales service system platform runs on the client, and the user initiates a device maintenance request through the client. The multi-modal fault data refers to multiple types of fault-related data provided by the user during the repair reporting process and derived from different perception channels, which includes but is not limited to at least one of text description data, voice description data, image data, and video data. The device identification information refers to an information carrier for uniquely identifying the identity of the intelligent device to be repaired, including but not limited to a device serial number, a MAC address, coding information contained in a device two-dimensional code, or a device number bound in a user asset library.

[0029] In a possible implementation, the user scans a two-dimensional code or a barcode on the device through the client to automatically obtain the device identification information; or the user selects the current device to be repaired from the device list already bound to the client to associate the device identification information; or the client automatically reads a near field communication tag of the device or obtains device electronic tag information through a Bluetooth protocol to acquire the device identification information.

[0030] In a specific implementation, a user finds that the refrigeration of the smart air conditioner at home is abnormal, opens the brand after-sales service APP on the mobile phone, clicks the "voice input" button on the repair reporting page to record a 15-second voice description "the air conditioner has been turned on for half an hour and still does not cool, and the air outlet makes louder noise than usual", then takes a photo of the flashing fault indicator light on the air conditioner panel and the error code "E5" displayed on the remote control screen, the APP automatically reads the device serial number of the air conditioner as the device identification information, packages and uploads the voice file, the photo and the device serial number to the service platform together, and the system receives the above data to complete the collection of the multi-modal fault data and the device identification information.

[0031] Step S20: performing feature extraction on the multi-modal fault data based on a pre-trained multi-modal large model to obtain a multi-modal fault feature, and performing fusion retrieval and reasoning in combination with the device identification information and a pre-constructed device-fault knowledge graph to generate a structured fault diagnosis report.

[0032] It should be noted that, in this embodiment, the multimodal large model refers to a pre-trained deep learning model capable of simultaneously processing and understanding multiple modalities of data (text, speech, images, video, etc.), possessing cross-modal semantic understanding and feature alignment capabilities. The multimodal large model includes, but is not limited to, multimodal Transformer architecture models, contrastive language-image pre-trained models, or multimodal large language models. Feature extraction refers to converting the original unstructured multimodal data into numerical feature representations in a high-dimensional vector space. Specifically, for text modal data, semantic encoding is performed using the text encoder in the multimodal large model to obtain text semantic feature vectors; for speech modal data, speech recognition is first performed to convert it into text, and then processed by the text encoder, or acoustic feature vectors are directly extracted using the speech encoder; for image modal data, convolutional encoding or attention encoding is performed using the visual encoder in the multimodal large model to extract visual semantic feature vectors; for video modal data, keyframes are extracted, and the keyframe sequence is visually encoded, fusing temporal information to obtain video feature vectors. Multimodal fault features refer to the comprehensive feature representation obtained by mapping feature vectors from different modes to a unified embedding space and then performing cross-modal fusion processing.

[0033] In one possible implementation, feature concatenation is used to directly concatenate the feature vectors of each modality in terms of dimension, forming a multimodal fusion feature; alternatively, a cross-modal attention mechanism is used to weightedly fuse features from different modalities, adaptively highlighting modal information with high relevance to fault diagnosis and suppressing redundant or noisy modal information. In this embodiment, feature extraction is used to convert heterogeneous multimodal input data into a unified vectorized representation, enabling subsequent retrieval and inference steps to measure similarity and perform association matching in the same semantic space.

[0034] Additionally, it should be noted that in this embodiment, the equipment-fault knowledge graph is a knowledge base organized in a graph structure, containing entities such as equipment model, fault phenomenon, fault cause, and repair solution, as well as their interrelationships. The entities and relationships in the equipment-fault knowledge graph are pre-constructed through structured extraction from equipment manuals, repair manuals, historical work order data, and expert experience. Specifically, the equipment-fault knowledge graph uses the equipment model as the root node, with each root node associated with a corresponding fault phenomenon node. Fault phenomenon nodes are connected to fault cause nodes through "representation" relationships, and fault cause nodes are connected to repair step sequence nodes, required spare parts list nodes, and standard working hours nodes through "solution" relationships, forming a complete reasoning chain of "fault phenomenon → fault cause → repair solution".

[0035] Additionally, it should be noted that fusion retrieval and reasoning refers to locating the root node of the corresponding equipment model in the knowledge graph based on the equipment identification information, thereby limiting the retrieval scope to the subgraph related to that equipment model; then, using multimodal fault features as query vectors, similarity retrieval is performed in the limited subgraph, calculating the similarity score between the query vector and the feature vectors corresponding to each fault phenomenon node, and obtaining a set of candidate fault phenomenon nodes whose similarity exceeds a preset threshold; then, graph reasoning traversal is performed along the association edges of the candidate fault phenomenon nodes in the knowledge graph to sequentially obtain the corresponding fault cause node, maintenance step sequence node, required spare parts list node, and standard working hour node. A structured fault diagnosis report refers to a data object with clearly defined fields, formed by organizing the retrieval and reasoning results according to a preset format.

[0036] In one possible implementation, the structured fault diagnosis report is assembled in JSON format and includes the following fields: Fault Location Description field, used to record the specific location and manifestation of the fault phenomenon; Root Cause field, used to record the reasoned cause of the fault; Maintenance Operation Sequence field, used to record the list of steps specified in the standard operating procedure; Spare Parts List field, used to record the names and model numbers of spare parts that need to be replaced or are recommended to be replaced during maintenance; Estimated Maintenance Duration field, used to record the value of standard working hours; and Safety Risk Level field, used to record the level of safety precautions for the maintenance operation.

[0037] This embodiment combines multimodal fault features with equipment identification information to perform precise retrieval and reasoning within a knowledge graph subgraph limited by equipment model. This can improve diagnostic accuracy while narrowing the retrieval range, avoid mismatches across models, and effectively enhance the accuracy of fault location.

[0038] In one specific implementation, the system performs speech recognition on the received air conditioner voice file to obtain the text "Air conditioner not cooling, abnormal noise from the air outlet." It then extracts image features from the "E5" error code image using a visual encoder. The system fuses the text and image features to obtain a multimodal fault feature vector. Based on the device serial number, the system determines the device model as "KFR-35GW / ABC," locates the corresponding subgraph in the knowledge graph, and searches this subgraph using the multimodal fault feature vector. It matches the fault phenomenon node "E5 error code + not cooling + abnormal noise." The associated fault cause of this node is "fan motor bearing wear leading to decreased speed and evaporator icing." The repair solution node includes two steps: "replace the fan motor bearing" and "check refrigerant pressure," along with the bearing model "6201Z" and an estimated repair time of 45 minutes. This information is then assembled in JSON format to generate a structured diagnostic report.

[0039] Step S30: Based on the user location information, a set of candidate engineers is obtained, and candidate engineers whose real-time load of service resource profiles exceeds a preset load threshold are removed from the set of candidate engineers.

[0040] It should be noted that in this embodiment, user location information refers to the coordinate data of the user's geographical location. The candidate engineer set obtained by filtering based on user location information is based on the user location information as the reference center, and engineers with geographical proximity are initially selected from all available engineers according to spatial distance conditions.

[0041] In one possible implementation, a circular geofence is drawn with the latitude and longitude coordinates of the user's location information as the center and a preset radius R. All engineers currently online on the service platform whose service address or current location falls within this circular geofence are queried and initialized as a candidate engineer set. Alternatively, based on traffic conditions and actual road distance instead of straight-line distance, engineers whose estimated commute time is within a preset time threshold are selected as the candidate engineer set. Each engineer in the candidate engineer set corresponds to a service resource profile, which is a set of digital tags describing the engineer's service capabilities and current service status. This profile includes static and dynamic attribute dimensions: static attribute dimensions include fixed information such as skill tag set, years of experience, and qualification certification level; dynamic attribute dimensions include real-time geographical location, number of currently pending work orders, and work order status.

[0042] This embodiment limits the service area to the user's vicinity through a spatial filtering step, ensuring that dispatching engineers can quickly arrive at the site, shortening user waiting time and improving service response efficiency.

[0043] Additionally, it should be noted that real-time load refers to the number of work orders that an engineer is currently processing or has accepted but not yet completed, reflecting the engineer's current workload.

[0044] In one possible implementation, real-time load is collected in real time through a work order status monitoring thread maintained by the service platform. This monitoring thread listens to the work order allocation and completion events of each engineer in real time and dynamically updates the real-time load count for each engineer. The preset load threshold is a pre-set upper limit value used to determine whether an engineer is in an overloaded state.

[0045] In one possible implementation, the preset load threshold is a fixed value, for example, uniformly set to a maximum of 5 orders per engineer at the same time; or it is dynamically adjusted according to the engineer's seniority or skill level, for example, the load threshold for senior engineers is higher than that for junior engineers, so as to fully release the service capacity of highly skilled engineers; or it is dynamically adjusted according to the current overall backlog of work orders on the service platform, with the threshold being appropriately raised when the overall system load is high to improve the system's concurrent throughput capacity, and the threshold being appropriately lowered when the overall system load is low to ensure service quality.

[0046] This embodiment performs pre-load pruning to prevent new maintenance tasks from being assigned to overloaded engineers, thus preventing a decline in their service quality or inability to respond in a timely manner. On the other hand, by pruning before the comprehensive score calculation rather than eliminating candidates after the score is calculated, the number of candidates participating in the comprehensive score calculation can be effectively reduced, thereby reducing the system's ineffective computational overhead and improving matching efficiency.

[0047] In one specific implementation, based on the coordinates (116.397, 39.908) reported by the user's mobile phone GPS, the service platform filters within a 5-kilometer radius of these coordinates, identifying 8 online engineers within this range, forming a candidate engineer set. Further querying the real-time load of these 8 engineers' service resource profiles reveals: Engineer A currently has 6 work orders in progress, Engineer B has 4, Engineer C has 3, and the remaining 5 engineers have 1-3 work orders in progress. With a preset load threshold of 5, since Engineer A's 6 in-progress work orders exceed the threshold of 5, Engineer A is removed from the candidate engineer set, and the remaining 7 engineers proceed to the subsequent scoring stage.

[0048] Step S40: Calculate a comprehensive score based on the skill matching degree, reliability score, and load penalty item of each candidate engineer after elimination. Determine the candidate engineer with the highest comprehensive score as the dispatch target and send the structured fault diagnosis report to the terminal of the dispatch target.

[0049] It should be noted that, in this embodiment, the skill matching degree refers to the quantified value of the degree of fit between the professional skills of the candidate engineer and the current fault repair needs. Specifically, the system extracts fault type tags or required skill tags from the structured fault diagnosis report, and simultaneously reads the set of professional skill tags and the proficiency level of each skill from the service resource profile of each candidate engineer. The system calculates the semantic similarity between the fault type tags and the skill tag set, and performs weighted correction based on the proficiency level to obtain the skill matching degree. In one possible implementation, the semantic similarity is obtained by calculating the cosine similarity between the skill embedding vector corresponding to the fault type tag and the embedding vector corresponding to the skill tags of each candidate engineer.

[0050] Additionally, it should be noted that in this embodiment, the reliability score refers to a comprehensive score calculated based on the engineer's historical service performance data, used to characterize the reliability of the engineer's service quality. In one possible implementation, the reliability score is obtained by weighted aggregation of historical service quality data. Specifically, this includes the historical first-time repair rate (the percentage of times the engineer completed a repair on their last visit); the user satisfaction rate (the percentage of users who gave positive feedback on the engineer's service); and the historical on-time arrival rate (the percentage of times the engineer arrived on-site within the promised timeframe). In another possible implementation, a time decay factor is introduced to correct for the timeliness of the aforementioned historical data, giving greater weight to recent service performance than to long-term service performance, thus more accurately reflecting the engineer's current service capability level.

[0051] Additionally, it should be noted that in this embodiment, the load penalty item refers to a negative adjustment factor on the overall score based on the engineer's current load status. In one possible implementation, the value of the load penalty item is the ratio of the engineer's current load rate to the baseline load rate. The higher the current load rate, the larger the load penalty item, and the lower the overall score, thereby suppressing the continued allocation of tasks to engineers with high loads during task dispatching.

[0052] Furthermore, it should be noted that in this embodiment, the comprehensive score is obtained by weighting and summing the above three dimensions according to preset weight coefficients. In one possible implementation, the preset weight coefficients are dynamically adjusted based on the fault difficulty level contained in the structured fault diagnosis report; if the fault difficulty level is higher than the preset difficulty threshold, the weight ratio of skill matching is increased and the weight ratio of load penalty is decreased to prioritize the allocation of high-difficulty faults to engineers with matching skills; if the fault difficulty level is lower than the preset difficulty threshold, the weight ratio of skill matching is appropriately decreased and the weight ratio of load penalty is increased to achieve efficient and balanced allocation of simple faults. The comprehensive score calculation in this embodiment comprehensively considers three dimensions: skill matching, service quality reliability, and real-time load, achieving optimal dispatch decision under multi-objective constraints.

[0053] Additionally, it should be noted that the dispatch recipient refers to the engineer with the highest score after comprehensive evaluation and ranking. In this embodiment, the system pushes the structured fault diagnosis report to the dispatch recipient's terminal, i.e., the work terminal used by that engineer. The terminal runs a dedicated work client for the engineer. After receiving the structured fault diagnosis report, the client displays it to the engineer in a structured format, specifically including fault location and phenomenon description, root cause explanation, maintenance operation procedure guidance, required spare parts list and numbers, estimated maintenance time, and safety precautions.

[0054] In one possible implementation, the terminal is a smartphone or tablet, and the structured fault diagnosis report is presented on the terminal's client interface in the form of a card list and step-by-step guide. In another possible implementation, the terminal is augmented reality glasses, and the maintenance guidance information in the structured fault diagnosis report is overlaid on the corresponding location of the physical equipment using augmented reality.

[0055] This embodiment directly sends structured fault diagnosis reports to the terminal of the dispatching party, enabling engineers to fully grasp fault information and repair plans after receiving an order. This allows them to accurately prepare the necessary spare parts and tools before departure, avoiding the problem of inaccurate spare parts preparation caused by opaque diagnostic information in traditional processes, and effectively improving the first-time repair rate.

[0056] Furthermore, the step of receiving multimodal fault data and device identification information uploaded by the client includes steps A201-A202 before the following steps: Step A201: In response to a maintenance request initiated by the user through the client, collect multimodal fault data, which includes at least one of the following: text description of the fault, voice description, equipment images, and operation logs. Step A202: Obtain the device identification information of the device, and encapsulate the multimodal fault data and the device identification information before uploading them to the service platform.

[0057] It should be noted that, in this embodiment, multimodal fault data refers to various types of data related to device faults originating from different sensing channels. Specifically, textual descriptions of faults refer to the fault phenomenon description entered by the user in natural language through the client's input box; voice descriptions refer to audio files of the user's verbal description of the fault situation recorded by the user through the client's recording function; device images refer to photos of the device's appearance, abnormal indicator lights, screen error codes, or short videos of abnormal device operation taken by the user through the client; and operation logs refer to the operating status data recorded and stored by the smart device itself, including but not limited to device operating parameters, sensor-collected data, abnormal event records, and fault code information. Association encapsulation refers to establishing a logical association between multimodal fault data and device identification information, organizing them as different fields within the same maintenance request data packet. This ensures that the device identification information and the corresponding multimodal fault data maintain a binding relationship during transmission and storage, avoiding data confusion. Uploading to the service platform refers to the client sending the encapsulated data packet to the after-sales service system platform deployed in the cloud via a wireless network. After receiving the data packet, the service platform parses and stores it, using the storage address as the unique data index for the maintenance request.

[0058] In one possible implementation, encapsulation is performed according to a preset data protocol format. The client fills the device identification information into the device identity field of the protocol header and fills the multimodal fault data into the corresponding data blocks of the protocol body according to different types, such as text data blocks, audio data blocks, image data blocks and log data blocks. The data blocks are distinguished by type identifiers, and finally a complete upload data packet is formed.

[0059] This application collects multimodal fault data in response to maintenance requests, and uploads the multimodal fault data to the service platform after associating and encapsulating it with equipment identification information. This enables the system to obtain far richer fault clues than a single text description. By binding unstructured data with equipment identity information, it effectively solves the problem of limited and scarce fault information in the traditional repair process, and improves the accuracy and efficiency of diagnosis.

[0060] Furthermore, referring to Figure 2 The second embodiment of the fault diagnosis and dynamic matching method for intelligent devices in this application provides a flowchart, based on the above. Figure 2 The embodiment shown further refines the step S20, "extracting features from the multimodal fault data based on a pre-trained multimodal large model to obtain multimodal fault features," including steps A301-A304: Step A301: Perform speech recognition and semantic parsing on the speech description of the multimodal fault data to extract the fault semantic feature vector.

[0061] It should be noted that in this embodiment, speech recognition refers to the system calling the speech recognition engine to preprocess the audio file, including noise reduction, endpoint detection, and feature extraction, and then decoding the audio signal into the corresponding text sequence through acoustic and language models. Semantic parsing refers to performing natural language understanding processing on the text content obtained after speech recognition to extract key semantic information related to fault diagnosis, including but not limited to fault phenomenon keywords (such as "not cooling", "abnormal noise", "water leakage"), equipment operating status descriptions (such as "automatically shuts down after power-on", "burning smell during operation"), and the time and environmental conditions of the fault occurrence (such as "occurred on rainy days", "occurred after running for half an hour"). The fault semantic feature vector refers to the numerical feature representation obtained by mapping the key information in the semantic parsing results to a high-dimensional vector space.

[0062] In one possible implementation, the system uses a pre-trained language encoder to encode the semantically parsed text. This encoder, based on the Transformer architecture, captures the contextual semantic relationships within the text, converting fault description texts of varying lengths into fixed-dimensional semantic feature vectors. In another possible implementation, the system first performs word segmentation, part-of-speech tagging, and dependency parsing on the speech recognition text to extract triples representing the fault subject, fault action, and fault attribute. These triples are then input into the encoder to obtain the semantic feature vector. This step, through direct acquisition and automatic escaping of speech descriptions, allows users to complete fault descriptions without manually inputting text, significantly lowering the barrier to reporting repairs.

[0063] Step A302: Perform target detection and image classification on the equipment images of the multimodal fault data to identify the fault visual feature vector.

[0064] It should be noted that, in this embodiment, target detection refers to the process of locating and identifying target object regions related to fault diagnosis in the device image. In one possible implementation, the system employs a target detection model based on a deep convolutional neural network to detect regions of interest such as indicator light areas, display screen areas, and device casing areas in the image, and outputs the category labels and bounding box positions of each region.

[0065] Additionally, it should be noted that in this embodiment, image classification refers to the process of classifying and identifying the detected target area or the entire image to determine the category of the fault state it represents. For example, for a detected indicator light area, the image classification model further determines the color (red, yellow, green) and state (solid, flashing, off) of the indicator light; for a detected screen display area, the image classification model identifies the displayed error code characters or fault icons. The fault visual feature vector refers to the feature representation obtained by vectorizing and encoding the visual recognition results obtained from target detection and image classification, which encodes all observable visual fault cues in the image.

[0066] This step uses automatic recognition and analysis of device images to capture fault characteristics that users cannot accurately describe in words from a visual perspective, such as flashing indicator lights of specific colors and error code displays.

[0067] Step A303: Perform abnormal event analysis on the operation log of the multimodal fault data and extract the fault numerical feature vector.

[0068] It should be noted that, in this embodiment, abnormal event parsing refers to the process of automatically scanning and analyzing the operation log data to identify parameters that exceed normal thresholds or event records marked as abnormal. The fault numerical feature vector refers to the feature representation obtained by numerically encoding the abnormal parameters, abnormal values, and their occurrence times extracted from the abnormal event parsing results.

[0069] In one possible implementation, the system reads the normal range of values ​​for each sensor and the baseline values ​​of operating parameters corresponding to the device model from the configuration library, compares the measured values ​​in the log with the baseline values, and marks all events exceeding the threshold. In another possible implementation, the system uses a time-series anomaly detection algorithm to perform sliding window analysis on the time-series data in the log to identify abrupt changes, abnormal trends, and periodic deviations in parameter values.

[0070] In one possible implementation, the fault numerical feature vector includes the identifier code of the over-threshold parameter and the normalized value of its deviation magnitude; in another possible implementation, the fault numerical feature vector includes the vectorized representation of the error code recorded in the equipment operation log.

[0071] This application can obtain objective and accurate operating status data from the device through automatic parsing of the operating log, effectively overcoming the possible deviation between the user's subjective description and the actual status of the device.

[0072] Step A304: Map the semantic feature vector, visual feature vector and numerical feature vector to a unified embedding space to perform cross-modal feature fusion and generate a multimodal fault feature vector.

[0073] It should be noted that, in this embodiment, the unified embedding space refers to a high-dimensional vector space. Feature vectors from different modalities are mapped into this space through their respective projection layers, so that semantically similar vectors from different modalities have similar distances in this space. Cross-modal feature fusion refers to the process of integrating the three modal feature vectors mapped to the unified embedding space according to a preset fusion strategy to generate a fused feature vector containing multi-dimensional comprehensive information.

[0074] In one possible implementation, cross-modal feature fusion employs a feature concatenation method, directly concatenating semantic feature vectors, visual feature vectors, and numerical feature vectors along their vector dimensions to form a fused feature vector with added dimensions. Alternatively, an attention-weighted fusion method can be used. This method utilizes a cross-modal attention mechanism to calculate the importance weight of each modal feature for the current fault diagnosis task, and then weights and sums the modal features according to these weights. This adaptively highlights modal information highly relevant to the fault while suppressing redundant or noisy modal information. For example, when the speech description includes explicit fault code descriptions, the attention mechanism can increase the weight of the speech modal features.

[0075] This embodiment integrates three feature vectors from different sources and in different forms into a unified multimodal fault feature vector, realizing the comprehensive utilization of all fault clues in user voice, equipment images and operation logs, and significantly improving the completeness and accuracy of fault characterization.

[0076] Furthermore, referring to Figure 3 The third embodiment of the fault diagnosis and dynamic matching method for intelligent devices in this application provides a flowchart, based on the above. Figure 3 The embodiment shown further refines the step S20, "combining the device identification information and the pre-built device-fault knowledge graph for fusion retrieval and reasoning to generate a structured fault diagnosis report," including steps A401-A404: Step A401: Determine the root node of the device model in the knowledge graph based on the device identification information.

[0077] It should be noted that, in this embodiment, the equipment-fault knowledge graph refers to a knowledge base organized in a graph structure, containing entities such as equipment model, fault phenomenon, fault cause, and repair plan, as well as their interrelationships. This knowledge graph is pre-constructed through structured extraction from equipment manuals, repair manuals, historical work order data, and expert experience. The knowledge graph is stored and managed using a graph database, where nodes represent entities and edges represent the relationships between entities. The equipment model root node refers to the top-level node in the knowledge graph that matches the equipment model corresponding to the equipment identification information. This node serves as the index root node for all fault knowledge of that model in the knowledge graph, and it contains child nodes for all possible fault phenomena of that model. The equipment model field included in the equipment identification information is used for precise matching in the knowledge graph to locate the corresponding equipment model root node.

[0078] In one possible implementation, the system first queries the model index layer of the knowledge graph for nodes that exactly match the device model field; if no exact match is found, a fuzzy match is performed to find the closest model node through the character similarity or synonym mapping relationship of the model name, and the matching confidence is recorded for subsequent diagnostic reference.

[0079] This embodiment determines the root node of the device model as the starting point for retrieval, and narrows the subsequent graph retrieval scope from the entire knowledge graph to the subgraph corresponding to that model. This effectively avoids cross-model mismatches that may occur between different models of devices due to similar fault phenomena, and significantly improves retrieval efficiency and accuracy.

[0080] Step A402: Using the multimodal fault feature vector as the query condition, starting from the root node of the device model, perform path retrieval in the knowledge graph to obtain the corresponding set of fault phenomenon nodes.

[0081] It should be noted that, in this embodiment, path retrieval refers to the process of starting from the root node of the device model, traversing downwards along the edge relationships defined in the knowledge graph, and searching for target nodes that match the query conditions within a limited range. Specifically, the system uses the multimodal fault feature vector as the query vector, calculates the similarity between the query vector and the feature vectors associated with each fault phenomenon node in the knowledge graph, and takes the fault phenomenon nodes with similarity exceeding a preset threshold as the matching results.

[0082] In one possible implementation, each fault phenomenon node in the knowledge graph has a corresponding feature vector pre-stored. The feature vector is extracted from multimodal data in historical fault work orders. The dimension of the feature vector of each fault phenomenon node is consistent with the dimension of the multimodal fault feature vector. The two are located in the same vector space, making the vector similarity calculation comparable.

[0083] In one possible implementation, the system uses a vector similarity retrieval method to calculate the cosine similarity or Euclidean distance between the query vector and the feature vectors of each fault phenomenon node. The nodes are then sorted by similarity from highest to lowest, and the top N fault phenomenon nodes are selected to form a candidate set. For phenomenon nodes with similarity scores below a threshold, the system marks them as low-confidence matches and reduces their path weight in subsequent inference. In another possible implementation, path retrieval supports multi-hop retrieval. Starting from the device model root node, it first retrieves first-level child nodes (directly associated fault phenomena). If the matching degree is insufficient, it expands to second-level child nodes (sub-phenomena associated with the first-level nodes) to cover fuzzy matching scenarios where user descriptions are not precise enough.

[0084] This embodiment obtains a set of fault phenomenon nodes through path retrieval, establishes an association mapping between the multimodal fault feature vector and the pre-stored structured fault knowledge in the knowledge graph, and realizes the transformation from unstructured fault description to structured fault nodes.

[0085] Step A403: Starting from the set of fault phenomenon nodes, perform reasoning traversal along the predefined association edges in the knowledge graph to obtain the fault cause nodes and the corresponding maintenance solution nodes.

[0086] It should be noted that, in this embodiment, the association edge refers to a directed edge with a band type that connects different entity nodes in the knowledge graph and represents the semantic relationship between entities. Specifically, a fault phenomenon node is connected to a fault cause node through a "root cause relationship" edge, representing which fault causes may lead to a certain fault phenomenon; a fault cause node is connected to a maintenance plan node through a "solution relationship" edge, representing the maintenance operation plan corresponding to a certain fault cause. Reasoning traversal refers to the process of gradually expanding outward from the starting node along the direction of the association edges to obtain all or part of the target nodes associated with the starting node. Specifically, for each fault phenomenon node in the fault phenomenon node set, the system obtains one or more fault cause nodes along its "root cause relationship" edge, and then obtains the corresponding maintenance plan node along the "solution relationship" edge of each fault cause node.

[0087] In one possible implementation, when multiple fault phenomenon nodes are matched, the system performs reasoning traversal on each fault phenomenon node to obtain multiple fault cause nodes and repair solution nodes. The reasoning results are then weighted and sorted according to the similarity scores of each fault phenomenon node, and the reasoning path with the highest overall score is output as the primary diagnostic result, while other candidate paths are retained as alternative diagnoses. In another possible implementation, if a fault cause node is pointed to by multiple fault phenomenon nodes simultaneously, the confidence level of that fault cause is increased accordingly, and the recommendation priority of that cause is increased when generating the diagnostic report.

[0088] This embodiment, by reasoning and traversing along the associated edges of the knowledge graph, can automatically deduce possible causes of failure and their corresponding maintenance solutions from the fault phenomena, realizing a complete reasoning chain from "phenomenon description" to "cause location" and then to "solution output".

[0089] Step A404: Extract the maintenance step sequence, required spare parts list and standard operating parameters based on the maintenance plan nodes, and assemble and generate a structured fault diagnosis report according to the preset data structure.

[0090] It should be noted that in this embodiment, the repair plan node is an entity node in the knowledge graph that stores repair plan information. Its attribute fields contain complete repair operation information corresponding to the cause of the fault. The repair step sequence refers to a list of standard operating procedure steps extracted from the repair plan node and arranged in execution order. Each repair step includes a step number, operation content description, operation precautions, and expected operation time. The required spare parts list refers to a list of spare parts that need to be replaced or are recommended to be replaced for repairing the fault. Each spare part information includes the spare part name, spare part model number, recommended replacement quantity, and spare part inventory status. Standard operating parameters refer to the technical parameter standards involved in the repair process, including but not limited to quantitative indicators such as standard working hours, torque requirements, pressure range, and temperature threshold. The preset data structure refers to the predefined diagnostic report data format specification, which specifies the name, data type, value range, and required attributes of each field in the report.

[0091] In one possible implementation, the preset data structure is in JSON format, which includes the following top-level fields: Fault Location field, used to record the specific description of the fault phenomenon and its location; Root Cause field, used to record the cause of the fault; Maintenance Operation Sequence field, used to store the sequence of maintenance steps; Spare Parts List field, used to store the list of required spare parts; Estimated Maintenance Duration field, used to store the standard working hours; and Safety Risk Level field, used to record safety tips and protection requirements for maintenance operations.

[0092] In another possible implementation, when assembling the diagnostic report, the step descriptions in the maintenance plan nodes are also adapted and adjusted according to the firmware version field in the device identification information. For example, step branch descriptions are made for the differences in operation under different firmware versions, so that the generated diagnostic report accurately matches the actual device status.

[0093] This embodiment assembles the knowledge graph retrieved from the knowledge graph according to a preset data structure, converting the graph structure knowledge into structured data objects for information transmission and engineer work guidance in the subsequent order dispatch process, thus achieving efficient conversion from knowledge graph data to business application data.

[0094] Furthermore, the step of filtering the candidate engineer set based on user location information includes A501-A503: Step A501: Obtain the geographic coordinates corresponding to the user's location information.

[0095] In one possible implementation, the system obtains the user's location information through the latitude and longitude coordinates reported by the client's built-in GPS module; or by querying the location database corresponding to the Wi-Fi hotspot connected to the client; or by performing triangulation positioning through the base station signal accessed by the client; or by using the address information manually entered by the user and calling the geocoding service to perform coordinate transformation to obtain the user's location information.

[0096] Step A502: Determine the service coverage area with the geographic coordinates as the center and the preset service distance as the radius.

[0097] It should be noted that in this embodiment, the preset service distance is a value pre-set by the system to define the radius of the service area, and can be configured according to business needs and service strategies.

[0098] In one possible implementation, the preset service distance is dynamically adjusted based on the density of engineers in the user's area. When there are many online engineers in the area, the preset service distance is shortened to control the size of the candidate set; or it can be configured differently based on the geographical type of the user's area.

[0099] Step A503: Query the engineers in the service platform who are currently available for service and whose service location is within the service coverage area, and generate the candidate engineer set.

[0100] It should be noted that, in this embodiment, "serviceable status" refers to the engineer's current working status on the service platform, which is marked as "available to accept orders." Specifically, serviceable status includes a combination of conditions such as the engineer being currently online, not on vacation or offline, and not being temporarily locked by the system. Secondly, each engineer in the generated candidate engineer set corresponds to a service resource profile, which includes comprehensive information such as the engineer's skill tag set, real-time load status, and historical credit score.

[0101] This step combines spatial filtering with status filtering, excluding unavailable engineers who are too far away or offline in the initial stage. This effectively reduces the data scale for comprehensive score calculation and improves overall matching efficiency.

[0102] Furthermore, referring to Figure 4 The fourth embodiment of the fault diagnosis and dynamic matching method for intelligent devices in this application provides a flowchart, based on the above. Figure 4 The embodiment shown further refines the step of "removing candidate engineers whose real-time load on the service resource profile in the candidate engineer set exceeds a preset load threshold" in step S30, including steps A601-A603: Step A601: The current number of tasks in progress for each engineer in the candidate engineer set is collected in real time through the load monitoring thread.

[0103] It should be noted that, in this embodiment, the load monitoring thread refers to an independently running background service process within the service platform, used to continuously monitor and update the real-time work order load of each engineer; specifically, it executes the data collection task cyclically at preset time intervals, or triggers data collection updates in response to work order status change events. The current number of tasks in transit refers to the total number of work orders currently assigned to engineers but not yet completed. This number reflects the current workload of engineers and available service capacity; specifically, the current number of tasks in transit includes pending work orders that engineers have accepted but have not yet dispatched, work orders that have been dispatched and are en route, and work orders currently under on-site repair, but does not include completed work orders awaiting user confirmation.

[0104] Step A602: Dynamically configure the corresponding load limit threshold based on the seniority level of each engineer.

[0105] It should be noted that in this embodiment, the qualification level refers to the level label used to characterize the engineer's professional and technical level and the richness of his / her work experience. It is based on a comprehensive evaluation of the engineer's years of service, skill certification level, and historical repair quality. Specifically, the qualification level is divided into primary level and secondary level. The load limit threshold refers to a pre-set critical value used to determine whether the engineer is in an overloaded state. When the engineer's current on-the-go workload reaches or exceeds this value, the system determines that the engineer is temporarily not capable of undertaking new tasks.

[0106] Additionally, it should be noted that dynamic configuration refers to the system calculating the load limit threshold differently for each engineer based on their seniority level, rather than applying a uniform fixed threshold to all engineers. In one possible implementation, the system presets a load limit threshold mapping relationship for different seniority levels. For example, the load limit threshold for junior engineers is 3 orders, for intermediate engineers it is 5 orders, for senior engineers it is 7 orders, and for expert engineers it is 9 orders. Engineers with higher seniority levels are configured with higher load limit thresholds because they have higher work efficiency and more experience, so as to fully release the service capacity of highly skilled engineers.

[0107] This embodiment achieves differentiated load management for engineers at different skill levels by dynamically configuring the load limit threshold based on their seniority level. This avoids both the idle waste of service capacity of highly skilled engineers and the excessive order pressure on junior engineers that leads to a decline in service quality.

[0108] Step A603: Remove the engineers whose current number of tasks in transit exceeds the load limit threshold from the candidate engineer set.

[0109] This embodiment performs pre-load pruning before performing complex comprehensive scoring calculations. This early stage excludes overloaded engineers who lack the capacity to handle the workload, effectively reducing the number of candidates required for subsequent comprehensive scoring and lowering unnecessary computational overhead. Simultaneously, this rigid pruning strategy ensures the actual serviceability of assigned tasks through rigid constraints, avoiding service response delays and quality degradation that could result from assigning new tasks to already overloaded engineers, thus guaranteeing the rationality of task assignment decisions.

[0110] Furthermore, the reliability rating specifically includes steps A701-A703: Step A701: Read the historical service quality data of each candidate engineer based on the engineer profile database. The historical service quality data includes historical first-time repair rate, historical user rating and historical on-time arrival rate. Step A702: Based on the preset time decay function, the historical one-time repair rate, historical user rating and historical on-time arrival rate are respectively weighted by time efficiency to generate time efficiency corrected scores for each dimension. Step A703: The timeliness-corrected scores of each dimension are weighted and aggregated to generate a reliability score.

[0111] It should be noted that in this embodiment, the reliability score refers to a comprehensive score calculated based on the engineer's historical service performance data, used to characterize the engineer's service quality and reliability. A higher score indicates better historical service performance. The engineer profile database is a database within the service platform that stores static attribute information and dynamic historical data for each engineer. It stores a complete service profile for each engineer, including basic personal information, skill tag set, qualification certification information, historical work order records, service quality indicators, and real-time status data. The historical first-time repair rate refers to the proportion of all on-site repair work orders completed by an engineer within a preset time period where the fault was completely resolved on the first visit without requiring a second visit. This indicator reflects the engineer's diagnostic accuracy and comprehensive repair capabilities, and is a core indicator for measuring the engineer's technical level. This embodiment comprehensively evaluates the historical service performance of each candidate engineer through three dimensions: technical level, user satisfaction, and on-time performance.

[0112] Additionally, it should be noted that in this embodiment, the preset time decay function is a mathematical function that reduces the weight coefficient of historical data as the time span increases. This function addresses the time sensitivity issue of historical service quality data. Since an engineer's service capabilities may improve or fluctuate over time, historical data that is too old cannot accurately reflect the engineer's current actual service level; therefore, timeliness correction is necessary. This embodiment uses the time decay function for timeliness correction, enabling the reliability score to more accurately reflect the engineer's recent actual service level. This prevents engineers with excellent historical performance but recent declines from continuously receiving high scores, thus improving the timeliness and accuracy of the score.

[0113] In one possible implementation, the time decay function is an exponential decay function, expressed as w(t) = e^(-λt), where t is the number of days between the data point and the current time, and λ is the decay rate control parameter. This function makes the weight of recent data much higher than that of older data. The system applies the time decay function to the time series data of historical first-time repair rate, historical user rating, and historical on-time arrival rate, respectively. After multiplying each historical data point by the corresponding time-sensitivity decay weight, a weighted average is performed to obtain the time-sensitivity corrected first-time repair rate score, user rating score, and on-time arrival rate score.

[0114] Additionally, it should be noted that in this embodiment, weighted aggregation is the sum of the first repair rate score, user rating score, and on-time arrival rate score after timeliness correction, according to their respective preset aggregation weight coefficients, to form a comprehensive single score value.

[0115] In one possible implementation, the weighted aggregation calculation formula is: Reliability score = α × First repair rate score + β × User rating score + γ × On-time arrival rate score, where α, β, and γ are preset aggregation weight coefficients. The system multiplies the timeliness-corrected scores of each dimension with the corresponding aggregation weight coefficients and then sums them to finally obtain the reliability score of each candidate engineer.

[0116] In one possible implementation, the technical matching degree first extracts the fault type identifier from the structured fault diagnosis report, obtains the set of professional skill tags and the proficiency level corresponding to each skill from the profile of each candidate engineer, then calculates the semantic matching degree between the fault type identifier and the set of skill tags in the preset skill system, and performs weighted correction on the semantic matching degree according to the proficiency level to obtain the skill matching degree.

[0117] Furthermore, referring to Figure 5 The fifth embodiment of the fault diagnosis and dynamic matching method for intelligent devices in this application provides a flowchart, based on the above. Figure 5The embodiment shown further refines the step S40, "calculating a comprehensive score based on the skill matching degree, reliability score, and workload penalty item of each candidate engineer after elimination," including steps A801-A803: Step A801: Analyze the fault complexity level in the structured fault diagnosis report; Step A802: Based on the fault complexity level, dynamically determine the fusion weight coefficients of the skill matching degree, reliability score, and load penalty item. Step A803: The skill matching degree, reliability score and load penalty item are weighted and summed according to the fusion weight coefficient to obtain a comprehensive score.

[0118] It should be noted that in this embodiment, the skill matching degree refers to the quantitative value of the fit between the candidate engineer's professional skills and the current fault repair needs. It is calculated based on the semantic matching degree between the skill tag set and fault type tag in the engineer profile, combined with a proficiency level weighted correction. The higher the skill matching degree, the higher the degree of fit between the engineer's technical expertise and the current fault. The load penalty item refers to the negative adjustment factor generated by the engineer's current on-the-go tasks on the comprehensive score. It is used to reflect the impact of the engineer's current remaining service capacity in the comprehensive score. The higher the current load, the larger the value of the load penalty item, and the more obvious the reduction effect on the comprehensive score. Dynamic determination refers to adaptively adjusting the fusion weight coefficients of each dimension according to different fault complexity levels, so that the focus of the comprehensive score is different in different fault scenarios.

[0119] In one possible implementation, when the fault complexity level is complex or extremely complex, the system increases the fusion weight coefficient of skill matching to 0.7, while reducing the reliability score weight to 0.2 and the load penalty item weight to 0.1. This ensures that in complex fault scenarios, the overall score is primarily determined by skill matching, guaranteeing that high-difficulty faults are preferentially assigned to engineers with the best skill matching.

[0120] This embodiment dynamically determines the fusion weight coefficient based on the fault complexity level, and implements differentiated dispatch strategies under different fault difficulty scenarios. This not only ensures the technical matching requirements of complex faults, but also takes into account the balanced allocation efficiency of simple faults.

[0121] Additionally, it should be noted that in this embodiment, weighted summation refers to multiplying the scores of each dimension by their corresponding fusion weight coefficients and then summing them to obtain the final comprehensive score. In one possible implementation, the comprehensive score is calculated as follows: Comprehensive Score = W1 × Skill Matching Degree + W2 × Reliability Score (normalized) + W3 × (1 - Load Penalty Item), where W1, W2, and W3 are the fusion weight coefficients corresponding to skill matching degree, reliability score, and load penalty item, respectively. Simultaneously, the load penalty item, as a negative adjustment factor, uses its inverse value (i.e., 1 minus the load penalty item) in the weighted summation, ensuring that engineers with lower load and higher remaining service capacity receive higher comprehensive scores.

[0122] The comprehensive score obtained by weighted summation in this embodiment is a comprehensive quantitative expression of the candidate engineer's multi-dimensional adaptability. The higher the score, the higher the engineer's comprehensive adaptability in the current fault scenario.

[0123] Furthermore, the step of sending the structured fault diagnosis report to the terminal of the dispatching object includes A901-A903: Step A901: In response to the on-site arrival signal sent by the dispatch object terminal, load the corresponding digital twin model based on the device identification information.

[0124] It should be noted that, in this embodiment, the dispatching object refers to the engineer determined as the final service provider after comprehensive scoring and ranking; the digital twin model refers to a virtual device model constructed in digital form corresponding to the physical device, which accurately reproduces the structural features, component composition, and spatial assembly relationship of the physical device in three-dimensional space. The digital twin model is stored in association with the device identification information, and the system retrieves and loads the corresponding model file from the digital twin model library through the device identification information.

[0125] In one possible implementation, the on-site arrival signal is triggered by the dispatcher clicking the "Arrival at Site" button on the work terminal. After receiving the signal, the system records the arrival timestamp for subsequent on-time rate statistics and service timeliness monitoring.

[0126] In one possible implementation, the digital twin model is a lightweight 3D model that can be smoothly rendered and displayed on a mobile terminal. The model includes independent identifiers and spatial coordinate information for each detachable component.

[0127] Step A902: Generate a fault identifier on the digital twin model based on the fault location information in the structured fault diagnosis report.

[0128] It should be noted that, in this embodiment, fault location information refers to the specific location description of the fault phenomenon on the device recorded in the structured fault diagnosis report, including which component, area, or interface of the device the fault occurred in. Fault identification refers to the visual graphic element that marks the location of the fault point on the digital twin model.

[0129] In one possible implementation, the fault location information is a structured component identification code that corresponds one-to-one with the component identification in the digital twin model.

[0130] This embodiment converts the text-based fault location description into an intuitive three-dimensional spatial visualization marker, enabling dispatch objects to quickly locate the fault location in complex equipment structures, avoiding misjudgments or wasted time searching for fault points due to unfamiliarity with the equipment structure.

[0131] Step A903: The disassembly and assembly instructions of the fault identifier and the maintenance step sequence in the structured fault diagnosis report are overlaid and displayed on the real-time scanning screen of the device on the dispatching target terminal using augmented reality.

[0132] It should be noted that in this embodiment, the disassembly and assembly instructions refer to the specific operational guidance information corresponding to each step in the maintenance procedure sequence, including technical parameters such as disassembly direction, insertion position, cable sequence, and tightening torque. Augmented reality refers to a display technology that overlays virtually generated digital information onto a real-world screen. Specifically, the dispatching party activates the camera on its work terminal, and the terminal displays the real-time physical image of the equipment captured by the camera on the screen. The system uses image recognition technology to identify the actual equipment and its posture in the image, and performs real-time registration and projection transformation of the fault identifiers and disassembly and assembly instructions in the digital twin model according to the spatial pose of the actual equipment in the image, generating a superimposed image of virtual and real fusion and displaying it on the terminal screen.

[0133] In one possible implementation, the overlaid content includes a highlighted box for the faulty component, a numerical label indicating the component disassembly sequence, a dynamic arrow indicating the disassembly direction, magnified detailed views of key operating parts, and safety warning information.

[0134] This embodiment uses augmented reality to overlay fault indicators and disassembly / reassembly instructions onto the real-time scanning screen of the equipment, achieving precise integration of virtual diagnostic information and physical equipment. This eliminates the need for dispatchers to repeatedly compare paper manuals or two-dimensional views on terminal screens with the actual equipment; instead, they can directly see the overlaid operation instructions in their field of vision, significantly reducing the cognitive load and operational difficulty of maintenance operations and improving maintenance efficiency.

[0135] In one possible implementation, the system receives the actual maintenance data packet returned by the dispatching terminal. The actual maintenance data packet includes the actual time spent at each step, the actual spare parts consumed, the fault description supplemented on-site, diagnostic deviation information and operation photos, as well as the electronic signature and satisfaction evaluation submitted by the user through the client. Then, adaptive optimization is performed based on the actual maintenance data packet, specifically including: first, updating the predicted working time model of the maintenance task for the corresponding equipment model using the actual time spent; then, adjusting the reliability score of the corresponding engineer for this type of fault based on the maintenance quality and the comparison results of the actual time spent and the expected maintenance time; then, performing a structured comparison between the diagnostic deviation information and the structured fault diagnosis report, and when the deviation frequency meets the preset conditions, updating the association between fault phenomena and fault causes in the equipment-fault knowledge graph, and fine-tuning the diagnostic parameters of the multimodal large model.

[0136] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the fault diagnosis and dynamic matching method of an intelligent device in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0137] This application also provides a fault diagnosis and dynamic matching system for intelligent devices. Please refer to [link / reference]. Figure 6 The fault diagnosis and dynamic matching system for the intelligent device includes: Data acquisition module 10 is used to receive multimodal fault data and equipment identification information uploaded by the client; The fault diagnosis module 20 is used to extract features from the multimodal fault data based on a pre-trained multimodal large model to obtain multimodal fault features, and to perform fusion retrieval and reasoning by combining the equipment identification information and the pre-built equipment-fault knowledge graph to generate a structured fault diagnosis report. The load matching module 30 is used to filter and obtain a set of candidate engineers based on user location information, and remove candidate engineers whose real-time load of service resource profile exceeds a preset load threshold from the set of candidate engineers. The dispatching execution module 40 is used to calculate a comprehensive score based on the skill matching degree, reliability score and load penalty item of each candidate engineer after elimination, determine the candidate engineer with the highest comprehensive score as the dispatching object, and send the structured fault diagnosis report to the terminal of the dispatching object.

[0138] This application provides a fault diagnosis and dynamic matching system for intelligent devices, employing a fault diagnosis and dynamic matching method for intelligent devices as described in the above embodiments. This system addresses the technical problem that existing technologies cannot comprehensively utilize multimodal data to achieve accurate fault diagnosis and dynamic matching, resulting in diagnostic accuracy and first-time repair rates that fail to meet user needs. Compared to existing technologies, the beneficial effects of the fault diagnosis and dynamic matching system for intelligent devices provided in this application are the same as those of the fault diagnosis and dynamic matching method for intelligent devices provided in the above embodiments. Furthermore, other technical features of the fault diagnosis and dynamic matching system for intelligent devices are the same as those disclosed in the methods of the above embodiments, and will not be elaborated upon here.

[0139] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. Any equivalent structural transformations made based on the technical concept of this application and the content of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are permitted.

Claims

1. A method for fault diagnosis and dynamic matching of intelligent devices, characterized in that, The method includes: Receive multimodal fault data and device identification information uploaded by the client; Based on a pre-trained multimodal large model, feature extraction is performed on the multimodal fault data to obtain multimodal fault features. These features are then combined with the equipment identification information and a pre-built equipment-fault knowledge graph for fusion retrieval and reasoning to generate a structured fault diagnosis report. A candidate engineer set is obtained by filtering based on user location information, and candidate engineers whose real-time load of service resource profile exceeds a preset load threshold are removed from the candidate engineer set. Based on the skill matching degree, reliability score, and load penalty of each candidate engineer after elimination, a comprehensive score is calculated. The candidate engineer with the highest comprehensive score is determined as the dispatch target, and the structured fault diagnosis report is sent to the terminal of the dispatch target.

2. The fault diagnosis and dynamic matching method for intelligent devices as described in claim 1, characterized in that, Before the step of receiving multimodal fault data and device identification information uploaded by the client, the following steps are included: In response to a maintenance request initiated by a user through a client, multimodal fault data is collected, including at least one of the following: text description of the fault, voice description, equipment images, and operation logs. Obtain the device identification information, and then encapsulate the multimodal fault data and the device identification information and upload them to the service platform.

3. The fault diagnosis and dynamic matching method for intelligent devices as described in claim 2, characterized in that, The step of extracting features from the multimodal fault data based on a pre-trained multimodal large model to obtain multimodal fault features includes: The speech description of the multimodal fault data is subjected to speech recognition and semantic parsing to extract the fault semantic feature vector; Target detection and image classification are performed on the equipment images of the multimodal fault data to identify the fault visual feature vector; The operation logs of the multimodal fault data are analyzed for abnormal events, and fault numerical feature vectors are extracted. The semantic feature vector, visual feature vector, and numerical feature vector are mapped to a unified embedding space for cross-modal feature fusion to generate a multimodal fault feature vector.

4. The fault diagnosis and dynamic matching method for intelligent devices as described in claim 3, characterized in that, The step of combining the device identification information and the pre-built device-fault knowledge graph for fusion retrieval and reasoning to generate a structured fault diagnosis report includes: The root node for the device model in the knowledge graph is determined based on the device identification information. Using the multimodal fault feature vector as the query condition, starting from the root node of the equipment model, a path retrieval is performed in the knowledge graph to match and obtain the corresponding set of fault phenomenon nodes; Starting from the set of fault phenomenon nodes, reasoning and traversal are performed along the predefined association edges in the knowledge graph to obtain the fault cause nodes and the corresponding maintenance solution nodes. Based on the maintenance plan nodes, the maintenance step sequence, the required spare parts list, and the standard operating parameters are extracted, and a structured fault diagnosis report is generated according to the preset data structure.

5. The fault diagnosis and dynamic matching method for intelligent devices as described in claim 1, characterized in that, The step of obtaining the candidate engineer set based on user location information includes: Obtain the geographic coordinates corresponding to the user's location information; The service coverage area is defined with the geographic coordinates as the center and the preset service distance as the radius; The query service platform identifies engineers who are currently available for service and whose service location is within the service coverage area, generating the candidate engineer set.

6. The fault diagnosis and dynamic matching method for intelligent devices as described in claim 1, characterized in that, The step of removing candidate engineers from the candidate engineer set whose real-time load exceeds a preset load threshold for their service resource profile includes: The current number of tasks in progress for each engineer in the candidate engineer set is collected in real time through the load monitoring thread; The corresponding load limit threshold is dynamically configured based on each engineer's seniority level; Remove engineers whose current number of tasks in progress exceeds the load limit threshold from the candidate engineer set.

7. The fault diagnosis and dynamic matching method for intelligent devices as described in claim 1, characterized in that, The reliability rating specifically includes: Based on the engineer profile database, the historical service quality data of each candidate engineer is read. The historical service quality data includes the historical first-time repair rate, historical user rating, and historical on-time arrival rate. Based on a preset time decay function, the historical one-time repair rate, historical user rating and historical on-time arrival rate are respectively time-weighted to generate time-corrected ratings for each dimension. The timeliness-corrected scores of each dimension are weighted and aggregated to generate a reliability score.

8. The fault diagnosis and dynamic matching method for intelligent devices as described in claim 1, characterized in that, The steps for calculating the comprehensive score based on the skill matching degree, reliability score, and workload penalty of each candidate engineer after elimination include: Analyze the fault complexity level in the structured fault diagnosis report; Based on the fault complexity level, the fusion weight coefficients of the skill matching degree, reliability score and load penalty item are dynamically determined. The skill matching degree, reliability score, and load penalty item are weighted and summed according to the fusion weight coefficient to obtain a comprehensive score.

9. The fault diagnosis and dynamic matching method for intelligent devices as described in claim 1, characterized in that, The step of sending the structured fault diagnosis report to the terminal of the dispatching object includes: In response to the on-site arrival signal sent by the dispatching object terminal, the corresponding digital twin model is loaded based on the device identification information; Based on the fault location information in the structured fault diagnosis report, a fault identifier is generated on the digital twin model; The disassembly and assembly instructions of the fault identifier and the maintenance step sequence in the structured fault diagnosis report are overlaid and displayed on the real-time scanning screen of the device on the dispatching terminal using augmented reality.

10. A fault diagnosis and dynamic matching system for intelligent devices, characterized in that, The fault diagnosis and dynamic matching system for the intelligent device includes: The data acquisition module is used to receive multimodal fault data and device identification information uploaded by the client; The fault diagnosis module is used to extract features from the multimodal fault data based on a pre-trained multimodal large model to obtain multimodal fault features, and to perform fusion retrieval and reasoning by combining the equipment identification information and the pre-built equipment-fault knowledge graph to generate a structured fault diagnosis report. The load matching module is used to filter and obtain a set of candidate engineers based on user location information, and remove candidate engineers whose real-time load of service resource profile exceeds a preset load threshold from the set of candidate engineers. The dispatching module is used to calculate a comprehensive score based on the skill matching degree, reliability score, and load penalty item of each candidate engineer after elimination, determine the candidate engineer with the highest comprehensive score as the dispatching object, and send the structured fault diagnosis report to the terminal of the dispatching object.