New energy vehicle fault diagnosis method based on reinforcement learning and score feedback and related equipment

Abstract

The application discloses a new energy vehicle fault diagnosis method based on reinforcement learning and score feedback and related equipment, which can be applied to the technical field of intelligent interaction fault diagnosis. According to the first diagnosis state vector of the new energy vehicle constructed according to the current activated fault code set and the historical maintenance work order, the problem of single fault code modeling can be effectively avoided, then the target single-step atomic diagnosis action is determined from the hybrid action space through the strategy network, and after the multi-source user feedback information is encoded to obtain the multi-source user feedback feature vector and the reinforcement learning reward function value is calculated, the second diagnosis state vector is obtained by combining the data features of the diagnosis state vectors of multiple time steps and fusing, and the fault diagnosis trajectory report is generated after the diagnosis session is terminated, and then the strategy network is optimized combined with the reinforcement learning reward function value, so that the accuracy of the fault troubleshooting path can be effectively improved, and the fault troubleshooting efficiency of the staff is improved, and the fault troubleshooting time is shortened.

Description

Technical Field

[0001] This application relates to the field of intelligent interactive fault diagnosis technology, and in particular to a fault diagnosis method and related equipment for new energy vehicles based on reinforcement learning and scoring feedback. Background Technology

[0002] In current new energy vehicle fault diagnosis, the core method involves reading diagnostic trouble codes (DTCs) through the On-Board Diagnostics (OBD) system interface, combined with repair manuals, experience knowledge bases, or manufacturer technical support for troubleshooting. With the increasing complexity and intelligence of automotive electronic architectures, some intelligent diagnostic systems are attempting to introduce machine learning techniques. These systems train models based on offline historical repair work order data accumulated by manufacturers, outputting possible fault causes or fixed troubleshooting paths to assist repair technicians. However, models trained using existing methods often output fixed troubleshooting paths that cannot be dynamically optimized, potentially leading to incorrect troubleshooting paths and increased troubleshooting time.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main objective of this application is to propose a fault diagnosis method and related equipment for new energy vehicles based on reinforcement learning and scoring feedback, which can effectively improve the accuracy of fault diagnosis paths, thereby improving the efficiency of fault diagnosis by staff and shortening the fault diagnosis time.

[0005] To achieve the above objectives, one aspect of this application proposes a fault diagnosis method for new energy vehicles based on reinforcement learning and scoring feedback, the method comprising the following steps:

[0006] Obtain the current set of activation fault codes and historical repair work orders for new energy vehicles; Construct the first diagnostic status vector of the new energy vehicle based on the currently active fault code set and the historical repair work orders; Based on the first diagnostic state vector, the target single-step atomic diagnostic action is determined from the hybrid action space through the policy network; Obtain multi-source user feedback information corresponding to the execution of the target single-step atomic diagnostic action; The multi-source user feedback information is encoded to obtain a multi-source user feedback feature vector; The reinforcement learning reward function value for the current time step is calculated based on the multi-source user feedback information. The first diagnostic state vector from the previous time step, the diagnostic action embedding vector executed at the current time step, the supplementary data features of the fault diagnosis system collected after the action execution, and the multi-source user feedback feature vector are fused to obtain the second diagnostic state vector; and a fault diagnosis trajectory report is generated after the diagnostic session terminates. The policy network is optimized based on reinforcement learning experience replay and policy optimization algorithms, combined with the fault diagnosis trajectory report and the reinforcement learning reward function value.

[0007] In some embodiments, constructing the first diagnostic state vector of the new energy vehicle based on the currently active fault code set and the historical repair work orders includes: The active fault codes in the current active fault code set are encoded to obtain a fault code encoding vector; Semantic encoding is performed on the text information in the historical maintenance work orders to obtain historical maintenance semantic embedding vectors; The fault code encoding vector and the historical maintenance semantic embedding vector are normalized respectively; The normalized fault code encoding vector and the historical maintenance semantic embedding vector are integrated to obtain the first diagnostic state vector of the new energy vehicle.

[0008] In some embodiments, the hybrid action space includes underlying diagnostic command actions and graphic guidance prompt actions.

[0009] In some embodiments, the multi-source user feedback information includes preset feedback tag information, free text query information, and step utility rating information. Encoding the multi-source user feedback information to obtain a multi-source user feedback feature vector includes: The preset feedback label information is encoded using a one-hot encoding method to obtain a label feature vector; The free text query information is encoded using semantic encoding to obtain a semantic feature vector; The utility rating information of the steps is normalized and mapped to obtain a rating feature vector; The tag feature vector, the semantic feature vector, and the rating feature vector are fused and standardized to obtain the multi-source user feedback feature vector.

[0010] In some embodiments, calculating the reinforcement learning reward function value at the current time step based on the multi-source user feedback information includes: Obtain the execution result reward and execution efficiency reward for the current time step; The reinforcement learning reward function value for the current time step is calculated based on the execution result reward, the execution efficiency reward, and the step utility score information.

[0011] In some embodiments, optimizing the policy network based on reinforcement learning experience replay and policy optimization algorithms, combined with the fault diagnosis trajectory report and the reinforcement learning reward function value, includes: Based on the target activation fault code set corresponding to the current fault diagnosis session, the target trajectory in the fault diagnosis trajectory report corresponding to the current fault diagnosis session is classified and stored in the experience pool, and the average step score of the target trajectory is calculated based on the reinforcement learning reward function value. When the number of target trajectories in the same target activation fault code set in the experience pool is greater than or equal to the trajectory threshold, the target trajectories in the target activation fault code set are filtered according to the average step score, and the policy network is optimized using the near-end policy optimization algorithm.

[0012] In some embodiments, the process of optimizing the policy network using the near-end policy optimization algorithm is as follows: ; In the formula, To optimize the parameters of the policy network; To optimize the parameters of the original policy network; The learning rate; The gradient of the loss function is used to guide the parameters of the policy network to be updated in the target direction.

[0013] To achieve the above objectives, another aspect of this application proposes a fault diagnosis device for new energy vehicles based on reinforcement learning and scoring feedback, the device comprising: The first module is used to obtain the current set of activated fault codes and historical repair work orders of new energy vehicles; The second module is used to construct the first diagnostic status vector of the new energy vehicle based on the currently active fault code set and the historical repair work orders. The third module is used to determine the target single-step atomic diagnostic action from the hybrid action space through a policy network based on the first diagnostic state vector. The fourth module is used to obtain multi-source user feedback information corresponding to the execution of the target single-step atomic diagnostic action; The fifth module is used to encode the multi-source user feedback information to obtain a multi-source user feedback feature vector; The sixth module is used to calculate the reinforcement learning reward function value at the current time step based on the multi-source user feedback information; The seventh module is used to fuse the first diagnostic state vector of the previous time step, the diagnostic action embedding vector executed in the current time step, the supplementary data features of the fault diagnosis system collected after the action is executed, and the multi-source user feedback feature vector to obtain the second diagnostic state vector; and to generate a fault diagnosis trajectory report after the diagnostic session is terminated. The eighth module is used to optimize the policy network based on reinforcement learning experience replay and policy optimization algorithms, combined with the fault diagnosis trajectory report and the reinforcement learning reward function value.

[0014] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0016] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0017] The embodiments of this application include at least the following beneficial effects: This application provides a new energy vehicle fault diagnosis method and related equipment based on reinforcement learning and scoring feedback. This scheme obtains the current active fault code set and historical repair work orders of the new energy vehicle, and then constructs an initial diagnostic state vector of the new energy vehicle as the first diagnostic state vector based on the current active fault code set and historical repair work orders. This effectively avoids the problem of single fault code modeling. Next, based on the first diagnostic state vector, a target single-step atomic diagnostic action is determined from the hybrid action space through a policy network, and multi-source user feedback information corresponding to the execution of the target single-step atomic diagnostic action is obtained. Then, the multi-source user feedback information is encoded to obtain a multi-source user feedback feature vector. The reinforcement learning reward function value for the current time step is calculated based on multi-source user feedback information. The second diagnostic state vector is obtained by fusing the first diagnostic state vector from the previous time step, the embedding vector of the diagnostic action executed at the current time step, the supplementary data features of the fault diagnosis system collected after the action is executed, and the feature vector of multi-source user feedback. After the diagnostic session terminates, a fault diagnosis trajectory report is generated. Based on reinforcement learning experience replay and policy optimization algorithms, the policy network is optimized by combining the fault diagnosis trajectory report and the reinforcement learning reward function value. This allows for dynamic updates to the policy network and dynamic adjustments to the fault investigation path based on real-time conditions, effectively improving the accuracy of the fault investigation path and thus increasing the efficiency of fault investigation by staff and shortening the fault investigation time. Attached Figure Description

[0018] Figure 1 This is a flowchart of a new energy vehicle fault diagnosis method based on reinforcement learning and scoring feedback provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of the new energy vehicle fault diagnosis device based on reinforcement learning and scoring feedback provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0020] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0021] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0023] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0024] On-Board Diagnostics (OBD) for new energy vehicles is a standardized system used to monitor the operating status of key vehicle components in real time and diagnose faults. The OBD system uses sensors to monitor components related to emissions or powertrain systems (such as the engine, exhaust aftertreatment system, or, in new energy vehicles, the battery, motor, and electronic control system) in real time and sends the data to the vehicle's computer. If an anomaly is detected, the system stores a fault code (DTC) and alerts the driver via a warning light on the dashboard. Repair personnel can then quickly locate the problem by reading the code using specialized equipment.

[0025] In new energy vehicles, OBD-II (On-Board Diagnostics-II) is a core technology used to monitor and diagnose vehicle faults. OBD-II is an on-board diagnostic system standard jointly developed by the International Organization for Standardization (ISO) and the Society of Automotive Engineers (SAE). It is mainly used to monitor vehicle emissions and drive performance-related faults, and its hardware interface (DLC) and communication protocols (such as KWP2000, CAN-BUS, etc.) are unified.

[0026] In current technologies, the core approach to fault diagnosis in new energy vehicles involves reading fault codes through the interface of the onboard automatic diagnostic system and then conducting troubleshooting based on repair manuals, experience knowledge bases, or manufacturer technical support. With the increasing complexity and intelligence of automotive electronic architectures, some intelligent diagnostic systems are attempting to introduce machine learning techniques. These systems train models based on offline historical repair work order data accumulated by manufacturers, outputting possible fault causes or fixed troubleshooting paths to assist repair technicians. However, models trained using existing methods tend to output fixed troubleshooting paths that cannot be dynamically optimized, potentially leading to incorrect troubleshooting paths and increased troubleshooting time.

[0027] In view of this, this application provides a new energy vehicle fault diagnosis method and related equipment based on reinforcement learning and scoring feedback, which can effectively improve the accuracy of fault diagnosis path, thereby improving the fault diagnosis efficiency of staff and shortening the fault diagnosis time.

[0028] The new energy vehicle fault diagnosis method based on reinforcement learning and scoring feedback provided in this application relates to the field of intelligent interactive fault diagnosis technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the new energy vehicle fault diagnosis method based on reinforcement learning and scoring feedback, but is not limited to the above forms.

[0029] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0030] The embodiments of this application will be described in detail below with reference to the accompanying drawings: Figure 1 This is an optional flowchart of the new energy vehicle fault diagnosis method based on reinforcement learning and scoring feedback provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S110 to S180: Step S110: Obtain the current set of activated fault codes and historical repair work orders for the new energy vehicle; Step S120: Construct the first diagnostic state vector of the new energy vehicle based on the currently activated fault code set and the historical repair work orders; Step S130: Based on the first diagnostic state vector, determine the target single-step atomic diagnostic action from the hybrid action space through the policy network; Step S140: Obtain multi-source user feedback information corresponding to the execution of the target single-step atomic diagnostic action; Step S150: Encode the multi-source user feedback information to obtain the multi-source user feedback feature vector; Step S160: Calculate the reinforcement learning reward function value for the current time step based on multi-source user feedback information; Step S170: The first diagnostic state vector of the previous time step, the embedded vector of the diagnostic action executed in the current time step, the supplementary data features of the fault diagnosis system collected after the action is executed, and the feature vector of multi-source user feedback are fused to obtain the second diagnostic state vector; and a fault diagnosis trajectory report is generated after the diagnostic session is terminated. Step S180: Optimize the policy network based on reinforcement learning experience playback and policy optimization algorithms, combined with fault diagnosis trajectory reports and reinforcement learning reward function values.

[0031] It is understandable that the currently active fault code set D of a new energy vehicle can be read through the vehicle's standard OBD-II interface. The expression for the currently active fault code set D is as follows: ; In the formula, This represents the current number of active fault codes. For a single standard fault code; when no active fault code is found. .

[0032] Historical repair work orders carry textual information such as historical fault descriptions and repair measures. In this embodiment, after obtaining the current active fault code set and historical repair work orders, the active fault codes in the current active fault code set D are encoded to obtain fault code encoding vectors. Simultaneously, the textual information in the historical repair work orders is semantically encoded to obtain historical repair semantic embedding vectors. Then, the fault code encoding vectors and historical repair semantic embedding vectors are normalized respectively, and finally integrated to obtain the first diagnostic state vector of the new energy vehicle. Specifically, this embodiment can map the active fault codes in the current active fault code set into vector features using a pre-trained embedding model, and then generate a unified representation using feature aggregation. This information is used to participate in subsequent diagnostic status modeling processes. Simultaneously, a semantic encoding model is used to semantically encode the textual information in historical maintenance work orders, generating structured semantic embedding vectors. This is to ensure that textual information can participate in feature fusion.

[0033] It is understandable that, in this embodiment, after encoding the text information in the currently active fault code set and the historical maintenance work orders, the fault code encoding vector is... and historical maintenance semantic embedding vector Normalization is performed separately to eliminate differences in numerical scale between different feature sources; then, the feature fusion module is used. The two types of normalized feature vectors are integrated to generate an initial diagnostic state vector representing the current diagnostic scenario. This serves as the first diagnostic state vector for new energy vehicles. Among them, the feature fusion module... A fully connected network or a lightweight fusion architecture can be used to achieve a unified representation of multi-source features and output a standardized state vector with fixed dimensions, thereby ensuring stable input to the subsequent policy network.

[0034] It is understandable that in this embodiment, the first diagnostic state vector is obtained. Then, it can be done through a preset policy network. Select the unique and optimal single-step atomic diagnostic action from the mixed action space. This serves as the target single-step atomic diagnostic action in the current fault diagnosis process. Specifically, the target single-step atomic diagnostic action... The selection formula is as follows: ; In the formula, For hybrid motion space Any candidate action in the list.

[0035] Specifically, hybrid action space This includes, but is not limited to, two core actions: low-level diagnostic commands and graphical guidance prompts. Both types of core actions have been engineered to meet the execution requirements of the maintenance terminal. (1) Low-level diagnostic commands: Directly call the OBD-II interface communication protocol to perform data reading or status detection, and the command execution results are sent back to the diagnostic system in real time; (2) Graphic and text guidance prompts: Visual operation guides for maintenance technicians.

[0036] It is understood that, in this embodiment, after determining the target single-step atomic diagnostic action at the current time step, the user can perform fault diagnosis on the new energy vehicle based on the target single-step atomic diagnostic action, and can input feedback information in the hybrid interactive interface provided by the maintenance terminal (PAD / PC). This embodiment obtains multi-source user feedback information by encoding the multi-source user feedback information corresponding to the execution of the target single-step atomic diagnostic action, thus obtaining a multi-source user feedback feature vector. Specifically, the multi-source user feedback information includes, but is not limited to, three types of feedback information: preset feedback label information, free text query information, and step utility rating information. The preset feedback label information includes, but is not limited to, the user selecting a corresponding identifier from the three types of feedback: fault troubleshooting, invalid action, and execution anomaly. The free text query information may include, but is not limited to, the user inputting a natural language question, and the query type being determined by an intent classification model. The step utility rating information includes, but is not limited to, the user's evaluation of the practicality of the current action mapped to a numerical value. .

[0037] Understandably, in order to achieve unified modeling of multi-source user feedback information, this embodiment uses preset feedback tag information. Free text query information and step-by-step utility rating information Feature encoding is performed separately and then fused to generate a standardized multi-source user feedback feature vector. Specifically, this embodiment uses one-hot encoding to encode the preset feedback tag information. Encode the tag feature vector; use semantic encoding to process the free text query information. Encoding yields a fixed-dimensional semantic feature vector; step utility scoring information is then processed. After performing normalization mapping to eliminate dimensional differences, the rating feature vector is obtained. Subsequently, the label feature vector, semantic feature vector, and rating feature vector are fused and standardized to form a multi-source user feedback feature vector. The multi-source user feedback feature vector in this embodiment This is used to characterize the comprehensive features of user interaction feedback at the current time step, and serves as the input for subsequent reward calculation and diagnostic status update processes.

[0038] It is understandable that, in the process of calculating the reinforcement learning reward function value for the current time step based on multi-source user feedback information in this embodiment, the reinforcement learning reward function value for the current time step can be calculated based on the execution result reward and execution efficiency reward for the current time step, and then on the execution result reward, execution efficiency reward, and step utility score information. The calculation process is as follows: ; In the formula, Rewards for performance results Execution efficiency reward The reward weights are such that the sum is 1, which is used to balance the impact of rewards from different dimensions.

[0039] Specifically, the current time step Reinforcement learning rewards The feedback will be sent to the policy network in real time, providing a basis for the policy network to generate the next optimal diagnostic action; at the same time, this reward It is fully preserved and used as a core screening criterion for subsequent high-value diagnostic trajectories.

[0040] It is understood that, after completing the execution of the action corresponding to the current time step, this embodiment obtains the first diagnostic state vector of the previous time step, the embedded vector of the diagnostic action executed at the current time step, and the supplementary data features of the fault diagnosis system collected after the action execution. These are then fused with multi-source user feedback feature vectors to obtain the second diagnostic state vector. A fault diagnosis trajectory report is generated after the diagnostic session terminates. Specifically, this embodiment achieves continuity and closed-loop control of the diagnostic process by iterating the diagnostic state based on the structure and user feedback after the execution of the corresponding diagnostic action at each time step and defining the session termination boundary. This provides complete and effective interactive trajectory data support for subsequent network strategy optimization.

[0041] Specifically, in this embodiment, when updating the state vector, the diagnostic state vector can be dynamically updated based on the state transition mechanism of the Markov decision process in reinforcement learning. This is done after the diagnostic action corresponding to the current time step is executed. Specifically, in this embodiment, the first diagnostic state vector from the previous time step is updated. The embedding vector of the diagnostic action executed at the current time step Features of newly acquired OBD supplementary data after the action is executed and user feedback feature vectors The fusion process generates a new diagnostic state vector. This serves as the second diagnostic state vector. The update process in this embodiment can be achieved through a state update function. The newly added information from multiple sources is uniformly integrated and standardized to ensure the updated diagnostic state vector. It can fully reflect the current diagnostic process and interactive feedback results, and serve as the input basis for the subsequent strategy network to generate the next diagnostic action.

[0042] It is understandable that, in order to avoid invalid iterations and ensure diagnostic efficiency when determining the termination of a diagnostic session, this embodiment can set a clear session termination boundary based on closed-loop control theory, and terminate the current diagnostic session when any of the following conditions are met: (1) User feedback indicates that the troubleshooting goal has been achieved, i.e., the fault has been confirmed to be eliminated; (2) If feedback is received for three consecutive steps that the diagnostic action is invalid or cannot be executed, it is determined that the current strategy is not adaptable to the fault. (3) The number of interactive steps reaches the preset limit to avoid infinite loop.

[0043] This embodiment automatically generates a standardized fault diagnosis trajectory report after the diagnostic session terminates. This report includes, but is not limited to, information such as core fault information, action execution sequence, user feedback records, and scoring statistics, thus providing a complete data sample for subsequent strategy fine-tuning.

[0044] It is understood that, after obtaining the fault diagnosis trajectory report, this embodiment, based on reinforcement learning experience playback and policy optimization algorithms, combines the scoring data of the complete diagnostic session trajectory in the fault diagnosis trajectory report with the reinforcement learning reward function value to fine-tune and optimize the policy network, thereby realizing the online optimization process of the policy network and improving the accuracy of action recommendation and diagnostic efficiency in subsequent similar scenarios. Specifically, when optimizing the policy network, this embodiment can classify and store the target trajectory in the fault diagnosis trajectory report corresponding to the current fault diagnosis session into an experience pool according to the target activation fault code set corresponding to the current fault diagnosis session, thereby realizing centralized management of similar fault data, and calculating the average step score of the target trajectory according to the reinforcement learning reward function value to quantify the trajectory value. The formula for calculating the average step score is as follows: ; In the formula, The average step score for a single session trajectory, This represents the total number of interaction steps in the session. This represents the reinforcement learning reward function value at the current time step t.

[0045] This embodiment calculates the average step score for each individual session trajectory and then performs high-value trajectory sampling and policy network fine-tuning optimization. Specifically, when the number of target trajectories in the same target activation fault code set in the experience pool is greater than or equal to the trajectory threshold, the target trajectories in the target activation fault code set are prioritized for sampling and filtering based on the average step score. High-value trajectories with an average step score of not less than 3.5 are preferentially selected as reference data for subsequent policy network fine-tuning optimization. Then, the Proximal Policy Optimization (PPO) algorithm is used to perform online fine-tuning optimization of the policy network to ensure the stability and effectiveness of policy network updates. The parameter optimization process of the policy network in this embodiment is as follows: ; In the formula, To optimize the parameters of the policy network; To optimize the parameters of the original policy network; The learning rate; The gradient of the loss function is used to guide the parameters of the policy network to be updated in the target direction.

[0046] In this embodiment, after fine-tuning and optimizing the policy network, the policy network can automatically prioritize the diagnostic actions in high-scoring trajectories when encountering the same or similar sets of fault codes in the future. This can gradually reduce invalid guidance, optimize the diagnostic path, and achieve the goal of continuously evolving the diagnostic strategy according to the actual use scenario.

[0047] As can be seen from the above, the method of this application integrates fault codes and textual semantic features from historical maintenance work orders through multi-source data fusion to construct a comprehensive initial diagnostic state, which can significantly improve the accuracy of fault characterization and reduce misjudgments caused by incomplete information; by relying on reinforcement learning policy network to output single-step optimal atomic diagnostic actions and dynamically updating the diagnostic state with feedback, the diagnostic process can be dynamically adapted, reducing the threshold for novice operation and shortening the fault troubleshooting cycle; by constructing a complete closed loop from feedback collection to policy fine-tuning, the policy network has self-evolution capabilities, which can improve the success rate of diagnosing similar faults in the long term and reduce invalid guidance; by clearly controlling the session termination boundary, invalid iterations are avoided, further ensuring diagnostic efficiency and improving the interactive experience of maintenance technicians.

[0048] Please see Figure 2 This application also provides a fault diagnosis device for new energy vehicles based on reinforcement learning and scoring feedback. The device includes: The first module is used to obtain the current set of activated fault codes and historical repair work orders of new energy vehicles; The second module is used to construct the first diagnostic state vector of new energy vehicles based on the currently active fault code set and historical repair work orders. The third module is used to determine the target single-step atomic diagnostic action from the hybrid action space through the policy network based on the first diagnostic state vector. The fourth module is used to obtain multi-source user feedback information corresponding to the execution of the target single-step atomic diagnostic action; The fifth module is used to encode multi-source user feedback information to obtain multi-source user feedback feature vectors; The sixth module is used to calculate the reinforcement learning reward function value for the current time step based on multi-source user feedback information; The seventh module is used to fuse the first diagnostic state vector of the previous time step, the diagnostic action embedding vector executed in the current time step, the supplementary data features of the fault diagnosis system collected after the action is executed, and the multi-source user feedback feature vector to obtain the second diagnostic state vector; and to generate a fault diagnosis trajectory report after the diagnostic session is terminated. The eighth module is used to optimize the policy network based on reinforcement learning experience replay and policy optimization algorithms, combined with fault diagnosis trajectory reports and reinforcement learning reward function values.

[0049] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0050] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0051] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0052] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0053] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0054] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0055] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0056] This application provides a new energy vehicle fault diagnosis method and related equipment based on reinforcement learning and scoring feedback. After obtaining the current active fault code set and historical repair work orders of the new energy vehicle, an initial diagnostic state vector is constructed based on these data as the first diagnostic state vector. This effectively avoids the problem of modeling a single fault code. Then, based on the first diagnostic state vector, a target single-step atomic diagnostic action is determined from the hybrid action space through a policy network. Multi-source user feedback information corresponding to the execution of the target single-step atomic diagnostic action is obtained. This multi-source user feedback information is then encoded to obtain a multi-source user feedback feature vector. Finally, based on the multi-source user feedback... The system calculates the reinforcement learning reward function value for the current time step, fuses the first diagnostic state vector from the previous time step, the embedding vector of the diagnostic action executed at the current time step, the supplementary data features of the fault diagnosis system collected after the action execution, and the multi-source user feedback feature vectors to obtain the second diagnostic state vector, and generates a fault diagnosis trajectory report after the diagnostic session terminates. Based on reinforcement learning experience replay and policy optimization algorithms, the system optimizes the policy network by combining the fault diagnosis trajectory report and the reinforcement learning reward function value, thereby dynamically updating the policy network and adjusting the fault investigation path according to real-time conditions, effectively improving the accuracy of the fault investigation path, thereby improving the fault investigation efficiency of staff and shortening the fault investigation time.

[0057] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0058] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0059] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0060] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0061] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0062] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0063] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0064] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0066] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A fault diagnosis method for new energy vehicles based on reinforcement learning and rating feedback, characterized in that, The method includes the following steps: Obtain the current set of activation fault codes and historical repair work orders for new energy vehicles; Construct the first diagnostic status vector of the new energy vehicle based on the currently active fault code set and the historical repair work orders; Based on the first diagnostic state vector, the target single-step atomic diagnostic action is determined from the hybrid action space through the policy network; Obtain multi-source user feedback information corresponding to the execution of the target single-step atomic diagnostic action; The multi-source user feedback information is encoded to obtain a multi-source user feedback feature vector; The reinforcement learning reward function value for the current time step is calculated based on the multi-source user feedback information. The first diagnostic state vector from the previous time step, the diagnostic action embedding vector executed at the current time step, the supplementary data features of the fault diagnosis system collected after the action execution, and the multi-source user feedback feature vector are fused to obtain the second diagnostic state vector; and a fault diagnosis trajectory report is generated after the diagnostic session terminates. The policy network is optimized based on reinforcement learning experience replay and policy optimization algorithms, combined with the fault diagnosis trajectory report and the reinforcement learning reward function value.

2. The method according to claim 1, characterized in that, The step of constructing the first diagnostic status vector of the new energy vehicle based on the currently active fault code set and the historical repair work orders includes: The active fault codes in the current active fault code set are encoded to obtain a fault code encoding vector; Semantic encoding is performed on the text information in the historical maintenance work orders to obtain historical maintenance semantic embedding vectors; The fault code encoding vector and the historical maintenance semantic embedding vector are normalized respectively; The normalized fault code encoding vector and the historical maintenance semantic embedding vector are integrated to obtain the first diagnostic state vector of the new energy vehicle.

3. The method according to claim 1, characterized in that, The hybrid action space includes underlying diagnostic command actions and graphic guidance prompt actions.

4. The method according to claim 1, characterized in that, The multi-source user feedback information includes preset feedback tag information, free text query information, and step utility rating information. Encoding the multi-source user feedback information to obtain a multi-source user feedback feature vector includes: The preset feedback label information is encoded using a one-hot encoding method to obtain a label feature vector; The free text query information is encoded using semantic encoding to obtain a semantic feature vector; The utility rating information of the steps is normalized and mapped to obtain a rating feature vector; The tag feature vector, the semantic feature vector, and the rating feature vector are fused and standardized to obtain the multi-source user feedback feature vector.

5. The method according to claim 4, characterized in that, The step of calculating the reinforcement learning reward function value at the current time step based on the multi-source user feedback information includes: Obtain the execution result reward and execution efficiency reward for the current time step; The reinforcement learning reward function value for the current time step is calculated based on the execution result reward, the execution efficiency reward, and the step utility score information.

6. The method according to claim 1, characterized in that, The reinforcement learning experience replay and policy optimization algorithm, which optimizes the policy network by combining the fault diagnosis trajectory report and the reinforcement learning reward function value, includes: Based on the target activation fault code set corresponding to the current fault diagnosis session, the target trajectory in the fault diagnosis trajectory report corresponding to the current fault diagnosis session is classified and stored in the experience pool, and the average step score of the target trajectory is calculated based on the reinforcement learning reward function value. When the number of target trajectories in the same target activation fault code set in the experience pool is greater than or equal to the trajectory threshold, the target trajectories in the target activation fault code set are filtered according to the average step score, and the policy network is optimized using the near-end policy optimization algorithm.

7. The method according to claim 6, characterized in that, The process of optimizing the policy network using the near-end policy optimization algorithm is as follows: ； In the formula, To optimize the parameters of the policy network; To optimize the parameters of the policy network before implementation; The learning rate; The gradient of the loss function is used to guide the parameters of the policy network to be updated in the target direction.

8. A fault diagnosis device for new energy vehicles based on reinforcement learning and rating feedback, characterized in that, The device includes: The first module is used to obtain the current set of activated fault codes and historical repair work orders of new energy vehicles; The second module is used to construct the first diagnostic status vector of the new energy vehicle based on the currently active fault code set and the historical repair work orders. The third module is used to determine the target single-step atomic diagnostic action from the hybrid action space through a policy network based on the first diagnostic state vector. The fourth module is used to obtain multi-source user feedback information corresponding to the execution of the target single-step atomic diagnostic action; The fifth module is used to encode the multi-source user feedback information to obtain a multi-source user feedback feature vector; The sixth module is used to calculate the reinforcement learning reward function value at the current time step based on the multi-source user feedback information; The seventh module is used to fuse the first diagnostic state vector of the previous time step, the diagnostic action embedding vector executed in the current time step, the supplementary data features of the fault diagnosis system collected after the action is executed, and the multi-source user feedback feature vector to obtain the second diagnostic state vector; and to generate a fault diagnosis trajectory report after the diagnostic session is terminated. The eighth module is used to optimize the policy network based on reinforcement learning experience replay and policy optimization algorithms, combined with the fault diagnosis trajectory report and the reinforcement learning reward function value.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images