Vehicle fault visual guiding method, system and device and storage medium
By generating fault indicators when a vehicle malfunctions and using a generative artificial intelligence model to generate dynamic visualization videos, the problem of the lack of real-time dynamic visualization in existing vehicle fault indication methods is solved, thereby improving user experience and fault handling efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG LINGAI FUTURE TECHNOLOGY CO LTD
- Filing Date
- 2025-12-31
- Publication Date
- 2026-05-05
AI Technical Summary
In existing technologies, vehicle fault indication methods lack real-time dynamic visualization, resulting in a poor user experience and insufficient explanatory power, operability, personalization, and interactivity.
By generating fault indicators when a vehicle malfunctions, and using a pre-trained target video generation model, dynamic and visual video prompts are generated based on the fault indicators, user category, and style. Combined with real-time sensor data and generative artificial intelligence models, real-time dynamic and visual reminders of vehicle malfunctions are achieved.
It enables real-time dynamic visual alerts for vehicle malfunctions, enhancing the user experience, and provides personalized operation guidance videos, thereby improving user interactivity and fault handling efficiency.
Smart Images
Figure CN121982801A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of vehicle technology, specifically to vehicle fault visualization guidance methods, systems, devices, and storage media. Background Technology
[0002] When a vehicle malfunctions, timely alerts help users quickly understand and address the problem. However, current fault indication technologies typically use static icons or text prompts (such as dashboard warning lights). These methods fail to provide real-time, dynamic visualization of the fault, impacting the user experience. Summary of the Invention
[0003] This invention provides a vehicle fault visualization guidance method, system, device, and storage medium to achieve real-time dynamic visualization of faults and improve user experience.
[0004] Firstly, a vehicle fault visualization guidance method is provided, including the following steps: If the vehicle's current operating status data meets the preset fault judgment conditions, the vehicle is determined to have a fault, and a target fault identifier corresponding to the fault is generated. Based on the target fault identifier, the vehicle's user category label, and the preset mapping relationship between fault identifiers and operation instructions, determine the target prompt words for the target fault identifier; The target style is determined based on user category tags and the preset correspondence between user and style tags; Using a target video generation model, target videos corresponding to target fault identifiers are generated based on target prompts and target styles.
[0005] In some embodiments, the preset fault identifier and operation guide mapping relationship includes: a preset fault identifier instruction association rule base, a preset basic operation instruction base, and a preset user profile adaptation rule base; Based on the target fault identifier, the vehicle's user category label, and the preset mapping relationship between fault identifiers and operation instructions, the target prompt words for the target fault identifier are determined, including: The target natural language paragraph is determined based on the target fault identifier, the preset fault identifier instruction association rule base, and the preset basic operation instruction base. Based on user category tags and a preset user profile adaptation rule library, the target natural language paragraph is converted into target prompt words.
[0006] In some embodiments, determining the target natural language paragraph based on the target fault identifier, a preset fault identifier instruction association rule base, and a preset basic operation instruction base includes: Based on the target fault identifier, search the preset fault identifier instruction association rule library to determine the target instruction rule; Based on the target instruction rules and the preset basic operation instruction library, the target fault identifier is translated into the target natural language paragraph.
[0007] In some embodiments, the target video generation model includes an encoder, an adapter, and a diffusion model; using the target video generation model, a target video corresponding to the target fault identifier is generated based on the target cue words and the target style, including: The target prompt words are parsed using an encoder to extract conditional embedding vectors; The target weights corresponding to the target style are configured through the adapter to obtain the style feature vector; The conditional embedding vector and style feature vector are denoised iteratively using a diffusion model until the preset number of iterations is met, at which point the target video is generated.
[0008] In some embodiments, the target video generation model further includes a temporal attention layer and a temporal convolutional layer; the method further includes: Temporal attention layers and temporal convolutional layers are used to process the conditional embedding vectors and style feature vectors in the temporal dimension.
[0009] In some embodiments, the output target video includes: The target video is compressed and encoded, and then the encoded target video is cut into multiple video segments using a preset streaming media transmission protocol; Output each video clip.
[0010] In some embodiments, the target fault identifier includes at least: the system to which the fault belongs, the fault type, and the fault number.
[0011] Secondly, this application also provides a vehicle fault visualization guidance system, comprising: The first determining module is used to determine that a vehicle has malfunctioned if the vehicle's current operating status data meets preset fault judgment conditions. The first generation module is used to generate the target fault identifier corresponding to the fault. The second determining module is used to determine the target prompt word of the target fault identifier based on the target fault identifier, the user category label of the vehicle, and the preset mapping relationship between fault identifier and operation guide; The third determination module is used to determine the target style based on user category tags and the preset correspondence between user and style tags; The second generation module is used to generate target videos corresponding to target fault identifiers based on target prompt words and target style using the target video generation model. The output module is used to output the target video.
[0012] Thirdly, this application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the computer program, when executed by the processor, implements the method described in the first aspect.
[0013] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps of the method described in the first aspect.
[0014] Beneficial Effects: This application provides a vehicle fault visualization guidance method, system, device, and storage medium. The vehicle fault visualization guidance method includes: determining that a vehicle fault has occurred when the vehicle's current operating status data meets preset fault judgment conditions, and generating a target fault identifier corresponding to the fault; determining a target prompt word for the target fault identifier based on the target fault identifier, the vehicle's user category label, and a preset mapping relationship between fault identifiers and operation guidelines; determining a target style based on the user category label and a preset correspondence between user and style labels; and generating a target video corresponding to the target fault identifier using a target video generation model based on the target prompt word and target style. The target video is then output. The vehicle fault visualization guidance method provided in this application generates a fault identifier when a vehicle fault occurs, generates corresponding prompt words based on the fault identifier and the preset mapping relationship between fault identifiers and operation guidelines, and generates a corresponding target style. A pre-trained target video generation model is then used to generate a video based on the target prompt word and target style, thereby achieving timely and dynamic visual reminders of vehicle faults and improving user experience. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a flowchart of a vehicle fault visualization guidance method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the overall process of a vehicle fault visualization guidance method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the principle structure of a vehicle fault visualization guidance system provided in the embodiments of this application. Detailed Implementation
[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0018] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.
[0019] "A and / or B" includes the following three combinations: A only, B only, and a combination of A and B.
[0020] The use of "applies to" or "configured to" in this application implies open and inclusive language, which does not exclude the applicability to or configuration to devices performing additional tasks or steps. Additionally, the use of "based on" implies openness and inclusivity, because processes, steps, calculations, or other actions "based on" one or more of the stated conditions or values may in practice be based on additional conditions or values beyond those stated.
[0021] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.
[0022] The applicant's research revealed that current intelligent vehicles generally use static icons or text prompts (such as dashboard malfunction lights) for fault indication. However, this method has the following shortcomings: 1. Poor explanatory power: Users find it difficult to accurately understand the cause of the malfunction.
[0023] Second, it lacks operability: it cannot guide users to complete actual repair operations.
[0024] Third, lack of personalization: It is unable to dynamically adjust the prompts based on different drivers' experience and environmental conditions.
[0025] Fourth, poor interactivity: It lacks humanized interaction and makes it difficult to improve the user experience.
[0026] 5. Static icon and text prompt solutions can only push fixed text descriptions and lack visual effects. Local fixed video prompt solution: relies on a local video library and cannot achieve real-time dynamic generation and user profile adaptation.
[0027] In view of this, embodiments of this application provide a vehicle fault visualization guidance method, system, device, and storage medium. The vehicle fault visualization guidance method provided in this application generates a fault identifier when a vehicle fault occurs, generates corresponding prompt words based on the fault identifier and a preset fault identifier-operation guidance mapping relationship, and generates a corresponding target style. A pre-trained target video generation model is then used to generate a video based on the target prompt words and target style, thereby achieving timely dynamic visualization reminders of vehicle faults and improving the user experience.
[0028] Figure 1 This is a flowchart illustrating a vehicle fault visualization guidance method provided in this embodiment. On one hand, this embodiment provides a vehicle fault visualization guidance method applicable to vehicle control systems, enabling real-time dynamic visualization of vehicle faults to improve user experience. This method can be executed by a vehicle fault visualization guidance system, which can be implemented in software and / or hardware and can be configured in the processor or controller of the vehicle control system. Please refer to... Figure 1 The method includes the following steps: Step 110: If the current operating status data of the vehicle meets the preset fault judgment conditions, determine that the vehicle has a fault and generate the target fault identifier corresponding to the fault.
[0029] The current operating status data includes vehicle engine parameters, vehicle posture data, environmental perception data, and environmental condition data (such as in-vehicle temperature and humidity data).
[0030] The current operating status data is obtained in real time through onboard sensors. For example, vehicle engine parameters can be obtained in real time through on-board diagnostics (OBD). Vehicle attitude data can be obtained in real time through inertial measurement unit (IMU). Environmental perception data can be obtained in real time through cameras. In-vehicle temperature and humidity data can be obtained in real time through in-vehicle temperature and humidity sensors.
[0031] In some embodiments, the target fault identifier includes at least: the system to which the fault belongs, the fault type, and the fault number.
[0032] The target fault is identified by a fault code. A fault code includes the system to which the fault belongs, the fault type, and a specific number. For example, fault code P0xxx indicates a powertrain system fault, fault code C0xxx indicates a chassis system fault, fault code B0xxx indicates a body system fault, and fault code U0xxx indicates a network communication fault.
[0033] The preset fault judgment conditions include preset judgment logic and threshold ranges, which are related to the specific parameters of the current operating status data. For example, when the current operating status data is in-vehicle temperature and humidity data, the corresponding preset fault judgment condition is: if the in-vehicle temperature and humidity exceed the temperature and humidity threshold range, then the temperature and humidity are abnormal. For example, if the current operating status data is engine parameters (e.g., power), the corresponding preset fault judgment condition is: if the engine power exceeds the preset power threshold range, then the engine power is abnormal.
[0034] For example, taking current operating status data including vehicle engine parameters, vehicle posture data, environmental perception data, and environmental condition data as an example, the specific implementation process of determining that a vehicle fault has occurred and generating a target fault identifier corresponding to the fault when the current operating status data meets preset fault judgment conditions is as follows: Each electronic control unit (ECU) of the vehicle (such as the engine ECU, transmission ECU, vehicle stability system ECU, etc.) acquires raw, continuous physical signals (such as voltage, resistance, and frequency) in real time through connected sensors (OBD sensors monitor engine parameters, IMU monitors vehicle posture, cameras perceive the environment, temperature and humidity sensors monitor environmental conditions, etc.) and converts them into digital signals. The ECU pre-stores preset fault judgment conditions (including preset judgment logic and threshold ranges) corresponding to each operating status data. The system compares the real-time calculated operating parameters with these preset fault judgment conditions. When one or more parameters continuously and repeatedly exceed the threshold range, or when the logical relationship between parameters is contradictory, the ECU determines that the subsystem or component is abnormal (i.e., a fault has occurred) and triggers an internal fault flag (i.e., the target fault identifier). Once the fault flag is triggered, the ECU will generate a Diagnostic Trouble Code (DTC) according to the format specified by the recommended specifications for fault codes of the On-Board Diagnostic (OBD) system (e.g., SAE J2012), such as a 5-digit standard fault code.
[0035] Step 120: Determine the target prompt word for the target fault identifier based on the target fault identifier, the vehicle's user category label, and the preset fault identifier and operation guide mapping relationship.
[0036] The vehicle's user category tag indicates the type of user, such as novice driver, experienced (or seasoned) driver, or probationary driver. This user category tag can be set and entered in real-time based on the actual users of the vehicle at any given time.
[0037] The preset fault identifier and operation guide mapping relationship is a preset fault code operation guide mapping library.
[0038] Specifically, after determining that a vehicle has malfunctioned and generating a corresponding target fault identifier, the target fault code is translated into a natural language prompt by calling the fault code operation guidance mapping library. Customized keywords (i.e., target prompt words) are generated by combining environmental factors (such as, but not limited to, the weather conditions at the time, geographical location, and other external conditions perceived by various sensors when the fault occurred) and user profile.
[0039] In some embodiments, the preset fault identifier and operation guidance mapping relationship includes: a preset fault identifier instruction association rule base, a preset basic operation instruction base, and a preset user profile adaptation rule base; determining the target prompt word for the target fault identifier based on the target fault identifier, the vehicle's user category label, and the preset fault identifier and operation guidance mapping relationship includes: determining the target natural language paragraph based on the target fault identifier, the preset fault identifier instruction association rule base, and the preset basic operation instruction base; and converting the target natural language paragraph into the target prompt word based on the user category label and the preset user profile adaptation rule base.
[0040] The fault code operation guidance mapping library is a structured, scalable knowledge graph or database. Its core structure consists of several interrelated data tables: a preset basic operation instruction library, a preset fault code instruction association rule library, and a preset user profile adaptation rule library. The preset basic operation instruction library stores all basic operation units, each with a unique identifier (ID) and operation description. The preset fault code instruction association rule library is the core rule library; it maps a fault code to a series of basic operation instructions through decision trees or conditional logic. It defines the logical order and conditional branches of the operations. The preset user profile adaptation rule library dynamically adjusts the style, detail, and content of the prompts based on user category tags.
[0041] Specifically, after generating the target fault identifier, user category tags are obtained. Based on a preset fault identifier instruction association rule base and a preset basic operation instruction base, the target fault identifier is translated into a target natural language paragraph. Then, based on a preset user profile adaptation rule base and user category tags, the target natural language paragraph is converted into target prompt words. This facilitates the subsequent generation of target videos based on the target prompt words, thereby achieving dynamic visualization of vehicle faults.
[0042] In some embodiments, determining the target natural language paragraph based on the target fault identifier, a preset fault identifier instruction association rule base, and a preset basic operation instruction base includes: searching the preset fault identifier instruction association rule base to determine the target instruction rule based on the target fault identifier; and translating the target fault identifier into the target natural language paragraph based on the target instruction rule and the preset basic operation instruction base.
[0043] For example, after generating the target fault identifier, such as when the system receives fault code P0171 and the current user's category tag is "novice driver," the operation flow of the prompt word generator (i.e., the target prompt word and target style) is as follows: Step 1: Matching rules. Based on the target fault identifier, search the preset fault identifier instruction association rule library to lock the target instruction rule corresponding to the target fault identifier, for example, locking the instruction rule Rule_P0171 corresponding to fault code P0171. Step 2: Based on the target instruction rule and the preset basic operation instruction library, generate an operation sequence containing all basic instructions, and concatenate the corresponding operation description, tools, and precautions into a basic natural language paragraph (i.e., the target natural language paragraph). Step 3: Based on the preset user profile adaptation rule library and user category tag (such as the "novice" tag), convert the basic natural language into a detailed and patient tone, and add tool recognition and safety reminders to each operation step, while setting style tags, such as setting it to cartoon animation. Step 4: Output the processed descriptive words to the target video generation model.
[0044] Step 130: Determine the target style based on user category tags and the preset correspondence between user and style tags.
[0045] The target styles include cartoon animation, realistic style, etc. The default user-style tag mapping is a user-style mapping table. For example, novice drivers correspond to cartoon style, and experienced drivers correspond to realistic style, etc. The specific settings can be adjusted according to the actual situation, and no specific restrictions are made here.
[0046] Step 140: Using the target video generation model, generate the target video corresponding to the target fault identifier based on the target prompt words and target style.
[0047] The target video generation model is trained based on a generative artificial intelligence (AIGC) model.
[0048] The AIGC model comprises a first backbone network, a first temporal attention layer, and a first temporal convolutional layer. The first backbone network employs a 3D U-Net structure as its denoising backbone. This first backbone network not only processes the image content of each frame spatially but also jointly models inter-frame relationships temporally by introducing the first temporal attention layer and the first temporal convolutional layer. This ensures that the generated video is highly coherent in time and can accurately depict continuous dynamic processes such as "smooth rotation of the sleeve" and "uniform spraying of cleaning agent."
[0049] Specifically, the process of training the target video generation model based on the AIGC model is as follows: Historical vehicle operating status data and fault identifiers corresponding to faults occurring in the historical operating status data are obtained. The corresponding prompt words and styles are obtained according to the target prompt word and target style generation methods provided in the above embodiments of this application. The prompt words and styles are then input into the AIGC model, specifically into the first backbone network. The first backbone network processes the process spatial dimension, and the first temporal attention layer and first temporal convolutional layer process the temporal dimension. Iterative processing is then performed. Under the condition that preset iteration conditions are met (e.g., model convergence or reaching a preset number of iterations), the trained target video generation model is obtained.
[0050] The target video generation model is a pre-trained AIGC model. This model includes an encoder, an adapter, and a diffusion model. The diffusion model serves as the second backbone network, employing a 3D U-Net structure as the denoising backbone. The encoder is a text encoder.
[0051] The adapter used is a Low-Rank Adaptation (LoRA) adapter. To achieve multi-style output, this application does not train a complete model for each style, but strictly follows the principle of efficient parameter fine-tuning. For example, a separate set of LoRA adapter weights is trained for each target style, such as "cartoon animation" and "realistic style". During inference, the corresponding LoRA weights are dynamically loaded into the pre-trained 3D U-Net based on the style label selected by the user profile, thereby achieving precise control over the visual style of the output video without increasing the inference cost.
[0052] In some embodiments, the target video generation model includes an encoder, an adapter, and a diffusion model. Using the target video generation model, a target video corresponding to a target fault identifier is generated based on target prompts and a target style. This includes: parsing the target prompts using the encoder to extract conditional embedding vectors; configuring target weights corresponding to the target style using the adapter to obtain style feature vectors; and iteratively denoising the conditional embedding vectors and style feature vectors using the diffusion model until a preset number of iterations is met, thereby generating the target video.
[0053] The preset number of iterations is N, where N is a positive integer. The specific value can be set according to the actual situation, and no specific limit is set here.
[0054] The core function of the target video generation model is to generate personalized, visual operation guidance videos in real time. For example, taking the target prompt "A mechanic is using a 10mm socket to loosen the throttle body fixing screw counterclockwise" and the target style as cartoon-style, the process of generating the target video corresponding to the target fault identifier includes the following steps: Step 1: Input preprocessing and conditional encoding. The natural language prompts are parsed by a text encoder to extract conditional embedding tensors. For example, the natural language prompt "A repairman is using a 10mm socket to loosen the throttle body fixing screw counterclockwise" is parsed to capture the conditional embedding tensors of action (loosening), tool (10mm socket), and object (screw, throttle body).
[0055] Step 2: Configure the target weights corresponding to the target style through the adapter to obtain the style label after configuring the target weights, i.e., obtain the style feature vector. In LoRA, style feature vectors are stored as key-value pairs, for example, cartoon:[], oil_paint:[]. The method to obtain the style feature vector is as follows: The corresponding style feature vector needs to be found using the input style label. For example, using the keyword "cartoon", the style table is queried to the vector set corresponding to "cartoon". Then, this style feature vector, along with the text encoder's conditional embedding tensor, is input into the diffusion model to jointly influence the final denoising result.
[0056] Step 3: Iteratively denoise the conditional embedding vectors and style feature vectors received by the diffusion model (3D U-Net).
[0057] Step 4: After N iterations, the original video tensor is generated.
[0058] Step 5: Output the target video, for example, a 10-second cartoon-style video: in the video, a cartoonish hand uses a distinctive socket to smoothly rotate the throttle screw counterclockwise.
[0059] In some embodiments, the target video generation model further includes a temporal attention layer and a temporal convolutional layer; the vehicle fault visualization guidance method further includes: using the temporal attention layer and the temporal convolutional layer to perform temporal dimension processing on the conditional embedding vector and style feature vector.
[0060] The target video generation model includes a temporal attention layer (second temporal attention layer) and a temporal convolutional layer (second temporal convolutional layer). The second temporal attention layer is the first temporal attention layer trained on the AIGC model. The second temporal convolutional layer is the first temporal convolutional layer trained on the AIGC model.
[0061] Step 150: Output the target video.
[0062] In some embodiments, outputting the target video includes: compressing and encoding the target video, and cutting the encoded target video into multiple video segments using a preset streaming media transmission protocol; and outputting each video segment.
[0063] The compression encoding method involves the cloud server performing compression encoding using the video compression encoding standard h264 or h265. h264 is also known as Advanced Video Coding (AVC), and h265 is also known as High Efficiency Video Coding (HEVC).
[0064] The default streaming media transmission protocol is either HLS or RTMP. HLS is an HTTP-based streaming media transmission protocol that uses "slice transmission" (dividing the video into small segments) and has extremely high compatibility (supporting almost all devices such as mobile phones, computers, and in-vehicle terminals). RTMP is a TCP-based real-time streaming media transmission protocol with extremely low transmission latency (typically 1-3 seconds), making it suitable for real-time interactive scenarios.
[0065] Specifically, after generating the target video, it interacts with the vehicle. The in-vehicle interactive output process is as follows: After the target video generation model generates the target video, the cloud server compresses and encodes it using H.264 or H.265, and then uses HLS or RTMP streaming media transmission protocols to cut the video file into a series of small video segments. The vehicle terminal can start playing the first segment immediately after downloading it, while the subsequent segments continue to be downloaded on the backend. This method achieves "downloading and playing" of the video, greatly reducing user waiting time. Each video segment is digitally signed before transmission, and the vehicle terminal verifies the validity of the signature upon receipt to prevent data tampering during transmission and ensure the accuracy of the operation guide. After receiving the data, the vehicle system will launch a dedicated "maintenance guidance" interactive interface with voice broadcast. Users can drag the video by clicking the progress bar on the interface, and can click the zoom in / out buttons or use two fingers to zoom in and out of the screen to observe the video content carefully.
[0066] It is understood that the vehicle fault visualization guidance method provided in this application generates a fault identifier when a vehicle malfunctions, generates corresponding prompt words based on the fault identifier and the preset fault identifier-operation guidance mapping relationship, and generates a corresponding target style. The method then uses a pre-trained target video generation model to generate a video based on the target prompt words and target style, thereby achieving timely dynamic visualization reminders of vehicle faults and improving the user experience.
[0067] Figure 2 This is a schematic diagram of the overall process of a vehicle fault visualization guidance method provided in this application embodiment. For example, see [link to relevant documentation]. Figure 2 The system detects the vehicle's current operating status data using vehicle sensors. If the current operating status data meets preset fault judgment conditions, a vehicle fault is identified, and a corresponding target fault identifier, such as a fault code, is generated. The fault code is converted into a natural language paragraph. Based on the fault code and user profile, corresponding prompts are generated. These prompts are input into the AIGC model. The AIGC model generates an operation guidance video. The operation guidance video is transmitted by encoding the video data into MP4 format. Finally, it is played on the vehicle's display screen.
[0068] For example, taking fault code P0171: fuel mixture too lean as an example, the process of generating the target video for this fault includes the following steps: Step 1: The OBD detected an abnormal fuel-air ratio and output fault code P0171.
[0069] Step 2: The local mapping library converts P0171 into the prompt word: "Check for air leaks in the intake system. Cleaning the throttle body is recommended."
[0070] Step 3: With an external temperature of 35℃, the system prompts "Please avoid operating during high-temperature periods"; the user profile is "novice driver", and the system selects a cartoon style.
[0071] Step 4: The cloud-based AIGC model generates a 10-second cartoon video and displays it. Use a flashlight to locate cracks in the air intake pipe; Use a 10mm socket to remove the throttle body counterclockwise; Clean the throttle body with a spray; Step 5: Transmit the generated target video (i.e., the 10-second cartoon video mentioned above) to the vehicle's infotainment system for playback, while simultaneously announcing: "Please use a 10mm socket to loosen the throttle body screw."
[0072] In summary, the vehicle fault visualization guidance method provided in this application, by combining real-time sensor data and an AIGC model to generate dynamic guidance videos, enhances the vehicle's self-diagnosis and interactive capabilities. Furthermore, it achieves the following effects: 1. High real-time performance: It can generate video playback in real time from fault detection.
[0073] II. Personalized Adaptation: Different videos can be generated for different users and in different environments.
[0074] 3. Enhanced interactivity: Supports voice control and controllable video playback for a better user experience.
[0075] IV. Improved maintenance efficiency: Helps non-professional users quickly understand and handle common faults.
[0076] Figure 3 This is a schematic diagram of the structural principle of a vehicle fault visualization guidance system provided in this application embodiment. On the other hand, this embodiment provides a vehicle fault visualization guidance system, see [link / reference]. Figure 3 The vehicle fault visualization guidance system includes: a first determination module 101, used to determine that a vehicle fault has occurred when the vehicle's current operating status data meets preset fault judgment conditions; a first generation module 102, used to generate a target fault identifier corresponding to the fault; a second determination module 103, used to determine the target prompt word for the target fault identifier based on the target fault identifier, the vehicle's user category label, and a preset mapping relationship between fault identifiers and operation guidance; a third determination module 104, used to determine the target style based on the user category label and a preset correspondence between user and style labels; a second generation module 105, used to generate a target video corresponding to the target fault identifier using a target video generation model, based on the target prompt word and target style; and an output module 106, used to output the target video.
[0077] The technical solution of this application provides a vehicle fault visualization guidance system. This application generates a fault identifier when a vehicle malfunctions, generates corresponding prompt words based on the fault identifier and a preset fault identifier-operation guidance mapping relationship, and generates a corresponding target style. A pre-trained target video generation model is then used to generate a video based on the target prompt words and target style, thereby achieving timely dynamic visualization reminders of vehicle faults and improving the user experience.
[0078] In some embodiments, the preset fault identifier and operation guide mapping relationship includes: a preset fault identifier instruction association rule base, a preset basic operation instruction base, and a preset user profile adaptation rule base; The first determining module 101 is also used for: The target natural language paragraph is determined based on the target fault identifier, the preset fault identifier instruction association rule base, and the preset basic operation instruction base. Based on user category tags and a preset user profile adaptation rule library, the target natural language paragraph is converted into target prompt words.
[0079] In some embodiments, the first determining module 101 is further configured to: Based on the target fault identifier, search the preset fault identifier instruction association rule base to determine the target instruction rule; Based on the target instruction rules and the preset basic operation instruction library, the target fault identifier is translated into a target natural language paragraph.
[0080] In some embodiments, the target video generation model includes an encoder, an adapter, and a diffusion model; the second generation module 105 is further configured to: The target prompt words are parsed using an encoder to extract conditional embedding vectors; The target weights corresponding to the target style are configured through the adapter to obtain the style feature vector; The conditional embedding vector and style feature vector are denoised iteratively using a diffusion model until the preset number of iterations is met, at which point the target video is generated.
[0081] In some embodiments, the target video generation model further includes a temporal attention layer and a temporal convolutional layer; the vehicle fault visualization guidance system 100 further includes a processing module, which is used for: Temporal attention layers and temporal convolutional layers are used to process the conditional embedding vectors and style feature vectors in the temporal dimension.
[0082] In some embodiments, the output module 106 further includes: The encoding unit is used to compress and encode the target video, and to cut the encoded target video into multiple video segments using a preset streaming media transmission protocol; The transmission unit is used to output each video segment.
[0083] In some embodiments, the target fault identifier includes at least: the system to which the fault belongs, the fault type, and the fault number.
[0084] This embodiment also provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, it implements the method of any of the above embodiments.
[0085] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps of any of the methods in the above embodiments.
[0086] In the embodiments of this application, the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.
[0087] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0088] The above provides a detailed description of a vehicle fault visualization guidance method, system, device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A vehicle fault visualization guidance method, characterized in that, Includes the following steps: If the vehicle's current operating status data meets the preset fault judgment conditions, it is determined that the vehicle has a fault, and a target fault identifier corresponding to the fault is generated. Based on the target fault identifier, the user category label of the vehicle, and the preset fault identifier and operation guidance mapping relationship, the target prompt word of the target fault identifier is determined; The target style is determined based on the user category tags and the preset correspondence between users and style tags; Using a target video generation model, a target video corresponding to the target fault identifier is generated based on the target prompt words and the target style; Output the target video.
2. The method according to claim 1, characterized in that, The preset fault identifier and operation guide mapping relationship includes: a preset fault identifier instruction association rule library, a preset basic operation instruction library, and a preset user profile adaptation rule library; The step of determining the target prompt word for the target fault identifier based on the target fault identifier, the user category tag of the vehicle, and the preset fault identifier and operation guidance mapping relationship includes: The target natural language paragraph is determined based on the target fault identifier, the preset fault identifier instruction association rule base, and the preset basic operation instruction base. Based on the user category tags and the preset user profile adaptation rule base, the target natural language paragraph is converted into the target prompt word.
3. The method according to claim 2, characterized in that, The step of determining the target natural language paragraph based on the target fault identifier, the preset fault identifier instruction association rule base, and the preset basic operation instruction base includes: Based on the target fault identifier, search the preset fault identifier instruction association rule base to determine the target instruction rule; Based on the target instruction rules and the preset basic operation instruction library, the target fault identifier is translated into a target natural language paragraph.
4. The method according to claim 1, characterized in that, The target video generation model includes an encoder, an adapter, and a diffusion model; the step of generating the target video corresponding to the target fault identifier using the target video generation model, based on the target cue words and the target style, includes: The encoder parses the target prompt words to extract conditional embedding vectors; The target weights corresponding to the target style are configured through the adapter to obtain the style feature vector; The conditional embedding vector and the style feature vector are iteratively denoised using the diffusion model until a preset number of iterations is met, at which point the target video is generated.
5. The method according to claim 4, characterized in that, The target video generation model further includes a temporal attention layer and a temporal convolutional layer; the method further includes: The temporal attention layer and the temporal convolutional layer are used to perform temporal dimension processing on the conditional embedding vector and the style feature vector.
6. The method according to claim 1, characterized in that, Output the target video, including: The target video is compressed and encoded, and the encoded target video is cut into multiple video segments using a preset streaming media transmission protocol; Output each of the aforementioned video clips.
7. The method according to claim 1, characterized in that, The target fault identifier includes at least: the system to which the fault belongs, the fault type, and the fault number.
8. A vehicle fault visualization guidance system, characterized in that, include: The first determining module is used to determine that the vehicle has malfunctioned if the vehicle's current operating status data meets preset fault judgment conditions. The first generation module is used to generate the target fault identifier corresponding to the fault. The second determining module is used to determine the target prompt word of the target fault identifier based on the target fault identifier, the user category label of the vehicle, and the preset fault identifier and operation guidance mapping relationship; The third determining module is used to determine the target style based on the user category tags and the preset correspondence between user and style tags; The second generation module is used to generate a target video corresponding to the target fault identifier based on the target prompt words and the target style using a target video generation model. The output module is used to output the target video.
9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps of the method as described in any one of claims 1-7.