Fault auxiliary diagnosis method and system for network connection automobile installation and adjustment

Abstract

The invention provides a fault auxiliary diagnosis method and system for network connection automobile installation and adjustment, and belongs to the technical field of automobile installation and adjustment. The method comprises the following steps: collecting visual information data of each component in a networked automobile assembling and adjusting process, and providing visual and high-precision fault feature data for a system; integrating cases of the installation and adjustment process of the network-connected automobile, and constructing a fault knowledge graph; and performing real-time analysis on the visual information data of each component by using a trained deep learning model, extracting fault features, and marking the position and reason of a corresponding fault component according to the fault knowledge graph. Therefore, rapid identification and accurate positioning of a complex fault mode can be realized, so that the fault diagnosis efficiency and accuracy in the process of assembling and adjusting the networked automobile can be effectively improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of automobile adjustment, more particularly, relates to a fault auxiliary diagnosis method and system for networked automobile adjustment. BACKGROUND

[0002] Compared with traditional automobiles, networked automobiles integrate a large number of intelligent hardware and complex software systems, and the adjustment process involves electronic and electrical, communication protocol, software adaptation and other multi-field cooperation. The faults occurring in the adjustment process present the characteristics of "multi-source, correlation and concealment", and simple manual fault troubleshooting cannot meet the efficient operation and maintenance requirements.

[0003] In addition, the traditional manual fault troubleshooting has three bottlenecks: first, fault identification depends on experience accumulation, and new types of faults of networked automobiles lack mature troubleshooting standards, so that new operators need long-term training to be competent, and the implementation cost is high; second, the correlation of faults in each module makes fault positioning difficult, and manual troubleshooting needs to be verified one by one, so that the average time consumption is much higher than that of traditional automobile fault troubleshooting; third, the data analysis efficiency is low, and the sensor data, communication logs and other data generated in the adjustment process of networked automobiles can reach hundreds of GB per hour, so that it is difficult for manual analysis to extract effective fault features from the data, and it is easy to miss or misjudge. SUMMARY

[0004] The main purpose of the present application is to provide a fault auxiliary diagnosis method and system for networked automobile adjustment, which can reduce the manual detection time, improve the fault source positioning accuracy and ensure the efficient adjustment of networked automobiles.

[0005] In order to achieve the above purpose, the present application provides a fault auxiliary diagnosis method for networked automobile adjustment, comprising: Collecting visual information data of each component in the adjustment process of networked automobiles to provide intuitive and high-precision fault feature data for the system; integrating cases of past networked automobile adjustment processes to construct a fault knowledge graph; using a trained deep learning model to analyze the visual information data of each component in real time, extracting fault features and labeling the corresponding fault component position and reason according to the fault knowledge graph; to realize the rapid identification and accurate positioning of complex fault modes, thereby effectively improving the fault diagnosis efficiency and accuracy in the adjustment process of networked automobiles.

[0006] Further, the use of a trained deep learning model to analyze the visual information data of each component in real time comprises: obtaining local or global fault features of the visual information data of each component, mapping low-level detailed information to high-level semantics of fault features, and labeling the position and reason of the fault component through similarity comparison of semantic vectors and according to expert knowledge base experience.

[0007] Furthermore, the acquisition of visual information data of each component during the assembly and adjustment process of the connected vehicle includes: acquiring hardware component data from different angles and in different scenarios, and acquiring continuous video streams to capture the component status during the assembly and adjustment process of the connected vehicle.

[0008] Furthermore, the types of fault characteristics include radios without antennas, radios without power cords, radios without DB9 plugs, inertial navigation systems without positioning antennas, inertial navigation systems without directional antennas, inertial navigation systems without 4G antennas, inertial navigation systems without integrated wiring harnesses, Miwen systems without integrated wiring harnesses, Miwen systems without HDMI cables, and Miwen systems without network cables.

[0009] This invention also proposes a fault-aided diagnostic system for the assembly and adjustment of connected vehicles, comprising: A visual acquisition device is used to collect visual information data of various components during the assembly and adjustment process of connected vehicles, providing the system with intuitive and high-precision fault feature data; an expert knowledge base is used to integrate past cases of connected vehicle assembly and adjustment processes to construct a fault knowledge graph; a data analysis and processing system is used to perform real-time analysis of the visual information data of each component using a pre-trained deep learning model, extract fault features, and label the corresponding fault component locations and causes according to the fault knowledge graph; in order to achieve rapid identification and accurate positioning of complex fault modes, thereby effectively improving the efficiency and accuracy of fault diagnosis during the assembly and adjustment process of connected vehicles.

[0010] Furthermore, it also includes a user visualization interface and interactive device, used to present the diagnostic results of the data analysis and processing module to the operator in the form of charts or numbers.

[0011] Furthermore, the real-time analysis of the visual information data of each component using the pre-trained deep learning model includes: obtaining local or global fault features of the visual information data of each component, mapping low-level detail information to high-level semantics of fault features, and labeling the location and cause of faulty components by comparing the similarity of semantic vectors and based on the experience of the expert knowledge base.

[0012] Furthermore, the acquisition of visual information data of each component during the assembly and adjustment process of the connected vehicle includes: acquiring hardware component data from different angles and in different scenarios, and acquiring continuous video streams to capture the component status during the assembly and adjustment process of the connected vehicle.

[0013] Furthermore, the types of fault characteristics include radios without antennas, radios without power cords, radios without DB9 plugs, inertial navigation systems without positioning antennas, inertial navigation systems without directional antennas, inertial navigation systems without 4G antennas, inertial navigation systems without integrated wiring harnesses, Miwen systems without integrated wiring harnesses, Miwen systems without HDMI cables, and Miwen systems without network cables.

[0014] Compared with existing technologies, the technical solution of this invention has the following advantages: by collecting visual information from each component, it provides the system with intuitive and high-precision fault feature data; by integrating cases from previous connected vehicle assembly and debugging processes, it constructs a fault knowledge graph to provide decision support for the system; and by using data collected by a pre-trained deep learning model for real-time analysis, it extracts the fault features of faulty components and labels the location and cause of faulty components based on expert knowledge base experience; thus, it achieves rapid identification and accurate positioning of complex fault modes, thereby effectively improving the efficiency and accuracy of fault diagnosis in the connected vehicle assembly and debugging process. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a fault-assisted diagnosis method for connected vehicle assembly and adjustment provided in an embodiment of the present invention. Figure 2 This is a logical diagram of a fault auxiliary diagnosis system for connected vehicle assembly and adjustment provided in an embodiment of the present invention. Detailed Implementation

[0016] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0017] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0018] Example Compared to traditional vehicles, connected vehicles integrate a large number of intelligent hardware and complex software systems. Their assembly and commissioning process involves the collaboration of multiple fields such as electronics, electrical systems, communication protocols, and software adaptation. The faults that occur are characterized by "multi-source, correlation, and concealment". Simply relying on manual troubleshooting is difficult to meet the needs of efficient operation and maintenance.

[0019] Furthermore, traditional manual fault diagnosis faces three major bottlenecks: First, fault identification relies on accumulated experience. New types of faults in connected vehicles lack mature troubleshooting standards, requiring novice operators to undergo extensive training before they can perform the tasks effectively, resulting in high implementation costs. Second, the correlation between faults in various modules makes fault localization difficult, requiring manual troubleshooting to verify each fault individually, which takes significantly longer than traditional vehicle fault diagnosis. Third, data analysis efficiency is low. During the assembly and commissioning process of connected vehicles, hundreds of gigabytes of sensor data and communication logs are generated every hour, making it difficult for manual personnel to extract effective fault characteristics, easily leading to missed or incorrect diagnoses. Based on this, this embodiment provides a fault-assisted diagnosis method and system for the assembly and adjustment of connected vehicles, which can reduce manual inspection time, improve the accuracy of fault source location, and ensure the efficient assembly and adjustment requirements of connected vehicles.

[0020] like Figure 1 As shown, this embodiment proposes a fault-assisted diagnosis method for connected vehicle assembly and adjustment, including steps S100 to S300: S100: Collects visual information data of each component during the assembly and adjustment process of connected vehicles.

[0021] For example, using a high-definition camera to collect on-site data, firstly, by fixing the camera position, data on hardware components (faulty / normal) are acquired from different angles and under different lighting conditions to expand the diversity of data and facilitate the comparative analysis of fault feature vectors; then, the high-definition camera is fixed to the operator's chest to collect continuous video streams and capture the status of components during operation (e.g., inertial navigation with / without directional antenna), while avoiding blind spots in the field of view of the fixed camera.

[0022] S200 integrates past cases of connected vehicle assembly and adjustment processes to construct a fault knowledge graph.

[0023] For example, this invention summarizes the practical experience of senior assembly and debugging engineers and technical experts, and collects detailed information such as troubleshooting steps and solutions for typical faults resolved during the assembly and debugging of connected vehicles, forming an initial case library. This invention only summarizes the types of hardware fault characteristics that frequently occur during the assembly and debugging of connected vehicles, as shown in Table 1: S300: Utilize a pre-trained deep learning model to perform real-time analysis of the visual information data of each component, extract fault features, and label the corresponding fault component location and cause according to the fault knowledge graph; thereby achieving rapid identification and accurate positioning of complex fault modes, effectively improving the efficiency and accuracy of fault diagnosis during the assembly and adjustment process of connected vehicles.

[0024] The step of using a pre-trained deep learning model to perform real-time analysis of the visual information data of each component includes: obtaining local or global fault features of the visual information data of each component, mapping low-level detail information to high-level semantics of fault features, and labeling the location and cause of faulty components by comparing the similarity of semantic vectors and based on the experience of expert knowledge base.

[0025] For example, the model was built, trained, and tested using Python 3.5 and the PyTorch deep learning framework on a Windows 10 operating system. The number of neurons in the output layer was adjusted based on the set number of fault categories; the appropriate batch size, learning rate, and number of iterations were set according to the dataset size. In this invention, the number of samples for each state was 200, the number of neurons in the output layer was 11, the batch size was 32, the learning rate was 0.0002, the number of iterations was 200, and the average classification accuracy was used as the evaluation metric.

[0026] The trained YOLOv5 model is converted into a myYolo.engine file, and then deployed and run on the server. During runtime, a RESTful service is built using the FastAPI framework, defining an interface to receive image data. Real-time images of components are captured by industrial cameras and transmitted to the server via the local area network. The model processes the received files and returns the inference results in a monitor.json file.

[0027] This embodiment also provides a fault-aided diagnostic system for the assembly and adjustment of connected vehicles, such as... Figure 2 As shown, it includes: a visual acquisition device, an expert knowledge base, a data analysis and processing system, and a user visualization interface and interaction device, which are used to achieve rapid identification and accurate location of complex fault modes, thereby effectively improving the efficiency and accuracy of fault diagnosis during the assembly and adjustment process of connected vehicles.

[0028] The visual acquisition device uses a high-definition camera to collect on-site data. First, by fixing the camera position, it acquires hardware component (faulty / normal) data from different angles and lighting conditions to expand data diversity and facilitate comparative analysis of fault feature vectors. Then, the high-definition camera is fixed to the operator's chest to capture continuous video streams, recording the component status during operation (e.g., inertial navigation system with / without directional antenna), while avoiding blind spots in the fixed camera's field of view. The acquired data is saved to the local hard drive in both image and video formats. Blurry, overexposed, and duplicate images are manually filtered. After data quality confirmation, annotators label the acquired data, checking for complete coverage of targets by annotation boxes, consistency of annotations for similar targets, and any omissions.

[0029] The expert knowledge base is constructed by: summarizing the practical experience of senior assembly and debugging engineers and technical experts, collecting detailed information such as troubleshooting steps and solutions for typical faults resolved during the assembly and debugging of connected vehicles, and forming an initial case library. This embodiment only summarizes the types of hardware faults that frequently occur during the assembly and debugging of connected vehicles, including: radio without antenna, radio without power cord, radio without DB9 plug, inertial navigation system without positioning antenna, inertial navigation system without directional antenna, inertial navigation system without 4G antenna, inertial navigation system without integrated wiring harness, Mi-Wen without integrated wiring harness, Mi-Wen without HDMI cable, and Mi-Wen without network cable.

[0030] The data analysis and processing system is used to perform real-time analysis of the visual information data of each component using a pre-trained deep learning model, extract fault features, and label the location and cause of the corresponding faulty component according to the fault knowledge graph.

[0031] The model construction and deployment included: using Python 3.5 and the PyTorch deep learning framework, the model was built, trained, and tested on a Windows 10 operating system. The number of neurons in the output layer was adjusted according to the set number of fault categories; appropriate batch size, learning rate, and number of iterations were set based on the dataset size. In this invention, the number of samples for each state was 200, the number of neurons in the output layer was 11, the batch size was 32, the learning rate was 0.0002, the number of iterations was 200, and the average classification accuracy was used as the evaluation metric.

[0032] The trained YOLOv5 model is converted into a myYolo.engine file, and then deployed and run on the server. During runtime, a RESTful service is built using the FastAPI framework, defining an interface to receive image data. Real-time images of components are captured by industrial cameras and transmitted to the server via the local area network. The model processes the received files and returns the inference results as a monitor.json file to the data analysis module. The model's detection results are then fed back to the user's visualization interface via the local area network.

[0033] The user visualization interface and interactive device are used to present the diagnostic results of the data analysis and processing module to the operator in the form of charts or numbers. The user visualization interface can be written in JAVA language; at runtime, the user interface receives and parses the monitor.json file, displays the real-time screen of the operator's troubleshooting process, uses JavaFX to overlay fault annotation boxes on the real-time screen, and provides necessary measures to be taken.

[0034] In the description of this invention, it should be understood that the terms "middle," "length," "upper," "lower," "front," "rear," "vertical," "horizontal," "inner," "outer," "radial," "circumferential," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0035] In this invention, unless otherwise expressly specified and limited, the first feature "on" the second feature may be in direct contact with the first and second features, or indirect contact with the first and second features through an intermediate medium. "A plurality of" means at least two, such as two, three, etc., unless otherwise expressly and specifically limited.

[0036] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0037] The above description is merely illustrative of the embodiments of the present invention and is not intended to limit the present invention. For those skilled in the art, any modifications, equivalent substitutions, improvements, etc., made without creative effort within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fault-aided diagnosis method for the assembly and adjustment of connected vehicles, characterized in that, include: Collect visual information data of each component during the assembly and adjustment process of connected vehicles; Integrate past cases of connected vehicle assembly and adjustment processes to construct a fault knowledge graph; The visual information data of each component is analyzed in real time using a pre-trained deep learning model to extract fault features and label the location and cause of the corresponding faulty component according to the fault knowledge graph.

2. The fault auxiliary diagnosis method for connected vehicle assembly and adjustment according to claim 1, characterized in that, The real-time analysis of the visual information data of each component using a pre-trained deep learning model includes: obtaining local or global fault features of the visual information data of each component, mapping low-level detail information to high-level semantics of fault features, and labeling the location and cause of faulty components by comparing the similarity of semantic vectors and based on the experience of the expert knowledge base.

3. The fault auxiliary diagnosis method for connected vehicle assembly and adjustment according to claim 1, characterized in that, The process of collecting visual information data of each component during the assembly and adjustment of connected vehicles includes: acquiring hardware component data from different angles and in different scenarios, and collecting continuous video streams to capture the component status during the assembly and adjustment of connected vehicles.

4. The fault auxiliary diagnosis method for connected vehicle assembly and adjustment according to claim 1, characterized in that, The fault characteristics include radios without antennas, radios without power cords, radios without DB9 plugs, inertial navigation systems without positioning antennas, inertial navigation systems without directional antennas, inertial navigation systems without 4G antennas, inertial navigation systems without integrated wiring harnesses, Miwen systems without integrated wiring harnesses, Miwen systems without HDMI cables, and Miwen systems without network cables.

5. A fault-aided diagnostic system for the assembly and adjustment of connected vehicles, characterized in that, include: A visual acquisition device is used to collect visual information data of various components during the assembly and adjustment process of connected vehicles. An expert knowledge base is used to integrate past cases of connected vehicle assembly and adjustment processes to build a fault knowledge graph; The data analysis and processing system is used to perform real-time analysis of the visual information data of each component using a pre-trained deep learning model, extract fault features, and label the location and cause of the corresponding faulty component according to the fault knowledge graph. This enables rapid identification and precise location of complex fault modes, thereby effectively improving the efficiency and accuracy of fault diagnosis during the assembly and commissioning process of connected vehicles.

6. The fault auxiliary diagnostic system for connected vehicle assembly and adjustment according to claim 5, characterized in that, It also includes a user visualization interface and interactive device, which presents the diagnostic results of the data analysis and processing module to the operator in the form of charts or numbers.

7. The fault auxiliary diagnostic system for connected vehicle assembly and adjustment according to claim 5, characterized in that, The real-time analysis of the visual information data of each component using a pre-trained deep learning model includes: obtaining local or global fault features of the visual information data of each component, mapping low-level detail information to high-level semantics of fault features, and labeling the location and cause of faulty components by comparing the similarity of semantic vectors and based on the experience of the expert knowledge base.

8. The fault auxiliary diagnostic system for connected vehicle assembly and adjustment according to claim 5, characterized in that, The process of collecting visual information data of each component during the assembly and adjustment of connected vehicles includes: acquiring hardware component data from different angles and in different scenarios, and collecting continuous video streams to capture the component status during the assembly and adjustment of connected vehicles.

9. The fault auxiliary diagnostic system for connected vehicle assembly and adjustment according to claim 5, characterized in that, The fault characteristics include radios without antennas, radios without power cords, radios without DB9 plugs, inertial navigation systems without positioning antennas, inertial navigation systems without directional antennas, inertial navigation systems without 4G antennas, inertial navigation systems without integrated wiring harnesses, Miwen systems without integrated wiring harnesses, Miwen systems without HDMI cables, and Miwen systems without network cables.

Images