Vehicle diagnosis log data real-time access system and method

Abstract

The invention discloses a vehicle diagnosis log data real-time access system and method, and the method comprises the steps: generating a vehicle diagnosis log through a log generation and arrangement module, and writing the generated log data into a disk file; the log file path is transmitted to an SOVD server; the SOVD server receives the log file path and generates log entries, and the streaming transmission module transmits log data to the SOVD client in a streaming mode through an HTTP streaming transmission technology; the SOVD client sends a real-time log request to the SOVD server; and the SOVD client receives the log entry returned by the SOVD server, directly accesses the log data acquisition address, and acquires the log data in real time through an HTTP streaming transmission technology. According to the invention, the HTTP streaming transmission technology is combined with the SOVD protocol, so that the real-time, flexible and efficient log transmission capability can be provided; real-time transmission of log data is ensured, and delay and data loss are reduced.

Description

Technical Field

[0001] The present invention belongs to the field of data transmission, and particularly relates to a system and method for real-time access to vehicle diagnostic log data. Background Art

[0002] With the complexity of vehicle electronic architectures, especially the introduction of high-performance computing units (HPCs), vehicle diagnostic requirements have shifted from traditional hardware fault detection to real-time monitoring and analysis of software functions. Existing diagnostic protocols (such as UDS, ODX, etc.) mainly target hardware fault diagnosis and cannot meet the real-time monitoring requirements of modern vehicles for software running status and system resource usage (such as CPU load, memory usage, etc.). The SOVD protocol, as a vehicle diagnostic interface based on HTTP REST, provides a unified access method for vehicle diagnostic content and supports remote, near-field, and in-vehicle diagnostic scenarios. The SOVD protocol returns detailed log entries through the / {entity-path} / logs / entries interface, and each log entry contains an href field pointing to the access URL (Uniform Resource Locator, hereinafter referred to as URL) of the log. However, although the existing SOVD protocol provides the href field, it only points to static log resources and does not consider the requirements of real-time log transmission.

[0003] In the field of the Internet, HTTP streaming transmission technology has been maturely applied to achieve real-time data transmission and low-latency access. However, in the field of vehicle diagnosis, the existing UDS protocol, due to its request-response communication mode, lack of streaming transmission mechanism, low data transmission efficiency, non-support for long connections, and lack of flexible log filtering and query functions, cannot take advantage of the advantages of HTTP streaming transmission technology, resulting in delays and data loss problems in real-time monitoring and analysis of vehicle operating status. Especially in remote diagnosis and near-field diagnosis scenarios, the existing diagnostic protocols cannot provide efficient real-time log transmission capabilities and cannot meet the real-time log transmission requirements of modern vehicles. Summary of the Invention

[0004] Object of the Invention: Aiming at the problems existing in the prior art, the present invention proposes a real-time access system for vehicle diagnostic log data that makes full use of the existing SOVD protocol interface to implement HTTP streaming transmission, while maintaining protocol compatibility and improving the real-time performance and efficiency of log transmission.

[0005] Technical Solution: To achieve the above object, the present invention provides a real-time access system for vehicle diagnostic log data, including a log generation and collation module, an SOVD server, a streaming transmission module, and an SOVD client; wherein, The streaming module is integrated in the SOVD server. The SOVD server receives the log file path passed by the log generation and collation module through the API interface of the SOVD dynamic library, generates log entries, reads the log file according to the log file path, and divides the log data into data blocks. The streaming module transmits the data blocks to the SOVD client in a streaming manner through HTTP streaming technology; The SOVD client sends a real-time log request to the SOVD server through the REST API interface of the SOVD protocol; the SOVD client receives the log entries returned by the SOVD server, directly accesses the log data acquisition address, and obtains the log data in real time through HTTP streaming technology.

[0006] Furthermore, the href field included in the log entry points to the log data acquisition address that supports HTTP streaming. This can achieve better compatibility and obtain dynamic logs without making changes to the SOVD protocol.

[0007] Furthermore, the log generation and collation module also includes log file rotation. When the size of the current log file is greater than the file size threshold, or the time interval of the data received in the current log file is greater than the time interval threshold, a new log file is automatically created, and the old file path is passed to the SOVD server. In this way, a single log file remains within a relatively small range, which is beneficial to improving the timeliness of log transmission, reducing the memory and I / O pressure at the same time, avoiding problems such as file handle resource inflation and increased write latency caused by long-term writing to a single large log file. At the same time, the peak memory occupancy during log reading is reduced.

[0008] Furthermore, the SOVD server also includes an adaptive transmission module, a data cache and retransmission module, and a priority queue module; among them, the adaptive transmission module uses the AIMD algorithm to dynamically adjust the data transmission rate according to the network conditions; the data cache and retransmission module caches the data that has not been successfully transmitted when the network is interrupted; the priority queue module sets priorities for different types of log data and transmits according to the priorities. This can effectively ensure the real-time and integrity of data transmission and provide a stable and reliable data foundation for the log collection and monitoring system.

[0009] Furthermore, the adaptation transmission module uses the transmission result of the data block as the adjustment basis. When all the data blocks in the previous round are transmitted successfully, the amount of data block pushing in the next round is increased additively; when all the data blocks in the previous round are transmitted unsuccessfully, the amount of data block pushing in the next round is reduced multiplicatively. This can effectively cope with network fluctuations and balance resource overhead; at the same time, improve the memory usage efficiency.

[0010] Further, the division of log data into data blocks divides the logs into two categories: critical logs and non-critical logs, and they are respectively packaged into independent data blocks. This realizes the efficient utilization and intelligent scheduling of resources while ensuring the transmission order and stability.

[0011] The present invention also provides a method for real-time access to vehicle diagnostic log data, including the following steps: Step 1: The SOVD client sends a real-time vehicle diagnostic log request to the SOVD server; Step 2: After receiving the vehicle diagnostic log request, the SOVD server reads the log file according to the log file path, divides the log data into data blocks, and at the same time generates log entries and sends them to the SOVD client; Step 3: After receiving the log entries, the SOVD client directly accesses the log data acquisition address; Step 4: The SOVD server sends the data blocks to the SOVD client in the form of HTTP streaming transmission.

[0012] The present invention also provides a computer-readable medium storing software, where the software includes instructions executable by one or more computers, and these instructions, through such execution, cause the one or more computers to perform operations, and the operations include the process of the above-mentioned method for real-time access to vehicle diagnostic log data.

[0013] The present invention also provides a computer system, including: One or more processors; A memory storing operable instructions, and these instructions, when executed by the one or more processors, cause the one or more processors to perform operations, and the operations include the process of the above-mentioned method for real-time access to vehicle diagnostic log data.

[0014] Working principle: The present invention utilizes the log access interface and standard fields (href in LogEntry) provided in the existing SOVD protocol, and sets the resource address corresponding to this field as the server URL supporting HTTP streaming transmission. Without making any changes to the SOVD protocol itself, it realizes the real-time acquisition of log data, with good compatibility and convenience in system integration.

[0015] Beneficial effects: Compared with the prior art, the present invention combines the HTTP streaming transmission technology with the SOVD protocol, can provide real-time, flexible, and efficient log transmission capabilities; ensures the real-time transmission of log data, reduces latency and data loss; at the same time, based on the standardized interface of the SOVD protocol, the present invention is compatible with existing vehicle diagnostic systems, does not require additional hardware support, and also has good scalability, supporting more log types and transmission protocols in the future. Description of the Drawings

[0016] Figure 1 It is a schematic structural diagram of the vehicle diagnostic log data real-time access system disclosed in Embodiment 1; Figure 2 It is a schematic structural diagram of the vehicle diagnostic log data real-time access system disclosed in Embodiment 2. Detailed Embodiments

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention. Embodiment 1

[0018] As Figure 1 shown, in this embodiment, a vehicle diagnostic log data real-time access system is disclosed, which includes a log generation and collation module, an SOVD server, a streaming transmission module, and an SOVD client; wherein the log generation and collation module and the SOVD server are integrated in the application program, and the application program runs in the computing unit of the vehicle; the streaming transmission module is integrated in the SOVD server.

[0019] During the operation of the application program, the log generation and collation module generates vehicle diagnostic logs, which include data such as vehicle system operation logs and application program logs, and writes the logs into disk files. The log data includes but is not limited to: 1) System resource usage (such as CPU load, memory usage, disk I / O, etc.).

[0020] 2) Application program running status (such as start time, running duration, error logs, etc.).

[0021] 3) Network communication status (such as network latency, packet loss rate, etc.).

[0022] The naming and storage path of the log file can be set according to the configuration; for example, the log file can be named by timestamp and stored in a specified directory. The log file path is passed to the SOVD server through the API interface of the SOVD dynamic library. The log generation and organization module also supports the log file rotation mechanism. When the size of the current log file is greater than, for example, 5MB, or the time interval of the data received in the current log file is greater than 1 minute, a new log file is automatically created, and the old file path is passed to the SOVD server. Keeping a single log file within a smaller range is beneficial to improving the timeliness of log transmission, while reducing the memory and I / O pressure, and avoiding problems such as file handle resource inflation and increased write latency caused by long-term writing to a single large log file. At the same time, it reduces the peak memory occupancy when reading logs.

[0023] The SOVD server mainly manages the diagnostic data of the vehicle, provides a REST API interface based on the SOVD protocol, and supports real-time streaming transmission of log data. The SOVD server receives the log file path passed by the log generation and organization module through the API interface of the SOVD dynamic library and generates log entries; reads the log file according to the log file path, divides the log data into data blocks, and sends them out through the streaming transmission module. Among them, the href field contained in the log entry points to a log access URL that supports HTTP streaming transmission. The SOVD server divides the log data into data blocks mainly according to the severity level of the logs. The logs are divided into two categories: critical logs and non-critical logs, and are respectively packaged into independent data blocks. Among them, the ERROR level and the FATAL level are critical logs, and the INFO level, DEBUG level, and TRACE level are non-critical logs; a single critical log can be set as a data block. If there are multiple consecutive critical logs in terms of time, then these critical logs can be organized into a data block. Each data block contains some metadata, such as an identifier, a priority (critical log or non-critical log), the content of the log, the number of retransmissions, etc.

[0024] The streaming transmission module mainly transmits the data blocks to the SOVD client in a streaming manner through HTTP streaming transmission technology. The streaming transmission module maintains continuous communication with the SOVD client through an HTTP long connection, avoiding the overhead of frequent connection establishment and disconnection; the log data is gradually sent to the client through chunked transfer encoding to ensure the real-time and continuity of the data.

[0025] The SOVD client mainly sends real-time log requests to the SOVD server through the REST API interface of the SOVD protocol. The real-time log requests include the type of logs, the time range of the logs required, and the filtering conditions. The type of logs is the type of logs to be obtained, such as system logs, application logs, etc.; the time range of the logs required is the time range of the logs to be obtained, such as the last 1 hour, the last 24 hours, etc.; the filtering conditions are the filtering conditions of the logs to be obtained, such as the log level (INFO, WARN, ERROR, etc.). The SOVD client receives the log entries returned by the SOVD server and extracts the href field therein. Directly access the URL pointed to by href, and obtain the log data in real time through HTTP streaming technology for real-time processing and analysis. Store the received log data in a local or remote server for subsequent analysis and auditing. Embodiment 2

[0026] As Figure 2 shown, in this embodiment, the SOVD server also includes an adaptive transmission module, a data cache and retransmission module, and a priority queue module.

[0027] Among them, the adaptive transmission module dynamically adjusts the data transmission rate according to the network conditions to ensure high real-time performance even in the case of unstable networks. It mainly controls the time interval for sending each data block through a timer. Whenever the timer triggers a callback function, it will dynamically adjust the number of data blocks sent this time according to the current network state; mainly adopts the AIMD (Additive Increase, Multiplicative Decrease) algorithm for flow control management; specifically, the AIMD algorithm uses the transmission result of the data block as the adjustment basis: when all the data blocks in the previous round are transmitted successfully, the algorithm will increase the data block push volume in the next round in an additive manner to gradually improve the utilization efficiency of the transmission bandwidth; when transmission failures occur (such as timeouts, packet losses, etc.), the algorithm will quickly reduce the data block push volume in the next round in a multiplicative manner to relieve possible network congestion problems. The data blocks that fail to be sent should be cached in memory until the transmission is successful. AIMD will adjust the number of data blocks, and once the data blocks are transmitted successfully, they can be removed from the cache, reducing the memory usage.

[0028] The priority queue module sets priorities for different types of log data to ensure that critical log data is transmitted first. All data blocks will be queued in a unified transmission cache queue according to the timestamps of log generation and sent strictly in chronological order, thus ensuring good time consistency and traceability of the log content received by the client. At the same time, the priority queue module will set different retransmission priorities based on the importance of the data blocks. Data blocks of critical logs will be given a higher retransmission priority, which can be configured to retransmit infinitely until successful, or set a maximum number of retransmissions higher than that of non-critical logs to ensure that important information can be reliably delivered under any network conditions. Non-critical logs adopt a relatively loose retransmission policy to reduce system resource occupancy and improve overall transmission efficiency. Through this mechanism, while ensuring the high availability and reliability of critical log transmission, the system performance and the consistency of transmission order can be taken into account, providing a stable and reliable data basis for the log collection and monitoring system.

[0029] The data caching and retransmission module mainly caches the data blocks that have not been successfully transmitted during network interruption and retransmits them after the network resumes to ensure data integrity. It realizes the dynamic management of the log cache queue through the time window control strategy. Each data block will be automatically marked with a timestamp when it is generated and stored in the cache queue in chronological order. The cache management module continuously maintains a sliding time window, for example, set to the last N minutes (such as 5 minutes). Data blocks of non-critical logs are only retained within the time window, and data blocks of non-critical logs that exceed the window time will be cleared and no longer retransmitted to avoid occupying resources with invalid data; data blocks of critical logs are not restricted by the time window. If they are in a state of transmission failure or waiting for retransmission, they will be preferentially retained until successful, or released after reaching the set maximum number of retransmissions. Embodiment 3

[0030] This embodiment discloses a method for real-time access to vehicle log data, which mainly includes the following two parts: Vehicle log generation part: During the operation of the application, the log generation and collation module generates vehicle diagnostic logs in real time. The vehicle diagnostic logs include data such as vehicle system operation logs and application logs, and writes the logs into a disk file. The naming and storage path of the log file can be set according to the configuration; for example, the log file can be named according to the timestamp and stored in a specified directory. And the log file path is passed to the SOVD server through the API interface of the SOVD dynamic library.

[0031] When the size of the current log file is greater than the file size threshold, or the time interval of the data received in the current log file is greater than the time interval threshold, a new log file is automatically created, and the path of the old file is passed to the SOVD server. In this embodiment, the file size threshold is 5M, and the time interval threshold is 1 minute.

[0032] Vehicle log request part: Step 1: The SOVD client sends a real-time log request to the SOVD server through the REST API interface of the SOVD protocol; Step 2: After receiving the log request, the SOVD server reads the log file according to the log file path and divides the log data into data blocks. At the same time, it generates log entries and sends them to the SOVD client; among them, the href field included in the log entry points to a log access URL that supports HTTP streaming.

[0033] Step 3: After receiving the log entry, the SOVD client extracts the href field therein and directly accesses the URL; Step 4: The SOVD server sends the data blocks to the SOVD client in the way of HTTP streaming; the SOVD client chooses to store the received log data locally or on a remote server for subsequent analysis and auditing.

[0034] Among them, in the process of the SOVD server sending the data blocks to the SOVD client in the way of HTTP streaming, it first maintains continuous communication with the SOVD client through an HTTP long connection, and gradually sends the data blocks to the SOVD client through chunked transfer encoding. The SOVD server divides the log data into data blocks mainly according to the severity level of the logs. The logs are divided into two categories: critical logs and non-critical logs, and are respectively packaged into independent data blocks. Among them, the ERROR level and the FATAL level are critical logs, and the INFO level, the DEBUG level, and the TRACE level are non-critical logs; a single critical log can be set as a data block. If there are multiple consecutive critical logs in terms of time, then these critical logs can be organized into a data block. Each data block contains some metadata, such as an identifier, a priority (critical log or non-critical log), the content of the log, the number of retransmissions, etc. Embodiment 4

[0035] In this embodiment, a method for real-time access to vehicle log data is disclosed. In this embodiment, during the process of the SOVD server sending the data blocks to the SOVD client in the way of HTTP streaming in Embodiment 3, an adaptive network transmission, a priority queue, and a data cache and retransmission part are added.

[0036] Among them, the priority queue part sets priorities for different types of log data to ensure that critical log data is transmitted first. All data blocks will be queued into a unified transmission cache queue according to the timestamps of the logs generated, and will be sent strictly in chronological order, so as to ensure good time consistency and traceability of the log content received by the client. At the same time, the priority queue module will set different retransmission priorities according to the importance of the data blocks. The data blocks of critical logs will be given a higher retransmission priority, which can be configured to retransmit infinitely until successful, or set a maximum retransmission times higher than that of non-critical logs to ensure that important information can be reliably delivered under any network conditions. Non-critical logs adopt a relatively loose retransmission strategy to reduce system resource occupancy and improve overall transmission efficiency.

[0037] The adaptive network transmission part mainly uses the AIMD (Additive Increase, Multiplicative Decrease) algorithm for flow control management; specifically, the AIMD algorithm uses the transmission results of data blocks as the adjustment basis: when all data blocks in the previous round are transmitted successfully, the algorithm will increase the data block push volume in the next round in an additive manner, gradually improving the utilization efficiency of the transmission bandwidth, so as to ensure high real-time performance even under unstable network conditions.

[0038] The data caching and retransmission part mainly caches the data blocks that have not been successfully transmitted when the network is interrupted and retransmits them after the network resumes to ensure data integrity. It realizes the dynamic management of the log cache queue through the time window control strategy. Each data block will be automatically marked with a timestamp when it is generated and stored in the cache queue in chronological order. The cache management module continuously maintains a sliding time window, for example, set to the most recent N minutes (such as 5 minutes). The data blocks of non-critical logs are only retained within the time window, and the data blocks of non-critical logs that exceed the window time will be cleared and no longer retransmitted to avoid invalid data occupying resources; the data blocks of critical logs are not restricted by the time window. If they are in a transmission failure or waiting for retransmission state, they will be preferentially retained until successful, or released after reaching the set maximum retransmission times.

[0039] In this embodiment, the security and privacy protection of log data during transmission are also ensured through TLS / SSL encryption and strict access control mechanisms.

[0040] The present invention also provides a computer system, including: one or more processors; a memory storing operable instructions, the instructions, when executed by the one or more processors, cause the one or more processors to perform operations, the operations including the process of the above vehicle diagnostic log data real-time access method.

[0041] It should be understood that the foregoing examples of the real-time access method based on the above vehicle diagnostic log data of the present invention can be implemented in any computer system including data storage and data processing. The foregoing computer system can be at least one electronic processing system or electronic device including a processor and a memory, such as a PC computer, whether it is a personal PC computer, a commercial PC computer, a graphics processing PC computer, or a server-level PC computer. These PC computers achieve wired and / or wireless data transmission, especially image data, through a data interface and / or a network interface.

[0042] In some other embodiments, the computer system can also be a server, especially a cloud server, with data storage, processing, and network communication functions.

[0043] A computer system as an example generally includes at least one processor, a memory, and a network interface connected by a system bus. The network interface is used to communicate with other devices / systems.

[0044] The processor is used to provide the computing and control of the system.

[0045] The memory includes non-volatile memory and a cache.

[0046] The non-volatile memory usually has a large storage capacity and can store an operating system and computer programs. These computer programs can include operable instructions that, when executed by one or more processors, enable the one or more processors to execute the process of the real-time access method based on the above vehicle diagnostic log data in the foregoing embodiments of the present invention.

[0047] In a required or reasonable implementation manner, the foregoing computer system, whether it is a PC device or a server, can also include more or fewer components than shown in the figure, or combinations, or adopt different hardware, software, and other different components or different deployment methods.

Claims

1. A real-time access system for vehicle diagnostic log data, characterized in that: It includes a log generation and sorting module, a SOVD server, a streaming module and a SOVD client; among which, The log generation and sorting module generates vehicle diagnostic logs and writes the generated log data into disk files; the log file path is passed to the SOVD server through the API interface of the SOVD dynamic library; The streaming module is integrated in the SOVD server. The SOVD server receives the log file path passed by the log generation and collation module through the API interface of the SOVD dynamic library and generates log entries. It reads the log file according to the log file path and divides the log data into data blocks. The streaming module streams the data blocks to the SOVD client through HTTP streaming technology. The SOVD client sends a real-time log request to the SOVD server through the REST API interface of the SOVD protocol; the SOVD client receives the log entries returned by the SOVD server, directly accesses the log data acquisition address, and obtains the log data in real time through HTTP streaming technology.

2. The vehicle diagnostic log data real-time access system according to claim 1, characterized in that: The href field contained in the log entry points to the log data acquisition address that supports HTTP streaming.

3. The vehicle diagnostic log data real-time access system according to claim 1, characterized in that: The log generation and arrangement module also includes the rotation of log files. When the size of the current log file is larger than the file size threshold, or the time interval of the data received in the current log file is larger than the time interval threshold, a new log file is automatically created and the old file path is passed to the SOVD server.

4. The vehicle diagnostic log data real-time access system according to claim 1, characterized in that: The SOVD server also includes an adaptive transmission module, a data cache and retransmission module, and a priority queue module; the adaptive transmission module uses the AIMD algorithm to dynamically adjust the data transmission rate according to network conditions; the data cache and retransmission module caches data that has not been successfully transmitted when the network is interrupted; the priority queue module sets priorities for different types of log data and transmits them according to the priority.

5. The vehicle diagnostic log data real-time access system according to claim 4, characterized in that: The adaptive transmission module uses the transmission results of the data blocks as the adjustment basis. When the data blocks of the previous round are all successfully transmitted, the data block push amount of the next round is increased in an additive manner; when the data blocks of the previous round are all failed to be transmitted, the data block push amount of the next round is reduced in a multiplicative manner.

6. The vehicle diagnostic log data real-time access system according to claim 1, characterized in that: Dividing the log data into data blocks means dividing the logs into two categories: key logs and non-key logs, and packaging them into independent data blocks respectively.

7. A method for real-time access to vehicle diagnostic log data, characterized in that: The following steps are involved: Step 1: SOVD client sends a real-time vehicle diagnostic log request to SOVD server; Step 2: After receiving the vehicle diagnostic log request, the SOVD server reads the log file according to the log file path and divides the log data into data blocks, and generates log entries and sends them to the SOVD client; Step 3: After receiving the log entry, the SOVD client directly accesses the log data acquisition address; Step 4: The SOVD server sends the data chunks to the SOVD client in HTTP streaming mode.

8. The method for real-time access to vehicle diagnostic log data according to claim 7, characterized in that: The href field contained in the log entry points to the log data acquisition address that supports HTTP streaming.

9. A computer-readable medium storing software, characterized in that: The software includes instructions that can be executed by one or more computers, and the instructions, through such execution, enable the one or more computers to perform operations, wherein the operations include the process of the vehicle diagnostic log data real-time access method as described in any one of claims 7-8.

10. A computer system, characterized in that: include: one or more processors; A memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors are caused to perform operations, wherein the operations include the process of the method for real-time access to vehicle diagnostic log data as described in any one of claims 7-8.

Images