Data transmission method and device, computer equipment and storage medium

Abstract

The invention relates to a data transmission method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring vehicle-end full-amount bus data, and filtering redundant information in the vehicle-end full-amount bus data by adopting a preset data acquisition rule to obtain filtered bus data; performing structure optimization on the filtered bus data according to a preset data structure to obtain vehicle-end bus data; wherein the data volume of the vehicle-end bus data is smaller than the data volume of the filtered bus data; and reporting the vehicle end bus data to a cloud end. By adopting the method, the traffic occupied by data transmission can be saved.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to a data transmission method, apparatus, computer equipment and storage medium. Background Art

[0002] With the advent of the era of intelligent vehicle networking, cockpit architecture adjustments have become increasingly complex, resulting in an increasing number of types of data that need to be collected by on-board data collection terminals.

[0003] In traditional vehicle-side data collection technology, the data collection method is often integrated into the vehicle-mounted terminal, and the collection method is relatively fixed. When the types of data to be collected are diverse, if the original data collection method is still maintained, the total amount of collected data will be too large, greatly increasing the traffic required for data transmission between the vehicle and the cloud.

[0004] Therefore, traditional technologies still have the problem of large traffic usage in transmitting data between the vehicle and the cloud. Summary of the invention

[0005] Based on this, it is necessary to provide a vehicle-side bus data processing method, device, computer equipment and storage medium that can save the traffic occupied by transmission data in order to address the above technical problems.

[0006] In a first aspect, the present application provides a data transmission method, which is applied to a vehicle side, and the data transmission method includes:

[0007] Acquire the full amount of bus data on the vehicle side, and use the preset data collection rules to filter out the redundant information in the full amount of bus data on the vehicle side to obtain filtered bus data;

[0008] Optimizing the structure of the filtered bus data according to a preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data;

[0009] Report the vehicle-side bus data to the cloud.

[0010] In one embodiment, the structure optimization of the filtered bus data according to the preset data structure to obtain the vehicle-side bus data includes:

[0011] Based on preset optimization rules, structural optimization is performed on the filtered bus data to obtain optimized bus data;

[0012] The optimized bus data is compressed to obtain vehicle-side bus data.

[0013] In one embodiment, the preset optimization rule includes structured data serialization; based on the preset optimization rule, the structure of the filtered bus data is optimized to obtain optimized bus data:

[0014] The filtered bus data is stored according to a structured data storage format to obtain optimized bus data.

[0015] In one embodiment, the preset optimization rule includes deduplication of identifiers; and the structure optimization of the filtered bus data based on the preset optimization rule to obtain the optimized bus data includes:

[0016] Identifiers for identifying a plurality of filter bus sub-data in the filter bus data;

[0017] Based on the multiple identifiers, the filtering bus sub-data with the same identifier are grouped together, and each group of the filtering bus sub-data retains one identifier;

[0018] Based on a plurality of sets of the filtered bus sub-data, optimized bus data is obtained.

[0019] In one embodiment, the preset optimization rule includes matrix inversion; the structure optimization based on the filtered bus data based on the preset optimization rule to obtain the optimized bus data includes:

[0020] The rows and columns of the matrix are inverted for each byte message data in the filtered bus data to obtain optimized bus data.

[0021] In one embodiment, compressing the optimized bus data to obtain vehicle-side bus data includes:

[0022] Compressing the optimized bus data to obtain compressed bus data;

[0023] The compressed bus data is marked with a version number to obtain vehicle-side bus data.

[0024] In one embodiment, the preset data collection rule includes identifier filtering; the method of filtering redundant information in the vehicle-side full bus data using the preset data collection rule to obtain filtered bus data includes:

[0025] Based on the preset identifier to be filtered, the data corresponding to the preset identifier to be filtered in the full amount of bus data on the vehicle side is filtered out to obtain filtered bus data.

[0026] In one embodiment, the preset data collection rule includes frame extraction collection; the method of filtering redundant information in the full amount of bus data on the vehicle side using the preset data collection rule to obtain filtered bus data includes:

[0027] Based on the preset frame extraction frequency, the full amount of bus data on the vehicle side is re-collected to obtain filtered bus data after the re-collection.

[0028] In one embodiment, the preset data collection rule includes change upload; the vehicle-side full bus data includes a plurality of full bus sub-data; the preset data collection rule is used to filter out redundant information in the vehicle-side full bus data to obtain filtered bus data, including:

[0029] Calculate the difference between the current full bus sub-data and the previous full bus sub-data in chronological order;

[0030] If the difference between the current full bus sub-data and the previous full bus sub-data is zero, the current full bus sub-data is filtered out to obtain filtered bus data.

[0031] In a second aspect, the present application provides a data transmission device, the data transmission device comprising:

[0032] The acquisition and filtering module is used to obtain the full amount of bus data on the vehicle side, and filter out the redundant information in the full amount of bus data on the vehicle side using the preset data acquisition rules to obtain filtered bus data;

[0033] A structure optimization module, used for optimizing the structure of the filtered bus data according to a preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data;

[0034] The data reporting module is used to report the vehicle-side bus data to the cloud.

[0035] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the data transmission method described above when executing the computer program.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the data transmission method as described above.

[0037] The above-mentioned vehicle-side bus data processing method, device, computer equipment and storage medium obtain the full amount of vehicle-side bus data, and use preset data acquisition rules to filter out redundant information in the full amount of vehicle-side bus data to obtain filtered bus data; optimize the structure of the filtered bus data according to the preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data; report the vehicle-side bus data to the cloud, which can achieve filtering of unnecessary data at the data acquisition level, and realize integration of redundant data in the process of data structure optimization to adjust the occupied space of the data to be transmitted to the ideal size, which can achieve the effect of saving the traffic occupied by the transmitted data. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A diagram showing an application environment of a data transmission method in an embodiment;

[0039] Figure 2 A schematic diagram of a flow chart of a data transmission method in an embodiment;

[0040] Figure 3 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0041] Figure 4 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0042] Figure 5 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0043] Figure 6 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0044] Figure 7 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0045] Figure 8 A schematic diagram of a flow chart of a data transmission method in another embodiment;

[0046] Fig. 9 A schematic diagram of a message data structure in another embodiment;

[0047] Fig.10 is a structural block diagram of a data transmission device in one embodiment;

[0048] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The data transmission method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 obtains the full amount of bus data on the vehicle side, and uses the preset data acquisition rules to filter out the redundant information in the full amount of bus data on the vehicle side to obtain the filtered bus data; the structure of the filtered bus data is optimized according to the preset data structure to obtain the vehicle side bus data; wherein the data volume of the vehicle side bus data is less than the data volume of the filtered bus data; reporting the vehicle side bus data to the cloud can be reporting the vehicle side bus data to the server 104. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, vehicle-mounted devices, etc. on the vehicle side. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0051] In one embodiment, Figure 2 As shown, a data transmission method is provided, which is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0052] Step S100, obtaining the full amount of bus data on the vehicle side, and using a preset data acquisition rule to filter out redundant information in the full amount of bus data on the vehicle side to obtain filtered bus data.

[0053] Among them, the full amount of bus data on the vehicle side can be obtained by collecting relevant data during the operation of the vehicle through multiple sensors and control units on the vehicle. Exemplarily, the full amount of bus data on the vehicle side can be collected through automobile bus protocols such as CAN (Controller Area Network) bus and LIN (Local Interconnect Network) bus. It can be collected through a specific data acquisition module, or the sensor can be connected to the bus system to obtain the full amount of collected data on the vehicle side. The full amount of bus data on the vehicle side includes but is not limited to data such as speed, acceleration, and throttle position.

[0054] The preset data collection rules may be collection rules for vehicle-side bus data pre-set on the vehicle side. It is understandable that the vehicle-side full bus data records the complete bus data collected by each sensor, but the total size of data reported by the vehicle side to the cloud is limited. As the vehicle side is upgraded through firmware or hardware, the limited bandwidth cannot meet the transmission of the required collected data. In this embodiment, by setting the preset data collection rules, the redundant information in the vehicle-side full bus data is filtered out to obtain the filtered necessary bus data, which can reduce the amount of data required for data transmission and save bandwidth.

[0055] Exemplarily, the preset data collection rules filter out redundant information of the full bus data on the vehicle side. The preset data collection rules may be used to resample the full bus data on the vehicle side. For example, the preset collection frequency, the preset data type that needs to be collected, and other rules may be followed.

[0056] Exemplarily, the preset data collection rules filter out redundant information of the full bus data on the vehicle side, and may be resampled according to a preset collection frequency, a preset collection object, a preset collection data type, etc., thereby filtering out redundant information that has not been resampled and recorded during the resampling process.

[0057] Step S200, optimizing the structure of the filtered bus data according to a preset data structure to obtain vehicle-side bus data.

[0058] The data volume of the vehicle-side bus data is smaller than the data volume of the filtering bus data. The filtering bus data is structurally optimized according to a preset data structure, which may be a structural rearrangement of the filtering bus data using a preset data structure. For example, the structural optimization of the filtering bus data may be implemented using data formats such as JSON and Protobuf, so that the data volume of the optimized vehicle-side bus data is smaller than the filtering bus data.

[0059] Furthermore, in the process of structural optimization, for data with a fixed pattern or periodic changes, differential encoding can be used to transmit only the changing part of the value, thereby reducing the amount of data.

[0060] Furthermore, for data with a fixed pattern or periodic changes, differential encoding can be used to obtain the changing part of the transmission value, thereby reducing the amount of data.

[0061] Step S300, reporting the vehicle-side bus data to the cloud.

[0062] The vehicle-side bus data can be reported to the cloud using a preset network protocol, such as HTTP protocol, FTP protocol, etc. To ensure the secure transmission of data, the vehicle-side bus data can also be encrypted so that the cloud uses a checksum to verify the integrity of the data, thereby achieving the effect of secure reporting to the cloud.

[0063] A data transmission method provided in this embodiment obtains full-volume bus data on the vehicle side, and uses preset data acquisition rules to filter out redundant information in the full-volume bus data on the vehicle side to obtain filtered bus data; structurally optimizes the filtered bus data according to a preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data; and reports the vehicle-side bus data to the cloud, thereby filtering unnecessary data at the data acquisition level, and integrating redundant data in the process of data structure optimization to compress the space occupied by the data to be transmitted to an ideal size, thereby achieving the effect of saving the traffic occupied by the transmitted data.

[0064] Among them, the vehicle-side bus data can be reported to the cloud using protocols such as HTTPS and MQTT. Furthermore, considering the security of data transmission, the vehicle-side bus data can also be encrypted so that the cloud can verify the integrity of the data through a checksum to achieve secure reporting.

[0065] A data transmission method provided in this embodiment obtains full-volume bus data on the vehicle side, and uses preset data acquisition rules to filter out redundant information in the full-volume bus data on the vehicle side to obtain filtered bus data; structurally optimizes the filtered bus data according to a preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data; and reports the vehicle-side bus data to the cloud, thereby filtering unnecessary data at the data acquisition level, and integrating redundant data in the process of data structure optimization, adjusting the space occupied by the data to be transmitted to an ideal size, and achieving the effect of saving the traffic occupied by the transmitted data.

[0066] In one embodiment, if Figure 3 As shown, the structure of the filtered bus data is optimized according to the preset data structure to obtain the vehicle-side bus data including:

[0067] Step S210, based on a preset optimization rule, structurally optimizing the filtered bus data to obtain optimized bus data;

[0068] Step S220, compressing the optimized bus data to obtain vehicle-side bus data.

[0069] Among them, the preset optimization rules can be optimization rules determined according to the characteristics and requirements of filtering bus data. Exemplarily, the preset optimization rules can be determined according to characteristics such as data type, data volume, data frequency, etc., and can also be determined according to the working status of the vehicle, environmental conditions and other requirements. Exemplarily, structural optimization can be achieved by using one or more methods of data reorganization, data type optimization, data deduplication, and specific data encoding. Among them, data sufficiency can be reordering data fields according to preset rules; data type optimization can be converting data into a more space-saving format. For example, for 32-bit integer data, when the value range is small, such as the values ​​are all within 8 bits, the placeholders of integer data can be reduced by type optimization; data deduplication can be to remove duplicate data records, and when duplicate data records appear again in the future, they can be represented by difference or reference; data encoding can be, for example, variable-length encoding, so as to dynamically adjust the encoding length according to the size of the value, thereby reducing the overall data volume.

[0070] Optimized bus data refers to filtered bus data after structural optimization. Exemplarily, optimized bus data may be filtered bus data obtained by processing according to preset optimization rules such as data type conversion, data volume compression, and data frequency adjustment.

[0071] The optimized bus data may be compressed by using a compression algorithm. For example, compression algorithms such as LZ77, Huffman, LZMA, and Brotli may be used.

[0072] The present embodiment provides a data transmission method, which can achieve the effect of saving the traffic occupied by the transmission data by converting the originally large filtering bus data into optimized bus data and further reducing the data volume through a compression algorithm.

[0073] In one embodiment, based on a preset optimization rule, the structure of the filtered bus data is optimized to obtain optimized bus data:

[0074] The filtered bus data is stored according to a structured data storage format to obtain optimized bus data.

[0075] Among them, the preset optimization rules include structured data serialization, which is used to convert unstructured data into structured data. Structured data serialization can be implemented in a variety of ways, such as serialization using data formats such as JSON, XML, and CSV. Through structured data serialization, the original data can be converted into structured data that is easy to process and analyze, thereby facilitating data storage, transmission, and processing.

[0076] Exemplarily, the structured data storage format may be a data storage format using Protobuf to achieve data serialization, which encodes data in a binary format, thereby improving transmission efficiency and data compression ratio.

[0077] A data transmission method provided in this embodiment can not only improve the readability of data, but also reduce the volume occupied by data and reduce the traffic consumption of data transmission by serializing filtered bus data according to a structured data storage format to obtain optimized bus data.

[0078] In one embodiment, if Figure 4 As shown, the structure optimization is performed on the filtered bus data based on the preset optimization rules to obtain the optimized bus data, including:

[0079] Step S310, identifying identifiers of a plurality of filter bus sub-data in the filter bus data;

[0080] Step S320, based on the multiple identifiers, grouping the filtered bus sub-data with the same identifier, and each group of the filtered bus sub-data retains one identifier;

[0081] Step S330, obtaining optimized bus data based on a plurality of sets of the filtered bus sub-data.

[0082] The preset optimization rules include identifier deduplication, which is used to remove multiple filter bus sub-data with the same identifier, retain one identifier, reduce data redundancy, and improve data processing efficiency.

[0083] It can be understood that each filtered bus sub-data corresponds to an identifier. In the full bus data on the vehicle side, multiple filtered bus sub-data will also have multiple repeated identifiers. Therefore, the identifiers can be deduplicated by grouping the filtered bus sub-data with the same identifier and retaining only one identifier.

[0084] A data transmission method provided in this embodiment can further reduce the size of the data volume by deduplicating identifiers, control the space occupied by the data to be transmitted to an ideal size, and achieve the effect of saving the traffic occupied by the transmitted data.

[0085] In one embodiment, the structure optimization based on the filtered bus data is performed based on a preset optimization rule to obtain the optimized bus data, including:

[0086] The rows and columns of the matrix are inverted for each byte message data in the filtered bus data to obtain optimized bus data.

[0087] Wherein, the preset optimization rule includes matrix inversion. Matrix inversion may be to change the rows of the matrix into columns and the columns into rows. Performing matrix row and column inversion on each byte message data in the filtering bus data may be to organize each byte message data in the filtering bus data into a matrix, which may be in the form of a one-dimensional array, a two-dimensional array or a multi-dimensional array, and performing matrix row and column inversion on each byte message data.

[0088] Furthermore, after the matrix is ​​inverted, the optimized bus data is obtained, and the optimized bus data can be further compressed. It can be understood that under some compression algorithms, the column data with generally different values ​​and with small differences can be converted into row data by inverting the matrix. When the compression algorithm compresses the row data, the data volume of the row data with small differences can be compressed to a smaller size, thereby controlling the space occupied by the data to be transmitted to an ideal size, which can achieve the effect of saving the traffic occupied by the transmitted data.

[0089] A data transmission method provided in this embodiment can optimize the structure of filtering bus data through matrix inversion, and also help data processing in subsequent processing to achieve the effect of improving data processing efficiency.

[0090] In one embodiment, if Figure 5 As shown, the optimized bus data is compressed to obtain vehicle-side bus data, including:

[0091] Step S410, compressing the optimized bus data to obtain compressed bus data;

[0092] Step S420, marking the compressed bus data with a version number to obtain vehicle-side bus data.

[0093] The data may be compressed using a corresponding compression algorithm according to the data characteristics, and the optional compression algorithms include but are not limited to the LZ77 algorithm, the LZMA algorithm, the Brotli algorithm, and the like.

[0094] The data transmission method provided in this embodiment can optimize bus data processing into a more compact form through the processing of a compression algorithm, thereby achieving the effect of improving data processing efficiency.

[0095] In one embodiment, the method of filtering out redundant information in the vehicle-side full bus data using a preset data collection rule to obtain filtered bus data includes:

[0096] Based on the preset identifier to be filtered, the data corresponding to the preset identifier to be filtered in the full amount of bus data on the vehicle side is filtered out to obtain filtered bus data.

[0097] Among them, the preset data collection rules include identifier filtering. It is understandable that when reporting the bus data of the vehicle side to the cloud, in the current scene (such as parking, driving scene), or in the current vehicle state (such as charging state), such as the current time, current location or even the vehicle-related firmware version, there are data corresponding to the identifier in the full amount of bus data on the vehicle side that does not need to be reported or is not necessary to be reported, and the identifiers of these data are defined as identifiers to be filtered.

[0098] In this embodiment, a preset identifier to be filtered is provided. By setting the preset identifier to be filtered, redundant parts of the identifier in the full bus data on the vehicle side can be filtered out.

[0099] A data transmission method provided in this embodiment can achieve the effect of saving the traffic occupied by the transmission data by filtering out the data corresponding to the preset identifier to be filtered in the full bus data on the vehicle side to obtain filtered bus data.

[0100] In one embodiment, the method of filtering out redundant information in the vehicle-side full bus data using a preset data collection rule to obtain filtered bus data includes:

[0101] Based on the preset frame extraction frequency, the full amount of bus data on the vehicle side is re-collected to obtain filtered bus data after the re-collection.

[0102] Among them, the preset data collection rules include frame collection, which means secondary collection of the full bus data on the vehicle side according to the preset frame frequency. The frame frequency is the number of times data is collected per unit time, and the frequency can be determined according to actual needs.

[0103] The full amount of bus data on the vehicle side is collected for the second time according to the preset frame sampling frequency. The time point corresponding to each collection at the preset frame sampling frequency can be calculated, and each data at the corresponding time point in the full amount of bus data on the vehicle side is searched for the second collection.

[0104] A data transmission method provided in this embodiment performs secondary collection of the full bus data on the vehicle side based on a preset frame extraction frequency to obtain filtered bus data after the secondary collection. This method can reduce the amount of data while retaining the data at key time points, thereby achieving the effect of saving the traffic occupied by the transmitted data.

[0105] In one embodiment, if Figure 6 As shown, the redundant information in the vehicle-side full bus data is filtered out by using preset data collection rules to obtain filtered bus data, including:

[0106] Step S510, calculating the difference between the current full bus sub-data and the previous full bus sub-data in chronological order;

[0107] Step S520: If the difference between the current full bus sub-data and the previous full bus sub-data is zero, the current full bus sub-data is filtered out to obtain filtered bus data.

[0108] Wherein, the preset data collection rules include change upload. Change upload can be uploaded when there is a change in the bus data. Exemplarily, the vehicle-side full bus data includes multiple full bus sub-data, and the multiple full bus sub-data are arranged in sequence at corresponding adjacent time points. The difference between the current full bus sub-data and the previous full bus sub-data is calculated in chronological order. If the difference between the current full bus sub-data and the previous full bus sub-data is zero, it means that the difference between the current full bus sub-data and the previous full bus sub-data has not changed, then the current full bus sub-data can be filtered out, and if the difference between the current full bus sub-data and the previous full bus sub-data is not zero, then the current full bus sub-data can be recorded as the difference during data collection.

[0109] It can be understood that the full bus sub-data of this embodiment can be data with the same identifier at multiple time points, and the vehicle-side full bus data can include multiple full bus sub-data groups, each full bus sub-data group includes multiple full bus sub-data.

[0110] A data transmission method provided in this embodiment calculates the difference between the current full bus sub-data and the previous full bus sub-data in chronological order; if the difference between the current full bus sub-data and the previous full bus sub-data is zero, the current full bus sub-data is filtered out to obtain filtered bus data, and the obtained filtered bus data only includes the changed data points, thereby reducing the data volume while retaining the data at the key time points, thereby achieving the effect of saving the traffic occupied by the transmitted data.

[0111] In order to more clearly illustrate the technical solution of the present application, the present application also provides a detailed embodiment.

[0112] In one embodiment, Figure 7 As shown, a data transmission method is provided, comprising:

[0113] The vehicle side collects all CAN messages of T-BOX in different data scenarios, which may include charging, parking, driving and other scenarios.

[0114] Data collection strategy: When collecting CAN message data on the vehicle side, the SDK (Software Development Kit) performs CAN ID filtering, CAN message frame extraction, CAN signal masking, CAN message change transmission and CAN message index deduplication to obtain filtered CAN messages.

[0115] Data optimization strategy: The vehicle side optimizes data according to version number, matrix inversion, Protobuf data format and time offset, that is, timestamp custom data structure, and deduplicates repeated CAN IDs in the data structure to obtain CAN messages after the optimization strategy.

[0116] Data compression strategy: compress data using the data compression algorithm ZSTD 12.

[0117] Version addition: Add a custom data structure version number to the compressed data, and finally upload the CAN message to the cloud.

[0118] Among them, the CAN message is the full bus data on the vehicle side, and the CAN ID is the identifier.

[0119] In a specific implementation of the preset data collection rules, data compression strategy, CAN ID filtering strategy, CAN message change transmission strategy, CAN message frame extraction strategy and CAN signal mask strategy are set. Among them, the data compression strategy adopts the ZSTD compression algorithm; the CAN ID filtering strategy is to collect CAN message data with a specified identifier; the CAN message change transmission is used to set whether to collect changes; the CAN message frame extraction strategy can be set to extract one frame every N seconds, where N is a positive number; the CAN signal mask strategy is to set the mask to be ignored, so that the signal mask is set according to the corresponding bit of the CAN message byte for some unused signals in the CAN message. Through this data structure, the collection time of the CAN message, the data compression level, the CAN message change transmission, and the message collection of the specified CAN ID can be preset, so as to prepare for the structural optimization and compression of the CAN message data.

[0120] like Figure 8As shown, after structural optimization, a data packet includes a fixed value, a version number, a timestamp and message data, wherein the fixed value is a 2-byte magic value, the version number is a 1-byte SDK version, and the timestamp is an 8-byte total absolute timestamp of the data packet. The message data includes multiple groups of CAN messages, wherein each group of CAN messages shares an identifier CAN ID. For example, the figure includes CAN messages from groups A to X, and the CAN messages of group A share a 4-byte A group identifier A CAN ID, and the CAN messages of group X share a 4-byte X group identifier X CAN ID. Each group of CAN messages is a CAN message set and a timestamp difference between the upper and lower frames of the CAN messages at different periodic frequencies, including CAN messages 1, CAN messages 2 to CAN messages n, and the timestamp difference is also represented by time offset 1, time offset 2 to time offset n, respectively. In this embodiment, the sampling period of a data packet is a period of 60 seconds.

[0121] The CAN message data of different CAN IDs in one data packet are assembled according to the above structure. Using this structure can avoid the traffic occupation problem caused by repeated storage of CAN IDs.

[0122] In a specific implementation of the data structure optimization strategy, a CAN ID deduplication strategy, a time difference strategy and a matrix conversion strategy are set, wherein the full packet timestamp can be defined first, and the map structure can be used to remove the identifier CAN ID of the CAN message with the same CAN ID for deduplication to reduce storage and traffic consumption. For CAN messages with the same CAN ID, the difference between the upper and lower frame timestamps is calculated to reduce storage and traffic consumption. CAN message data of different CAN IDs are collected within a preset period, and the matrix inversion strategy is performed on the collected data packets, and the values ​​with obvious regular characteristics of the CAN message are transposed to the same row as much as possible to improve the compression efficiency. Using the Protobuf data format for processing can reduce data storage and traffic consumption to the optimal state. Adding a version number to the data packet can distinguish data optimization strategies at different stages for targeted data restoration in the cloud. The above-mentioned map structure is an associative container in the C++ programming language, which stores data in the form of key-value pairs. It can also be achieved by using other containers or tools that can reflect the mapping relationship to achieve identifier deduplication, which is not limited in this embodiment.

[0123] Furthermore, for CAN ID deduplication, a map set may be used to deduplicate the same CANID within a unit time, thereby achieving the effect of saving storage and traffic.

[0124] From the perspective of CAN message data structure, the important data parts are mainly CAN ID, CAN message and timestamp. Assembling them into sets according to periodic data cannot minimize storage and traffic. By observing CAN messages, it can be found that the timestamp can set a time offset strategy; in a certain period of time, the CAN message set finds that the data in the same row is irregular, and the data values ​​in the same column are more similar. If the column data is converted into row data, the data compression efficiency will be higher. At the same time, CAN ID duplication in the custom data structure can avoid this problem.

[0125] The time offset is the difference between the absolute timestamps of the upper and lower frames of the same signal. By using the time offset instead of the absolute timestamp, the storage and traffic occupied by time recording can be reduced.

[0126] Matrix inversion is row-column conversion. Since the byte data of all CAN messages in the same cycle change differently, the compression effect will be poor if no regular pattern can be found. After observing the CAN message data, the inventor found that the change pattern of each frame is often the middle message. Therefore, if the matrix is ​​inverted, such as Fig. 9 As shown, the 8-byte matrix of N frames can be inverted 90 degrees clockwise, so that most of the row data after inversion presents a regular arrangement or change, so that the compression algorithm can achieve a better compression effect during the compression process due to obvious regularity.

[0127] This embodiment provides a data transmission method, based on the CAN message mode, specifies preset data collection rules in the CAN message collection stage to prepare for reducing storage space and traffic consumption of CAN message data; in the CAN message data stage, a custom data storage structure is used to solve the problem of duplicate IDs of the same CAN message, and combined with the time difference between the upper and lower frames of the same CAN message and the inversion strategy of the matrix of all CAN messages in a unit period, the data with the same numerical rules of the CAN message is transposed in the same row as much as possible to provide data compression efficiency. Through the above data structure means, the cost of CAN message data storage and traffic consumption can be significantly reduced.

[0128] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0129] Based on the same inventive concept, the embodiment of the present application also provides a data transmission device for implementing the data transmission method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data transmission device embodiments provided below can refer to the limitations on the data transmission method above, and will not be repeated here.

[0130] In one embodiment, Fig.10 As shown, a data transmission device is provided, including: a collection and filtering module 100, a structure optimization module 200 and a data reporting module 300, wherein:

[0131] The collection and filtering module 100 is used to obtain the full amount of bus data on the vehicle side, and filter out the redundant information in the full amount of bus data on the vehicle side using a preset data collection rule to obtain filtered bus data;

[0132] A structure optimization module 200, configured to perform structure optimization on the filtered bus data according to a preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data;

[0133] The data reporting module 300 is used to report the vehicle-side bus data to the cloud.

[0134] In one embodiment, the structure optimization module 200 is further used to:

[0135] Based on preset optimization rules, structural optimization is performed on the filtered bus data to obtain optimized bus data;

[0136] The optimized bus data is compressed to obtain vehicle-side bus data.

[0137] In one embodiment, the structure optimization module 200 is further used to:

[0138] The filtered bus data is stored according to a structured data storage format to obtain optimized bus data.

[0139] In one embodiment, the preset optimization rule includes identifier deduplication; the structure optimization module 200 is further used for:

[0140] Identifiers for identifying a plurality of filter bus sub-data in the filter bus data;

[0141] Based on the multiple identifiers, the filtering bus sub-data with the same identifier are grouped together, and each group of the filtering bus sub-data retains one identifier;

[0142] Based on a plurality of sets of the filtered bus sub-data, optimized bus data is obtained.

[0143] In one embodiment, the preset optimization rule includes matrix inversion; the structure optimization module 200 is further used for:

[0144] The rows and columns of the matrix are inverted for each byte message data in the filtered bus data to obtain optimized bus data.

[0145] In one embodiment, the structure optimization module 200 is further used to:

[0146] Compressing the optimized bus data to obtain compressed bus data;

[0147] The compressed bus data is marked with a version number to obtain vehicle-side bus data.

[0148] In one embodiment, the preset data collection rule includes identifier filtering; the collection and filtering module 100 is further used to:

[0149] Based on the preset identifier to be filtered, the data corresponding to the preset identifier to be filtered in the full amount of bus data on the vehicle side is filtered out to obtain filtered bus data.

[0150] In one embodiment, the preset data collection rule includes frame extraction collection; the collection and filtering module 100 is further used for:

[0151] Based on the preset frame extraction frequency, the full amount of bus data on the vehicle side is re-collected to obtain filtered bus data after the re-collection.

[0152] In one embodiment, the preset data collection rule includes change upload; the vehicle-side full bus data includes a plurality of full bus sub-data; the collection and filtering module 100 is further used for:

[0153] Calculate the difference between the current full bus sub-data and the previous full bus sub-data in chronological order;

[0154] If the difference between the current full bus sub-data and the previous full bus sub-data is zero, the current full bus sub-data is filtered out to obtain filtered bus data.

[0155] Each module in the above data transmission device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0156] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data transmission method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0157] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the data transmission method of any of the above embodiments is implemented:

[0159] Acquire the full amount of bus data on the vehicle side, and use the preset data collection rules to filter out the redundant information in the full amount of bus data on the vehicle side to obtain filtered bus data;

[0160] Optimizing the structure of the filtered bus data according to a preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data;

[0161] Report the vehicle-side bus data to the cloud.

[0162] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data transmission method of any of the above embodiments is implemented:

[0163] Acquire the full amount of bus data on the vehicle side, and use the preset data collection rules to filter out the redundant information in the full amount of bus data on the vehicle side to obtain filtered bus data;

[0164] Optimizing the structure of the filtered bus data according to a preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data;

[0165] Report the vehicle-side bus data to the cloud.

[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0168] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data transmission method, characterized in that: Applied to the vehicle side, the data transmission method includes: Acquire the full amount of bus data on the vehicle side, and use the preset data collection rules to filter out the redundant information in the full amount of bus data on the vehicle side to obtain filtered bus data; Optimizing the structure of the filtered bus data according to a preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data; Report the vehicle-side bus data to the cloud.

2. The data transmission method according to claim 1, characterized in that: The structure of the filtered bus data is optimized according to the preset data structure to obtain the vehicle-side bus data, including: Based on preset optimization rules, structural optimization is performed on the filtered bus data to obtain optimized bus data; The optimized bus data is compressed to obtain vehicle-side bus data.

3. The data transmission method according to claim 2, characterized in that: The preset optimization rules include structured data serialization; based on the preset optimization rules, the structure of the filtered bus data is optimized to obtain optimized bus data: The filtered bus data is stored according to a structured data storage format to obtain optimized bus data.

4. The data transmission method according to claim 2, characterized in that: The preset optimization rule includes deduplication of identifiers; the structure optimization of the filtered bus data based on the preset optimization rule to obtain the optimized bus data includes: Identifiers for identifying a plurality of filter bus sub-data in the filter bus data; Based on the multiple identifiers, the filtering bus sub-data with the same identifier are grouped together, and each group of the filtering bus sub-data retains one identifier; Based on a plurality of sets of the filtered bus sub-data, optimized bus data is obtained.

5. The data transmission method according to claim 2, characterized in that: The preset optimization rule includes matrix inversion; the structure optimization based on the preset optimization rule is performed on the filtered bus data to obtain the optimized bus data, including: The rows and columns of the matrix are inverted for each byte message data in the filtered bus data to obtain optimized bus data.

6. The data transmission method according to claim 2, characterized in that: The performing data compression on the optimized bus data to obtain vehicle-side bus data comprises: Compressing the optimized bus data to obtain compressed bus data; The compressed bus data is marked with a version number to obtain vehicle-side bus data.

7. The data transmission method according to claim 1, characterized in that: The preset data collection rule includes identifier filtering; the preset data collection rule is used to filter out redundant information in the vehicle-side full bus data to obtain filtered bus data, including: Based on the preset identifier to be filtered, the data corresponding to the preset identifier to be filtered in the full amount of bus data on the vehicle side is filtered out to obtain filtered bus data.

8. The data transmission method according to claim 1, characterized in that: The preset data collection rules include frame extraction collection; The method of filtering out redundant information in the vehicle-side full bus data using a preset data collection rule to obtain filtered bus data includes: Based on the preset frame extraction frequency, the full amount of bus data on the vehicle side is re-collected to obtain filtered bus data after the re-collection.

9. The data transmission method according to claim 1, characterized in that: The preset data collection rules include change upload; the vehicle-side full bus data includes a plurality of full bus sub-data; The method of filtering out redundant information in the vehicle-side full bus data using a preset data collection rule to obtain filtered bus data includes: Calculate the difference between the current full bus sub-data and the previous full bus sub-data in chronological order; If the difference between the current full bus sub-data and the previous full bus sub-data is zero, the current full bus sub-data is filtered out to obtain filtered bus data.

10. A data transmission device, characterized in that: The data transmission device comprises: The acquisition and filtering module is used to obtain the full amount of bus data on the vehicle side, and filter out the redundant information in the full amount of bus data on the vehicle side using the preset data acquisition rules to obtain filtered bus data; A structure optimization module, used for optimizing the structure of the filtered bus data according to a preset data structure to obtain vehicle-side bus data; wherein the data volume of the vehicle-side bus data is smaller than the data volume of the filtered bus data; The data reporting module is used to report the vehicle-side bus data to the cloud.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the data transmission method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data transmission method according to any one of claims 1 to 9 is implemented.

Images