Data processing method

Abstract

The invention discloses a data processing method. The method comprises the steps of obtaining a target data type; based on a target data type, determining preparation data corresponding to the target data type; collecting target data based on the change rate of the preparatory data; and compressing the collected data, and sending the compressed data to the cloud. The data acquisition process of the vehicle end is accurately controlled through the cloud end, unnecessary data uploading is reduced, the pressure of network bandwidth is reduced, and meanwhile storage resources of the vehicle end are saved. The sampling frequency is flexibly adjusted according to the actual change condition of the data and the network condition, uploading of a large amount of redundant data is avoided, and the data transmission efficiency is improved. By compressing the collected data, the space and time required for data transmission are reduced, and the pressure of network bandwidth is further relieved.

Description

Technical Field

[0001] The present application relates to the field of automotive electronic technology, and in particular to a data processing method. Background Art

[0002] With the rapid development of intelligent transportation systems, Internet of Vehicles (IoV) technology has been widely used. In an IoV environment, vehicles can not only collect a large amount of driving data, such as location information, speed, acceleration, environmental perception data, etc., but also generate more data streams through on-board sensors and other devices. These data are essential for realizing intelligent driving, traffic management optimization, accident prevention and other functions. However, with the surge in data volume, data processing and transmission face unprecedented challenges.

[0003] In order to cope with the huge amount of data, existing data processing solutions usually use general compression algorithms to reduce the data volume. These algorithms achieve the purpose of compression by identifying repeated patterns in the data and using pointers to replace repeated data fragments. Although these methods reduce the storage space requirements of data to a certain extent, they are not optimized for the characteristics of Internet of Vehicles data. Specifically, the data generated by vehicles often contains a lot of redundant information. For example, in traffic congestion, multiple vehicles may report similar location and speed information; in a stable driving state, the status information of the vehicle changes little, and the difference in the continuously reported data is not large. Therefore, even after compression, these data still occupy a large volume. The huge amount of data not only increases the storage cost, but more importantly, it leads to a high occupancy rate of network bandwidth. In the case of limited network resources, the frequent transmission of large amounts of data may cause network congestion, which in turn causes data transmission delays.

[0004] Therefore, in the Internet of Vehicles environment, traditional data processing methods are difficult to meet the needs of efficient and fast data transmission.

[0005] Application Contents

[0006] Based on the above problems, the present application provides a data processing method.

[0007] The embodiments of the present application disclose the following technical solutions:

[0008] Obtaining a target data type, where the target data type is the data type that the vehicle indicates needs to be collected;

[0009] Based on the target data type, determining the preliminary data corresponding to the target data type;

[0010] Based on the change rate of the prepared data, collecting target data;

[0011] Compress the collected data and send the compressed data to the cloud.

[0012] In a possible implementation, collecting target data based on the change rate of the preparation data includes:

[0013] When the change rate of the prepared data is greater than a first threshold, obtaining an initial sampling frequency;

[0014] When the change rate of the prepared data is greater than a second threshold, increasing the initial sampling frequency;

[0015] When the change rate of the prepared data is less than a third threshold and greater than a second threshold, lowering the initial sampling frequency;

[0016] The target data is collected based on the adjusted initial sampling frequency.

[0017] In a possible implementation, collecting target data based on the change rate of the preparation data includes:

[0018] When the change rate of the prepared data is not greater than a first threshold, determining whether the target data is preset key data;

[0019] When the target data is preset key data, collecting the target data;

[0020] In a possible implementation, the method further includes:

[0021] When the target data is not preset key data, the target data is collected based on a preset period.

[0022] In a possible implementation, the method further includes:

[0023] When the target data is not the preset key data, when the target data changes, the changed data is collected.

[0024] In a possible implementation, collecting target data based on the change rate of the preparation data includes:

[0025] When the change rate of the prepared data is greater than a first threshold and the initial sampling frequency cannot be received from the cloud, obtaining the locally stored prepared sampling frequency;

[0026] The target data is collected based on the prepared sampling frequency.

[0027] In a possible implementation, before compressing the collected data and sending the compressed data to the cloud, the method further includes:

[0028] The compressed data is written into the local cache area to obtain a data copy.

[0029] In a possible implementation, after compressing the collected data and sending the compressed data to the cloud, the method further includes:

[0030] When the cloud fails to receive the compressed data, the compressed data is written to the local cache area to obtain a copy of the data.

[0031] In a possible implementation, the method further includes:

[0032] When the compressed data has been uploaded to the cloud, the data copy in the local cache area is cleared.

[0033] In a possible implementation, the method further includes:

[0034] When the cloud fails to receive the compressed data after a preset period of time, and / or the local cache space reaches a preset threshold, a circular overwriting strategy is used to adjust the data in the local cache area, wherein the circular overwriting strategy is used to indicate that the latest data is to be acquired to overwrite the earliest data.

[0035] Compared with the prior art, this application has the following beneficial effects:

[0036] First, identify and obtain the target data type that needs to be collected. Based on the target data type, determine the preliminary data associated with the target data type. Based on the change rate of the preliminary data, collect the target data; compress the collected data, and send the compressed data to the cloud. By accurately controlling the data collection process on the vehicle side through the cloud, unnecessary data uploads are reduced, thereby alleviating the pressure on the network bandwidth and saving storage resources on the vehicle side. The sampling strategy is flexibly adjusted according to the actual changes in the data to avoid uploading a large amount of redundant data at the same time, thereby improving the efficiency of data transmission. By compressing the collected data, the space and time required for data transmission are reduced, further alleviating the pressure on the network bandwidth and improving the efficiency of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 An interactive schematic diagram provided for an embodiment of the present application;

[0039] Figure 2A flowchart of a data processing method provided in an embodiment of the present application;

[0040] Figure 3 A data collection logic schematic diagram provided for an embodiment of the present application;

[0041] Figure 4 A structural diagram of a data processing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the professional terms involved in the embodiments of the present application will be explained below.

[0044] The multi-master serial communication protocol (Controller Area Network, CAN) is widely used in automobiles and other embedded systems. It allows microcontrollers and devices to exchange information with each other without the need for a central node. The CAN bus has the characteristics of high reliability, low cost and easy implementation, making it one of the preferred field buses in the automotive industry.

[0045] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0046] As mentioned above, in modern smart connected cars, uploading vehicle-side data to the cloud has become an indispensable technical means. This data transmission method not only helps vehicle manufacturers and service providers better understand the operating status of the vehicle, but also provides strong support for improving the performance, safety and reliability of the vehicle. In the existing Internet of Vehicles technology, the car directly compresses the collected data (such as vehicle CAN data, buried points, etc.) using compression algorithms such as LZ77 encoding (Lempel-Ziv-77, LZ77) or LZ78 encoding (Lempel-Ziv-78, LZ78), and then uploads it to the cloud.

[0047] Although compression algorithms such as LZ77 or LZ78 are used to reduce the data volume, the amount of data generated by vehicles is huge, especially containing a lot of redundant information, so the compressed data is still huge. This leads to high occupancy of network bandwidth, which may cause network delays and affect user experience, especially in poor network conditions, where this impact is more obvious.

[0048] In order to ensure that data is not lost when network conditions are poor, data is usually stored in the vehicle system first. However, as the amount of data grows, the storage space of the vehicle system will gradually become tight, and there may even be insufficient storage space, which not only affects the recording of new data, but also may affect the normal operation of other functions of the vehicle.

[0049] Existing methods mainly focus on how to reduce the bandwidth occupied during data transmission, but not enough on how to process and utilize this data more effectively. A large amount of duplicate data uploaded to the cloud increases the data processing burden of the cloud server and reduces the efficiency of data processing.

[0050] Moreover, as more and more sensitive data such as personal driving habits and vehicle location information are uploaded, without proper encryption measures, it may cause user privacy leakage and security risks.

[0051] In order to solve the above problems, the embodiment of the present application accurately controls the data collection type and frequency on the vehicle side through the cloud, reduces unnecessary data uploads, thereby alleviating the pressure on network bandwidth and saving storage resources on the vehicle side. The sampling frequency is flexibly adjusted according to the actual changes in the data and the network conditions, avoiding the upload of a large amount of redundant data and improving the efficiency of data transmission. Through an efficient lossless data compression algorithm, the space and time required for data transmission are reduced, further alleviating the pressure on network bandwidth. The encryption algorithm is used to encrypt data, which enhances the security of data transmission, prevents data from being intercepted or tampered with during transmission, and protects user privacy. By caching data locally on the vehicle side, the problem of data upload failure when the network is unstable is solved, ensuring the integrity and availability of the data.

[0052] In summary, through a series of technical means such as cloud-based configuration, intelligent dynamic sampling, data compression optimization, security enhancement and data caching, the problems of high network bandwidth occupancy, insufficient storage space, low data processing efficiency and privacy security existing in the existing technology have been effectively solved, the amount of data uploaded to the car's Internet of Vehicles has been reduced, the data transmission efficiency has been improved, the impact of the vehicle-side network environment on the user's Internet experience has been avoided, the bandwidth problem of the car's Internet of Vehicles uploading data has been solved, and user privacy and data security have been guaranteed.

[0053] It should be noted that the data processing method provided by the present application can be applied to the field of automotive electronic technology. The above is only an example and does not limit the application field of the data processing method provided by the present application. In addition, the embodiments of the present application may not limit the execution subject of data processing. For example, the data processing method of the embodiments of the present application can be applied to data processing devices such as terminal devices or servers. Among them, the terminal device can be an electronic device such as a computer, a personal digital assistant (PDA), etc. The server can be a stand-alone server, a cloud server, or a cluster server composed of multiple servers.

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] The present application embodiment involves the interaction between the cloud, the vehicle-side gateway and the CAN bus. Figure 1 , Figure 1 Interaction diagram provided for an embodiment of the present application. Among them, the cloud is responsible for sending data upload configuration to the vehicle-side gateway. The vehicle-side gateway is used to receive data upload configuration from the cloud. Perform data collection and edge computing according to the configuration. Report the processed data to the cloud. The CAN bus is part of the vehicle-mounted communication system and is used to connect various electronic control units (ECUs), such as engine control units, body control systems, etc. In this scenario, the CAN bus is responsible for collecting various operating data of the vehicle, such as sensor data such as speed, rotation speed, and throttle position. The vehicle-side gateway sends and receives data with the CAN bus to obtain various operating data of the vehicle.

[0056] First, the cloud sends a command to the vehicle-side gateway to "send data upload configuration". This command contains configuration information about data collection, upload frequency, data type, etc. After receiving the configuration command from the cloud, the vehicle-side gateway performs corresponding data sending and receiving operations to prepare for subsequent data collection and upload. The collected data is transmitted to the vehicle-side gateway via the CAN bus. The vehicle-side gateway reads the data and processes it accordingly according to the configuration sent from the cloud.

[0057] The following is an example of an embodiment to illustrate a data processing method provided by the present application. Figure 2 ,Should Figure 2 A flowchart of a data processing method provided in an embodiment of the present application, the method comprising:

[0058] S101. Obtain target data type.

[0059] The target data type is the data type reported by the vehicle side through the configuration sent by the cloud. That is, the vehicle side needs to determine what type of data needs to be collected, such as temperature, humidity, location information, etc. This step helps to select appropriate preliminary data and formulate corresponding data collection plans in subsequent steps.

[0060] The data collection configuration can formulate a set of data collection rules for the cloud platform according to specific needs or standards, and send these rules in electronic form to the device installed on the vehicle - the vehicle-side gateway. These rules usually include but are not limited to: Data type: refers to the type of data that needs to be collected, such as the vehicle's location information, speed, fuel consumption, engine temperature, tire pressure, etc. Data length: The maximum or minimum record length of each data item, which helps to optimize storage space and reduce unnecessary data transmission. Collection frequency: The time interval for data collection determines the freshness of the data. Threshold for adjusting the collection frequency: Automatically adjust the boundary value of the collection frequency based on certain conditions (such as changes in vehicle status, network conditions, etc.).

[0061] The cloud platform first generates a data collection configuration based on the application scenario or user needs. This configuration is organized in a structured way, and commonly used formats include JavaScript Object Notation (JSON) and eXtensible Markup Language (XML). Both formats are easy for machines to read and parse, and are also convenient for humans to read, and are suitable for cross-platform data exchange.

[0062] The configuration file is sent to the vehicle-side gateway via a mobile network (such as 4G, 5G, or future 6G). The mobile network is chosen as the transmission medium because it can cover a wide geographical area while providing relatively stable connection quality to ensure the reliability and timeliness of data transmission. In order to enhance the security of the configuration file, encryption technology such as Secure Sockets Layer (SSL) or Transport Layer Security (TLS) can be used during the transmission of the configuration file to protect the security of the data and prevent the data from being intercepted or tampered with by a third party.

[0063] After receiving the data collection configuration sent by the cloud, the vehicle-side gateway parses it and stores it in the local memory or hard disk. This step is crucial to ensure that subsequent data collection activities can be executed according to the predetermined rules.

[0064] After completing the storage of the configuration information, the vehicle-side gateway sends a confirmation message to the cloud through the same mobile network channel. This confirmation message is used to inform the cloud that the configuration information has been correctly received and saved. It can also serve as a feedback mechanism for the communication between the two parties to help detect and correct possible transmission errors.

[0065] S102: Based on the target data type, determine the preliminary data corresponding to the target data type.

[0066] For each target data type, a certain amount of historical data is collected and stored in advance as preliminary data. The preliminary data is used to calculate the data change rate and thus determine the subsequent sampling frequency.

[0067] According to different application scenarios and application requirements, determine the type of target data that needs to be collected, such as sensor data, environmental monitoring data, etc.

[0068] S102: Based on the change rate of the prepared data, target data is collected.

[0069] By analyzing the prepared data, the change rate is calculated. The change rate can be determined by the difference between adjacent data points or other statistical methods.

[0070] Before starting data collection, the vehicle needs to monitor the change rate of the prepared data. The prepared data can be historical data, predicted data, or other auxiliary information that is closely related to the target data. By continuously monitoring the change rate of the prepared data, the vehicle can understand the change trend of the target data in real time.

[0071] See also Figure 3 , Figure 3 A data collection logic diagram provided for an embodiment of the present application. Among them, the process starts with receiving a data upload configuration instruction from the cloud. According to the configuration of the cloud, the data collection process is started. Determine whether the data to be collected currently belongs to "frequently changing data" or "non-frequently changing data". For frequently changing data, set the initial sampling frequency and threshold. If the rate of change of the data is lower than the threshold, reduce the sampling frequency. If the rate of change of the data is higher than the threshold, increase the sampling frequency. Finally decide whether to collect data. For non-frequently changing data, determine whether it is critical data. If it is non-critical data, determine whether the cycle time has arrived. If the cycle time has arrived, data collection is performed; otherwise, no collection is performed. If it is critical data, data collection is performed directly. After completing the above decisions, the process ends. The data after the above screening will be processed by an efficient lossless compression algorithm to further reduce the data volume. The compressed data is then securely uploaded to the cloud server for subsequent analysis.

[0072] Through the above process, this application can not only significantly reduce unnecessary data transmission, but also greatly improve data processing efficiency and service response speed while ensuring data quality, thereby improving the overall user experience. In addition, this method also helps to extend the battery life of vehicle-mounted equipment and reduce operating costs.

[0073] The following describes how to collect target data in various situations.

[0074] In a possible implementation, collecting target data based on the change rate of the preparation data includes:

[0075] When the change rate of the prepared data is greater than a first threshold, an initial sampling frequency is obtained; when the change rate of the prepared data is greater than a second threshold, the initial sampling frequency is increased; when the change rate of the prepared data is less than a third threshold and greater than the second threshold, the initial sampling frequency is lowered; and target data is collected based on the adjusted initial sampling frequency.

[0076] The first threshold is a preset critical value used to determine whether the prepared data begins to change significantly. Once the change rate of the prepared data exceeds this threshold, the vehicle will start the data collection process and obtain an initial sampling frequency. The initial sampling frequency can be determined based on the specific application scenario and data type, or it can be provided by the cloud, such as collecting data once per second.

[0077] The second threshold is higher than the first threshold, indicating that the change of the prepared data is more drastic. In this case, in order to capture more data details, the vehicle end can automatically increase the sampling frequency, for example, from once per second to twice per second. This ensures that important information is not missed during the stage of rapid data changes.

[0078] The third threshold is lower than the second threshold but higher than the first threshold, indicating that the change of the prepared data has stabilized but still has certain fluctuations. At this time, the vehicle side can appropriately reduce the sampling frequency, for example, from twice per second to once per second. This can reduce unnecessary data collection and save resources.

[0079] Once the final sampling frequency is determined, the system will collect the target data according to this frequency. The collected data is stored in the local buffer and compressed and transmitted to the cloud after reaching a certain amount.

[0080] By dynamically adjusting the sampling frequency, this method can better adapt to data changes in different situations, ensuring that more data is collected when data changes dramatically, and reducing unnecessary collection when data changes slowly, thereby improving the efficiency of data collection. Reasonable adjustment of the sampling frequency can reduce unnecessary data collection, reduce equipment energy consumption, and extend the service life of the equipment. Increasing the sampling frequency when data changes dramatically can more accurately capture the details of data changes and improve the quality and value of data.

[0081] For data that does not change frequently, that is, the change rate of the prepared data is not greater than the first threshold, different collection strategies are adopted by distinguishing the importance of the data to improve the efficiency and quality of data collection. Specifically, for data that changes less frequently, identify which are key data and which are non-key data, and formulate corresponding collection strategies accordingly. In a possible implementation, the target data is collected based on the change rate of the prepared data, including:

[0082] When the change rate of the prepared data is not greater than the first threshold, it is determined whether the target data is the preset key data; when the target data is the preset key data, the target data is collected; when the target data is not the preset key data, the target data is collected based on a preset period. Alternatively, when the target data is not the preset key data, when the target data changes, the changed data is collected.

[0083] When the change rate of the prepared data is not greater than the first threshold, the vehicle can enter a more cautious collection mode. At this time, the system checks whether the target data belongs to the preset key data. The preset key data refers to those data that are very important even at a low rate of change and need to be monitored regularly or at any time. These data are usually closely related to the safety, performance or core functions of the system.

[0084] If the target data is confirmed as preset key data, the vehicle side can collect it regardless of the change rate of the prepared data. This ensures the integrity and timeliness of key data, even when the system as a whole is relatively stable.

[0085] For non-preset critical data, if the change rate of the prepared data is not greater than the first threshold, the system will collect data according to the preset period. The preset period can be set according to the specific application scenario and needs, for example, collect data once an hour, and force data reporting at the end of this period. This method can reduce unnecessary data collection and save resources. In addition, it ensures that even if the data has not changed significantly for a long time, the data status can be checked regularly to prevent data loss due to network failures and other reasons.

[0086] In addition to collecting data based on preset cycles, the system can also monitor changes in target data. For non-critical data, the initial value is collected and recorded for the first time. After that, only when the data changes, the system immediately collects the changed data and reports it. This mechanism ensures that any changes in the data are captured in time to avoid missing important data updates.

[0087] By identifying and focusing on the collection of preset key data, the integrity and timeliness of key data are ensured, which helps the stable operation and troubleshooting of the system. For non-critical data, the collection strategy combining preset cycles and change triggers can effectively reduce unnecessary data collection, reduce system load and resource consumption.

[0088] In a possible implementation, when the threshold for adjusting the sampling frequency is not received from the cloud, the system should automatically switch to using the default threshold stored locally to adjust the sampling frequency. This ensures that data collection activities can continue and the sampling frequency remains at a reasonable level even when the cloud connection cannot be reached.

[0089] In one possible implementation, at the initial startup of the system, regardless of whether the current data has reached the regular data reporting sampling period, the system should perform a full sampling of all data. This operation helps ensure that the latest data status can be quickly restored after the system is restarted, and provides a benchmark for subsequent incremental data collection.

[0090] S103: compress the collected data and send the compressed data to the cloud.

[0091] The collected data is compressed to reduce the data volume.

[0092] For example, an efficient lossless data compression algorithm such as LZ77, LZ78, etc. can be used. These algorithms can significantly reduce the amount of data while maintaining data integrity. Data compression is achieved by finding and replacing repeated sequences. Specifically, the algorithm finds repeated data fragments in the data stream and replaces these repeated fragments with pointers or indexes, thereby reducing data redundancy.

[0093] A sliding window is a fixed-size buffer used to store fragments of historical data. The size of the window can be adjusted according to the actual application scenario, usually between a few kilobytes and a few megabytes. For each new character or sequence of characters, the algorithm searches for the longest repeating sequence in the historical data within the sliding window. This step is done by comparing the degree of match of the new character with the existing characters in the window. After finding the longest repeating sequence, the algorithm can replace this repeating sequence with a pointer (containing distance and length information), thereby achieving data compression.

[0094] After processing the current character or character sequence, the sliding window moves forward to continue processing the next character or character sequence. The sliding of the window ensures that the algorithm can continue to effectively find and replace repeated sequences. When the sliding window reaches the end of the data stream, appropriate boundary processing is required to ensure that all data can be compressed correctly.

[0095] After receiving the compressed data in the cloud, the corresponding decompression algorithm is used for lossless restoration. The decompression algorithm restores the original data sequence based on the pointer information generated during compression. In order to ensure the integrity and accuracy of the data, data verification can be performed after decompression, such as calculating and comparing the hash value of the data.

[0096] In the process of data compression and decompression, multi-threading or multi-process technology can be used for parallel processing to improve processing speed. Use cache mechanism to store frequently accessed data fragments to reduce repeated calculations and improve efficiency.

[0097] Therefore, through the efficient lossless data compression algorithm and sliding window mechanism, the amount of data is significantly reduced and the data transmission efficiency is improved. The compressed data can be restored to the original data without loss, ensuring the integrity and accuracy of the data. The parallel processing and caching mechanism are adopted to improve the speed of data compression and decompression, which is more suitable for large-scale data processing scenarios.

[0098] During data transmission, AES or other encryption algorithms can be used to encrypt data; end-to-end encrypted transmission is implemented to ensure the security of data throughout the entire transmission process.

[0099] Before uploading data, use Advanced Encryption Standard (AES) to encrypt the data. AES is a symmetric encryption algorithm with high security, high speed and low resource consumption, suitable for large-scale data encryption. In addition to AES, other mature encryption algorithms such as RSA, DES, etc. can also be selected according to actual needs and performance considerations.

[0100] Generate random encryption keys to ensure that the keys used for each encryption are different, which increases the difficulty of cracking. Use a secure key exchange protocol (such as the Diffie-Hellman key exchange protocol) to securely share keys between the client and the server. Keys should be stored properly to avoid disclosure. Hardware security modules (HSMs) or secure key management systems can be used to protect keys.

[0101] Use Transport Layer Security (TLS) or Secure Sockets Layer (SSL) to protect data during transmission. These protocols provide data encryption, authentication, and data integrity verification. Make sure to use a valid digital certificate and verify the certificate through a certificate authority (CA) to prevent man-in-the-middle attacks.

[0102] Calculate the hash value of the data before data transmission, and recalculate the hash value at the receiving end for comparison to ensure that the data has not been tampered with during transmission. Use digital signature technology to sign the data to ensure that the source of the data is credible and has not been tampered with.

[0103] During data transmission, the network status and transmission quality are monitored in real time to detect and handle abnormal situations in a timely manner. Key information of data transmission, such as transmission time, transmission status, transmission volume, etc., is recorded to facilitate subsequent auditing and troubleshooting.

[0104] By using encryption algorithms such as AES and end-to-end encrypted transmission, the security and privacy of data during transmission are ensured to prevent data from being stolen or tampered with. A variety of security measures are adopted to enhance the system's anti-attack capabilities and reduce security risks. Users do not need to worry about data security issues, which improves the reliability of the system and user trust. Comply with various data protection regulations and standards, such as GDPR, HIPAA, etc., to ensure compliance with data processing processes.

[0105] In one possible implementation, before data is reported, the system caches a copy of the data to be reported locally. Only when it is confirmed that the data has been successfully uploaded to the cloud can the local cached copy of the data be cleared. This strategy can effectively avoid data loss due to network interruptions or other technical failures, and ensure data integrity and consistency.

[0106] In a possible implementation, during the data upload process, the vehicle side can monitor the network status in real time. If it is detected that the network status is poor, which may cause data upload failure, the vehicle side can automatically switch to data cache mode.

[0107] When data upload fails due to poor network conditions, the vehicle can temporarily store the unsuccessfully uploaded data in a local cache. The local cache can use a file system, database, or other suitable storage methods to ensure data security and reliability. The vehicle can record the number of upload attempts and timestamps for each cached data so that the data can be processed first after the network is restored.

[0108] The vehicle can periodically check the network status, and once the network status returns to normal, immediately start the process of uploading cached data. The system can also trigger the upload of cached data through network status change events (such as network connection recovery events).

[0109] When the network status returns to normal, the vehicle can read the data that was not successfully uploaded from the local cache and retry uploading to the cloud. If network problems or other abnormalities are encountered again during the upload process, the vehicle can record the error information and continue to keep the data in the local cache, waiting for the next network recovery to try again.

[0110] When the cached data is successfully uploaded to the cloud, the vehicle can automatically clear the corresponding data in the local cache to free up storage space.

[0111] The system can also set up a regular cleanup mechanism to delete cached data that has not been successfully uploaded for more than a certain period of time to prevent the cache space from being occupied indefinitely.

[0112] Through the local cache and retransmission mechanism, data can be successfully uploaded to the cloud even in unstable network conditions. The cache mechanism ensures that data will not be lost due to network problems, ensuring data integrity and consistency. Users do not need to worry about data loss or delays caused by network problems, which improves system reliability and user experience. The regular cleanup mechanism prevents the cache space from being occupied without limit and optimizes the use of system resources.

[0113] In one possible implementation, when the cloud fails to receive the compressed data sent from the device within a preset time period. Or, when the local cache space reaches the preset usage threshold. The system automatically starts a cyclic overwrite mechanism, giving priority to the earliest cached data (that is, the data with the earliest acquisition time) for overwriting. The newly generated data will replace these earliest data to ensure that the data in the cache is kept as up-to-date as possible. Before each overwrite operation, the vehicle side can record key information of the overwritten data, such as the unique identifier of the data, the timestamp of the overwrite operation, and an overview of the basic content of the data, in preparation for subsequent data recovery or auditing needs.

[0114] Therefore, ensuring that the data in the cache is always up to date is particularly important for application scenarios with high real-time requirements. Even in the case of unstable network, the continuity of data collection tasks can be guaranteed by rationally utilizing limited local storage resources. Each overwrite operation is recorded in detail to facilitate subsequent data management and troubleshooting.

[0115] The above are some specific implementations of the data processing method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding data processing system. The system provided in the embodiment of the present application will be introduced from the perspective of functional modularization. Figure 4 A structural diagram of a data processing system provided in an embodiment of the present application.

[0116] The system comprises:

[0117] An acquisition unit 110 is used to acquire a target data type, where the target data type is a data type that the vehicle indicates needs to be collected;

[0118] A determining unit 111, configured to determine, based on a target data type, the preliminary data corresponding to the target data type;

[0119] A collection unit 112, configured to collect target data based on the change rate of the prepared data;

[0120] The sending unit 113 is used to compress the collected data and send the compressed data to the cloud.

[0121] The embodiment of the present application also provides a corresponding vehicle for implementing the data processing method provided in the embodiment of the present application.

[0122] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the data processing method provided in the embodiments of the present application.

[0123] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the data processing method described in any embodiment of the present application.

[0124] The computer storage medium stores codes, and when the codes are executed, the device executing the codes implements the data processing method described in any embodiment of the present application.

[0125] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0126] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0127] It should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0128] It should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0129] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0130] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0131] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, characterized in that: include: Obtaining a target data type, where the target data type is the data type that the vehicle indicates needs to be collected; Based on the target data type, determining the preliminary data corresponding to the target data type; Based on the change rate of the prepared data, collecting target data; Compress the collected data and send the compressed data to the cloud.

2. The method according to claim 1, characterized in that: The collecting of target data based on the change rate of the prepared data includes: When the change rate of the prepared data is greater than a first threshold, obtaining an initial sampling frequency; When the change rate of the prepared data is greater than a second threshold, increasing the initial sampling frequency; When the change rate of the prepared data is less than a third threshold and greater than a second threshold, lowering the initial sampling frequency; The target data is collected based on the adjusted initial sampling frequency.

3. The method according to claim 1, characterized in that The collecting of target data based on the change rate of the prepared data includes: When the change rate of the prepared data is not greater than a first threshold, determining whether the target data is preset key data; When the target data is preset key data, the target data is collected.

4. The method according to claim 3, characterized in that The method further comprises: When the target data is not preset key data, the target data is collected based on a preset period.

5. The method according to claim 3, characterized in that: The method further comprises: When the target data is not the preset key data, when the target data changes, the changed data is collected.

6. The method according to claim 1, characterized in that The collecting of target data based on the change rate of the prepared data includes: When the change rate of the prepared data is greater than a first threshold and the initial sampling frequency cannot be received from the cloud, obtaining the locally stored prepared sampling frequency; The target data is collected based on the prepared sampling frequency.

7. The method according to claim 1, characterized in that The step of compressing the collected data and sending the compressed data to the cloud further includes: The compressed data is written into the local cache area to obtain a data copy.

8. The method according to claim 1, characterized in that After compressing the collected data and sending the compressed data to the cloud, the method further includes: When the cloud fails to receive the compressed data, the compressed data is written to the local cache area to obtain a copy of the data.

9. The method according to claim 7 or 8, characterized in that: The method further comprises: When the compressed data has been uploaded to the cloud, the data copy in the local cache area is cleared.

10. The method according to claim 1, characterized in that The method further comprises: When the cloud fails to receive the compressed data after a preset period of time, and / or the local cache space reaches a preset threshold, a circular overwriting strategy is used to adjust the data in the local cache area, wherein the circular overwriting strategy is used to indicate that the latest data is to be acquired to overwrite the earliest data.

Images