Internet of vehicles data processing method and device, and computer readable storage medium

Abstract

The invention provides an Internet of Vehicles data processing method and device and a computer readable storage medium, an export request sent by a client of an Internet of Vehicles platform is received, the export request comprises a query condition, the query condition is decomposed based on a decomposition strategy corresponding to the query condition, and the decomposed query condition is sent to the client of the Internet of Vehicles platform. And acquiring the Internet of Vehicles data according to the segmented N sub-query conditions, thereby realizing batch query of the data, avoiding the problem of server performance and lagging caused by processing excessive data at one time, and acquiring and writing the data in time. And then N groups of queried data are written into a specified file according to a preset format and fed back to the client, so that the data export efficiency is improved on the whole, and accurate support is provided for related decisions.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a method and device for processing Internet of Vehicles data, and a computer-readable storage medium. Background Art

[0002] With the rapid development of Internet of Vehicles technology, Internet of Vehicles data presents significant characteristics such as large scale, high data application value, and complexity and diversity. However, real-time query and large-scale export of Internet of Vehicles data are huge difficulties in applications. Real-time query requires the system to respond quickly in order to provide key information to users in a timely manner, while large-scale export needs to cope with the challenges of huge data volume and complex processing.

[0003] When processing the export of ultra-large-scale data, traditional relational databases such as MySQL need to load a large amount of data into memory for processing and sorting, which will cause the database server's memory to be quickly filled up, making it difficult to meet the needs of real-time acquisition and affecting other running queries and businesses. When performing applications such as trip analysis, trajectory analysis, battery data prediction, and vehicle maintenance and repair, the speed of data acquisition is seriously lagging, and it is impossible to provide accurate support for relevant decisions in a timely manner. Summary of the invention

[0004] In order to overcome the problems existing in the related art, this specification provides a vehicle network data processing method, device and computer-readable storage medium.

[0005] According to a first aspect of an embodiment of this specification, a method for processing Internet of Vehicles data is provided, the method being applied to a server of an Internet of Vehicles platform, the method comprising:

[0006] Receiving an export request sent by a client of the Internet of Vehicles platform, wherein the export request includes a query condition;

[0007] Decomposing the query condition based on the decomposition strategy corresponding to the query condition, and acquiring Internet of Vehicles data according to the N sub-query conditions after the division;

[0008] The N groups of data found out are written into a specified file according to a preset format and fed back to the client.

[0009] According to a vehicle networking data processing method provided by the present application, the query condition includes a vehicle identification code and a query start and end time, and the corresponding decomposition strategy includes division by time dimension,

[0010] The decomposing the query condition based on the decomposition strategy corresponding to the query condition, and acquiring Internet of Vehicles data according to at least one sub-query condition after the division, includes:

[0011] The query start and end times are divided into N sub-query times in an incrementing manner every hour;

[0012] According to the sub-query condition consisting of the vehicle identification code and N sub-query times, N groups of Internet of Vehicles data are obtained in batches in chronological order.

[0013] According to a vehicle networking data processing method provided by the present application, the export request includes a header data item;

[0014] The step of writing the queried data into a designated file in a preset format to feed back to the client includes:

[0015] The N groups of Internet of Vehicles data are written in batches into the positions corresponding to the header data items in the designated file, wherein the content written into the designated file for the first time includes the header data items and the first group of Internet of Vehicles data, and the additionally written content includes the other groups of Internet of Vehicles data after the first group.

[0016] According to a vehicle networking data processing method provided by the present application, the queried data is written into a specified file according to a preset format to be fed back to the client, including:

[0017] Call the global single-row blocking queue, and put the queried data into the global single-case blocking queue if the global single-case blocking queue has not reached the set maximum capacity;

[0018] When there is data available for retrieval in the global singleton blocking queue, the data is retrieved from the global singleton blocking queue and written into a designated file in a preset format to be fed back to the client.

[0019] According to a vehicle networking data processing method provided by the present application, the method further includes:

[0020] When the global singleton blocking queue reaches a set maximum capacity, stop putting data into the global singleton blocking queue;

[0021] When the time corresponding to the stop operation reaches the blocking time of the global single-column blocking queue, the queried data is put into the global singleton blocking queue.

[0022] According to a vehicle networking data processing method provided by the present application, the method further includes:

[0023] In the process of writing the queried N groups of data into the designated file according to the preset format, the writing status of each group of data is fed back to the client to be displayed on the client.

[0024] According to a method for processing Internet of Vehicles data provided by the present application, before decomposing the query condition based on the decomposition strategy corresponding to the query condition, the method further includes:

[0025] Based on the query condition, determine the total amount of the Internet of Vehicles data that meets the query condition;

[0026] When the total data volume exceeds a threshold, a decomposition strategy corresponding to the query condition is executed to decompose the query condition.

[0027] According to a method for processing Internet of Vehicles data provided by the present application, after writing the queried N groups of data into a specified file according to a preset format and before feeding back to the client, the method further includes:

[0028] Convert the N groups of data based on a preset escape function to obtain converted target data;

[0029] The target data is fed back to the client.

[0030] According to a method for processing Internet of Vehicles data provided by the present application, after receiving an export request sent by a client of an Internet of Vehicles platform, the method further includes:

[0031] Performing a preliminary query in a preset cache area according to the query condition;

[0032] In the case where the query condition matches the historical query condition in the cache area, obtaining data corresponding to the historical query condition from the cache area as target data;

[0033] The cache area includes at least one query record of the first query based on the historical query condition, and the query record includes the historical query condition and the historical target data obtained by the first query.

[0034] According to a method for processing Internet of Vehicles data provided by the present application, the query condition is decomposed based on a decomposition strategy corresponding to the query condition, and before obtaining Internet of Vehicles data according to at least one sub-query condition after the division, the method further includes:

[0035] The user who initiates the export request on the client is subjected to an authority check. If the check passes, the query condition is decomposed based on a decomposition strategy corresponding to the query condition, and Internet of Vehicles data is obtained according to at least one sub-query condition after the division.

[0036] The present application also provides a vehicle networking data processing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vehicle networking data processing method described above is implemented.

[0037] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the vehicle networking data processing methods described above.

[0038] The Internet of Vehicles data processing method, device and computer-readable storage medium in the embodiments of this specification receive an export request sent by a client of an Internet of Vehicles platform, the export request including a query condition, decompose the query condition based on a decomposition strategy corresponding to the query condition, obtain Internet of Vehicles data according to the N sub-query conditions after segmentation, implement batch query of data, avoid server performance and lag problems caused by processing too much data at one time, obtain and write data in a timely manner, then write the N groups of data obtained by the query into a specified file according to a preset format, and feed back to the client, so as to improve the overall data export efficiency and provide accurate support for relevant decisions.

[0039] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0041] Figure 1 is a flow chart of a method for processing vehicle network data according to an exemplary embodiment of the present specification;

[0042] Figure 2 is a flowchart of another method for processing vehicle network data according to an exemplary embodiment of the present specification;

[0043] Figure 3 is a schematic diagram of the progress of data writing status of a client according to an exemplary embodiment of this specification;

[0044] Figure 4 is a schematic diagram of a vehicle networking data processing device according to an exemplary embodiment of this specification;

[0045] Figure 5 It is a schematic block diagram of a vehicle networking data processing device shown in this specification according to an exemplary embodiment. DETAILED DESCRIPTION

[0046] Here, the technical solutions in the embodiments (or "implementations") of the present application will be described clearly and completely in conjunction with the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0047] If there are terms involving directional indications or positional relationships in the embodiments of the present application (such as up, down, left, right, front, back, inside, outside, top, bottom, center, vertical, horizontal, longitudinal, transverse, length, width, counterclockwise, clockwise, axial, radial, circumferential, etc.), such terms are only used to explain the relative positional relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings); if the specific posture changes, the directional indication or positional relationship will also change accordingly. In addition, the terms "first", "second", etc. involved in the embodiments of the present application are only used for the purpose of convenience of description and cannot be understood as indicating or implying relative importance.

[0048] The present application provides a method, device and computer-readable storage medium for processing data of an Internet of Vehicles. The present application is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0049] Internet of Vehicles data is characterized by large scale, high data application value, and complexity and diversity. In terms of trip analysis, trajectory analysis, battery data prediction, vehicle maintenance and repair, the storage capacity of time series data and efficient, convenient, and sustainable operation management have become essential indicators in the system architecture. However, in actual use, real-time query and large-scale export are huge difficulties in the application of Internet of Vehicles data.

[0050] When processing the export of ultra-large-scale data, traditional relational databases such as MySQL need to load a large amount of data into memory for processing and sorting, which will cause the database server's memory to be quickly filled up, making it difficult to meet the needs of real-time acquisition and affecting other running queries and businesses. When performing applications such as trip analysis, trajectory analysis, battery data prediction, and vehicle maintenance and repair, the speed of data acquisition is seriously lagging, and it is impossible to provide accurate support for relevant decisions in a timely manner.

[0051] In order to solve the above technical problems, this specification provides a method for processing Internet of Vehicles data.

[0052] By setting up an HBase database on the Internet of Vehicles platform and connecting with the client and server through the HBase programming interface, the massive data of the Internet of Vehicles can be processed to ensure the accuracy and completeness of the query results of the Internet of Vehicles data.

[0053] Specifically, HBase is an open source, distributed, versioned, non-relational database based on Hadoop. It can process hundreds of millions of data and is very suitable for storing large-scale Internet of Vehicles data. HBase uses a columnar storage model to facilitate column expansion and can cope with the complexity of Internet of Vehicles data.

[0054] Combination Figure 1 and Figure 2 As shown, Figure 1 This is a flow chart of a method for processing vehicle network data provided by an embodiment of this specification, comprising the following steps:

[0055] S100: Receive an export request sent by a client of a vehicle networking platform, where the export request includes a query condition.

[0056] In the Internet of Vehicles platform, vehicle resources are allocated to different users according to certain rules and requirements, so that users can use Internet of Vehicles data to perform power performance analysis, battery analysis, driving information acquisition, etc. In some embodiments, the main path for vehicle allocation is to query and import vehicles into Excel tables. Specifically, vehicle information is obtained by querying databases and other methods and allocation decisions are made. That is, allocation can be based on real-time vehicle data obtained by query, or it can be based on data in a pre-prepared Excel table. When a user needs to view or apply Internet of Vehicles data, a query or export request is sent to the Internet of Vehicles platform to query or export the corresponding vehicle information in the HBase database.

[0057] In some embodiments, in order to have different vehicle query permissions for different users, corresponding person-vehicle relationships need to be assigned, so the system needs corresponding data permissions. Specifically, one situation is that the platform establishes users; another situation is that vehicles are assigned to users, mainly through querying vehicles (mainly for single-vehicle allocation) and importing vehicles (mainly for multiple-vehicle allocation); or through dynamic vehicle allocation through formulas, such as selecting vehicle brands and vehicle manufacturers.

[0058] Furthermore, when receiving an export request sent by a client of the Internet of Vehicles platform, the authority of the user who initiated the export request on the client is verified, and if the verification passes, the target data is obtained from the HBase database according to the query conditions, thereby ensuring that only legitimate users can access the relevant data.

[0059] Specifically, permission verification includes identity verification and protocol verification.

[0060] As an example, when using username verification, first there needs to be a permission management system or database table to store the permission association information between users and vehicles. When a query request is received, the user's identity (such as username, user ID, etc.) is searched in the permission system to determine whether the user has the permission to export and query the specified vehicle.

[0061] For example, you can create a permission table user_vehicle_permissions, which contains fields such as user_id, vehicle_id, and permission_flag. You can query this table to determine whether the user has permission.

[0062] As another example, the data item mainly depends on the protocol construction, mainly relying on GB / T32960, HJ1239, 17691, tsp, can, mscan protocols to construct the data item protocol tree structure, and performing protocol verification based on the data item protocol tree structure.

[0063] The client page provides a variety of data query options, such as new energy national standard detailed data query, heavy-duty vehicle National VI data detailed query, TSP detailed data query, CAN detailed data query, and ms-level CAN detailed data query. The heavy-duty vehicle National VI detailed data mainly includes two protocols 17691 and HJ 1239.

[0064] During the data query process, the vehicles will first be divided according to the reported protocol and the vehicle types supported by the protocol, and the data will be saved to redis (executed by the corresponding configured big data service redis). When querying the data, it will be determined whether the vehicle meets the relevant protocol based on the query conditions in the query request.

[0065] It should be noted that when designing the row key (RowKey), the needs of data query, the characteristics of the storage engine, and the data access mode are usually taken into consideration. In the current scenario of obtaining vehicle data for performance analysis, the required data can be quickly queried and obtained based on the vehicle identification code and the timestamp of the data. Therefore, the row key rowkey is built based on vin and time, and the data is stored in the HBase database as a row key rowkey. When querying, set the query condition to query the vehicle data based on the key value built by the export request, and the key value can contain the row key information. Of course, other HBase rowkeys can be set in other scenarios or requirements, and there is no limitation here.

[0066] Continuing from the previous embodiment, when the query conditions include the vehicle identification code and the start and end time, as an example, if the energy national standard detailed data is currently queried, on the client page, after the vehicle identification code vin is entered, the protocol interface will be called to determine whether the vehicle supports the query of 32960 data. If the vehicle does not meet the requirements, the front-end page will display a pop-up window. Generally, a vehicle will meet multiple protocol data.

[0067] If the aforementioned permission check and vehicle type check are passed, the system will start the query and export operation. This may involve interacting with the database or data storage system, using appropriate query / export statements or APIs to obtain the required data.

[0068] If the permission check and vehicle type check fail, an exception is thrown on the client side so that the user can re-enter the query conditions.

[0069] S200 , decompose the query condition based on a decomposition strategy corresponding to the query condition, and obtain Internet of Vehicles data according to the divided N sub-query conditions.

[0070] For a table containing billions of rows of data, if you want to filter and export part of the data according to a certain condition, the database may need to read a large number of rows to determine whether it meets the condition. In this process, a large amount of data needs to be operated in memory. If the memory is insufficient, it will cause a sharp drop in performance and may even cause the server to crash, thus affecting other running queries and businesses. Therefore, in this manual, the paging idea is adopted to decompose the query conditions, split them into N sub-query conditions, and perform batch queries based on the N sub-query conditions, thereby improving the overall performance, stability and scalability of the system and improving the data export efficiency of the system.

[0071] In some embodiments, the decomposing the query condition based on the decomposition strategy corresponding to the query condition, and acquiring Internet of Vehicles data according to at least one sub-query condition after the decomposition, includes:

[0072] S201 , dividing the query start and end time into N sub-query times in an hourly increment.

[0073] The query conditions include the vehicle identification code and the query start and end time, and the corresponding decomposition strategy includes division by time dimension. The main paging logic is set according to the start and end time, according to the idea of ​​one increment per hour. For example, if the query time is from 12:09:09 on 2024-04-06 to 18:09:09 on 2024-04-06, then each query increments by one hour, which is 13:09:09 on 2024-04-06, 14:09:09 on 2024-04-06, and so on. The sub-query time of each hour and the vehicle identification code vin constitute multiple groups of sub-query conditions for batch query.

[0074] S202. Acquire N groups of Internet of Vehicles data in batches in chronological order according to a sub-query condition consisting of a vehicle identification code and N sub-query times.

[0075] It is worth noting that when exporting single-vehicle data, the vin and query start and end times of the vehicle are divided into time dimensions to obtain multiple groups of sub-query conditions corresponding to the vehicle for subsequent query and data export. When exporting multiple-vehicle data, the vin and query start and end times of each vehicle are divided into time dimensions to obtain multiple groups of sub-query conditions for each vehicle, and data queries are performed on the sub-query conditions of each vehicle in turn.

[0076] In some other embodiments, before decomposing the query condition based on the decomposition strategy corresponding to the query condition, the method further includes:

[0077] S210, determining the total amount of the Internet of Vehicles data that meets the query condition based on the query condition;

[0078] S220: When the total data volume exceeds a threshold, executing a decomposition strategy corresponding to the query condition to decompose the query condition.

[0079] Determine the total amount of data that meets the requirements under the query conditions, that is, the total data volume, and decide the subsequent query method based on the total data volume. If the total data volume is within a specific range, all eligible data will be queried and obtained at one time based on the query conditions; if the total data volume exceeds the specific range, the query will be performed in a paging manner. The paging logic is based on the start and end time, and the query is performed step by step in increments of each hour.

[0080] The preset interval range may also be represented by a set data volume threshold.

[0081] In this embodiment, the query efficiency and system resource consumption are balanced by setting the query mode. When the amount of data is small, it is acquired all at once to improve efficiency. For a large amount of data, it is processed step by step through paging to avoid performance problems caused by processing too much data at one time.

[0082] As an example, the HBase database includes multiple regions divided based on rowkey (it can also be understood as based on partial query conditions), each region stores a certain range of row key-value pairs, and uses coprocessor technology, such as HBase's coprocessor AggregationClient, to perform aggregation operations to calculate the total data volume. It can be understood that this partitioning mechanism helps to achieve distributed storage and parallel processing of data, and improves data read and write performance.

[0083] Specifically, the coprocessor is an extension mechanism provided by HBase that allows custom code logic to be executed on RegionServer to enhance the functionality of HBase. By using HBase's coprocessor AggregationClient to calculate the total number of data, data statistics and aggregation can be processed more efficiently.

[0084] The coprocessor is tightly integrated with the RegionServer. When a client initiates a request, the RegionServer will decide whether to activate the corresponding coprocessor based on the type and conditions of the request. The coprocessor can access and process the data in the Region and return the results to the RegionServer, which will then return them to the client or the server of the Internet of Vehicles platform.

[0085] In this embodiment, when a coprocessor is used to query on a RegionServer, the coprocessor can operate directly within the relevant Region, avoiding unnecessary data scanning and transmission. The partitioning mechanism provides a basis for the efficient execution of the coprocessor, enabling it to process data within a specific range more specifically, reducing computing and network overhead.

[0086] S300: Write the N groups of data found out in the search into a designated file according to a preset format, and feed back the data to the client.

[0087] During the data export process, a specified file will be constructed to store the N sets of data obtained through the query. The multiple sets of data obtained through the query will be written into the specified file in a pre-set format, and the file will be fed back to the client. This setting helps to ensure the consistency and standardization of the data, and facilitates subsequent processing, analysis, use, and interpretation of the data. Moreover, a specific format may be compatible with certain specific software, systems, or tools, which can facilitate subsequent import and use, thereby improving the availability of data.

[0088] In some embodiments, a CSV writing method is constructed, which mainly includes initial writing and additional writing. The initial writing requires writing the table header and the first batch of query data, and the additional writing mainly refers to each subsequent query writing.

[0089] Specifically, writing the queried data into a designated file in a preset format to feed back to the client includes:

[0090] S301. Write the N groups of Internet of Vehicles data into the positions corresponding to the header data items in the designated file in batches, wherein the content written into the designated file for the first time includes the header data items and the first group of Internet of Vehicles data, and the content written additionally includes the other groups of Internet of Vehicles data after the first group.

[0091] The export request in this article includes a header data item, which is used to create a CSV file. When performing the first write operation, the header data item is written into the CSV file, followed by the first batch of queried Internet of Vehicles data.

[0092] In this embodiment, the CSV file is selected as the designated file for storing the N groups of data queried out. The advantages are: the CSV format structure is simple and easy to generate and parse. Most programming languages ​​and data analysis tools have good support for the CSV format and can easily read, write and process CSV files. CSV is a common format for data exchange. When transferring data between different systems and applications, the CSV format usually does not encounter compatibility issues.

[0093] It is worth emphasizing that the process of querying data according to query conditions and writing CSV files adopts the idea of ​​asynchronous multi-threading, and operates simultaneously in different threads to improve overall efficiency.

[0094] For example, in the query thread, after obtaining a portion of data, it is placed in a shared data structure (such as a queue). The write thread obtains data from this queue and writes it to CSV. In this way, the query thread can continue to obtain subsequent data, and the write thread can also process the already obtained data in a timely manner. The two threads work concurrently, improving the responsiveness and throughput of the system. Especially when processing large amounts of data or complex exports / queries, performance is significantly improved.

[0095] In other embodiments, in order to avoid the speed impact between query and write, where query is fast and write is slow, and to prevent Out of Memory exceptions caused by large amounts of data being exported, a global singleton blocking queue is set, and the blocking queue size is set. The blocking time can be set to a fixed time or to a blocking time set according to the time of querying and writing CSV, to avoid multiple tasks querying and writing data at the same time, occupying too much memory, and affecting service performance.

[0096] The blocking queue size is to limit the maximum capacity of the buffer in memory. If the size limit is not set, data may be added to the queue without restraint, which will quickly consume a lot of memory and increase the risk of OutofMemory exception.

[0097] The blocking time can be set to a fixed time to ensure that within a certain period of time, if the queue operation cannot be completed (such as writing or retrieving data), the thread will temporarily block and wait. The blocking time can also be set dynamically according to the time of querying and writing CSV. This is more flexible and adaptive. For example, if the speed of querying data is fast and the speed of writing CSV is slow, the blocking time can be appropriately extended to avoid the query thread frequently blocking because the queue is full, thereby improving overall efficiency. Conversely, if the writing speed is fast and the query speed is slow, the blocking time can be shortened to prevent data from accumulating in the queue for too long.

[0098] In this way, it is possible to avoid excessive memory usage when multiple tasks perform query and write operations at the same time. When one task is processing data, other tasks will wait in the blocking queue, thereby achieving coordination between tasks and reasonable allocation of resources, ensuring service performance and stability, and reducing service anomalies or performance degradation caused by excessive memory usage.

[0099] As an example, writing the queried data into a designated file in a preset format to feed back to the client includes:

[0100] Call the global single-row blocking queue, and put the queried data into the global single-case blocking queue if the global single-case blocking queue has not reached the set maximum capacity;

[0101] When there is data available for retrieval in the global singleton blocking queue, the data is retrieved from the global singleton blocking queue and written into a designated file in a preset format to be fed back to the client.

[0102] The query thread is responsible for getting data from the data source (such as a database, file, etc.) by calling the global singleton blocking queue. The acquired data will be put into the global singleton blocking queue. If there is enough space in the blocking queue when putting the data, the query thread will put the queried data into the global singleton blocking queue. The writing thread takes the data out of the global singleton blocking queue, and after taking out the data, writes it into the CSV file in the CSV format.

[0103] Furthermore, the writing thread takes data from the global singleton blocking queue. If the blocking queue is empty, the writing thread will be blocked until there is data to be taken out.

[0104] In other embodiments, the queried data is written into a designated file in a preset format to be fed back to the client, and the method further includes:

[0105] Calling a global single-queue blocking queue, and stopping the operation of putting data into the global single-queue blocking queue when the global single-queue blocking queue reaches a set maximum capacity;

[0106] When the time corresponding to the stop operation reaches the blocking time of the global single-column blocking queue, the queried data is put into the global singleton blocking queue.

[0107] If the blocking queue is full (the global singleton blocking queue reaches the set maximum capacity) when putting data, the query thread will be blocked and stop putting data into the global singleton blocking queue until there is enough space in the queue. That is, when the waiting time for the operation of putting data reaches the set blocking time, the operation of putting data into the blocking queue will continue.

[0108] In some examples, if the blocking waiting time reaches the set blocking time but there is still no space in the blocking queue, an exception will be thrown or other processing strategies will be adopted.

[0109] In this embodiment, the query thread and the write thread of the entire process can work concurrently, which improves efficiency. The blocking queue plays a role of buffering and coordination, ensuring the orderly transmission and processing of data and avoiding data loss or confusion.

[0110] In some embodiments, data is written to a server or cloud, and the front-end data is being written and the status of the current writing task is returned. Specifically, in the process of writing the N groups of queried data into a specified file according to a preset format, the writing status of each group of data is fed back to the client for display on the client.

[0111] like Figure 3 As shown, the task status will present progress information, which is mainly constructed based on the start and end time of the query. If the query is from 00:00:00 to 23:59:59 on July 2, it will be divided into 24 parts, each time written as a decimal. After writing, the database is updated and the front-end query is used to display the progress.

[0112] In some embodiments, after writing the N groups of data found out in the query into a designated file in a preset format and before feeding back to the client, the method further includes:

[0113] Convert the N groups of data based on a preset escape function to obtain converted target data;

[0114] The target data is fed back to the client.

[0115] For the returned N groups of data, escape according to specific rules or mappings to make them meet the requirements of front-end display or processing to prevent abnormal or erroneous display. Therefore, before encapsulation, the N groups of data are converted based on a preset escape function, which can be a mapping relationship function based on the constructed escape enumeration class before and after the escape, so as to solve the data escape problem by converting the value from the enumeration class later.

[0116] After the data is written, the front end downloads the data to the local computer by calling the interface. The download process is in the form of streaming. In this article, streaming means gradually transmitting the compressed package to the front end in a streaming manner through the network, rather than loading the entire compressed package into the memory at one time before transmitting it. This can further optimize memory usage and transmission efficiency.

[0117] If the CSV file is too large, the backend will compress the file and return the data in a compressed package to further optimize memory usage and transmission efficiency. After the frontend receives the streaming data, it will gradually save it as a complete compressed package file. Users can decompress it locally as needed to obtain the original CSV file.

[0118] In other embodiments, the acquired data is returned to the client and stored in a cache area for direct downloading during the next query export. Therefore, after reading the corresponding data from the database according to the export request sent by the client, the corresponding data can also be read from the cache area for storing the data for each query.

[0119] Specifically, after receiving the export request sent by the client of the Internet of Vehicles platform, the method further includes:

[0120] Performing a preliminary query in a preset cache area according to the query condition;

[0121] In the case where the query condition matches the historical query condition in the cache area, obtaining data corresponding to the historical query condition from the cache area as target data;

[0122] The cache area includes at least one query record of the first query based on the historical query condition, and the query record includes the historical query condition and the historical target data obtained by the first query.

[0123] Each time the IoV data is obtained based on the query conditions, it will be saved in the LRU (Least Recently Used) cache. If the previously saved IoV data is found in the LRU cache during a subsequent query, it will be directly used; if not found, the query conditions will be decomposed based on the decomposition strategy corresponding to the query conditions to obtain the IoV data, and then saved in the LRU cache.

[0124] It should be noted that the above data processing method is applied to the scenario of single-vehicle export. In the scenario of multiple-vehicle export, the idea of ​​queue will be adopted to create a queue for the export operation of each vehicle and process each vehicle in turn. This means that the export process of each vehicle will have an independent position in this queue waiting to be processed.

[0125] Specifically, for each vehicle, a CSV file will be constructed first. The content of this file is determined based on the vehicle identification code vin, the start and end time, and the corresponding user's userid. Each query and write also uses the single vehicle export method to complete the writing of each vehicle's CSV file. The addresses of these files will be stored in a queue. When the CSV files of all vehicles are processed, these addresses are taken out of the queue that stores the file addresses, and the corresponding CSV files are combined and compressed to form a compressed package. The front end obtains this compressed package from the back end by calling the download interface and downloads it locally.

[0126] In this embodiment, the process of exporting multi-vehicle data is made more orderly and efficient, avoiding confusion and performance problems caused by processing data of multiple vehicles at the same time.

[0127] The present application provides a method, device and computer-readable storage medium for processing Internet of Vehicles data, which receive an export request sent by a client of an Internet of Vehicles platform, wherein the export request includes a query condition, the query condition is decomposed and processed based on a decomposition strategy corresponding to the query condition, and Internet of Vehicles data is obtained according to the N sub-query conditions after segmentation, so as to implement batch query of data, avoid server performance and lag problems caused by processing too much data at one time, obtain and write data in time, and then write the N groups of data obtained by the query into a specified file according to a preset format, and feed back to the client, so as to improve the overall data export efficiency and provide accurate support for relevant decisions.

[0128] Based on the same application concept as the above method, the embodiment of the present application also proposes a vehicle network data processing device, such as Figure 4 shown.

[0129] The device comprises:

[0130] A request receiving module 402 is used to receive an export request sent by a client of the Internet of Vehicles platform, wherein the export request includes a query condition;

[0131] A data query module 404 is used to decompose the query condition based on the decomposition strategy corresponding to the query condition, and obtain Internet of Vehicles data according to the N sub-query conditions after the division;

[0132] The data encapsulation module 406 is used to write the N groups of data found out in the query into a designated file according to a preset format, and feed back the data to the client.

[0133] The implementation process of the functions and effects of each module / submodule / unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and the same technical effect can be achieved, so it will not be repeated here.

[0134] Figure 5 The following is a schematic diagram of the physical structure of a vehicle networking data processing device. Figure 5 As shown, the Internet of Vehicles data processing device may include: a processor (processor) 810, a communication interface (CommunicationsInterface) 820, a memory (memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the Internet of Vehicles data processing method.

[0135] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0136] On the other hand, the present application also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the Internet of Vehicles data processing methods provided by the above methods.

[0137] On the other hand, the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the vehicle network data processing method provided by the above-mentioned methods.

[0138] It should be noted that the technical solutions or technical features described in the above embodiments can be combined or supplemented with each other without causing conflicts. The scope of protection of this application is not limited to the precise structures described in the above embodiments and shown in the drawings; all modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A method for processing Internet of Vehicles data, characterized in that: The method is applied to the server side of the Internet of Vehicles platform, and the method includes: Receiving an export request sent by a client of the Internet of Vehicles platform, wherein the export request includes a query condition; Decomposing the query condition based on the decomposition strategy corresponding to the query condition, and acquiring Internet of Vehicles data according to the N sub-query conditions after the division; The N groups of data found out are written into a specified file according to a preset format and fed back to the client.

2. The method for processing Internet of Vehicles data according to claim 1, characterized in that: The query conditions include the vehicle identification code and the query start and end time, and the corresponding decomposition strategy includes division by time dimension. The decomposing the query condition based on the decomposition strategy corresponding to the query condition, and acquiring Internet of Vehicles data according to at least one sub-query condition after the division, includes: The query start and end times are divided into N sub-query times in an incrementing manner every hour; According to the sub-query condition consisting of the vehicle identification code and N sub-query times, N groups of Internet of Vehicles data are obtained in batches in chronological order.

3. The method for processing Internet of Vehicles data as claimed in claim 2, characterized in that: The export request includes a header data item; The step of writing the queried data into a designated file in a preset format to feed back to the client includes: The N groups of Internet of Vehicles data are written in batches into the positions corresponding to the header data items in the designated file, wherein the content written into the designated file for the first time includes the header data items and the first group of Internet of Vehicles data, and the additionally written content includes the other groups of Internet of Vehicles data after the first group.

4. The method for processing Internet of Vehicles data according to claim 1, characterized in that: The step of writing the queried data into a designated file in a preset format to feed back to the client includes: Call the global single-row blocking queue, and put the queried data into the global single-case blocking queue if the global single-case blocking queue has not reached the set maximum capacity; When there is data available for retrieval in the global singleton blocking queue, the data is retrieved from the global singleton blocking queue and written into a designated file in a preset format to be fed back to the client.

5. The method for processing Internet of Vehicles data as claimed in claim 4, characterized in that: The method further comprises: When the global singleton blocking queue reaches a set maximum capacity, stop putting data into the global singleton blocking queue; When the time corresponding to the stop operation reaches the blocking time of the global single-column blocking queue, the queried data is put into the global singleton blocking queue.

6. The method for processing Internet of Vehicles data according to claim 1, characterized in that: The method further comprises: In the process of writing the queried N groups of data into the designated file according to the preset format, the writing status of each group of data is fed back to the client to be displayed on the client.

7. The method for processing Internet of Vehicles data according to claim 1, characterized in that: Before decomposing the query condition based on the decomposition strategy corresponding to the query condition, the method further includes: Based on the query condition, determine the total amount of the Internet of Vehicles data that meets the query condition; When the total data volume exceeds a threshold, a decomposition strategy corresponding to the query condition is executed to decompose the query condition.

8. The method for processing Internet of Vehicles data as claimed in claim 1, characterized in that: After writing the N groups of data found out in the query into a designated file according to a preset format and before feeding back to the client, the method further includes: Convert the N groups of data based on a preset escape function to obtain converted target data; The target data is fed back to the client.

9. The method for processing Internet of Vehicles data as claimed in claim 1, characterized in that: After receiving the export request sent by the client of the Internet of Vehicles platform, the method further includes: Performing a preliminary query in a preset cache area according to the query condition; In the case where the query condition matches the historical query condition in the cache area, obtaining data corresponding to the historical query condition from the cache area as target data; The cache area includes at least one query record of the first query based on the historical query condition, and the query record includes the historical query condition and the historical target data obtained by the first query.

10. The method for processing Internet of Vehicles data according to claim 1, characterized in that: Before decomposing the query condition based on the decomposition strategy corresponding to the query condition and acquiring Internet of Vehicles data according to at least one sub-query condition after the division, the method further includes: The user who initiates the export request on the client is subjected to an authority check. If the check passes, the query condition is decomposed based on a decomposition strategy corresponding to the query condition, and Internet of Vehicles data is obtained according to at least one sub-query condition after the division.

11. A vehicle networking data processing device, characterized in that: It includes a memory, a processor, and a vehicle networking data processing program stored in the memory and executable on the processor. When the processor executes the vehicle networking data processing program, the steps of the vehicle networking data processing method as described in any one of claims 1 to 10 are implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a vehicle networking data processing program, and when the vehicle networking data processing program is executed, the steps of the vehicle networking data processing method as described in any one of claims 1 to 10 are implemented.

Images