Internet of vehicles data processing method and device, and computer readable storage medium

Abstract

According to the Internet of Vehicles data processing method and device and the computer readable storage medium provided by the invention, the vehicle data is stored based on the HBase database, column expansion is facilitated, so that mass data is accommodated, the demand for the complexity of the Internet of Vehicles data is met, and the data processing efficiency is improved on the basis of the HBase database. In the embodiment of the invention, the data query response is performed on the received query request sent by the client of the Internet of Vehicles platform, the target data is acquired from the HBase database based on the row key information in the query request, the target data is packaged, and the packaged data is fed back to the client for display, so that the accuracy and integrity of the query result are improved.

Description

Technical Field

[0001] The present application relates to the field of data query technology, and in particular to a method, device and computer-readable storage medium for processing Internet of Vehicles data. Background Art

[0002] With the rapid development of Internet of Vehicles technology, Internet of Vehicles data presents significant characteristics such as large scale, high data application value, and complexity and diversity. However, in the application of Internet of Vehicles data, there are a series of severe technical challenges.

[0003] In terms of data storage and query methods, existing technologies have many defects. Due to the extremely large amount of Internet of Vehicles data, traditional data storage uses relational databases or simple file systems. These methods have limited storage capacity when facing massive Internet of Vehicles data and are difficult to cope with the rapid expansion of data scale.

[0004] In terms of data query, existing technologies mostly rely on full table scan query methods, which are inefficient when processing massive data (such as hundreds of millions of IoV data) and difficult to meet the needs of real-time query. This results in a serious lag in data acquisition speed when performing applications such as trip analysis, trajectory analysis, battery data prediction, vehicle maintenance and repair, and is unable to provide accurate support for relevant decisions in a timely manner.

[0005] At the same time, the complexity of Internet of Vehicles data causes query conditions and requirements to change constantly, and existing technologies are not flexible enough to cope with such dynamic changes. They cannot quickly adapt to different query conditions and data structure changes, making it difficult to ensure the accuracy and completeness of query results. Summary of the invention

[0006] In order to overcome the problems existing in the related art, this specification provides a vehicle network data processing method, device and computer-readable storage medium.

[0007] According to a first aspect of an embodiment of this specification, a method for processing Internet of Vehicles data is provided, the method being applied to a server of an Internet of Vehicles platform, the method comprising:

[0008] Receiving a query request sent by a client of the Internet of Vehicles platform, wherein the query request includes row key information;

[0009] According to the row key information, target data is obtained from an HBase database, where the HBase database includes a plurality of Regions divided based on the row key information, and the Regions store Internet of Vehicles data;

[0010] The target data is encapsulated, and the encapsulated data is fed back to the client for display.

[0011] According to a vehicle networking data processing method provided by the present application, the target data includes total data and paging data;

[0012] The method further comprises:

[0013] Calling the asynchronous programming tool, creating a first thread and a second thread for asynchronous concurrent processing through the asynchronous programming tool; the first thread is used to obtain the total amount data, and the second thread is used to obtain the paging data;

[0014] The result of the first thread and the result of the second thread are combined to form target data.

[0015] According to a vehicle networking data processing method provided by the present application, the method further includes:

[0016] In the first thread, a coprocessor of the Internet of Vehicles platform HBase is called to obtain total amount data from the HBase database based on the row key information;

[0017] The HBase database includes a plurality of Regions divided based on the row key information, and the Internet of Vehicles data is stored in the Regions.

[0018] According to a vehicle networking data processing method provided by the present application, the query request also includes column family information and column information;

[0019] The method further comprises:

[0020] Based on the row key information, family information and column information, the total amount of data is obtained from the HBase database.

[0021] According to a method for processing Internet of Vehicles data provided by the present application, the query request further includes a data item for characterizing the data type of the Internet of Vehicles data;

[0022] The method further comprises:

[0023] In the second thread, target vehicle data is obtained from the HBase database according to the row key information and the data item;

[0024] Determine the data range based on preset query parameters; the query parameters include query page number and data size per page;

[0025] The paging data in the target vehicle data for feeding back to the client is determined according to the data range.

[0026] According to a method for processing Internet of Vehicles data provided by the present application, before encapsulating the target data and feeding back the encapsulated data to the client for display, the method further includes:

[0027] The paging data is converted based on a preset escape function to obtain converted target data.

[0028] According to a method for processing Internet of Vehicles data provided by the present application, the query request also includes index information;

[0029] The acquiring target data from the HBase database according to the row key information includes:

[0030] Determine the Region to be queried in the HBase database from a preset index table according to the index information;

[0031] Obtain target data from the region to be queried according to the row key information;

[0032] The index table is created by presetting the index information and the row key information corresponding to the Region.

[0033] According to a method for processing Internet of Vehicles data provided by the present application, after receiving a query request sent by a client of an Internet of Vehicles platform, the method further includes:

[0034] Performing a preliminary query in a preset cache area according to the row key information;

[0035] In the case where the row key information matches the historical row key information in the cache area, obtaining data corresponding to the historical row key information from the cache area as target data;

[0036] The cache area includes at least one query record based on the historical row key information when the HBase database is queried for the first time, and the query record includes the historical row key information and the historical target data queried for the first time.

[0037] According to a method for processing Internet of Vehicles data provided by the present application, before obtaining the target data from the HBase database according to the row key information, the method further includes:

[0038] The user who initiates the query request on the client is subjected to permission verification. If the verification passes, the target data is obtained from the HBase database according to the row key information. The permission verification includes a protocol verification.

[0039] According to a vehicle networking data processing method provided by the present application, the row key information includes a vehicle identification code and a start and end time.

[0040] The present application also provides a vehicle networking data processing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vehicle networking data processing method described above is implemented.

[0041] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the vehicle networking data processing methods described above.

[0042] The Internet of Vehicles data processing method, device and computer-readable storage medium in the embodiments of this specification store vehicle data based on the HBase database, which is convenient for column expansion to accommodate massive data and meet the demand for the complexity of Internet of Vehicles data. On the basis of the HBase database, the data query is responded to the query request sent by the client of the Internet of Vehicles platform, the target data is obtained from the HBase database based on the row key information in the query request, the target data is packaged, and the packaged data is fed back to the client for display, thereby improving the accuracy and completeness of the query results.

[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0045] Figure 1 is a flow chart of a method for processing vehicle network data according to an exemplary embodiment of the present specification;

[0046] Figure 2 is a flowchart of another method for processing vehicle network data according to an exemplary embodiment of the present specification;

[0047] Figure 3 This is a schematic diagram of vehicle networking data interaction according to an exemplary embodiment of this specification;

[0048] Figure 4 is a schematic diagram of a vehicle networking data processing device according to an exemplary embodiment of this specification;

[0049] Figure 5 It is a schematic block diagram of a vehicle networking data processing device shown in this specification according to an exemplary embodiment. DETAILED DESCRIPTION

[0050] Here, the technical solutions in the embodiments (or "implementations") of the present application will be described clearly and completely in conjunction with the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0051] If there are terms involving directional indications or positional relationships in the embodiments of the present application (such as up, down, left, right, front, back, inside, outside, top, bottom, center, vertical, horizontal, longitudinal, transverse, length, width, counterclockwise, clockwise, axial, radial, circumferential, etc.), such terms are only used to explain the relative positional relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings); if the specific posture changes, the directional indication or positional relationship will also change accordingly. In addition, the terms "first", "second", etc. involved in the embodiments of the present application are only used for the purpose of convenience of description and cannot be understood as indicating or implying relative importance.

[0052] The present application provides a method, device and computer-readable storage medium for processing data of an Internet of Vehicles. The present application is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0053] Internet of Vehicles data is characterized by large scale, high data application value, and complexity and diversity. In terms of trip analysis, trajectory analysis, battery data prediction, vehicle maintenance and repair, the storage capacity of time series data and efficient, convenient, and sustainable operation management have become essential indicators in the system architecture. However, in actual use, real-time query and large-scale export are huge difficulties in the application of Internet of Vehicles data.

[0054] Current storage uses relational databases or simple file systems, which have limited storage capacity when faced with massive amounts of Internet of Vehicles data and are difficult to cope with the rapid expansion of data size. In terms of data query, existing technologies mostly rely on full table scan query methods, which are inefficient when processing massive amounts of data (such as hundreds of millions of Internet of Vehicles data) and are difficult to meet the needs of real-time queries.

[0055] As a result, it is difficult to ensure the accuracy and completeness of the query results of Internet of Vehicles data.

[0056] In order to solve the above technical problems, this specification provides a method for processing Internet of Vehicles data.

[0057] By setting up an HBase database on the Internet of Vehicles platform and connecting with the client and server through the HBase programming interface, the massive data of the Internet of Vehicles can be processed to ensure the accuracy and completeness of the query results of the Internet of Vehicles data.

[0058] Specifically, HBase is an open source, distributed, versioned, non-relational database based on Hadoop. It can process hundreds of millions of data and is very suitable for storing large-scale Internet of Vehicles data. HBase uses a columnar storage model to facilitate column expansion and can cope with the complexity of Internet of Vehicles data.

[0059] Combination Figure 1 and Figure 2 As shown, Figure 1 This is a flow chart of a method for processing vehicle network data provided by an embodiment of this specification, comprising the following steps:

[0060] S100: Receive a query request sent by a client of the Internet of Vehicles platform, where the query request includes row key information.

[0061] In the Internet of Vehicles platform, vehicle resources are allocated to different users according to certain rules and requirements, so that users can use Internet of Vehicles data to perform power performance analysis, battery analysis, driving information acquisition, etc. In some embodiments, the main path for vehicle allocation is to query and import vehicles into Excel tables. Specifically, vehicle information is obtained and allocation decisions are made by querying databases and other methods. That is, allocation can be based on real-time vehicle data obtained by query, or it can be based on data in a pre-prepared Excel table. When a user needs to view or apply Internet of Vehicles data, a query request is issued through the Internet of Vehicles platform to query the corresponding vehicle information in the HBase database.

[0062] In some embodiments, in order to have different vehicle query permissions for different users, it is necessary to assign corresponding person-vehicle relationships, so the system needs corresponding data permissions. Specifically, one case is that the platform creates users; another case is that vehicles are assigned to users, mainly through querying vehicles (mainly for single-vehicle allocation) and importing vehicles (mainly for multiple-vehicle allocation); or through dynamic vehicle allocation through formulas such as selecting vehicle brands and vehicle manufacturers.

[0063] Furthermore, when receiving a query request sent by a client of the Internet of Vehicles platform, the permission of the user who initiated the query request on the client is verified, and if the verification passes, the target data is obtained from the HBase database according to the row key information, thereby ensuring that only legitimate users can access relevant data.

[0064] Specifically, permission verification includes identity verification and protocol verification.

[0065] As an example, when using username verification, first there needs to be a permission management system or database table to store the permission association information between users and vehicles. When a query request is received, the user's identity (such as username, user ID, etc.) is searched in the permission system to determine whether the user has the permission to query the specified vehicle.

[0066] For example, you can create a permission table user_vehicle_permissions, which contains fields such as user_id, vehicle_id, and permission_flag. You can query this table to determine whether the user has permission.

[0067] As another example, the data item mainly depends on the protocol construction, mainly relying on GB / T32960, HJ1239, 17691, tsp, can, mscan protocols to construct the data item protocol tree structure, and performing protocol verification based on the data item protocol tree structure.

[0068] The client page provides a variety of data query options, such as new energy national standard detailed data query, heavy-duty vehicle National VI data detailed query, TSP detailed data query, CAN detailed data query, and ms-level CAN detailed data query. The heavy-duty vehicle National VI detailed data mainly includes two protocols 17691 and HJ 1239.

[0069] During the data query process, the vehicles are first divided according to the reported protocol and the vehicle types supported by the protocol, and the data is saved to redis (executed by the corresponding configured big data service redis). When querying the data, the row key information in the query request will be used to determine whether the vehicle meets the relevant protocol.

[0070] It should be noted that when designing the row key, the requirements for data query, the characteristics of the storage engine, and the data access mode are usually taken into consideration. In the current scenario of obtaining vehicle data for performance analysis, the required data can be quickly queried and obtained based on the vehicle identification code and the timestamp of the data. Therefore, the row key is constructed based on vin and time, and the vehicle data is queried through the row key. Of course, other HBase row keys can be set in other scenarios or requirements, and are not limited here.

[0071] Continuing from the previous embodiment, when the row key information includes the vehicle identification code and the start and end time, as an example, when querying the energy national standard detailed data, on the client page, after entering the vehicle identification code vin, the protocol interface will be called to determine whether the vehicle supports querying 32960 data. If the vehicle does not meet the requirements, the front-end page will display a pop-up window. Generally, a vehicle will meet multiple protocol data.

[0072] If the aforementioned permission check and vehicle type check are passed, the system will initiate a query operation, which may involve interacting with a database or data storage system, using appropriate query statements or APIs to obtain the required data.

[0073] If the permission check and vehicle type check fail, an exception is thrown on the client so that the user can re-enter the row key information.

[0074] S200. Obtain target data from an HBase database according to the row key information, wherein the HBase database includes a plurality of Regions divided based on the row key information, and the Regions store Internet of Vehicles data.

[0075] The partitions in HBase described in this article are called Regions. The data in the table is automatically divided into multiple Regions based on the range of row keys. Each Region contains a certain range of row key-value pairs. This partitioning mechanism helps to achieve distributed storage and parallel processing of data, and improves data read and write performance.

[0076] The target data includes total data and paging data. The total data and paging data use asynchronous programming tools to implement concurrent multi-task execution, and the results are returned at one time, which can fully utilize the concurrent capabilities of the system, improve the efficiency of data acquisition, reduce waiting time, and avoid blocking caused by sequential execution.

[0077] In some embodiments, the method further comprises:

[0078] Calling the asynchronous programming tool, creating a first thread and a second thread for asynchronous concurrent processing through the asynchronous programming tool; the first thread is used to obtain the total amount data, and the second thread is used to obtain the paging data;

[0079] The result of the first thread and the result of the second thread are combined to form target data.

[0080] Specifically, create two CompletableFuture task threads, the first thread is used to obtain the total amount of data, and the second thread is used to obtain the paging data. These two tasks can be executed simultaneously in different threads. When both tasks are completed, the results are merged and returned to the caller at once. For example, you can use Java's CompletableFuture class to create two asynchronous tasks, each executing the logic of obtaining the total amount and paging data. After completion, get the results and merge them through the callback function or join method.

[0081] In this embodiment, a coprocessor technology is used, such as the AggregationClient of HBase, to perform aggregation operations to calculate the total number, and the result is directly returned to the client, avoiding the client from calculating on its own, thereby improving data query efficiency.

[0082] Based on the above embodiment, the process of obtaining the total amount data in the first thread includes:

[0083] In the first thread, a coprocessor of the Internet of Vehicles platform HBase is called to obtain total amount data from the HBase database based on the row key information;

[0084] The HBase database includes a plurality of Regions divided based on the row key information, and the Internet of Vehicles data is stored in the Regions.

[0085] Specifically, the coprocessor is an extension mechanism provided by HBase that allows custom code logic to be executed on RegionServer to enhance the functionality of HBase. By using HBase's coprocessor AggregationClient to calculate the total number of data, data statistics and aggregation can be processed more efficiently.

[0086] The coprocessor is tightly integrated with the RegionServer. When a client initiates a request, the RegionServer decides whether to activate the corresponding coprocessor based on the type and conditions of the request. The coprocessor can access and process the data in the Region and return the results to the RegionServer, which then returns them to the client.

[0087] For example, Figure 3 As shown in the figure, when performing data statistics operations, the client specifies the row key information according to business needs and sends a query request to the server through the scan operation. After receiving the query request, the server determines the region where the data is located according to the row key information and routes the request to the corresponding RegionServer. The RegionServer locates the specific Region to read the data and needs to use the coprocessor to calculate the total number of qualified data in the specific Region. After the coprocessor performs the calculation, it returns the result to the RegionServer, and the RegionServer then passes the result to the client.

[0088] The coprocessor can also interact with other components of HBase, such as the Master, to a limited extent, such as reporting status or requesting resources in some special cases. However, the main interaction is still concentrated between the RegionServer and the client to implement specific data processing and computing tasks.

[0089] In this embodiment, when a coprocessor is used to query on a RegionServer, the coprocessor can operate directly within the relevant Region, avoiding unnecessary data scanning and transmission. The partitioning mechanism provides a basis for the efficient execution of the coprocessor, enabling it to process data within a specific range more specifically, reducing computing and network overhead.

[0090] In other embodiments, when performing a scan operation, the addColumn method is used to specify a specific column family and column to be scanned to improve query efficiency. Among them, the can.addColumn operation is used to specify a specific column to be queried. Here, the column family "cf" and the column "vin" are specified, which helps to reduce unnecessary data transmission and processing and improve query efficiency. For example, in a vehicle management system, if you only care about the total number of vehicle identification codes (vin), you can quickly and accurately obtain them in this way.

[0091] Based on the above embodiment, the process of obtaining paging data in the second thread includes:

[0092] In the second thread, the target vehicle data is obtained from the HBase database according to the row key information and the data item; wherein the query request also includes a data item for characterizing the data type of the Internet of Vehicles data;

[0093] Determine the data range based on preset query parameters; the query parameters include query page number and data size per page;

[0094] The paging data in the target vehicle data for feeding back to the client is determined according to the data range.

[0095] The data items described in this article are mainly constructed according to the 32960 protocol. The front-end page will call the interface to display the data items that can be queried by the protocol. When querying, the English field corresponding to the data item will be passed in, such as the data type, such as vehicle driving data, charging data, etc.

[0096] As an example, assuming that the pageNum requested by the front end is 3 and the pageSize is 20, then the data range to be obtained is calculated to start from row 41 and obtain the next 20 rows of data, that is, rows 41 to 60. By encapsulating the setting of scan.setLimit in this way, the amount of data returned by each query can be dynamically adjusted according to the front end request, thereby optimizing the performance of paging queries.

[0097] In order to improve the query speed, the data range to be obtained will be calculated based on the paging parameters transmitted by the client, such as the query page number pagenum and the data size of each page pagesize, and the size scan.setLimit(size) will be set. By using scan.setLimit(size) to limit the amount of data returned by the scan, the required paging data can be accurately obtained, excessive data loading can be avoided, and paging performance issues can be solved, thereby improving the query speed.

[0098] In other embodiments, multi-vehicle query can be performed, that is, the input row key includes multiple vins and start and end times. In the process of obtaining the total amount of data, the coprocessor AggregationClient will use traversal of multiple vins and start and end time statistics. In the process of obtaining paginated data, it will be processed according to the input multiple vins and start and end times, as well as the query page number and the size of each page of data. It is worth noting that multi-vehicle query is the query and encapsulation of paginated data, and the query parameters are also set for feedback.

[0099] For example, when querying data for 10 vehicles, and setting the query page number to the first page and the data size of each page to 10, if the number of data items queried based on the vin number of the first vehicle within this start and end time exceeds 10, the data of the first vehicle will be fed back; if the number of data items queried based on the vin number of the first vehicle within this start and end time is less than 10 (assuming 5), then continue to query based on the vin number of the second vehicle within this start and end time, and only query five data items. The results of the two queries are packaged together for a total of 10 data items, and the data is returned, and so on. The data of different vehicles are effectively integrated to ensure the integrity and accuracy of each page of data, meet the needs of paging queries in the best way, and improve the system's response performance and resource utilization efficiency. At the same time, users can obtain the required information faster without waiting for all data processing of all vehicles to be completed, improving the user experience.

[0100] In some other embodiments, before encapsulating the target data and feeding the encapsulated data back to the client for display, the method further includes:

[0101] The paging data is converted based on a preset escape function to obtain converted target data.

[0102] For the returned data items, escape is performed according to specific rules or mappings to make them meet the requirements of front-end display or processing to prevent abnormal or erroneous display. Therefore, before encapsulation, the paging data is converted based on a preset escape function, which can be a mapping relationship function based on the constructed escape enumeration class before and after the escape, so as to convert the values ​​from the enumeration class later and solve the data escape problem.

[0103] For example, in the pagination of a product list on an e-commerce website, the corresponding data is accurately obtained based on the page number and the number of items displayed per page selected by the user, and special characters or formats are processed to ensure normal display of the page.

[0104] In some embodiments, the query request further includes index information; and obtaining target data from the HBase database according to the row key information includes:

[0105] Determine the Region to be queried in the HBase database from a preset index table according to the index information;

[0106] Obtain target data from the region to be queried according to the row key information;

[0107] The index table is created by presetting the index information and the row key information corresponding to the Region.

[0108] Specifically, we can speed up the query by creating an additional index table. The process of creating a secondary index in HBase is as follows:

[0109] First, a separate index table needs to be created. The row key of this index table can be designed as a vehicle feature, such as the year of vehicle production, and the column value can store the row key of the vehicle corresponding to the year of production (that is, the row key information of the corresponding row in the original HBase database, such as vin+start and end time). It can be understood that the index table is created by pre-setting the index information and the row key information corresponding to the Region.

[0110] When you want to query vehicles of a specific production year, first quickly locate the corresponding row in the index table by the production year, and obtain a row key list containing vehicles of that production year.

[0111] Then, according to the obtained row key list, the detailed vehicle information is obtained from the original HBase database.

[0112] The advantage of this is that in the original HBase database, vehicle data may be stored and partitioned according to other row key rules (such as vin+start and end time, etc.), and querying directly by production year may not be efficient. However, through the secondary index, the row key can be quickly filtered according to the production year, and then the specific data can be obtained, thereby improving the speed of querying vehicles of a specific production year.

[0113] For example, the row key of the original vehicle data table is vin, and the data is partitioned according to the vin range. If you query "2022" directly, you may need to traverse multiple partitions. However, with a secondary index table of production year, for example, the row key is "2022", and the column values ​​are "vin1", "vin3", "vin7", etc., you can directly obtain the vin of the relevant vehicle based on the row key "2022", and then go to the HBase database to find the detailed vehicle information corresponding to these vins, which greatly reduces the amount and scope of the query data and improves the query efficiency.

[0114] S300: Encapsulate the target data, and feed the encapsulated data back to the client for display.

[0115] The processed total amount data, paging data, and other possible related information are encapsulated in a predetermined data structure or format for easy transmission and processing by the caller. For example, an object containing the total amount, paging data, and other necessary metadata is created and serialized into JSON or other suitable formats and returned to the caller, so that different systems or programming languages ​​can easily parse and process the returned data.

[0116] Doing so can improve the readability and maintainability of the data, making it easier for the caller to perform subsequent processing and use.

[0117] In some embodiments, after receiving the query request sent by the client of the Internet of Vehicles platform, the method further includes:

[0118] Performing a preliminary query in a preset cache area according to the row key information;

[0119] In the case where the row key information matches the historical row key information in the cache area, obtaining data corresponding to the historical row key information from the cache area as target data;

[0120] The cache area includes at least one query record based on the historical row key information when the HBase database is queried for the first time, and the query record includes the historical row key information and the historical target data queried for the first time.

[0121] For the total data value, statistics are collected by traversing multiple vins and the corresponding start and end times, and the statistical values ​​are saved in the LRU (Least Recently Used) cache. If the corresponding value previously saved is found in the LRU cache during subsequent queries, it is directly used; if not found, the value is calculated by the coprocessor and then saved in the LRU cache.

[0122] At the same time, the data of each vehicle will also be saved. The purpose is to distinguish which page of vehicle data is insufficient in the subsequent data query processing, so as to supplement the data of another vehicle to meet the needs of page data display. Specifically, in other embodiments, when performing multi-vehicle queries, the row key information is decomposed when the row key information meets the preset conditions, and each vehicle is queried in batches based on the decomposed row key information to avoid excessive server memory usage when querying large amounts of data, affecting the stability and performance of the system. It should be noted that the row key information composed of vin and the start and end time is split according to the start and end time to obtain multiple groups of query conditions composed of the same vin and multiple decomposed start and end times, and batch queries of the data of a vehicle are implemented according to the query conditions. The implementation methods of multi-vehicle queries are basically the same, and the row key information of each vehicle is decomposed, and each vehicle is queried in batches in turn.

[0123] The preset condition can be that the start and end time of the query exceeds the set time threshold in the time dimension, that is, if the start and end time exceeds 24 hours, then asynchronous multi-threaded queries are performed in batches. For example, to query 20 vehicles at a time, enter the VIN and start and end time of 20 vehicles on the system page, and first determine whether the start and end time is within 24 hours. If not, the start and end period will be split into 24-hour cycles based on the 24-hour principle, and the data will be queried based on the split VIN and start and end time. The final result is to count the asynchronous multi-threaded query results and return them.

[0124] As an example, query three cars A, B, and C, with a start and end time of 05:00:00 on 2024-07-01 to 14:00:00 on 2024-07-06. The asynchronous multithreading is set to return the queried data based on a 24-hour cycle, that is, access car A from 05:00:00 on 2024-07-01 to 05:00:00 on 2024-07-02, 05:00:00 on 2024-07-02 to 05:00:00 on 2024-07-03, and so on, and obtain data in 5 times.

[0125] The total number of entries is counted through the results returned by asynchronous multi-threading, and the count data of each vin and start and end time are also saved in the LRU. When displayed on the client, they are displayed in the order of queried vehicles. If a page has insufficient data (for example, the amount of queried data is less than the pagesize of each page), it will be spliced ​​according to the data of the next vehicle to meet the page data display requirements.

[0126] Therefore, the query record of each first query based on the historical row key information in the HBase database can be named by row key information (such as vin and start and end time), and when other people export files with the same query conditions, it will be prompted that the file already exists and can be downloaded directly. The query record includes the historical row key information and the historical target data queried for the first time.

[0127] Specifically, in the aforementioned embodiment, a query is performed in the HBase database according to the row key information and the query result is saved in a preset cache area at the same time. After receiving a query request sent by a client of the Internet of Vehicles platform, first determine whether the required data exists in the cache area according to the key value constituted by the query request. If it exists, the data in the cache is directly used to avoid querying the original data again, thereby improving efficiency. It should be noted that when querying data from the HBase database through vin, start and end time and data items, the query record at the time of the first query is named according to vin, start and end time and data items, and the relevant data is saved in the LRU cache, so that after receiving a query request sent by a client of the Internet of Vehicles platform, first determine whether the required data exists in the cache area according to vin, start and end time and data items. If it exists, the data in the cache is directly used.

[0128] The present application provides a method, device and computer-readable storage medium for processing Internet of Vehicles data, which stores vehicle data based on an HBase database, facilitates column expansion, and thus accommodates massive data, meeting the demand for complexity of Internet of Vehicles data. On the basis of the HBase database, a data query is responded to a query request sent by a client of an Internet of Vehicles platform, and target data is obtained from the HBase database based on row key information in the query request, the target data is packaged, and the packaged data is fed back to the client for display, thereby improving the accuracy and completeness of the query results.

[0129] Based on the same application concept as the above method, the embodiment of the present application also proposes a vehicle network data processing device, such as Figure 4 shown.

[0130] The device comprises:

[0131] The request receiving module 402 is used to receive a query request sent by a client of the Internet of Vehicles platform, wherein the query request includes row key information;

[0132] A data query module 404 is used to obtain target data from an HBase database according to the row key information, wherein the HBase database includes a plurality of regions divided based on the row key information, and the regions store Internet of Vehicles data;

[0133] The data encapsulation module 406 is used to encapsulate the target data and feed the encapsulated data back to the client for display.

[0134] The implementation process of the functions and effects of each module / submodule / unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and the same technical effect can be achieved, so it will not be repeated here.

[0135] Figure 5 The following is a schematic diagram of the physical structure of a vehicle networking data processing device. Figure 5 As shown, the Internet of Vehicles data processing device may include: a processor (processor) 810, a communication interface (CommunicationsInterface) 820, a memory (memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the Internet of Vehicles data processing method.

[0136] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0137] On the other hand, the present application also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the Internet of Vehicles data processing methods provided by the above methods.

[0138] On the other hand, the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the vehicle network data processing method provided by the above-mentioned methods.

[0139] It should be noted that the technical solutions or technical features described in the above embodiments can be combined or supplemented with each other without causing conflicts. The scope of protection of this application is not limited to the precise structures described in the above embodiments and shown in the drawings; all modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A method for processing Internet of Vehicles data, characterized in that: The method is applied to the server side of the Internet of Vehicles platform, and the method includes: Receiving a query request sent by a client of the Internet of Vehicles platform, wherein the query request includes row key information; According to the row key information, the target data is obtained from an HBase database, where the HBase database includes a plurality of Regions divided based on the row key information, and the Regions store Internet of Vehicles data including the target data; The target data is encapsulated, and the encapsulated data is fed back to the client for display.

2. The method for processing Internet of Vehicles data according to claim 1, characterized in that: The target data includes total amount data and paging data; The method further comprises: Calling an asynchronous programming tool, and creating a first thread and a second thread for asynchronous concurrent processing through the asynchronous programming tool; the first thread is used to obtain the total amount data, and the second thread is used to obtain the paging data; The result of the first thread and the result of the second thread are combined to form target data.

3. The method for processing Internet of Vehicles data as claimed in claim 2, characterized in that: The first thread is used to obtain the total amount data, including: In the first thread, a coprocessor of the Internet of Vehicles platform HBase is called to obtain total amount data from the HBase database based on the row key information.

4. The method for processing Internet of Vehicles data as claimed in claim 3, characterized in that: The query request also includes column family information and column information; The obtaining of total amount data from the HBase database further includes: Based on the row key information, family information and column information, the total amount of data is obtained from the HBase database.

5. The method for processing Internet of Vehicles data as claimed in claim 2, characterized in that: The query request also includes a data item for characterizing the data type of the Internet of Vehicles data; The method further comprises: In the second thread, target vehicle data is obtained from the HBase database according to the row key information and the data item; Determine the data range based on preset query parameters; the query parameters include query page number and data size per page; The paging data in the target vehicle data for feeding back to the client is determined according to the data range.

6. The method for processing Internet of Vehicles data as claimed in claim 5, characterized in that: Before encapsulating the target data and feeding the encapsulated data back to the client for display, the method further includes: The paging data is converted based on a preset escape function to obtain converted target data.

7. The method for processing Internet of Vehicles data according to claim 1, characterized in that: The query request also includes index information; The acquiring target data from the HBase database according to the row key information includes: Determine the Region to be queried in the HBase database from a preset index table according to the index information; Obtain target data from the region to be queried according to the row key information; The index table is created by presetting the index information and the row key information corresponding to the Region.

8. The method for processing Internet of Vehicles data as claimed in claim 1, characterized in that: After receiving the query request sent by the client of the Internet of Vehicles platform, the method further includes: Performing a preliminary query in a preset cache area according to the row key information; In the case where the row key information matches the historical row key information in the cache area, obtaining data corresponding to the historical row key information from the cache area as target data; The cache area includes at least one query record based on the historical row key information when the HBase database is queried for the first time, and the query record includes the historical row key information and the historical target data queried for the first time.

9. The method for processing Internet of Vehicles data as claimed in claim 1, characterized in that: Before obtaining the target data from the HBase database according to the row key information, the method further includes: The user who initiates the query request on the client is subjected to permission verification. If the verification passes, the target data is obtained from the HBase database according to the row key information. The permission verification includes a protocol verification.

10. The method for processing Internet of Vehicles data according to any one of claims 1 to 9, characterized in that: The row key information includes a vehicle identification code and a start and end time.

11. A vehicle networking data processing device, characterized in that: It includes a memory, a processor, and a vehicle networking data processing program stored in the memory and executable on the processor. When the processor executes the vehicle networking data processing program, the steps of the vehicle networking data processing method as described in any one of claims 1 to 10 are implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a vehicle networking data processing program, and when the vehicle networking data processing program is executed, the steps of the vehicle networking data processing method as described in any one of claims 1 to 10 are implemented.

Images