Internet of vehicles database row key generation method, apparatus and device, and storage medium

Abstract

The invention discloses an Internet of Vehicles database row key generation method and device, equipment and a storage medium. Comprising the steps that an Internet of Vehicles database is acquired, a row key generation strategy is determined based on the Internet of Vehicles database, and the row key generation strategy comprises components and a sorting mode; generating a row key of each row of data in the Internet of Vehicles database according to the generation strategy; the method comprises the following steps: pre-partitioning an Internet of Vehicles database to generate data partitions, and carrying out continuous performance monitoring on the data partitions to dynamically adjust the data partitions and row keys. By customizing the row key generation strategy according to the data characteristics and requirements of the Internet of Vehicles database, the pertinence and applicability of the row key are improved, the data can be uniformly distributed through the pre-partitioning, the data hotspot and competition risk are reduced, and the concurrent processing capability is improved. Problems can be found in time through performance monitoring, and targeted adjustment is facilitated. The performance and the expandability of the Internet of Vehicles database can be improved, and the high requirement of Internet of Vehicles application for data processing can be better met.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Vehicles, and in particular to a method, device, equipment and storage medium for generating row keys in an Internet of Vehicles database. Background Art

[0002] The Internet of Vehicles has achieved rapid development in today's era. With the help of sensors and communication technology, many vehicles can upload large amounts of data in real time, thus creating a high-concurrency data processing scenario. However, in the actual high-concurrency application scenarios of the Internet of Vehicles, many difficult problems are exposed, such as data congestion, processing delays, and poor accuracy, which have brought great troubles to various business applications.

[0003] However, when processing massive amounts of data from the Internet of Vehicles, database performance faces huge challenges. Among them, the way row keys are generated plays a key role in database query efficiency, data storage, and distribution. Traditional row key generation methods often lack pertinence and optimization, and cannot fully adapt to the characteristics and application requirements of Internet of Vehicles data. For example, simple random generation or time-sequential generation of row keys may lead to uneven data distribution and prominent hot spots, thus affecting the read and write performance of the database. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for generating row keys in a vehicle networking database to solve the problems of data congestion, write failure and data inconsistency in the scenario of high-concurrency write to the database of the vehicle networking. It ensures the accuracy and integrity of data during writing and transmission, improves the reliability and stability of the data processing link, and provides a stable and reliable data foundation for the application fields related to the vehicle networking.

[0005] According to one aspect of the present invention, a method for generating row keys for a vehicle networking database is provided, the method comprising:

[0006] Obtaining an Internet of Vehicles database, and determining a row key generation strategy based on a data query scenario of the Internet of Vehicles database, wherein the row key generation strategy includes components and a sorting method;

[0007] Generate row keys for each row of data in the Internet of Vehicles database according to the generation strategy;

[0008] The Internet of Vehicles database is pre-partitioned to generate data partitions, and the data partitions are continuously monitored for performance to dynamically adjust data partitions and row keys.

[0009] Optionally, a row key generation strategy is determined based on the data query scenario of the Internet of Vehicles, including: determining the data query conditions and query frequency corresponding to the data query scenario of the Internet of Vehicles database; sorting the data query conditions in order from high to low data query frequency to generate a query condition list; selecting a specified number of data query conditions from the query condition list as target query conditions, and determining the components of the row key according to each target query condition; and determining the sorting method of the row key according to the order of each target query condition.

[0010] Optionally, a row key for each row of data in the Internet of Vehicles database is generated according to a generation strategy, including: taking each row of data in the Internet of Vehicles database as target data in turn; determining each key value corresponding to the target data according to the components; arranging each key value in a sorting manner to generate an initial row key sequence of the target data, and processing the initial row key sequence to generate a row key.

[0011] Optionally, the initial row key sequence is processed to generate a row key, including: controlling the length of the initial row key sequence according to a preset length to generate an intermediate row key sequence; hashing the intermediate row key sequence to generate a row key, wherein the hashing processing includes prefix reversal, hashing processing and salting processing.

[0012] Optionally, the Internet of Vehicles database is pre-partitioned to generate data partitions, including: determining data attributes corresponding to each row of data in the Internet of Vehicles database, wherein the data attributes include distribution pattern, vehicle type and time range; determining a partitioning strategy based on the data attributes, wherein the partitioning strategy includes prefix-based pre-partitioning, hash-based pre-partitioning, and range-based pre-partitioning; and pre-partitioning each row of data in the Internet of Vehicles database according to the partitioning strategy to generate data partitions.

[0013] Optionally, continuous performance monitoring is performed on data partitions to dynamically adjust data partitions and row keys, including: obtaining performance indicators and corresponding indicator thresholds, where performance indicators include partition size, data write speed and query response time; monitoring the performance data of each data partition based on performance indicators through monitoring tools; and dynamically adjusting data partitions and row keys based on performance data and indicator thresholds.

[0014] Optionally, data partitions and row keys are dynamically adjusted based on performance data and indicator thresholds, including: determining whether the partition size is greater than the corresponding indicator threshold, and if so, obtaining a partition splitting instruction, and splitting the data partition according to the partition splitting instruction; otherwise, when the data write speed or query response time is greater than the corresponding indicator threshold, obtaining a row key adjustment instruction, and dynamically adjusting the row key according to the row key adjustment instruction.

[0015] According to another aspect of the present invention, a device for generating a row key of Internet of Vehicles data is provided, the device comprising:

[0016] A row key generation strategy determination module is used to obtain a vehicle networking database and determine a row key generation strategy based on a data query scenario of the vehicle networking database, wherein the row key generation strategy includes components and a sorting method;

[0017] A row key generation module is used to generate row keys for each row of data in the Internet of Vehicles database according to a generation strategy;

[0018] The partition and row key adjustment module is used to pre-partition the Internet of Vehicles database to generate data partitions, and continuously monitor the performance of the data partitions to dynamically adjust the data partitions and row keys.

[0019] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0020] at least one processor;

[0021] and a memory communicatively coupled to the at least one processor;

[0022] Among them, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a method for generating row keys for Internet of Vehicles data as described in any embodiment of the present invention.

[0023] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for generating a row key for Internet of Vehicles data as described in any embodiment of the present invention when executed.

[0024] The technical solution of the embodiment of the present invention improves the pertinence and applicability of row keys by customizing the row key generation strategy according to the data characteristics and requirements of the Internet of Vehicles database. The pre-partitioning can reasonably distribute the data, reduce data hot spots and competition risks, and improve concurrent processing capabilities. Performance monitoring can timely discover existing problems, facilitating targeted adjustments and optimizations. It can improve the performance, efficiency and scalability of the Internet of Vehicles database and better meet the high requirements of Internet of Vehicles applications for data processing.

[0025] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 is a flowchart of a method for generating a row key of Internet of Vehicles data provided according to Embodiment 1 of the present invention;

[0028] Figure 2 is a flowchart of another method for generating a row key of Internet of Vehicles data provided according to Embodiment 2 of the present invention;

[0029] Figure 3 is a structural schematic diagram of a vehicle networking data row key generating device provided according to Embodiment 3 of the present invention;

[0030] Figure 4 It is a structural schematic diagram of an electronic device that implements a method for generating a row key for Internet of Vehicles data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] Figure 1A flowchart of a method for generating a row key for Internet of Vehicles data is provided for the first embodiment of the present invention. This embodiment is applicable to scenarios where Internet of Vehicles writes to a database with high concurrency. The method can be executed by an Internet of Vehicles data row key generating device. The Internet of Vehicles data row key generating device can be implemented in the form of hardware and / or software. The Internet of Vehicles data row key generating device can be configured in a vehicle controller. Figure 1 As shown, the method includes:

[0035] S110: Acquire an Internet of Vehicles database, and determine a row key generation strategy based on a data query scenario of the Internet of Vehicles database, wherein the row key generation strategy includes components and a sorting method.

[0036] Among them, the Internet of Vehicles is a comprehensive service system that uses the new generation of information and communication technology to achieve all-round network connections between vehicles and cloud platforms, vehicles and vehicles, vehicles and roads, and vehicles and people. The Internet of Vehicles database is a database used to store Internet of Vehicles related data, covering many aspects such as vehicle driving status, vehicle equipment information, traffic conditions information, and driving behavior information. The row key RowKey is used to uniquely identify the key value of each row of data in the HBase table of the Internet of Vehicles database. Similar to the primary key of a table in a relational database, a specific row of data can be quickly located, retrieved, and operated through the row key. The row key generation strategy is a method and set of rules for planning and determining how to generate row keys, including components and sorting methods.

[0037] Optionally, a row key generation strategy is determined based on the data query scenario of the Internet of Vehicles database, including: determining the data query conditions and query frequency corresponding to the data query scenario of the Internet of Vehicles database; sorting the data query conditions in descending order of query frequency to generate a query condition list; selecting a specified number of data query conditions from the query condition list as target query conditions, and determining the components of the row key according to the target query conditions; and determining the sorting method of the row key according to the order of the target query conditions.

[0038] Among them, the components are used to clarify which specific elements or information the row key is composed of. The components can include vehicle identification information, such as license plate number, frame number; time information; data type related information, etc. The sorting method specifies the order in which the various parts that make up the row key are arranged. A reasonable sorting method helps to quickly locate the required data rows according to the logical order of the row key when retrieving data, such as sorting by vehicle identification first, then by time, etc., so that the data storage structure is more logical, which is convenient for finding relevant data of a specific vehicle at different times.

[0039] It should be noted that the Internet of Vehicles database stores a large amount of vehicle-related data. In actual applications, users will perform various query operations on the database according to different needs. By obtaining past query operations, the corresponding data query conditions and the data used for each query condition can be determined.

[0040] Query frequency. When designing row keys, giving priority to high-frequency query requirements can greatly improve the efficiency of subsequent database queries. High-frequency query conditions mean that the need to obtain data in daily business is more urgent. Putting them in the forefront facilitates the subsequent targeted construction of row keys to better adapt to frequent query scenarios.

[0041] For example, data query condition A is "query the average speed of all vehicles in Beijing from October 1, 2024 to October 31, 2024", and the data query frequency is 50 times / month. Data query condition B is "query all fault alarm records of vehicles with frame number ABC123 in the past year", and the data query frequency is 30 times / month. Data query condition C is "query the battery power data of all new energy vehicles on November 15, 2024", and the data query frequency is 20 times / month.

[0042] It is known that it is usually impossible to fully integrate all data query conditions into the row key design, because that may make the row key too complex and difficult to manage. Therefore, when designing the row key, you can select a specified number of high-frequency data query conditions from the query condition list as the target query conditions. The key information involved in the target query condition becomes the component of the row key. Designing the components of the row key in this way can ensure that the row key contains the most frequently queried data feature information, so that in subsequent queries, the row key can be used to quickly locate the data rows that meet these high-frequency query requirements.

[0043] For example, for “querying all fault alarm records of vehicles with chassis number ABC123 in the past year”, the corresponding row key components may be “chassis number (ABC123), time range (past year), data type (fault alarm record)”.

[0044] Specifically, the order of row keys should be determined according to the order of the target query conditions, so that the row keys can be logically closely matched with high-frequency query scenarios. When performing query operations, the database can quickly filter out eligible data rows according to the logical order of the row keys, improving query efficiency. For example, if the row key order is determined first by region, then by time range, and finally by data type, then when querying a certain data type within a specific time period in a certain region, the target data row can be quickly located according to the row key order logic.

[0045] S120. Generate the row keys of each row of data in the vehicle networking database according to the generation strategy.

[0046] Optionally, generating the row keys of each row of data in the vehicle networking database according to the generation strategy includes: successively taking each row of data in the vehicle networking database as the target data; determining the respective key values corresponding to the target data according to the components; arranging the key values in a sorting manner to generate an initial row key sequence for the target data, and processing the initial row key sequence to generate the row keys.

[0047] Specifically, the controller processes each row of data one by one. When a certain row of data is taken as the target data, specific values corresponding to the components of the row key, that is, the key values, need to be extracted from this row of data. After determining each key value, arranging them according to the previously set sorting method can obtain the initial row key sequence.

[0048] Exemplarily, assume that the row key generation strategy stipulates that the row key consists of the vehicle license plate number, the acquisition timestamp, and the data type code, and the sorting method is first the license plate number, then the timestamp, and finally the data type code. Then for a vehicle with a license plate number of "Jing A12345", the vehicle condition data collected at 10 o'clock on December 2, 2024, and the corresponding data type code is "01", its initial row key sequence can be generated as "Jing A12345_20241202100000_01".

[0049] Optionally, processing the initial row key sequence to generate the row keys includes: controlling the length of the initial row key sequence according to a preset length to generate an intermediate row key sequence; performing a hashing process on the intermediate row key sequence to generate the row keys, where the hashing process includes prefix reversal, hashing, and salting.

[0050] It should be noted that in the vehicle networking database, the length of the row key usually needs to conform to certain specifications and limitations, and it is generally recommended to be within 16 bytes. On the one hand, an overly long row key may occupy too much storage space and increase the database storage cost; on the other hand, an overly long row key may affect the efficiency during data retrieval, comparison, etc., resulting in a longer processing time. Therefore, the length of the initial row key sequence needs to be controlled according to the preset length.

[0051] Among them, prefix reversal refers to performing a reversal operation on a part of the intermediate row key sequence (usually the prefix part with a certain pattern). In vehicle networking, if the initial row key sequence of a vehicle has a certain prefix pattern (such as a prefix divided by vehicle brand, model, or region), reversing these prefixes can disrupt the original order of the data and make the data more evenly distributed.

[0052] Exemplarily, assume that the middle row key sequence is "Jing A12345_20241202100000_vehicle driving speed". If the prefix "Jing A12345" is reversed to get "54321A Jing", then the row key sequence after prefix reversal processing may become "54321A Jing_20241202100000_vehicle driving speed".

[0053] Specifically, hashing refers to converting the input middle row key sequence, such as the VIN number of a vehicle, into a hash value of a fixed length through a hash function. The characteristic of a hash function is that for different inputs, it tries to generate evenly distributed outputs. Common hash functions include MD5, sha1, sha256, or sha512, etc.

[0054] Exemplarily, assume that the common MD5 hash function is used to hash the row key sequence "54321A Jing_20241202100000_vehicle driving speed" after prefix reversal processing, and a hash value similar to "32d56f8e79b86a9067b095697b2169c3" will be obtained.

[0055] Specifically, salting means adding a random "salt value" to the middle row key sequence and then performing storage or subsequent processing. This salt value can be a random number or a dynamic attribute related to the vehicle (such as the current location code of the vehicle).

[0056] S130. Pre-partition the vehicle networking database to generate each data partition, and continuously monitor the performance of the data partitions to dynamically adjust the data partitions and row keys.

[0057] Among them, the amount of data in the vehicle networking database is usually very large. To improve the read and write performance of the data, it is necessary to perform partition processing on it. Pre-partitioning is to divide the database into several different data partitions in advance according to the characteristics of vehicle networking data and business requirements before the data is formally stored. A data partition is each relatively independent data storage area formed after the pre-partitioning operation. Each data partition stores data with certain commonalities. For example, after partitioning by vehicle region, a certain partition stores all vehicle data from a specific region. By dividing the database into multiple data partitions, hierarchical management of the data can be achieved, the overall performance of the database can be improved, and specific operations and analyses can be conveniently performed on different partitions. Performance monitoring refers to the operation of continuously observing, measuring, and analyzing various performance indicators of the database during the operation of the database, which can include the read and write speed of the data, the occupancy of the storage space, the query response time, etc.

[0058] Optionally, the Internet of Vehicles database is pre-partitioned to generate data partitions, including: determining data attributes corresponding to each row of data in the Internet of Vehicles database, wherein the data attributes include distribution pattern, vehicle type and time range; determining a partitioning strategy based on the data attributes, wherein the partitioning strategy includes prefix-based pre-partitioning, hash-based pre-partitioning, and range- and size-based pre-partitioning; and pre-partitioning each row of data in the Internet of Vehicles database according to the partitioning strategy to generate data partitions.

[0059] Among them, data attributes include distribution rules, vehicle types and time ranges. Distribution rules refer to the fact that different types of data in the Internet of Vehicles database may present certain rules in storage. For example, vehicle data in certain areas may have specific distribution patterns in terms of collection frequency and data volume due to local traffic flow characteristics, economic activities and other factors. Understanding the distribution rules will help to reasonably divide the partitions according to the density of the data, so that the data volume of each partition is relatively balanced, which is convenient for management and query. Different types of vehicles have their own characteristics. For example, there are obvious differences between new energy vehicles and fuel vehicles in terms of battery power data and fuel consumption data. In addition, different types of vehicles may also have different usage scenarios, driving habits and other aspects, resulting in different data generation and change rules. Dividing data attributes according to vehicle types can facilitate special partition management and analysis of data for different types of vehicles. Data in different time ranges may have different importance, query frequency, etc. For example, recent data may be more frequently queried because it is closely related to current business decisions, while data with a long history may be more used for long-term trend analysis. By clarifying the time range, it is helpful to partition data according to the time dimension.

[0060] Among them, the process of determining the partition strategy (i.e., pre-partition strategy) refers to determining the number of initial data partitions of the final database, the start and end row keys of each data partition, and the size of each data partition. Common partition strategies include prefix-based pre-partitioning, hash-based pre-partitioning, and range-based and size-based pre-partitioning. Prefix-based pre-partitioning refers to dividing data partitions according to the prefix of the row key. It is suitable for situations where the row key has an obvious prefix. For example, if the first and second characters of the row key are both numbers from 0 to 9, we can divide the data into 10 partitions with partition names of 00, 10, 20, 30, 40, 50, 60, 70, 80, and 90, where the 00 partition stores the data of row keys from 00xxx to 09xxx, the 10 partition stores the data of row keys from 10xxx to 19xxx, and so on. Hash-based pre-partitioning refers to using a hash function to map the row key to a fixed range, and then dividing the partitions according to this range. For example, using the MD5 hash function, setting the number of pre-partitions to 20, and then calculating the starting row keys of each partition to divide the partitions. Pre-partitioning based on range and size means partitioning based on the value range of row keys and the size of each partition, which is applicable to the case where row keys are continuous. For example, if the row key is "region code reversal + vehicle frame number reversal + time", partitions are divided based on region codes, and the region codes are 101, 102, 103, 104, and 105, and the vehicle occupancy ratio of each region is 6:1:1:1:1, the number of data partitions is set to 10, and the partition names are 101_0xxx, 101_1xxx, 101_9x, 201_xxx, 301_xxx, 401_xxx, and 501_xxx, where the 101_0xxx partition includes data in the region 101 with row keys ranging from 101_00xx to 101_09xxx, and the 101_1xxx partition includes data in the region 101 with row keys ranging from 101_10xx to 101_19xx, and so on.

[0061] Specifically, when determining the partitioning strategy, if it is found that the vehicle data in some areas is densely distributed, while in other areas it is relatively sparse, you can consider dividing it by region. Divide the data-dense areas into separate partitions so that when querying the data in the area, it can be more efficiently located and read, avoiding problems such as slow queries due to excessive data volume. According to the data characteristics of different vehicle types, such as the difference between new energy vehicles and fuel vehicles, the partitioning strategy can be to divide the new energy vehicle data into one or more partitions, and the fuel vehicle data is also divided into separate partitions. In this way, when performing data query, analysis and other operations on a specific type of vehicle, you can directly locate the corresponding partition to improve efficiency. The partitioning strategy can also be divided according to time. For example, divide the monthly data into one partition, divide the data of nearly three months into one partition, and divide other historical data into another partition.

[0062] The technical solution of the embodiment of the present invention improves the pertinence and applicability of row keys by customizing the row key generation strategy according to the data characteristics and requirements of the Internet of Vehicles database. The pre-partitioning can reasonably distribute the data, reduce data hot spots and competition risks, and improve concurrent processing capabilities. Performance monitoring can timely discover existing problems, facilitating targeted adjustments and optimizations. It can improve the performance, efficiency and scalability of the Internet of Vehicles database and better meet the high requirements of Internet of Vehicles applications for data processing.

[0063] Embodiment 2

[0064] Figure 2 This is a flowchart of a method for generating row keys for a car networking database provided in the second embodiment of the present invention. This embodiment adds a specific process of continuous performance monitoring of data partitions to dynamically adjust data partitions and row keys based on the above-mentioned first embodiment. Among them, the specific contents of steps S210-S220 are roughly the same as steps S110-S120 in the first embodiment, so they will not be repeated in this embodiment. Figure 2 As shown, the method includes:

[0065] S210: Acquire an Internet of Vehicles database, and determine a row key generation strategy based on a data query scenario of the Internet of Vehicles database, wherein the row key generation strategy includes components and a sorting method.

[0066] Optionally, a row key generation strategy is determined based on the data query scenario of the Internet of Vehicles database, including: determining the data query conditions and query frequency corresponding to the data query scenario of the Internet of Vehicles database; sorting the data query conditions in order from high to low data query frequency to generate a query condition list; selecting a specified number of data query conditions from the query condition list as target query conditions, and determining the components of the row key according to each target query condition; and determining the sorting method of the row key according to the order of each target query condition.

[0067] S220: Generate a row key for each row of data in the Internet of Vehicles database according to the generation strategy.

[0068] Optionally, a row key for each row of data in the Internet of Vehicles database is generated according to a generation strategy, including: taking each row of data in the Internet of Vehicles database as target data in turn; determining each key value corresponding to the target data according to the components; arranging each key value in a sorting manner to generate an initial row key sequence of the target data, and processing the initial row key sequence to generate a row key.

[0069] Optionally, the initial row key sequence is processed to generate a row key, including: controlling the length of the initial row key sequence according to a preset length to generate an intermediate row key sequence; hashing the intermediate row key sequence to generate a row key, wherein the hashing processing includes prefix reversal, hashing processing and salting processing.

[0070] S230: pre-partition the Internet of Vehicles database to generate data partitions.

[0071] Optionally, the Internet of Vehicles database is pre-partitioned to generate data partitions, including: determining data attributes corresponding to each row of data in the Internet of Vehicles database, wherein the data attributes include distribution pattern, vehicle type and time range; determining a partitioning strategy based on the data attributes, wherein the partitioning strategy includes prefix-based pre-partitioning, hash-based pre-partitioning, and range- and size-based pre-partitioning; and pre-partitioning each row of data in the Internet of Vehicles database according to the partitioning strategy to generate data partitions.

[0072] S240. Obtain performance indicators and corresponding indicator thresholds, wherein the performance indicators include partition size, data writing speed, and query response time.

[0073] It should be noted that if the partition is too large, it may cause data retrieval, update and other operations to slow down, because the target data needs to be found or processed in more data. If the partition is too small, it may cause too many partitions, increase management complexity, and may also affect the overall storage efficiency. Therefore, it is necessary to clarify the appropriate partition size indicator threshold to determine whether the partition size is within a reasonable range. The data writing speed reflects how quickly the Internet of Vehicles database can collect new data into the corresponding partition. If the writing speed is too slow, data backlog may occur, resulting in subsequent queries and other operations unable to obtain the latest data in time. Setting the indicator threshold of the data writing speed can timely detect abnormal situations in the writing process and ensure timely data updates. The query response time is the time from submitting a query request to obtaining the result. A shorter query response time means that users can quickly get the required information and improve the user experience. If the query response time is too long, it means that there is a problem with the query efficiency of the database, which needs to be discovered and solved by monitoring this indicator and comparing it with the threshold.

[0074] S250. Monitor the performance data of each data partition based on the performance indicator through a monitoring tool.

[0075] Specifically, monitoring tools are used to collect real-time data on performance indicators of each data partition, such as partition size, data writing speed, and query response time. Monitoring tools can continuously obtain the actual performance of each partition by embedding monitoring code in the database system and interacting with the database interface.

[0076] For example, for partition size, the monitoring tool can regularly measure the storage space occupied by each partition; for data write speed, it can count the number of new data successfully written to the partition within a certain period of time, and calculate the average write speed accordingly; for query response time, the monitoring tool will record the processing time of each query request in each partition, and then derive the average query response time of the partition. Through continuous real-time monitoring by the monitoring tool, the performance status of each data partition can be fully and accurately grasped.

[0077] S260. Dynamically adjust data partitions and row keys based on performance data and indicator thresholds.

[0078] Optionally, data partitions and row keys are dynamically adjusted based on performance data and indicator thresholds, including: determining whether the partition size is greater than the corresponding indicator threshold, and if so, obtaining a partition splitting instruction, and splitting the data partition according to the partition splitting instruction; otherwise, when the data write speed or query response time is greater than the corresponding indicator threshold, obtaining a row key adjustment instruction, and dynamically adjusting the row key according to the row key adjustment instruction.

[0079] Among them, in the Internet of Vehicles database, each data partition has its reasonable storage capacity range. If the partition size is larger than the index threshold, it means that the partition stores too much data. Too much data piled up in a partition will greatly reduce the efficiency of operations such as data retrieval and update. For example, when querying the data in the partition, the database needs to search in a large amount of data, which increases the time cost of the query. At the same time, too large a partition may also affect the speed of data writing, because when writing new data, it may take more time to process and arrange the storage location.

[0080] Furthermore, when it is determined that the partition size is greater than the corresponding indicator threshold, the controller will obtain a partition splitting instruction. Then the data partition is split according to the partition splitting instruction. The splitting method is usually determined according to the characteristics of the data and business needs. For example, it can be split according to the time range, and the data of different time periods in the original partition can be divided into different new partitions. Through partition splitting, the amount of data in each new partition is relatively reasonable, which is convenient for subsequent management and operation, thereby improving the efficiency of data retrieval, writing and other operations.

[0081] Specifically, when the partition size does not exceed the corresponding indicator threshold, it means that there is no outstanding problem affecting performance in terms of partition size. However, you also need to pay attention to the data writing speed and query response time. If the data writing speed is greater than the corresponding indicator threshold, it means that the process of writing data to the partition is too slow, which may cause the new data to not be stored in the database in time, affecting the timeliness and integrity of the data. If the query response time is greater than the corresponding indicator threshold, it means that it takes too long to query the data in the partition, the user experience is poor, and it also reflects that the database has efficiency problems in query operations.

[0082] It can be known that when the data writing speed or query response time is greater than the corresponding indicator threshold, the controller will obtain the row key adjustment instruction, and the row key can be dynamically adjusted according to the row key adjustment instruction. The adjustment may include optimizing the components of the row key. For example, if the driving data of a certain vehicle in a specific area is often queried, but the original row key does not contain the regional information, then the regional information can be added to the components of the row key to more accurately filter out the required data. It may also include adjusting the sorting method of the row key. For example, if the query operation usually needs to locate the data in the order of vehicle type first and then time range, but the existing row key sorting method does not match it, then the position of the vehicle type in the row key can be advanced to make it more in line with common query logic. By adjusting the row key, the effectiveness of the row key in the data retrieval process can be improved, thereby reducing the data writing speed and query response time, and improving the overall performance of the database.

[0083] The technical solution of the embodiment of the present invention monitors the performance data of each data partition based on performance indicators through a monitoring tool, so as to understand the performance status of the data partition in real time and comprehensively, and discover potential problems in time. By dynamically adjusting the data partitions and row keys according to the performance data and indicator thresholds, dynamic optimization of the data partitions and row keys is achieved, ensuring that the database performance is always in a good state. The adaptability and flexibility of the database are improved, and it can cope with the ever-changing data access patterns and business needs. By splitting the data partitions according to the partition splitting instructions, the imbalance of data storage and access caused by too large partitions can be avoided, and the rationality of data distribution can be improved. It helps to improve the efficiency of data writing and querying and reduce performance bottlenecks.

[0084] Embodiment 3

[0085] Figure 3 This is a schematic diagram of the structure of a vehicle networking data row key generation device provided by the third embodiment of the present invention. Figure 3 As shown, the device includes: a row key generation strategy determination module 310, which is used to obtain a car networking database and determine a row key generation strategy based on a data query scenario of the car networking database, wherein the row key generation strategy includes components and a sorting method;

[0086] A row key generation module 320, used to generate a row key for each row of data in the Internet of Vehicles database according to a generation strategy;

[0087] The partition and row key adjustment module 330 is used to pre-partition the Internet of Vehicles database to generate data partitions, and continuously monitor the performance of the data partitions to dynamically adjust the data partitions and row keys.

[0088] Optionally, the row key generation strategy determination module 310 is specifically used to: determine the data query conditions and query frequencies corresponding to the data query scenarios of the Internet of Vehicles database; sort the data query conditions in order from high to low data query frequency to generate a query condition list; select a specified number of data query conditions from the query condition list as target query conditions, and determine the components of the row key according to the target query conditions; and determine the sorting method of the row key according to the order of the target query conditions.

[0089] Optionally, the row key generation module 320 specifically includes: a target data determination unit, which is used to take each row of data in the Internet of Vehicles database as the target data in turn; a key value determination unit, which is used to determine each key value corresponding to the target data according to the components; a row key generation unit, which is used to arrange each key value in a sorted manner to generate an initial row key sequence of the target data, and process the initial row key sequence to generate a row key.

[0090] Optionally, the row key generation unit is specifically used to: control the length of the initial row key sequence according to a preset length to generate an intermediate row key sequence; and hash the intermediate row key sequence to generate a row key, wherein the hash processing includes prefix reversal, hash processing and salting processing.

[0091] Optionally, the partitioning and row key adjustment module 330 specifically includes: a data partitioning unit, used to: determine the data attributes corresponding to each row of data in the Internet of Vehicles database, wherein the data attributes include distribution pattern, vehicle type and time range; determine the partitioning strategy according to the data attributes, wherein commonly used partitioning strategies include prefix-based pre-partitioning, hash-based pre-partitioning, and range- and size-based pre-partitioning; pre-partition each row of data in the Internet of Vehicles database according to the partitioning strategy to generate each data partition.

[0092] Optionally, the partition and row key adjustment module 330 specifically includes: a performance monitoring unit, used to: obtain performance indicators and corresponding indicator thresholds, where the performance indicators include partition size, data writing speed and query response time; monitor the performance data of each data partition based on the performance indicators through a monitoring tool; and dynamically adjust the data partitions and row keys according to the performance data and indicator thresholds.

[0093] Optionally, the performance monitoring unit specifically includes: a row key adjustment subunit, used to: determine whether the partition size is greater than the corresponding indicator threshold, if so, obtain a partition splitting instruction, and split the data partition according to the partition splitting instruction; otherwise, when the data write speed or query response time is greater than the corresponding indicator threshold, obtain a row key adjustment instruction, and dynamically adjust the row key according to the row key adjustment instruction.

[0094] The technical solution of the embodiment of the present invention improves the pertinence and applicability of row keys by customizing the row key generation strategy according to the data characteristics and requirements of the Internet of Vehicles database. The pre-partitioning can reasonably distribute the data, reduce data hot spots and competition risks, and improve concurrent processing capabilities. Performance monitoring can timely discover existing problems, facilitating targeted adjustments and optimizations. It can improve the performance, efficiency and scalability of the Internet of Vehicles database and better meet the high requirements of Internet of Vehicles applications for data processing.

[0095] A device for generating a row key for Internet of Vehicles data provided by an embodiment of the present invention can execute a method for generating a row key for Internet of Vehicles data provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0096] Embodiment 4

[0097] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0098] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0099] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0100] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for generating a row key for Internet of Vehicles data.

[0101] In some embodiments, a method for generating a row key for Internet of Vehicles data may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for generating a row key for Internet of Vehicles data described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute a method for generating a row key for Internet of Vehicles data by any other appropriate means (e.g., by means of firmware).

[0102] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0104] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0107] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0108] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0109] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating row keys for a vehicle networking database, characterized in that: include: Acquire a vehicle networking database, and determine a row key generation strategy based on a data query scenario of the vehicle networking database, wherein the row key generation strategy includes components and a sorting method; Generate a row key for each row of data in the Internet of Vehicles database according to the generation strategy; The Internet of Vehicles database is pre-partitioned to generate data partitions, and the data partitions are continuously performance-monitored to dynamically adjust the data partitions and the row keys.

2. The method according to claim 1, characterized in that The determining of a row key generation strategy based on a data query scenario of the Internet of Vehicles database includes: Determine data query conditions and query frequencies corresponding to data query scenarios of the Internet of Vehicles database; Sorting the data query conditions in descending order of the query frequency to generate a query condition list; Selecting a specified number of data query conditions from the query condition list as each target query condition, and determining a component of a row key according to each target query condition; The order of the row keys is determined according to the order of the target query conditions.

3. The method according to claim 1, characterized in that The step of generating row keys for each row of data in the Internet of Vehicles database according to the generation strategy includes: Taking each row of data in the Internet of Vehicles database as target data in turn; Determine each key value corresponding to the target data according to the components; The key values ​​are arranged according to the sorting method to generate an initial row key sequence of the target data, and the initial row key sequence is processed to generate a row key.

4. The method according to claim 3, characterized in that The processing of the initial row key sequence to generate a row key comprises: Controlling the length of the initial row key sequence according to a preset length to generate an intermediate row key sequence; The intermediate row key sequence is hashed to generate a row key, wherein the hashing process includes prefix reversal, hashing and salting.

5. The method according to claim 1, characterized in that The pre-partitioning of the Internet of Vehicles database to generate data partitions includes: Determine data attributes corresponding to each row of data in the Internet of Vehicles database, wherein the data attributes include distribution pattern, vehicle type and time range; Determining a partitioning strategy according to the data attributes, wherein the partitioning strategy includes prefix-based pre-partitioning, hash-based pre-partitioning, and range- and size-based pre-partitioning; Pre-partition each row of data in the Internet of Vehicles database according to the partitioning strategy to generate each data partition.

6. The method according to claim 1, characterized in that The continuously monitoring the performance of the data partition to dynamically adjust the data partition and the row key includes: Obtaining performance indicators and corresponding indicator thresholds, wherein the performance indicators include partition size, data writing speed, and query response time; Monitoring the performance data of each of the data partitions based on the performance indicator by using a monitoring tool; The data partition and the row key are dynamically adjusted according to the performance data and the indicator threshold.

7. The method according to claim 6, characterized in that The dynamically adjusting the data partition and the row key according to the performance data and the indicator threshold comprises: Determine whether the partition size is greater than the corresponding index threshold, and if so, obtain a partition splitting instruction, and split the data partition according to the partition splitting instruction; Otherwise, when the data writing speed or the query response time is greater than the corresponding indicator threshold, a row key adjustment instruction is obtained, and the row key is dynamically adjusted according to the row key adjustment instruction.

8. A device for generating row keys for Internet of Vehicles data, characterized in that: include: A row key generation strategy determination module, used to obtain a vehicle networking database and determine a row key generation strategy based on a data query scenario of the vehicle networking database, wherein the row key generation strategy includes components and a sorting method; A row key generation module, used to generate a row key for each row of data in the Internet of Vehicles database according to the generation strategy; The partition and row key adjustment module is used to pre-partition the Internet of Vehicles database to generate data partitions, and continuously monitor the performance of the data partitions to dynamically adjust the data partitions and the row keys.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 7 when executed.

Images