A vehicle data query method, device and equipment
By storing the time information of vehicle data according to a preset bit segment structure and reading the bit segments using a target mask, the problem of cross-time zone query and local time aggregation in vehicle data storage is solved, and efficient data query and statistics are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN STREAMING VIDEO TECH
- Filing Date
- 2026-04-24
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies cannot simultaneously support data aggregation and statistics based on local time and cross-time zone data when storing vehicle data, and they also incur significant processor overhead.
It adopts a time encoding format based on a preset bit segment structure, physically separates and stores date, time period, precise time and time zone information, and reads the corresponding bit segment through the target mask to aggregate data, supporting data query by local time dimension and cross time zone.
It enables efficient local time-dimensional data aggregation and statistics and cross-time zone data querying, reducing processor overhead and improving query performance.
Smart Images
Figure CN122432224A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vehicle data storage and processing technology, and in particular to a vehicle data query method, apparatus and equipment. Background Technology
[0002] Vehicle data storage requires continuous collection and reporting of vehicle location and operational status data via onboard terminals. Industrial vehicles (such as forklifts, logistics vehicles, and construction machinery) are widely distributed across different countries and regions globally, necessitating cloud platforms for data storage, statistical analysis, and querying. However, due to the vehicles' distribution across different time zones, data time processing is crucial. There are two approaches to data time processing: one is to convert all data time to UTC (Universal Time Coordinated) before storage. While this facilitates global sorting, it loses local time context information, making data queries based on local time ranges impossible. The other approach is to directly store the local string time. This method is convenient for querying local time but cannot handle time zone changes, making cross-time zone data queries difficult; furthermore, it consumes significant storage space. Summary of the Invention
[0003] In view of this, the purpose of this invention is to provide a vehicle data query method, apparatus, device, and storage medium that can simultaneously support data aggregation and statistics by local time dimension / cross-time zone, while reducing processor overhead. The specific solution is as follows: Firstly, this application discloses a method for querying vehicle data, including: Target sub-information is extracted based on the time information of vehicle data; the target sub-information includes date sub-information, time period sub-information, precise time sub-information, and time zone sub-information. The encoding of each target sub-information is stored in the corresponding bit segment according to the preset bit segment structure to obtain the encoded data of the time information; The target mask is determined based on the time dimension of the data query request, and the target bit segment in the encoded data is read using the target mask so as to perform data aggregation based on the target bit segment and the data query request.
[0004] Optionally, the preset bit structure includes, in sequence, a first bit segment for storing date sub-information, a second bit segment for storing time period sub-information, a third bit segment for storing precise time sub-information, and a fourth bit segment for storing time zone sub-information; wherein, the bit width of each bit segment is fixed.
[0005] Optionally, the time zone sub-information is the time zone offset between the time information and the reference time zone; Before reading the target bit segment in the encoded data using the target mask, the method further includes: Determine the encoding of the target time zone corresponding to the data query request; the encoding of the target time zone is the encoding of the time zone offset between the target time zone and the reference time zone; The time zone offset difference is calculated based on the encoding of the time zone sub-information in the encoded data and the encoding of the target time zone; The hour offset is determined based on the time zone offset difference; Modify the bit segment corresponding to the time period sub-information in the encoded data based on the hour offset; Modify the bit segment corresponding to the time zone sub-information in the encoded data to the encoding of the target time zone.
[0006] Optionally, determining the hour offset based on the time zone offset difference includes: The time zone offset difference is broken down into hour offset and minute offset; Determine whether the minute offset is zero. If not, modify the bit segment data corresponding to the precise time sub-information in the encoded data based on the minute offset.
[0007] Optionally, after modifying the bit segment corresponding to the time period sub-information in the encoded data based on the hour offset, the method further includes: Determine whether it spans a day based on the hourly offset; If so, the adjustment strategy is determined based on the value of the hour offset; Modify the bit segment data corresponding to the date sub-information in the encoded data according to the adjustment strategy, and modify the bit segment data corresponding to the time sub-information in the encoded data again.
[0008] Optionally, after obtaining the encoded data of the time information, the method further includes: The encoded data is stored in a columnar storage database in columnar format and physically sorted according to the target composite sort key; The target composite sorting key consists of the bit segment corresponding to the date sub-information, the bit segment corresponding to the time zone sub-information, and the bit segment corresponding to the time period sub-information, in that order.
[0009] Optionally, after performing physical sorting based on the target composite sorting key, the method further includes: Grouped by date and time zone, with encoded data in the same group having the same date and time zone; Based on the time period sub-information in the coded data within the group, a time period bitmap is constructed for the group; the time period bitmap indicates whether coded data exists within each time period.
[0010] Optionally, the data aggregation based on the target bit segment and the data query request includes: Extract the target date, target time zone, and target time period from the data query request; Based on the target date and the target time zone, a first set of encoded data is selected from the columnar storage database; Based on the target time period and the time period bitmap corresponding to the first coded data set, the target coded data is filtered out from the first coded data set; Based on the vehicle data corresponding to the target encoded data, the query results of the data query request are generated.
[0011] Secondly, this application discloses a vehicle data query device, comprising: The sub-information acquisition module is used to extract target sub-information based on the time information of vehicle data; the target sub-information includes date sub-information, time period sub-information, precise time sub-information, and time zone sub-information. The encoding and storage module is used to store the encoding of each target sub-information into the corresponding bit segment according to the preset bit segment structure, so as to obtain the encoded data of the time information; The query aggregation module is used to determine the target mask based on the time dimension of the data query request, and use the target mask to read the target bit segment in the encoded data so as to perform data aggregation based on the target bit segment and the data query request.
[0012] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned vehicle data query method.
[0013] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned vehicle data query method.
[0014] In this application, target sub-information is extracted based on the time information of vehicle data. This target sub-information includes date sub-information, time period sub-information, precise time sub-information, and time zone sub-information. The encoding of each target sub-information is stored in its corresponding bit segment according to a preset bit segment structure to obtain the encoded data of the time information. A target mask is determined based on the time dimension of the data query request, and the target bit segment in the encoded data is read using the target mask to perform data aggregation based on the target bit segment and the data query request. It is evident that by placing the date, time period, precise time, and time zone sub-information in fixed bit segments according to a preset bit segment structure, a corresponding target mask is generated based on the required query dimension during actual data querying. Using the target mask, only the data in the required bit segment is read, avoiding calls to the date parsing function and reducing processor overhead. Furthermore, by simultaneously storing the time period, precise time, and time zone, it supports both fast aggregation and statistics based on the local time dimension and cross-time zone data aggregation and statistics after time zone conversion, improving the query performance of vehicle data. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0016] Figure 1 A flowchart of a vehicle data query method provided in this application; Figure 2 A flowchart illustrating a specific time zone conversion method provided in this application; Figure 3 A flowchart illustrating a specific vehicle data query method provided in this application; Figure 4 This application provides a schematic diagram of the structure of a vehicle data query device; Figure 5 This application provides a structural diagram of an electronic device. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] In related technologies, storing data using a unified UTC format cannot directly answer business questions such as "vehicle operation status from 9:00 AM to 11:00 AM local time." Conversely, directly storing local string times makes it difficult to query data across time zones.
[0019] This application discloses a vehicle data query method. See [link to relevant documentation] Figure 1 As shown, the method may include the following steps: Step S11: Extract target sub-information based on the time information of vehicle data; the target sub-information includes date sub-information, time period sub-information, precise time sub-information, and time zone sub-information.
[0020] First, obtain the target sub-information included in the time information corresponding to the vehicle data. This target sub-information includes, but is not limited to, date sub-information, time period sub-information, precise time sub-information, and time zone sub-information. Specifically, the date sub-information includes the specific date, the time period sub-information is the hour information, the precise time sub-information includes minutes, seconds, and milliseconds, and the time zone sub-information is the time zone offset between the vehicle data's location and the standard time zone.
[0021] Step S12: Store the encoding of each target sub-information into the corresponding bit segment according to the preset bit segment structure to obtain the encoded data of the time information.
[0022] After acquiring the target sub-information, the target sub-information is encoded and stored in the corresponding bit segments according to a preset bit segment structure to obtain the final encoded time information data. The preset bit segment structure includes the order of the bit segments, i.e., the storage order of the sub-information, and the size of each bit segment; a bit segment refers to a data structure that specifies the length of a member in units of bits.
[0023] The aforementioned Hierarchical Time Code (HTC) format decomposes time information into four independent layers: date, time period, precision, and time zone. Compared to traditional Unix timestamps or ISO 8601 strings, this embodiment physically separates and stores different granularities of time information. During querying, bitmasking operations can be used to directly extract the corresponding layer's bit segments, enabling statistical operations at specific granularities without parsing the complete time. Furthermore, time zone information is embedded as an independent layer in the encoding, allowing each data entry to self-interpret its local context without recalculating the entire timestamp.
[0024] In a preferred embodiment, the encoded time information is stored using a compact 64-bit (8-byte) binary encoding, meaning the bit width of the encoded data is 64 bits. This utilizes a 64-bit four-level time encoding format to physically separate and store date, time period, precise time, and time zone information, ensuring full compatibility with standard 64-bit integer types and allowing direct utilization of existing database and programming language optimizations. While reducing storage space, zero-parse aggregation can be achieved using bitmasks, resolving the CPU bottleneck during large-scale data statistics.
[0025] In a preferred embodiment, the preset bit structure sequentially includes a first bit for storing date sub-information, a second bit for storing time period sub-information, a third bit for storing precise time sub-information, and a fourth bit for storing time zone sub-information; wherein the bit width of each bit is fixed. By placing the date information in the high bits, the overall monotonicity of the encoding can be guaranteed, which is beneficial for database indexing and sorting. Furthermore, storing the time period information and precise time information independently facilitates time-segmented statistics; for example, when statistics are only performed in hours, it is not necessary to read the precise time information.
[0026] A specific preset bit structure is: [L1:24bit][L2:8bit][L3:24bit][TZ:8bit]. The first bit (L1) serves as the date layer, storing date sub-information, with a bit width of 24 bits. The second bit (L2) serves as the time period layer, storing time period sub-information, i.e., specific hour information, with a bit width of 8 bits. The third bit (L3) serves as the precision layer, storing precise time sub-information, i.e., minutes, seconds, and milliseconds, with a bit width of 24 bits. The fourth bit (TZ) serves as the time zone layer, with a bit width of 8 bits, used to store time zone offset information. By embedding time zone information as an independent layer in the encoding, each time information record can self-interpret its local time context. The system can directly answer business questions such as "data for the vehicle's location at 9 AM," significantly shortening the response time for local time dimension queries in typical multi-time zone operational analysis tasks.
[0027] In some specific embodiments, the encoding of the above-mentioned types of sub-information can be carried out in the following manner. Of course, this embodiment does not limit the specific encoding method. The date sub-information has a total of 24 bits, and the encoding method is as follows: Where Y represents the year, with a value range of [2000, 6095], occupying 12 bits; M represents the month, with a value range of [1, 12], occupying 4 bits; and D represents the day, with a value range of [1, 31], occupying 5 bits. A total of 12 + 4 + 5 = 21 bits are used, with the remaining 3 bits reserved for expansion. During decoding, the bit segment is located based on its position, and the corresponding information is decoded through the inverse operation of the encoding.
[0028] The time period sub-information has a total of 8 bits, which are used to store the hour value. The encoding method can be: L2=H; where H is the hour, and the value range is [0,23]. In practice, only 5 bits are used, and the remaining 3 bits are reserved. The hour layer is separated to optimize the most common time period statistical query. Decoding is performed by the inverse operation of encoding to decode the corresponding information.
[0029] The precise time information segment consists of 24 bits, storing minutes, seconds, and milliseconds. The encoding method can be: Where Min represents minutes, with a value range of [0, 59], occupying 6 bits; Sec represents seconds, with a value range of [0, 59], occupying 6 bits; Ms represents milliseconds, with a value range of [0, 999], occupying 10 bits; a total of 6 + 6 + 10 = 22 bits are used, with the remaining 2 bits reserved for expansion; decoding is performed by inverse operation of encoding to decode the corresponding information.
[0030] The time zone sub-information consists of 8 bits, and the time zone offset is encoded using signed integers. The encoding method can be as follows: ;in, This represents the time zone offset relative to UTC, in 15-minute increments, with a value range of [-48, +56] (corresponding to UTC-12:00 to UTC+14:00). Adding 128 converts it to an unsigned number for storage, with a value range of [80, 184]. Adding time zone sub-information allows the local attribution of data to be determined without needing to join external tables. Decoding retrieves the corresponding information through the inverse operation of encoding. Additionally, it should be noted that data storage may involve batch storage of data from multiple vehicles; in such cases, batch encoding of multiple time information entries can be used to improve throughput.
[0031] Step S13: Determine the target mask based on the time dimension of the data query request, and use the target mask to read the target bit segment in the encoded data so as to perform data aggregation based on the target bit segment and the data query request.
[0032] During data aggregation and statistics, each statistical analysis does not require reading all time information. Only a corresponding mask needs to be generated based on the time dimension of the data query request. The time dimension can be date, time period, or precise time. The target bit segment in the encoded data (htc_code) is read using the target mask; this can be any single bit segment or multiple bit segments. It's understandable that traditional databases, when performing data aggregation (GROUP BY), need to execute parsing function calls on each row of data. This is because traditional databases record time using timestamps, which don't provide a clear view of date and time. For example, a traditional Unix timestamp is essentially a cumulative count of seconds / milliseconds from a certain starting point. Therefore, when aggregation is needed in days or hours, the database must convert this count value into a calendar date using a function call, resulting in significant computational overhead. In large-scale statistical analysis, each data point requires time zone conversion calculations. Taking tens of millions of daily active vehicles as an example, daily statistics require parsing and time zone conversion on billions of records one by one, resulting in enormous computational overhead. However, the hierarchical encoding structure of this application supports direct manipulation of binary data through bit masks, achieving zero-parse aggregation.
[0033] Specifically, target masks corresponding to different time dimensions can be pre-constructed based on the data structure of the encoded data of time information. After determining the time dimension, the target mask that fits the current time dimension is selected from all pre-constructed target masks, and then the bit segment is read using this target mask. Of course, after determining the time dimension, the corresponding target mask can also be generated by combining it with the data structure of the encoded data. For example, the time dimension for daily statistical queries is the date, that is, only the first segment of the encoded data needs to be read. Therefore, a date mask Mask_day=0xFFFFFF0000000000 is pre-constructed; the high 24 bits of this mask are all 1s, and the rest are all 0s; then the aggregation operation is converted to: GROUP BY(htc_code&Mask_day); & represents the AND operation. Since the bitwise AND operation is a CPU instruction-level operation and does not require memory object conversion, its speed is 1-2 orders of magnitude faster than function calls.
[0034] If hourly statistics are required, the first and second segments need to be read to construct a time period mask: Mask_hour = 0xFFFFFFFF00000000. The high 32 bits (24+8) of this mask are all 1s, and the rest are all 0s. The aggregation operation is then converted to: GROUP BY(htc_code&Mask_hour). In this way, daily statistics only require reading and comparing the high 24 bits of data, and hourly statistics only require reading and comparing the high 32 bits of data, significantly reducing the computational load compared to the complete parsing of traditional 64-bit timestamps.
[0035] In some embodiments, before reading the target bit segment in the encoded data using the target mask, the method further includes: modifying the relevant bit segment in the encoded data according to the encoding of the time zone sub-information in the encoded data and the target time zone corresponding to the data query request. That is, the data query request may be in a certain time zone (such as East Eight Zone) or in Universal Time (UTC), while the time information is usually stored according to the local time of the vehicle data. Therefore, time zone unification needs to be performed before querying, that is, the time zone of the encoded data is converted to the time zone in the data query request.
[0036] As can be seen from the above, this embodiment extracts target sub-information based on the time information of vehicle data. The target sub-information includes date sub-information, time period sub-information, precise time sub-information, and time zone sub-information. The encoding of each target sub-information is stored in its corresponding bit segment according to a preset bit segment structure to obtain the encoded data of the time information. A target mask is determined based on the time dimension of the data query request, and the target bit segment in the encoded data is read using the target mask to perform data aggregation based on the target bit segment and the data query request. It is evident that by placing the date, time period, precise time, and time zone sub-information in fixed bit segments according to a preset bit segment structure, a corresponding target mask is generated based on the required query dimension during actual data querying. The target mask is then used to read only the data in the required bit segments, avoiding calls to the date parsing function and reducing processor overhead. Furthermore, by simultaneously storing the time period, precise time, and time zone, it supports both fast aggregation and statistics based on the local time dimension and cross-time zone data aggregation and statistics after time zone conversion, improving the query performance of vehicle data.
[0037] Based on the above embodiments, for example Figure 2 As shown, the modification of relevant bit segments in the encoded data based on the encoding of time zone sub-information in the encoded data and the target time zone corresponding to the data query request can specifically include the following steps: Step S21: Determine the encoding of the target time zone corresponding to the data query request; the encoding of the target time zone is the encoding of the time zone offset between the target time zone and the reference time zone.
[0038] First, determine the encoding of the target time zone corresponding to the data query request. The target time zone is the time zone to which the data needs to be converted. The encoding of the target time zone is the encoding of the time zone offset between the target time zone and the reference time zone.
[0039] Step S22: Calculate the time zone offset difference based on the encoding of the time zone sub-information in the encoded data and the encoding of the target time zone; the time zone sub-information is the time zone offset between the time information and the reference time zone.
[0040] The bit segments corresponding to the time zone sub-information in the encoded data store the encoding of the time zone offset between the vehicle data's time zone and the reference time zone. The target time zone is encoded as the time zone offset between the target time zone and the reference time zone; both are referenced to the reference time zone, which can be UTC or a fixed time zone. Based on the encoding of the time zone sub-information and the target time zone in the encoded data, the time zone offset difference between the two time zones can be calculated.
[0041] Step S23: Determine the hour offset based on the time zone offset difference.
[0042] Step S24: Modify the bit segment corresponding to the time period sub-information in the encoded data based on the hour offset.
[0043] Since the time zone offset difference between most time zones is an integer number of hours, in most cases only the data of the corresponding bit segment of the time period sub-information needs to be changed. Therefore, the hour offset is determined based on the time zone offset difference, and then the bit segment corresponding to the time period sub-information in the encoded data is modified according to the hour offset.
[0044] Step S25: Modify the bit segment corresponding to the time zone sub-information in the encoded data to the encoding of the target time zone.
[0045] After the time zone offset is modified, the modified time information is already in the target time zone. Therefore, it is necessary to simultaneously modify the bit segments corresponding to the time zone sub-information in the encoded data to the target time zone encoding. It is understandable that traditional time zone conversion requires converting the timestamp to the target time zone and recalculating each time component, involving complex calendar calculations (such as determining leap years and months with long or short periods). This embodiment, through the aforementioned hierarchical time zone conversion, utilizes the hierarchical structure characteristics of HTC encoding and the independence of bit segments in the time period sub-information, modifying only the affected layers, significantly reducing computational overhead.
[0046] In some embodiments, determining the hour offset based on the time zone offset difference includes: splitting the time zone offset difference into hour offset and minute offset; determining whether the minute offset is zero; if it is not zero, modifying the bit segment data corresponding to the precise time sub-information in the encoded data based on the minute offset. Specifically, the time zone offset is split into hour offset and minute offset. If the minute offset is not zero, it indicates that a minute modification is needed, meaning the time difference between the two time zones is not an integer number of hours, and in this case, the minutes in the precise time sub-information need to be modified.
[0047] Furthermore, in some embodiments, after modifying the bit segment corresponding to the time period sub-information in the encoded data based on the hour offset, the method further includes: determining whether it crosses days based on the hour offset; if so, determining an adjustment strategy based on the value of the hour offset; modifying the bit segment data corresponding to the date sub-information in the encoded data according to the adjustment strategy, and modifying the bit segment data corresponding to the time period sub-information in the encoded data again. It can be understood that in some cases, time zone conversion involves crossing days, requiring adjustment of the date encoding in the encoded data. Therefore, in typical conversion scenarios with hourly offsets and no day crossings, only the L2 and TZ layers need to be modified to complete the time zone conversion, involving only integer addition and subtraction; for a small number of scenarios involving day crossings or half-hour time differences, only one date addition / subtraction and minute compensation are needed, still avoiding a complete calendar expansion for each record.
[0048] A specific time zone conversion method includes the following process: The encoded data of time information is These are the codes for the date sub-information, time period sub-information, precise time sub-information, and time zone sub-information, respectively; the code for the target time zone is... Then the time zone offset difference is: ;in, This represents the time zone offset difference, expressed in minutes, with a range of values. ; The target time zone encoded value, with a value range of... ; The source time zone encoded value, with a range of values. The time zone offset difference is broken down into hourly and minute increments: ; in, The offset is in integer hours, in units of hours. The remaining minute offset, with a value range of... The unit is minutes.
[0049] First, adjust the time period by hour: ;in, The target time zone hour value is calculated initially (may exceed). (range), in hours; The hour value is the source time zone, and the range of values is... ;when Before crossing the day (i.e.) )and At that time, the conversion result is: At this point, only modifications are needed. and Two levels, and Keeping everything unchanged, the conversion operation has a constant time complexity. It only involves integer addition and subtraction, and is typically represented by integer offset scenarios such as UTC±n.
[0050] when At that time, The layer performs an addition or subtraction compensation on the minute component, and adjusts the calculation based on whether the minute is carried over or borrowed to the hour. Do it again The corrections allow for precise support of time zones with offsets of half an hour or even 15 minutes, without requiring a full calendar calculation.
[0051] when Crossing the day (i.e.) or (Date layer needs adjustment) .like This indicates reverting to the previous day, obtaining the new date through a standard "previous day" calendar addition or subtraction operation. and order ;like This indicates that the next day has begun, and a new date is obtained through a calendar addition or subtraction operation for that "next day". and order .
[0052] In typical cross-time zone operation scenarios, most conversions are both hourly offsets and do not span days. In this case, only constant-time integer operations need to be performed on the L2 and TZ layers. Only a few scenarios require an additional date addition / subtraction and minute compensation. The overall computational load is still much less than recalculating the complete timestamp.
[0053] Through the above operations, cross-time zone vehicle data can support both efficient hierarchical sorting and precise querying, as well as rapid aggregation and statistics based on the local time dimension. At the same time, queries of different granularities can directly extract data at the corresponding level through bitmask operations, and statistical operations can be completed without parsing the complete time, thereby achieving on-demand reading and zero-parsing aggregation; avoiding storage redundancy and large-scale time zone conversion calculations at runtime.
[0054] Based on the above embodiments, after obtaining the encoded data of the time information, the method further includes: storing the encoded data in a columnar format to a columnar storage database, and physically sorting it according to a target composite sorting key; the target composite sorting key is, in sequence, the bit segment corresponding to the date sub-information, the bit segment corresponding to the time zone sub-information, and the bit segment corresponding to the time period sub-information. For example... Figure 3 As shown, after the time information is encoded to obtain encoded data, the encoded data will be stored in a columnar format on the cloud platform.
[0055] The upcoming code will be organized in columnar format, using a composite sort key (L1, TZ, L2) to achieve physical clustering. This means that data on the physical disk is first clustered by date, then by time zone within the same day, and finally by hour within the same time zone. After physical clustering by date / time zone / hour, data from the same day and time zone will be grouped together as closely as possible. Furthermore, each block will record the value range or summary information of certain key columns within that block, such as the date range and time zone range. When a query explicitly specifies a particular day and time zone, the engine first examines these block-level summaries to determine which blocks are unlikely to meet the conditions. Therefore, the query can target block-level pruning, skipping the reading of large amounts of irrelevant data blocks.
[0056] For example, when executing the query "Statistics on the operation data of all vehicles in the East 8 time zone (UTC+8) on January 1, 2024", the database engine can directly use the index to locate L1="Date"(2024-01-01) and TZ=160 (i.e., according to the time zone encoding rule UTC+8). The system can perform consecutive large I / O reads, completely skipping data blocks from other time zones and dates, as these data are physically stored contiguously. This mechanism significantly reduces disk I / O pressure.
[0057] In a preferred embodiment, after physically sorting according to the target composite sorting key, the method further includes: grouping by date and time zone, where the encoded data within the same group has the same date and time zone; constructing a time period bitmap for each group based on the time period sub-information in the encoded data within the group; the time period bitmap indicates whether encoded data exists within each time period. Specifically, the time period bitmap maintains a 24-bit bitmap for each "date + time zone" combination, marking which hours have data, with bit[i]=1 indicating data exists in the i-th hour; when querying a specific hour range, the bitmap is used to quickly filter out time periods without data, reducing I / O operations. By overlaying the hour bitmap on top of the above composite sorting for finer filtering, data reading can be further reduced.
[0058] Accordingly, the data aggregation based on the target bit segment and the data query request includes: extracting the target date, target time zone, and target time period from the data query request; filtering a first coded data set from the columnar storage database according to the target date and target time zone; filtering target coded data from the first coded data set according to the target time period and the time period bitmap corresponding to the first coded data set; and generating the query result of the data query request based on the vehicle data corresponding to the target coded data.
[0059] For example Figure 3As shown, after determining the target mask, the encoded data stored on the cloud platform is combined with data aggregation through date indexing and time-period bitmaps. Specifically, the date layer serves as the primary index, supporting fast date range queries; the time-period bitmap serves as the secondary index, supporting hourly query optimization. Physical clustered storage based on the encoded structure ensures that queries within specific time zones and time periods can accurately hit data blocks, significantly reducing the amount of invalid data scanned. The time-period bitmap index can further quickly filter out periods without data, reducing disk scans in multi-day statistical scenarios; and it can significantly reduce invalid data scans and disk I / O overhead when querying across time zones and multiple time periods.
[0060] Accordingly, this application also discloses a vehicle data query device, see [link to relevant documentation]. Figure 4 As shown, the device includes: The sub-information acquisition module 11 is used to extract target sub-information based on the time information of vehicle data; the target sub-information includes date sub-information, time period sub-information, precise time sub-information, and time zone sub-information. Encoding and storage module 12 is used to encode and store the target sub-information into corresponding bit segments according to a preset bit segment structure to obtain the encoded data of the time information; The query aggregation module 13 is used to determine the target mask according to the time dimension of the data query request, and use the target mask to read the target bit segment in the encoded data so as to perform data aggregation based on the target bit segment and the data query request.
[0061] In some specific embodiments, the preset bit segment structure includes, in sequence, a first bit segment for storing date sub-information, a second bit segment for storing time period sub-information, a third bit segment for storing precise time sub-information, and a fourth bit segment for storing time zone sub-information; wherein, the bit width of each bit segment is fixed.
[0062] In some specific embodiments, the time zone sub-information is the time zone offset between the time information and the reference time zone; The vehicle data query device may further include a time zone conversion module, which is used to modify the relevant bit segments in the encoded data according to the encoding of the time zone sub-information in the encoded data and the target time zone corresponding to the data query request before reading the target bit segment in the encoded data using the target mask.
[0063] In some specific embodiments, the time zone conversion module may specifically include: The encoding determination unit is used to determine the encoding of the target time zone corresponding to the data query request; the encoding of the target time zone is the encoding of the time zone offset between the target time zone and the reference time zone; The time zone offset difference determination unit is used to calculate the time zone offset difference based on the encoding of the time zone sub-information in the encoded data and the encoding of the target time zone; The hour offset determination unit is used to determine the hour offset based on the time zone offset difference; The first modification unit is used to modify the bit segment corresponding to the time period sub-information in the encoded data based on the hour offset; The second modification unit is used to modify the bit segment corresponding to the time zone sub-information in the encoded data to the encoding of the target time zone.
[0064] In some specific embodiments, the hour offset determination unit may specifically include: A splitting unit is used to split the time zone offset difference into hour offset and minute offset; The third modification unit is used to determine whether the minute offset is zero. If not, it modifies the bit segment data corresponding to the precise time minute information in the encoded data based on the minute offset.
[0065] In some specific embodiments, the time zone conversion module may specifically include: The cross-day determination unit is used to determine whether a cross-day occurs based on the hour offset after modifying the bit segment corresponding to the time period sub-information in the encoded data based on the hour offset. An adjustment strategy determination unit is used to determine an adjustment strategy based on the value of the hourly offset if the day spans multiple days. The fourth modification unit is used to modify the bit segment data corresponding to the date sub-information in the encoded data according to the adjustment strategy, and to modify the bit segment data corresponding to the time sub-information in the encoded data again.
[0066] In some specific embodiments, the vehicle data query device may specifically include: The storage unit is used to store the encoded data of the time information in columnar format to the columnar storage database after obtaining the encoded data of the time information, and to perform physical sorting according to the target composite sorting key; The target composite sorting key consists of the bit segment corresponding to the date sub-information, the bit segment corresponding to the time zone sub-information, and the bit segment corresponding to the time period sub-information, in that order.
[0067] In some specific embodiments, the vehicle data query device may specifically include: Grouping unit, used to group data by date and time zone after physical sorting based on target composite sorting key, with encoded data in the same group having the same date and time zone; The time period bitmap construction unit is used to construct a time period bitmap for a group based on the time period sub-information in the coded data within the group; the time period bitmap indicates whether coded data exists in each time period.
[0068] In some specific embodiments, the query aggregation module 13 may specifically include: The extraction unit is used to extract the target date, target time zone, and target time period from the data query request; The first filtering unit is used to filter out a first set of coded data from the columnar storage database based on the target date and the target time zone; The second filtering unit is used to filter out target encoded data from the first encoded data set based on the target time period and the time period bitmap corresponding to the first encoded data set. The query result generation unit is used to generate the query result of the data query request based on the vehicle data corresponding to the target encoded data.
[0069] Furthermore, this application also discloses an electronic device, see [link to relevant documentation]. Figure 5 As shown, the content in the figure should not be considered as any limitation on the scope of use of this application.
[0070] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the vehicle data query method disclosed in any of the foregoing embodiments.
[0071] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0072] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include operating system 221, computer program 222 and data 223 including coded data, etc., and the storage method can be temporary storage or permanent storage.
[0073] The operating system 221 manages and controls the various hardware devices on the electronic device 20 and the computer program 222 to enable the processor 21 to perform calculations and processing on the massive data 223 in the memory 22. The operating system 221 can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the vehicle data query method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.
[0074] Furthermore, this application also discloses a computer storage medium storing computer-executable instructions. When the computer-executable instructions are loaded and executed by a processor, they implement the vehicle data query method steps disclosed in any of the foregoing embodiments.
[0075] Furthermore, this application also discloses a computer program product, including a computer program that, when executed by a processor, implements the vehicle data query method steps disclosed in any of the foregoing embodiments.
[0076] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0077] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0078] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0079] The present invention provides a detailed description of a vehicle data query method, apparatus, device, and storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for querying vehicle data, characterized in that, include: Target sub-information is extracted based on the time information of vehicle data; the target sub-information includes date sub-information, time period sub-information, precise time sub-information, and time zone sub-information. The encoding of each target sub-information is stored in the corresponding bit segment according to the preset bit segment structure to obtain the encoded data of the time information; The target mask is determined based on the time dimension of the data query request, and the target bit segment in the encoded data is read using the target mask so as to perform data aggregation based on the target bit segment and the data query request.
2. The vehicle data query method according to claim 1, characterized in that, The preset segment structure includes, in sequence, a first segment for storing date sub-information, a second segment for storing time period sub-information, a third segment for storing precise time sub-information, and a fourth segment for storing time zone sub-information; wherein, the bit width of each segment is fixed.
3. The vehicle data query method according to claim 1, characterized in that, The time zone sub-information is the time zone offset between the time information and the reference time zone; Before reading the target bit segment in the encoded data using the target mask, the method further includes: Determine the encoding of the target time zone corresponding to the data query request; the encoding of the target time zone is the encoding of the time zone offset between the target time zone and the reference time zone; The time zone offset difference is calculated based on the encoding of the time zone sub-information in the encoded data and the encoding of the target time zone; The hour offset is determined based on the time zone offset difference; Modify the bit segment corresponding to the time period sub-information in the encoded data based on the hour offset; Modify the bit segment corresponding to the time zone sub-information in the encoded data to the encoding of the target time zone.
4. The vehicle data query method according to claim 3, characterized in that, The step of determining the hour offset based on the time zone offset difference includes: The time zone offset difference is broken down into hour offset and minute offset; Determine whether the minute offset is zero. If not, modify the bit segment data corresponding to the precise time sub-information in the encoded data based on the minute offset.
5. The vehicle data query method according to claim 3, characterized in that, After modifying the bit segment corresponding to the time period sub-information in the encoded data based on the hour offset, the method further includes: Determine whether it spans a day based on the hourly offset; If so, the adjustment strategy is determined based on the value of the hour offset; Modify the bit segment data corresponding to the date sub-information in the encoded data according to the adjustment strategy, and modify the bit segment data corresponding to the time sub-information in the encoded data again.
6. The vehicle data query method according to claim 1, characterized in that, After obtaining the encoded data of the time information, the method further includes: The encoded data is stored in a columnar storage database in columnar format and physically sorted according to the target composite sort key; The target composite sorting key consists of the bit segment corresponding to the date sub-information, the bit segment corresponding to the time zone sub-information, and the bit segment corresponding to the time period sub-information, in that order.
7. The vehicle data query method according to claim 6, characterized in that, After performing physical sorting based on the target composite sorting key, the process also includes: Grouped by date and time zone, with encoded data in the same group having the same date and time zone; Based on the time period sub-information in the coded data within the group, a time period bitmap is constructed for the group; the time period bitmap indicates whether coded data exists within each time period.
8. The vehicle data query method according to claim 7, characterized in that, The data aggregation based on the target bit segment and the data query request includes: Extract the target date, target time zone, and target time period from the data query request; Based on the target date and the target time zone, a first set of encoded data is selected from the columnar storage database; Based on the target time period and the time period bitmap corresponding to the first coded data set, the target coded data is filtered out from the first coded data set; Based on the vehicle data corresponding to the target encoded data, the query results of the data query request are generated.
9. A vehicle data query device, characterized in that, include: The sub-information acquisition module is used to extract target sub-information based on the time information of vehicle data; the target sub-information includes date sub-information, time period sub-information, precise time sub-information, and time zone sub-information. The encoding and storage module is used to store the encoding of each target sub-information into the corresponding bit segment according to the preset bit segment structure, so as to obtain the encoded data of the time information; The query aggregation module is used to determine the target mask based on the time dimension of the data query request, and use the target mask to read the target bit segment in the encoded data so as to perform data aggregation based on the target bit segment and the data query request.
10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the vehicle data query method as described in any one of claims 1 to 8.