A method for designing and querying a TD-based inland AIS data storage model

Abstract

The application relates to a TD-based inland river AIS data storage model design and query method, and belongs to the technical field of ship space-time trajectory data processing, and comprises the following steps: a sub-table table building mechanism of a TDengine time series database is used to build an independent MMSI sub-table for each ship stored in a same super-table, and an uploading time is stored as a time stamp; according to a given MMSI-time range query condition, MMSI-time super-tables are sequentially sub-queried, and a comprehensive space-time trajectory data set meeting all query conditions is obtained through combination; a space partition mapping function of an S2 algorithm is used to decompose and map an effective navigation area of a river channel into an approximate space grid area, so as to build a sub-table; according to a given space-time query condition, a space-time super-table is acquired, the space range is decomposed and mapped, and parallel query is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of ship space-time trajectory data processing, and relates to a TD-based inland river AIS data storage model design and query method. BACKGROUND

[0002] An automatic identification system (AIS) is a widely used digital navigation system and equipment. Ship dynamic information such as ship position, speed, change of heading rate and heading, and ship static information such as ship name, call sign, draft, dangerous goods, etc. can be broadcasted and communicated between AIS ship platforms and AIS base stations via very high frequency waves, thereby ensuring maritime traffic safety. At present, hundreds of thousands of ships worldwide are equipped with AIS, and the global monthly AIS data volume can reach several hundred GB. Among them, each AIS dynamic message can be regarded as a space-time trajectory point of a ship, and therefore, the accumulated AIS data contains a large amount of historical ship trajectory data. Through analysis and mining of the historical ship trajectory data, good solutions can be provided for route optimization, abnormal behavior monitoring, false target identification, port throughput statistical analysis, etc.

[0003] An efficient space-time trajectory data storage model is the key to effectively analyzing and mining massive historical ship trajectory data. For AIS data storage, there is currently a NoSQL database solution, which is known for its high flexibility, supports high-performance read and write operations and horizontal expansion, and adapts to large-scale data storage needs. However, NoSQL databases have some shortcomings in terms of accuracy and analysis capabilities when dealing with time series data. In particular, for inland waterways with obvious strip characteristics that are different from the obvious planar distribution of marine AIS, the distribution of AIS base stations and AIS data has a strip distribution feature, and the NoSQL data storage mode cannot meet the fast query requirements of massive historical ship trajectory data.

[0004] The AIS data collection network of the inland waterway can essentially be regarded as an Internet of Things, in which the shipborne AIS terminal acts as a sensor to continuously send data, the AIS base station deployed along the waterway acts as a data collection terminal, and the data of all terminals is converged to the data center. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a TD-based inland river AIS data storage model design and query method, which utilizes the unique sub-table building mechanism of the TDengine time series database and the time-indexed columnar storage feature to realize the query requirements centered on ships and spatial ranges, while providing efficient data writing and query performance.

[0006] To achieve the above purpose, the present application provides the following technical solutions:

[0007] A TD-based inland AIS data storage model design and query method, comprising the following steps:

[0008] S1: Using the sub-table table building mechanism of TDengine time series database, an independent mobile service identification code (MMSI) sub-table is established for each ship stored in the same super table, and then the upload time is stored as a timestamp, thereby constructing an MMSI-time storage model;

[0009] S2: According to the given MMSI-time range query condition, the MMSI-time super table is queried one by one, and finally the comprehensive space-time trajectory data set satisfying all query conditions is obtained by merging;

[0010] S3: Using the spatial partitioning mapping function of S2 algorithm, the effective navigation area of the river channel is decomposed and mapped into an approximate spatial grid area, thereby establishing a sub-table, and thus constructing a space-time storage model;

[0011] S4: According to the given space-time query condition, the space-time super table is obtained, the spatial range is decomposed and mapped, and parallel query is realized.

[0012] Further, the construction step of the MMSI-time storage model in step S1 is as follows:

[0013] S11: Create a database and configure its virtual group number to x, and set the buffer size to y;

[0014] S12: Design the MMSI-time super table, and set the first field of the MMSI-time super table as the receiving timestamp of the space-time trajectory data point, and set it as the primary key;

[0015] S13: For each ship, name the sub-table according to its MMSI, and let the sub-table inherit the format of the corresponding MMSI-time super table through automatic table building;

[0016] S14: The space-time trajectory data received by each AIS base station is uploaded to the data center, the data received by adjacent AIS base stations is de-duplicated and processed, and then classified according to MMSI, and stored in the MMSI-time super table sub-table respectively.

[0017] Further, step S2 specifically comprises the following steps:

[0018] S21: Given the MMSI-time range query condition, including the query object m and the query time range T rang =[T start ,T end ], wherein T start and T endstart time and end time of the query, respectively;

[0019] S22: Obtain MMSI-time super table;

[0020] S23: Query all query objects m in time period T rang , respectively;

[0021] S24: Merge the results by MMSI to obtain a comprehensive spatio-temporal trajectory data set Tras = {tra1, tra2, …, tra n}.

[0022] Further, the step S3 of constructing the space-time storage model specifically comprises the following steps:

[0023] S31: Create a database and configure the number of virtual groups thereof to be x and set the buffer size to be y;

[0024] S32: Design a space-time super table, set the first field of each super table to be the receiving time stamp of the spatio-temporal trajectory data point, and set it as the primary key;

[0025] S33: Map the input spatio-temporal trajectory data to space grid encoding by using the S2 algorithm;

[0026] S34: For each spatio-temporal trajectory data, automatically classify and store the data into the corresponding named sub-table by using the space grid encoding thereof, and each sub-table will inherit the structure and format of the super table to which it belongs through the automatic table building mechanism.

[0027] Further, the step S4 specifically comprises the following steps:

[0028] S41: Given the space-time query condition, including query space range s and query time range T rang = [T start , T end ], wherein T start and T end are the start time and end time of the query, respectively;

[0029] S42: Obtain the space-time super table;

[0030] S43: Decompose and map the s space range to n space grid encodings cell by using the Polygon decomposition function in the S2 algorithm;

[0031] S44: Perform parallel query on the n cell objects obtained through decomposition.

[0032] The application has the beneficial effects that: the application aims at the problems of backward storage model and low query efficiency existing in the use of the existing ship AIS trajectory data storage device and query method, and according to the characteristics of the ship trajectory space-time data structure, the ordered organization and storage of the ship AIS trajectory data are completed.

[0033] For the ship-centered MMSI-time query, the sub-table building mechanism of the TDengine time series database is used to build an independent MMSI sub-table for each ship stored in the same super table, that is, one ship one sub-table. At the same time, the TDengine time-based column storage feature is used to store the upload time as a timestamp. Such design not only takes advantage of the convenience brought by the super table template, but also ensures the quickness and traceability of data query.

[0034] For each space-time trajectory data, the system will use its spatial grid code to automatically classify and store the data into the corresponding named sub-table. Through the automatic table building mechanism, each sub-table will inherit the structure and format of the super table it belongs to, so as to maintain the consistency and efficiency of data organization. By adopting hierarchical index and setting query threshold to adjust the query precision, the spatial partition is optimized, and the overall query performance is improved.

[0035] Other advantages, objects and features of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the methods and instrumentalities particularly pointed out in the description. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to make the purpose, technical scheme and advantages of the application clearer, the preferred detailed description of the application will be combined with the drawings as follows, in which:

[0037] Figure 1 MMSI-time query method flow chart;

[0038] Figure 2 Effective navigation area space partition schematic diagram;

[0039] Figure 3 Space-time query method flow chart. DETAILED DESCRIPTION

[0040] Following, the advantages and effects of the present application will be easily understood by those skilled in the art from the present description. The present application can also be implemented or applied by other different specific embodiments, and the details in the present description can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the drawings provided in the following embodiments only illustrate the basic concepts of the present application in a schematic manner, and the drawings only show the components related to the present application without drawing the number, shape and size of the components in actual implementation. The shapes, numbers and proportions of the components in actual implementation can be changed arbitrarily, and the component layout pattern can be more complex.

[0041] It should be noted that the drawings provided in the following embodiments only illustrate the basic concepts of the present application in a schematic manner, and the drawings only show the components related to the present application without drawing the number, shape and size of the components in actual implementation. The shapes, numbers and proportions of the components in actual implementation can be changed arbitrarily, and the component layout pattern can be more complex.

[0042] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application, however, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details, and in other embodiments, the known structures and devices are shown in the form of block diagrams rather than in the form of details to avoid making the embodiments of the present application difficult to understand.

[0043] The present application provides a TD-based inland AIS data storage model design and query method, including a water mobile service identification code MMSI-time storage model and query method and a space-time storage model and query method.

[0044] In the inland waterway AIS data query, the space-time trajectory query centered on the ship is a common requirement. Such queries can be used for navigation history tracing, navigation behavior analysis, and logistics tracking and optimization, etc. The basis for the query is the MMSI of the ship and a certain time range, and the goal is to extract the space-time trajectory data of the ship within a certain time range.

[0045] AIS receiving devices are deployed along the inland waterway, and the receiving ranges of adjacent AIS devices overlap to ensure full coverage of the waterway AIS. Each device decodes the AIS message immediately after receiving it and publishes it through the MQTT protocol according to the dynamic information topic, static information topic, and heartbeat information topic. The data center subscribes to each topic and uses the TDengine time series database to establish an AIS receiving device super table and an MMSI super table to store AIS information. For each AIS receiving device stored in the AIS receiving device super table, an independent AIS device sub-table is established, i.e., "one device one sub-table," and each AIS receiving device is also assigned receiving range information, such as latitude and longitude or waterway mileage line range. For each ship stored in the ship super table, an independent MMSI sub-table is established, i.e., "one ship one sub-table." The TDengine time series database is developed by Tidewise Technology Co., Ltd. and is an open-source time series big data processing engine for the rapidly growing Internet of Things big data market.

[0046] The MMSI-time query can be described as:

[0047] In a given spatio-temporal trajectory dataset F, for a specific ship query object m and a determined time range T rang = [T start , T end ], where T start and T end are the start time and end time of the query, respectively. The goal of this query is to find all the spatio-temporal trajectory data segments Tras = {tra1, tra2,..., tra n} of the query object m within this time period.

[0048] Ship-centered queries aim to reflect the movement trajectory of each ship individually. Given the close relationship between inland AIS spatio-temporal trajectory data and ship objects, the TDengine time series database's sub-table creation mechanism is used to establish an independent MMSI sub-table for each ship stored in the same super table, i.e., "one ship one sub-table." Additionally, the TDengine's time-indexed columnar storage feature is used to store the upload time as a timestamp. This design leverages the convenience of super table templates while ensuring fast and traceable data queries. The specific design steps are as follows:

[0049] ① Create a database named MMSI_TS and configure its virtual group (vgroup) number to 12 and set the buffer size to 10MB. This design considers data sharding and memory management to support concurrent data writing and optimize query performance;

[0050] ② Design MMSI-time super table, create super table named MS. The first field of the super table is set to the time stamp of the space-time trajectory data point, and it is set as the primary key. This step ensures that the data can be effectively sorted and queried according to the time sequence. Other specific information design is shown in Table 1;

[0051] ③ For each ship, the system will name the sub-table with its MMSI, and through automatic table building, let the sub-table inherit the format of the corresponding super table;

[0052] ④ According to the collected space-time trajectory data, the data is classified according to its MMSI, and stored in the super table sub-table respectively. MMSI-time super table is shown in Table 1.

[0053] Table 1

[0054]

[0055] MMSI-time query method is as follows:

[0056] According to the given MMSI-time range query condition (query object m and query time range T rang ), first, get the super table MS. Then, the system will process these sub-queries one by one, and through MMSI, the results are merged to obtain a comprehensive space-time trajectory data set that meets all query conditions. This method not only considers the range and accuracy of the query, but also optimizes the data acquisition process, ensuring query efficiency and accuracy. The execution process of the query is shown in Figure 1 .

[0057] Space-time query can be described as:

[0058] In a given space-time trajectory data set F, facing a specific spatial range s and a determined time range T rang =[T start ,T end ], where T start and T end are the start time and end time of the query respectively. The goal of this query is to find all space-time trajectory data segments Tras={tra1,tra2,…,tra n} within the time period and spatial range s.

[0059] The spatial range in space-time query has a polygon, containing the longitude range and the latitude range, greatly increasing the complexity of the query. Using the spatial partitioning mapping function of the S2 algorithm, the effective navigation area of the river is decomposed and mapped into an approximate spatial grid area, to establish a sub-table, such as Figure 2As shown, these partitions cover the entire control river section in the shape of a spatial grid, with each region representing a different spatial range. The S2 algorithm refers to a geocoding processing technology developed by Google, which is mainly used to optimize the storage and query of large-scale geographic data. Its core is to divide the earth's surface into a series of hierarchical cells through mapping, thereby achieving efficient encoding and fast access of spatial data. Hierarchical indexing and setting query thresholds are used to adjust query precision, thereby optimizing spatial partitioning and improving overall query performance. The specific design steps of the database table are as follows:

[0060] ① Create a database named S2_TS and configure its virtual group (vgroup) number to be 12 and set the buffer size to 10MB.

[0061] ② Design a space-time super table and create a super table named S2_partition. The first field of each super table is set to the receive timestamp of the space-time trajectory data point, and it is set as the primary key. The space-time super table information is shown in Table 2.

[0062] Table 2

[0063]

[0064] ③ Use the S2 algorithm to map the input space-time trajectory data to spatial grid encoding;

[0065] ④ For each space-time trajectory data, use its spatial grid encoding to automatically classify and store the data into the corresponding named sub-table. Through the automatic table building mechanism, each sub-table will inherit the structure and format of its parent super table, thereby maintaining the consistency and efficiency of data organization.

[0066] The space-time query method is as follows:

[0067] Given the space-time query condition (query space range s and query time range Trange). First, get the S2_partition super table, then use the Polygon decomposition function in the S2 algorithm to decompose and map the s space range into spatial grid encoding cells, and perform parallel query on the obtained cell objects, which can greatly reduce the query time. The query process is shown in Figure 3 .

[0068] In the above embodiments, the description of "the present embodiment" in the specification means that the specific features, structures or characteristics described in the embodiment are included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "the present embodiment" do not necessarily all refer to the same embodiment.

[0069] In the foregoing embodiments, although the present application has been described in conjunction with specific embodiments thereof, numerous alternatives, modifications and variations will be readily apparent to those of ordinary skill in the art in the light of the foregoing descriptions. For example, other storage structures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed. The embodiments of the present application are intended to embrace all such alternatives, modifications and variations as falling within the scope of the appended claims.

[0070] The embodiment further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement any one of the methods in the embodiment.

[0071] The embodiment further provides an electronic terminal, which comprises a processor and a memory.

[0072] The memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory, so that the terminal executes any one of the methods in the embodiment.

[0073] The computer readable storage medium in the embodiment can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by a computer program related hardware. The foregoing computer program can be stored in a computer readable storage medium. The program is executed to perform the steps of the above-mentioned method embodiments; and the foregoing storage medium includes ROM, RAM, magnetic disk or optical disk and various media that can store program codes.

[0074] The electronic terminal provided in the embodiment comprises a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected with the processor and the transceiver and complete communication between each other. The memory is configured to store a computer program, the communication interface is configured to communicate, and the processor and the transceiver are configured to run the computer program, so that the electronic terminal executes each step of the method.

[0075] In the embodiment, the memory can include random access memory (RAM), and can also include non-volatile memory, such as at least one disk memory.

[0076] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0077] The present application can be applied to numerous general or special computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, etc.

[0078] The present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media including memory storage devices.

[0079] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should be covered in the scope of the claims of the present application.

Claims

1. A TD-based inland river AIS data storage model design and query method, characterized by: The following steps are involved: S1: Using the sub-table construction mechanism of the TDengine time series database, an independent MMSI sub-table is created for each ship stored in the same super table. The upload time is then stored as a timestamp to build an MMSI-time storage model. S2: Based on the given MMSI-time range query condition, the MMSI-time super table is subqueried one by one, and finally merged to obtain a comprehensive spatiotemporal trajectory dataset that meets all query conditions; S3: Using the spatial partition mapping function of the S2 algorithm, the effective navigation area of ​​the river is decomposed and mapped into approximate spatial grid areas, and sub-tables are established to build a space-time storage model; S4: Based on the given space-time query conditions, obtain the space-time super table, decompose and map the spatial range, and implement parallel query; The steps for constructing the MMSI-time storage model in step S1 are as follows: S11: Create a database, configure the number of virtual groups to be x, and set the buffer size to y; S12: Design an MMSI-time super table, wherein the first field of the MMSI-time super table is set to the receiving timestamp of the spatiotemporal trajectory data point, and set it as the primary key; S13: For each ship, name the subtable according to its MMSI, and automatically create a table so that the subtable inherits the format of the corresponding MMSI-time supertable; S14: The spatiotemporal trajectory data received by each AIS base station is uploaded to the data center. The data center deduplicates the data received by adjacent AIS base stations, classifies them by MMSI, and stores them in the MMSI-time super table subtable. Step S2 specifically includes the following steps: S21: Given MMSI-time range query conditions, including query object m and query time range ,in and are the start time and end time of the query respectively; S22: Obtain MMSI-time super table; S23: Query all the time periods separately The query object m within; S24: Merge the result set by MMSI to obtain a comprehensive spatiotemporal trajectory data set that meets all query conditions ; The step S3 of constructing the space-time storage model specifically includes the following steps: S31: Create a database, configure the number of virtual groups to be x, and set the buffer size to y; S32: Design a space-time hypertable, where the first field of each hypertable is set to the receiving timestamp of the space-time trajectory data point and set it as the primary key; S33: Use the S2 algorithm to map the input spatiotemporal trajectory data into a spatial grid code; S34: For each spatiotemporal trajectory data, its spatial grid encoding is used to automatically classify the data and store it in a corresponding named subtable. Through the automatic table creation mechanism, each subtable will inherit the structure and format of the supertable to which it belongs.

2. The TD-based inland river AIS data storage model design and query method according to claim 1 is characterized by: Step S4 specifically includes the following steps: S41: Given a space-time query condition, including the query space range s and the query time range ,in, and are the start time and end time of the query respectively; S42: Obtaining a space-time super table; S43: Use the Polygon decomposition function in the S2 algorithm to decompose the s spatial range into n spatial grid coding cells; S44: Perform parallel query on the n cell objects obtained by decomposition.

Images