Data dimension table generation method and device, server, and storage medium

Abstract

The application relates to the technical field of big data, in particular to a data dimension table generation method and device, a server and a storage medium, the method comprises the following steps: creating an automatic configuration project, and introducing remote procedure call dependencies, geographic indexes and group transaction monitor dependencies; providing a location-based service (LBS) service interface, receiving a calling request by using the LBS service interface, obtaining consumption data of a distributed publish / subscribe message system Kafka according to the calling request, and parsing the consumption data to obtain positioning data and a unique identifier of a vehicle; querying a topological tree by using the positioning data to obtain location coding data of the vehicle, querying a database by using the unique identifier to obtain vehicle information of the vehicle, generating a data dimension table according to the coding data and the vehicle information, and synchronizing the data dimension table to the database. Therefore, the problems that in the related art, the external access pressure is too large during the data dimension table generation process, the data delay is relatively long, and the scalability is poor are solved.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a method, apparatus, server and storage medium for generating data dimension tables. Background Technology

[0002] In the actual production of connected vehicle big data analysis, it is often necessary to use the raw data stream as a foundation and then associate it with a large number of external base tables to supplement some attributes. In Flink streaming computing, these dimension table data are updated periodically and need to be associated according to business requirements. Based on the timeliness requirements of business for dimension table data association, the following related technologies exist:

[0003] (1) Real-time query of dimension tables: generally leads to thread blocking and Task waiting for data to return, affecting the overall throughput of the task.

[0004] (2) Preloading full data: Since the dimension table data is cached in memory, the memory consumption of the computing node is very high, so it cannot be applied to a large number of dimension tables.

[0005] (3) LRU caching: This method has a certain data delay and requires additional settings for the expiration time of each data. Since hot data is frequently used, it is always in the cache. Therefore, once the dimension table changes, the data change cannot be detected.

[0006] (4) A cache-based big data processing dimension table storage and calculation system and method: A cache-based big data processing dimension table storage and calculation method is provided, which mainly includes dimension table data writing and dimension table association calculation. The dimension table data writing can obtain data from batch or streaming data sources, clean and transform the data and store it in the Redis cache service. The dimension table association calculation is used to complete data information. Business data is obtained from batch or streaming data sources, and the corresponding extended data is read from the Redis cache according to the index key to complete the missing fields of the business data. However, the cache may have the problem of memory overflow caused by excessively large associated data, and Redis itself has a limit on the size of a single key. Summary of the Invention

[0007] This application provides a method, apparatus, server, and storage medium for generating data dimension tables, in order to solve the problems of excessive external access pressure, long data latency, and poor scalability in the data dimension table generation process in related technologies.

[0008] The first aspect of this application provides a method for generating a data dimension table. The method is applied to a server, which includes a topology tree set in memory and a memory-based database. The method includes the following steps: creating an auto-configuration project and introducing remote procedure call dependencies, a geographic index, and a component transaction monitor dependency; configuring data to the topology tree and the database using the auto-configuration project; implementing serialization and network communication using the remote procedure call dependencies; finding service locations within a vehicle's target range using the geographic index; and using the component transaction monitor dependency to provide algorithms in the topology service to calculate spatial location relationships and minimum diameters. The method also provides a location-based services (LBS) interface, whereby the LBS interface receives call requests, obtains consumption data from a distributed publish-subscribe messaging system (Kafka) based on the call requests, parses the consumption data to obtain vehicle location data and a unique identifier, queries the topology tree using the location data to obtain the vehicle's location encoding data, queries the database using the unique identifier to obtain the vehicle's vehicle information, generates a data dimension table based on the encoding data and the vehicle information, and synchronizes the data dimension table to the database.

[0009] Optionally, in one embodiment of this application, configuring data to the topology tree and the database using the automatic configuration project includes: automatically adding conditional annotations to classes under the current classpath using the automatic configuration project; storing the location encoding data into the topology tree and the vehicle information into the database according to the conditional annotations.

[0010] Optionally, in one embodiment of this application, the serialization uses a binary stream, and the network communication uses a Socket communication mechanism.

[0011] Optionally, in one embodiment of this application, the method for generating the data dimension table further includes: introducing a circular buffer, using the circular buffer to write the analysis results of the vehicle historical trajectory into the database, wherein the circular buffer is a data structure in an embedded system, and when a pointer is used to access the position after the last memory location of the buffer, it returns to the starting point of the circular buffer.

[0012] A second aspect of this application provides a method for generating a data dimension table. The method is applied to a server, which includes a topology tree set in memory and a memory-based database. The method includes the following steps: identifying a call request for a location-based service (LBS) interface; obtaining consumption data from a distributed publish-subscribe messaging system (Kafka) based on the call request; parsing the consumption data to obtain vehicle location data and a unique identifier; querying the topology tree using the location data to obtain the vehicle's location encoding data; querying the database using the unique identifier to obtain the vehicle's vehicle information; generating a data dimension table based on the encoding data and the vehicle information; and synchronizing the data dimension table to the database.

[0013] Optionally, in one embodiment of this application, before identifying the call request of the location-based service (LBS) interface, the method further includes: creating an auto-configuration project and using the auto-configuration project to configure data to the topology tree and the database; introducing a remote procedure call dependency and using the remote procedure call dependency to implement serialization and network communication; introducing a geographic index and using the geographic index to find service locations within the target range of the vehicle; and introducing a component transaction monitor dependency and using the component transaction monitor dependency to provide algorithms in the topology service with corresponding calculations of spatial location relationships and minimum diameters.

[0014] A third aspect of this application provides a data dimension table generation apparatus. The apparatus is applied to a server, which includes a topology tree in memory and a memory-based database. The apparatus includes: a creation module for creating an auto-configuration project and introducing remote procedure call dependencies, a geographic index, and a component transaction monitor dependency. The auto-configuration project configures data to the topology tree and the database; the remote procedure call dependency enables serialization and network communication; the geographic index locates service locations within a vehicle's target range; and the component transaction monitor dependency provides algorithms for topology services, calculating spatial relationships and minimum diameters. A providing module for providing a location-based service (LBS) interface. The LBS interface receives call requests, obtains consumption data from a distributed publish-subscribe messaging system (Kafka) based on the call requests, parses the consumption data to obtain vehicle location data and a unique identifier, queries the topology tree using the location data to obtain the vehicle's location encoding data, queries the database using the unique identifier to obtain the vehicle's vehicle information, generates a data dimension table based on the encoding data and the vehicle information, and synchronizes the data dimension table to the database.

[0015] Optionally, in one embodiment of this application, the creation module is further configured to: automatically add conditional annotations to classes under the current classpath using the automatic configuration project; and store the location encoding data into the topology tree and the vehicle information into the database according to the conditional annotations.

[0016] Optionally, in one embodiment of this application, the serialization uses a binary stream, and the network communication uses a Socket communication mechanism.

[0017] Optionally, in one embodiment of this application, the data dimension table generation device further includes: a writing module for introducing a circular buffer and using the circular buffer to write the analysis results of the vehicle historical trajectory into the database, wherein the circular buffer is a data structure in an embedded system, and when a pointer is used to access the position after the last memory location of the buffer, it returns to the starting point of the circular buffer.

[0018] A fourth aspect of this application provides an apparatus for generating a data dimension table. The apparatus is applied to a server, which includes a topology tree stored in memory and a memory-based database. The apparatus includes: an identification module for identifying a call request to a location-based service (LBS) interface; an acquisition module for acquiring consumption data from a distributed publish-subscribe messaging system (KAFKA) based on the call request, parsing the consumption data to obtain vehicle location data and a unique identifier; and a generation module for querying the topology tree using the location data to obtain the vehicle's location encoding data, querying the database using the unique identifier to obtain the vehicle's vehicle information, generating a data dimension table based on the encoding data and the vehicle information, and synchronizing the data dimension table to the database.

[0019] Optionally, in one embodiment of this application, the data dimension table generation apparatus further includes: an introduction module, configured to create an auto-configuration project before identifying a call request for a location-based service (LBS) interface, and to configure data to the topology tree and the database using the auto-configuration project; introduce a remote procedure call dependency, and to use the remote procedure call dependency to implement serialization and network communication; introduce a geographic index, and to use the geographic index to find service locations within the vehicle's target range; and introduce a component transaction monitor dependency, and to use the component transaction monitor dependency to provide algorithms in the topology service with corresponding calculations of spatial location relationships and minimum diameters.

[0020] A fifth aspect of this application provides a server, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data dimension table generation method as described in the above embodiments.

[0021] A sixth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the method for generating a data dimension table as described in the above embodiments.

[0022] Therefore, this application has at least the following beneficial effects:

[0023] This application embodiment can effectively reduce the pressure and data latency of external system access by creating automatic configuration projects and introducing remote procedure call dependencies, geographic indexes, and component transaction monitor dependencies. It can also obtain vehicle location data and unique identifiers based on location-based service (LBS) interfaces, thereby effectively improving its scalability. This results in the vehicle's location coding data and vehicle information, generating a data dimension table. This application has relevant timeliness and is easy to scale horizontally, improving data concurrency processing capabilities and thus effectively enhancing data processing capabilities and efficiency.

[0024] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0025] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0026] Figure 1 This is a flowchart illustrating the method for generating a data dimension table according to an embodiment of this application;

[0027] Figure 2 This is a flowchart of a method for generating a local data dimension table according to an embodiment of this application;

[0028] Figure 3 This is a schematic diagram of the project architecture provided according to an embodiment of this application;

[0029] Figure 4 This is a flowchart illustrating the workflow provided according to an embodiment of this application;

[0030] Figure 5 This is a block diagram of a data dimension table generation apparatus provided according to an embodiment of this application;

[0031] Figure 6 This is an example diagram of a data dimension table generation apparatus provided according to an embodiment of this application;

[0032] Figure 7 This is a schematic diagram of the structure of a server provided according to an embodiment of this application. Detailed Implementation

[0033] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0034] The following description, with reference to the accompanying drawings, details a method, apparatus, server, and storage medium for generating data dimension tables according to embodiments of this application. Addressing the shortcomings of related technologies mentioned in the background section, such as real-time dimension table querying, preloading of full data, LRU caching, and cache-based big data processing dimension table storage systems and methods, this application provides a method for generating data dimension tables. In this method, embodiments of this application can effectively reduce the pressure on external system access and data latency by creating automatic configuration projects and introducing remote procedure call dependencies, geographic indexes, and component transaction monitor dependencies. Furthermore, by obtaining vehicle location data and unique identifiers based on location-based services (LBS) interfaces, its scalability is effectively improved, thereby obtaining vehicle location coding data and vehicle information, thus generating a data dimension table. This application features timely correlation and facilitates horizontal scaling, improving data concurrency processing capabilities, thereby effectively enhancing data processing capabilities and efficiency.

[0035] Specifically, Figure 1 This is a flowchart illustrating a method for generating a data dimension table according to an embodiment of this application. The method is applied to a server, which includes a topology tree set in memory and a memory-based database.

[0036] like Figure 1 As shown, the method for generating this data dimension table includes the following steps:

[0037] In step S101, an auto-configuration project is created, and remote procedure call dependencies, geographic indexes, and component transaction monitor dependencies are introduced. The auto-configuration project is used to configure data to the topology tree and database, the remote procedure call dependencies are used to implement serialization and network communication, the geographic indexes are used to find service locations within the target range of the vehicle, and the component transaction monitor dependencies are used to provide algorithms in the topology services to calculate spatial location relationships and minimum diameters.

[0038] Serialization uses binary streams, and network communication uses the Socket communication mechanism.

[0039] There are various types of geographic indexes, such as Uber H3 geographic indexes, and no specific limitations are specified.

[0040] The component transaction monitor dependency can include a JTS dependency, without specific restrictions.

[0041] The minimum diameter can be the diameter of the smallest circle that contains all the points.

[0042] It is understood that the embodiments of this application can configure data to the topology tree and database, realize serialization and network communication, and calculate the corresponding spatial location relationship and minimum diameter by creating automatic configuration projects and introducing remote procedure call dependencies, geographic indexes and component transaction monitor dependencies, thereby effectively improving data processing capabilities and efficiency and providing a foundation for the subsequent generation of data tables.

[0043] Specifically, the embodiments of this application can use technologies such as Spring Boot, Apache Dubbo, R-tree, and JTS to implement location services, including spatial computing, grid computing, administrative division computing, and information interface services such as basic information of external vehicle owners and names of provinces, cities, and districts.

[0044] Optionally, in one embodiment of this application, configuring data to the topology tree and database using an automatic configuration project includes: automatically adding conditional annotations to classes under the current classpath using an automatic configuration project; and storing location encoding data into the topology tree and the vehicle information storage database based on the conditional annotations.

[0045] It is understood that the embodiments of this application can store location-coded data into a topology tree and a vehicle information storage database through conditional annotations, which can effectively simplify the development model.

[0046] Specifically, when creating a Spring Boot project, Spring Boot can automatically configure beans based on the classes and JAR files in the current classpath. For example, adding a spring-boot-starter-web starter can enable web functionality, all of which can be accomplished with the help of conditional annotations.

[0047] In step S102, a location-based service (LBS) interface is provided. The LBS interface is used to receive call requests, and the consumption data of the distributed publish-subscribe messaging system Kafka is obtained according to the call requests. The consumption data is parsed to obtain the vehicle's location data and unique identifier. The location data is used to query the topology tree to obtain the vehicle's location encoding data. The unique identifier is used to query the database to obtain the vehicle's vehicle information. A data dimension table is generated based on the encoding data and vehicle information, and the data dimension table is synchronized to the database.

[0048] It is understood that the embodiments of this application can obtain the vehicle's location data and unique identifier based on the location service LBS service interface, and obtain the vehicle's location code data and vehicle information based on the location data and unique identifier, thereby generating a data dimension table. This can reduce the access pressure and data latency of external systems, give it good scalability, and improve data processing capabilities and efficiency.

[0049] Specifically, in this embodiment of the application, a call request can be received through the LBS service interface. After receiving the call request, the consumption data of the distributed publish-subscribe messaging system Kafka can be obtained through the call request. The consumption data is parsed to obtain the vehicle's location data and unique identifier. Then, the vehicle's location coding data is obtained by querying the topology tree through the vehicle location data. The vehicle's location information is obtained by querying the database through the unique identifier. A data dimension table is generated through the coding data and the vehicle information. Finally, the data dimension table is synchronized to the database.

[0050] Optionally, in one embodiment of this application, the method for generating the data dimension table further includes: introducing a circular buffer, using the circular buffer to write the analysis results of the vehicle's historical trajectory into the database, wherein the circular buffer is a data structure in an embedded system, and when the pointer is used to access the position after the last memory location of the buffer, it returns to the starting point of the circular buffer.

[0051] It is understood that the embodiments of this application can effectively improve data processing capabilities and efficiency by introducing a circular buffer to write the analysis results of vehicle historical trajectories into the database.

[0052] The data dimension table generation method proposed in this application can effectively reduce the pressure of external system access and data latency by creating an automatic configuration project and introducing remote procedure call dependencies, geographic indexes, and component transaction monitor dependencies. It also obtains vehicle location data and unique identifiers based on the location-based service (LBS) interface, thereby effectively improving its scalability. This results in the vehicle's location coding data and vehicle information, which are then used to generate the data dimension table. This application has relevant timeliness and is easy to scale horizontally, improving data concurrency processing capabilities and thus effectively enhancing data processing capabilities and efficiency.

[0053] This application also provides a method for generating a data dimension table, wherein the method is applied to a server, the server including a topology tree set in memory and a memory-based database, wherein, as... Figure 2 As shown, the method includes the following steps:

[0054] In step S201, the call request for the location-based service (LBS) interface is identified.

[0055] It is understood that the embodiments of this application can be recognized through a human-computer interaction interface or an intelligent voice system, and no specific limitations are made.

[0056] In step S202, the consumption data of the distributed publish-subscribe messaging system Kafka is obtained according to the call request, and the vehicle's location data and unique identifier are obtained by parsing the consumption data.

[0057] It is understood that the embodiments of this application can obtain consumption data by calling requests and obtain vehicle location data and unique identifiers by parsing, thereby improving data processing capabilities and efficiency.

[0058] In step S203, the location data is used to query the topology tree to obtain the vehicle's location code data, the unique identifier is used to query the database to obtain the vehicle's information, a data dimension table is generated based on the code data and the vehicle information, and the data dimension table is synchronized to the database.

[0059] It is understood that the embodiments of this application can obtain vehicle location code data and vehicle information based on location data and unique identifiers, thereby generating a data dimension table, effectively reducing external system access pressure and data latency, improving scalability, and enhancing data processing capabilities and efficiency.

[0060] Optionally, in one embodiment of this application, before identifying the call request of the location-based service (LBS) interface, the method further includes: creating an auto-configuration project and using the auto-configuration project to configure data to the topology tree and database; introducing a remote procedure call dependency and using the remote procedure call dependency to implement serialization and network communication; introducing a geographic index and using the geographic index to find service locations within the vehicle's target range; and introducing a component transaction monitor dependency and using the component transaction monitor dependency to provide algorithms in the topology service with corresponding calculations of spatial location relationships and minimum diameters.

[0061] It is understood that the embodiments of this application can configure data to the topology tree and database, realize serialization and network communication, and calculate the corresponding spatial location relationship and minimum diameter by creating automatic configuration projects and introducing remote procedure call dependencies, geographic indexes and component transaction monitor dependencies, thereby effectively improving data processing capabilities and efficiency and providing a foundation for the subsequent generation of data tables.

[0062] The method for generating the data dimension table proposed in this application will be further illustrated below through another embodiment, wherein, as... Figure 3 As shown, the specific content of the project architecture in this application embodiment includes:

[0063] The T-Box messages from the vehicle network are uploaded to the cloud service. The cloud service synchronizes the data to Kafka via the StreamSet synchronization tool. Kafka consumes the data in Flink through topic subscription. Flink calls LBS in the rich function of the map function. LBS returns the vehicle's announcement number, brand, model, owner information, and GPS (Global Positioning System) coordinates to Mars coordinates based on the Vin code. Flink processes the data through other operators and finally writes the data to Kudu and MySQL databases through the Sink operator.

[0064] Furthermore, based on the above architecture and Figure 3 , Figure 4 This application further elaborates on a Flink method for associating dimension tables based on LBS service interfaces, as proposed in its embodiments. Flink real-time computation consumes data from Kafka and parses the message data according to parsing rules. Additionally, it remotely calls the LBS service via Dubbo, querying the STRtree based on the incoming GPS latitude and longitude to obtain the province, city, district / county, and city code. It also queries the H2 database based on the incoming VIN code to obtain the brand, vehicle series, model, and announcement number. The application employs technologies such as Spring Boot, Apache Dubbo, R-tree, and JTS to implement location services, including spatial computing, grid computing, administrative division calculation, and external interface services for vehicle owner basic information and province / city / district names. The specific details of the method are as follows:

[0065] Step 1: Create a Spring Boot project. Spring Boot can automatically configure beans based on classes and JAR files in the current classpath. For example, adding a spring-boot-starter-web starter can provide web functionality. All of this can be accomplished with the help of conditional annotations. Using Spring Boot can greatly simplify the development process. It has corresponding component support for all commonly used frameworks that you want to integrate. Specifically, through component initialization, the latitude and longitude of the province, city / district, and county are stored in the R tree of STRtree, and the basic information of the vehicle owner is synchronized to the H2 database.

[0066] Step 2: Introduce the Dubbo dependency, based on RPC (Remote Procedure Call): The two most time-consuming and critical technologies in remote procedure call are serialization and network communication. Serialization mainly includes XML, JSON, and binary streams, among which Dubbo uses binary streams for the highest efficiency. For network communication, Dubbo uses the Socket communication mechanism. If the IP address and port number are known, a connection can be established directly.

[0067] Step 3: Introduce Uber H3 geographic indexes to search for vehicles based on range for 4S stores and service stations, and to find service stations and 4S stores within the smallest range for vehicles with alarms or malfunctions.

[0068] Step 4: Introduce JTS dependencies. First, define the Coordinate class to represent spatial coordinates. In addition, the JTS com.vividsolutions.jts.algorithm package provides commonly used algorithms in topology services, which are used to calculate the relationship between Point and Area, Line and Point, and to calculate the minimum diameter of Geometry. The concept of minimum diameter is: the diameter of the smallest circle containing all points.

[0069] Furthermore, since STRtree spatial objects are stored in memory, and H2 is a memory-based database that does not involve accessing external databases, large stream processing data will not put significant pressure on external systems or affect the overall task throughput. When dimension tables change, the data is synchronized to the H2 / MySQL database. Provincial, municipal, and county administrative regions change annually; after a change, the corresponding configuration file is replaced to obtain the latest administrative geographic location information. Currently, a single LBS service processes 5,000 data entries per second and uses 32GB of memory. Therefore, when the data volume increases, simply deploying more LBS services and horizontally scaling up can double the real-time data processing capacity.

[0070] Furthermore, a ring buffer is introduced. Ring buffers are a crucial data structure in embedded systems. Accessing the memory location following the last memory location in the buffer returns to the beginning of the ring. It is set to `RingBuffer.createMultiProducer(ValueEvent.EVENT_FACTORY, 8192, new BlockingWaitStrategy())` for subsequent processing of vehicle history trajectory analysis and writing to HBase.

[0071] In summary, this application addresses the shortcomings of existing methods for real-time dimension table querying, preloading full data, LRU caching, and cache-based big data processing dimension table storage systems and methods by associating dimension table base data through LBS service interfaces. It features low latency, high concurrency, timely association, easy horizontal scaling to improve data concurrency processing capabilities, strong processing power, and fast implementation speed. It is recommended for widespread adoption and can be used in Flink data stream association dimension table scenarios.

[0072] Next, the apparatus for generating a data dimension table according to an embodiment of this application is described with reference to the accompanying drawings.

[0073] Figure 5 This is a block diagram of a data dimension table generation apparatus according to an embodiment of this application, wherein the apparatus is applied to a server, and the server includes a topology tree set in memory and a memory-based database.

[0074] like Figure 5 As shown, the data dimension table generation device 10 includes a creation module 101 and a provision module 102.

[0075] The module 101 is used to create an auto-configuration project and introduce remote procedure call dependencies, geographic indexes, and component transaction monitor dependencies. The auto-configuration project configures data to the topology tree and database; the remote procedure call dependency enables serialization and network communication; the geographic index locates service locations within the vehicle's target range; and the component transaction monitor dependency provides algorithms for topology services, calculating spatial relationships and minimum diameters. The module 102 provides a location-based service (LBS) interface. This interface receives call requests, retrieves consumption data from the Kafka distributed publish-subscribe messaging system, parses the consumption data to obtain vehicle location data and unique identifiers, queries the topology tree using the location data to obtain vehicle location encoding data, queries the database using the unique identifier to obtain vehicle information, generates a data dimension table based on the encoding data and vehicle information, and synchronizes the data dimension table to the database.

[0076] Optionally, in one embodiment of this application, the creation module 101 is further configured to: automatically add conditional annotations to classes under the current classpath using the automatic configuration project; and store the location encoding data into the topology tree and the vehicle information storage database according to the conditional annotations.

[0077] Optionally, in one embodiment of this application, serialization uses a binary stream, and network communication uses a Socket communication mechanism.

[0078] Optionally, in one embodiment of this application, the apparatus 10 further includes a writing module.

[0079] The writing module is used to introduce a circular buffer and write the analysis results of the vehicle's historical trajectory into the database. The circular buffer is a data structure in the embedded system. When accessing the memory location after the last memory location of the buffer using a pointer, the system returns to the beginning of the circular buffer.

[0080] It should be noted that the foregoing explanation of the method for generating data dimension tables also applies to the apparatus for generating data dimension tables in this embodiment, and will not be repeated here.

[0081] The data dimension table generation apparatus proposed in this application can effectively reduce the pressure and data latency of external system access by creating automatic configuration projects and introducing remote procedure call dependencies, geographic indexes and component transaction monitor dependencies. It can also obtain vehicle location data and unique identifiers based on location-based service (LBS) interfaces, thereby effectively improving its scalability. This results in obtaining vehicle location coding data and vehicle information, thus generating a data dimension table. This application has relevant timeliness and is easy to scale horizontally, improving data concurrency processing capabilities, thereby effectively enhancing data processing capabilities and efficiency.

[0082] This application embodiment also provides a data dimension table generation apparatus. The apparatus is applied to a server, which includes a topology tree configured in memory and a memory-based database, wherein, as... Figure 6 As shown, the device 20 includes: an identification module 201, an acquisition module 202, and a generation module 203.

[0083] The system includes: an identification module 201 for identifying call requests to location-based service (LBS) interfaces; an acquisition module 202 for acquiring consumption data from the distributed publish-subscribe messaging system Kafka based on the call request, parsing the consumption data to obtain vehicle location data and a unique identifier; and a generation module 203 for querying the topology tree using the location data to obtain the vehicle's location encoding data, querying the database using the unique identifier to obtain the vehicle's vehicle information, generating a data dimension table based on the encoding data and vehicle information, and synchronizing the data dimension table to the database.

[0084] Optionally, in one embodiment of this application, the apparatus 20 further includes an introduction module.

[0085] The module introduces an auto-configuration project to create data in the topology tree and database before identifying the call request of the location-based service (LBS) interface. It also introduces a remote procedure call dependency to implement serialization and network communication. A geographic index is introduced to find service locations within the vehicle's target range. Finally, a component transaction monitor dependency is introduced to provide algorithms in the topology service with the corresponding calculations of spatial location relationships and minimum diameters.

[0086] It should be noted that the foregoing explanation of the method for generating data dimension tables also applies to the apparatus for generating data dimension tables in this embodiment, and will not be repeated here.

[0087] The data dimension table generation apparatus proposed in this application can effectively reduce the pressure and data latency of external system access by creating automatic configuration projects and introducing remote procedure call dependencies, geographic indexes and component transaction monitor dependencies. It can also obtain vehicle location data and unique identifiers based on location-based service (LBS) interfaces, thereby effectively improving its scalability. This results in obtaining vehicle location coding data and vehicle information, thus generating a data dimension table. This application has relevant timeliness and is easy to scale horizontally, improving data concurrency processing capabilities, thereby effectively enhancing data processing capabilities and efficiency.

[0088] Figure 7 A schematic diagram of the structure of a server provided in an embodiment of this application. The server may include:

[0089] The memory 701, the processor 702, and the computer program stored on the memory 701 and executable on the processor 702.

[0090] When the processor 702 executes the program, it implements the data dimension table generation method provided in the above embodiments.

[0091] Furthermore, the vehicle also includes:

[0092] Communication interface 703 is used for communication between memory 701 and processor 702.

[0093] The memory 701 is used to store computer programs that can run on the processor 702.

[0094] The memory 701 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0095] If the memory 701, processor 702, and communication interface 703 are implemented independently, then the communication interface 703, memory 701, and processor 702 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0096] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0097] The processor 702 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0098] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method for generating data dimension tables.

[0099] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0100] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0101] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0102] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0103] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0104] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for generating a data dimension table, characterized in that, The method is applied to a server, the server including a topology tree set in memory and a memory-based database, wherein the method includes the following steps: An auto-configuration project is created, and remote procedure call dependencies, geographic indexes, and component transaction monitor dependencies are introduced. The auto-configuration project configures data to the topology tree and the database. The remote procedure call dependencies are used for serialization and network communication. The geographic index is used to locate service locations within the vehicle's target range. The component transaction monitor dependencies are used to provide algorithms for topology services, corresponding to the calculation of spatial location relationships and minimum diameters. Specifically, configuring data to the topology tree and the database using the auto-configuration project includes automatically adding conditional annotations to classes in the current classpath using the auto-configuration project, storing location-encoded data in the topology tree based on the conditional annotations, and storing vehicle information in the database. A location-based service (LBS) interface is provided, wherein the LBS interface is used to receive a call request, and the consumption data of the distributed publish-subscribe messaging system Kafka is obtained according to the call request. The consumption data is parsed to obtain the vehicle's location data and unique identifier. The location data is used to query the topology tree to obtain the vehicle's location encoding data. The unique identifier is used to query the database to obtain the vehicle's vehicle information. A data dimension table is generated based on the encoding data and the vehicle information, and the data dimension table is synchronized to the database.

2. The method according to claim 1, characterized in that, The serialization uses a binary stream, and the network communication uses a Socket communication mechanism.

3. The method according to claim 1, characterized in that, Also includes: A circular buffer is introduced, and the analysis results of the vehicle's historical trajectory are written into the database using the circular buffer. The circular buffer is a data structure in the embedded system. When the pointer is used to access the position after the last memory location of the buffer, it returns to the starting point of the circular buffer.

4. A method for generating a data dimension table, characterized in that, The method is applied to a server, the server including a topology tree set in memory and a memory-based database, wherein the method includes the following steps: Create an auto-configuration project and use it to configure data to the topology tree and the database; introduce a remote procedure call dependency and use it to implement serialization and network communication; introduce a geographic index and use it to find service locations within the vehicle's target range; introduce a component transaction monitor dependency and use it to provide algorithms in the topology service to calculate spatial location relationships and minimum diameter. Identify call requests for location-based service (LBS) interfaces; Based on the call request, the consumer data of the distributed publish-subscribe messaging system Kafka is obtained, and the consumer data is parsed to obtain the vehicle's location data and unique identifier; The location data is used to query the topology tree to obtain the vehicle's location code data. The unique identifier is used to query the database to obtain the vehicle's vehicle information. A data dimension table is generated based on the code data and the vehicle information, and the data dimension table is synchronized to the database.

5. A device for generating a data dimension table, characterized in that, The apparatus is applied to a server, the server including a topology tree configured in memory and a memory-based database, wherein the apparatus includes: A creation module is used to create an auto-configuration project and introduce remote procedure call dependencies, geographic indexes, and component transaction monitor dependencies. The auto-configuration project is used to configure data to the topology tree and the database. The remote procedure call dependencies are used to implement serialization and network communication. The geographic index is used to find service locations within the target range of the vehicle. The component transaction monitor dependencies are used to provide algorithms in the topology service to calculate spatial location relationships and minimum diameters. A module is provided for providing a location-based service (LBS) interface. This LBS interface receives a call request, retrieves consumption data from a Kafka distributed publish-subscribe messaging system based on the call request, parses the consumption data to obtain vehicle location data and a unique identifier, queries the topology tree using the location data to obtain the vehicle's location encoding data, queries the database using the unique identifier to obtain the vehicle's vehicle information, generates a data dimension table based on the encoding data and the vehicle information, and synchronizes the data dimension table to the database.

6. A device for generating a data dimension table, characterized in that, The apparatus is applied to a server, the server including a topology tree configured in memory and a memory-based database, wherein the apparatus includes: The identification module is used to identify call requests for location-based service (LBS) interfaces. The acquisition module is used to acquire consumption data from the distributed publish-subscribe messaging system Kafka according to the call request, and parse the consumption data to obtain the vehicle's location data and unique identifier; The generation module is used to query the topology tree using location data to obtain the location coding data of the vehicle, query the database using the unique identifier to obtain the vehicle information of the vehicle, generate a data dimension table based on the coding data and the vehicle information, and synchronize the data dimension table to the database.

7. A server, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method for generating a data dimension table as described in any one of claims 1-4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method for generating data dimension tables as described in any one of claims 1-4.

Images