Data assembly method and device, electronic equipment, storage medium and product

Abstract

The invention discloses a data assembly method and device, electronic equipment, a storage medium and a product. Relates to the technical field of databases. The method comprises the steps of obtaining a data query request; querying information of each dimension from a multi-dimensional header table under the condition that the data list corresponding to the data query request does not exist in the cache; querying the to-be-assembled data from the multi-dimensional data table according to the information, the index and the query condition of each dimension; and mapping the to-be-assembled data to the positions of the corresponding dimensions in sequence to obtain an assembled data list. According to the scheme, the information of each dimension is stored by utilizing the multi-dimensional header table, the to-be-assembled data is stored by utilizing the multi-dimensional data table, the corresponding dimension can be automatically matched and the to-be-assembled data can be inquired and mapped based on the data inquiry request, and the assembled data list is obtained, so that the dynamic and real-time adjustment of the data dimension is realized, and the assembling efficiency is improved. The system does not need to be restarted or upgraded, the database structure does not need to be redesigned, and the flexibility and the assembly efficiency of multi-dimensional data assembly are improved.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a data assembly method, apparatus, electronic device, storage medium, and product. Background Technology

[0002] With the development of big data, data dimensions are increasing, and data display needs to support dynamic expansion at runtime. In complex data environments, through flexible parameter settings and algorithm mechanisms, data with multiple dimensions and dynamic changing characteristics can be collected, stored, analyzed, and visualized in real time or near real time to support data-driven decision-making and system optimization in multiple scenarios. Its core lies in building scalable data models, efficient processing engines, and intelligent configuration strategies to meet the diverse requirements of different business needs for data structure, processing logic, and response speed.

[0003] Currently, adjustments to multidimensional data dimensions are primarily based on static database table structure design methods, extraction, transformation, and loading (ETL) of predefined dimensions, online analytical processing (OLAP) relying on pre-compiled multidimensional cubes, pre-computation and caching through middleware, or manually writing Structured Query Language (SQL) statements for data querying and processing. These methods have several drawbacks. For example, they rely on fixed database structures, requiring a redesign of the database structure when adjusting data dimensions, increasing the workload of front-end and back-end development and maintenance. For the adjusted data dimensions to take effect, the entire business system usually needs to be restarted or upgraded, severely impacting system flexibility and real-time responsiveness. Using middleware for multidimensional data analysis often consumes significant server resources, resulting in high operational and usage costs. Manually writing SQL statements requires a high level of technical expertise from the user, necessitating SQL writing skills for data querying.

[0004] In summary, existing methods cannot achieve dynamic and real-time adjustment and display of data dimensions during website operation; the dynamic adjustment of data dimensions is inflexible and inefficient. How to flexibly and efficiently achieve dynamic adjustment of data dimensions without restarting or upgrading the system or redesigning the database structure is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a data assembly method, apparatus, electronic device, storage medium, and product to achieve dynamic and real-time adjustment of data dimensions, thereby improving the flexibility and efficiency of multidimensional data assembly.

[0006] In a first aspect, embodiments of this application provide a data assembly method, including:

[0007] Obtain a data query request, which is used to specify the dimensions, metrics, and query conditions of the data to be queried;

[0008] If the data list corresponding to the data query request does not exist in the cache, information for each dimension is queried from the multidimensional header table according to the data query request;

[0009] Based on the information of each dimension, the indicators, and the query conditions, query the data to be assembled from the multidimensional data table;

[0010] Based on the order of the dimensions defined in the multidimensional header table, the data to be assembled is mapped to the corresponding dimension positions to obtain the assembled data list.

[0011] Secondly, embodiments of this application also provide a data assembly apparatus, comprising:

[0012] The request acquisition module is used to acquire data query requests, which specify the dimensions, indicators, and query conditions of the data to be queried.

[0013] The dimension query module is used to query information for each dimension from the multidimensional header table according to the data query request if the data list corresponding to the data query request does not exist in the cache.

[0014] The data query module is used to query the data to be assembled from the multidimensional data table based on the information of each dimension, the indicators, and the query conditions.

[0015] The data assembly module is used to map the data to be assembled to the corresponding dimension positions according to the order of the dimensions defined in the multidimensional header table, so as to obtain the assembled data list.

[0016] Thirdly, embodiments of this application provide an electronic device, including:

[0017] One or more processors;

[0018] Storage device for storing one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the data assembly method as described in the first aspect.

[0020] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the data assembly method as described in the first aspect.

[0021] Fifthly, embodiments of this application also provide a computer program product, including a computer program and / or instructions, which, when executed by a processor, implement the data assembly method as described in any of the above embodiments.

[0022] This application provides a data assembly method, apparatus, electronic device, storage medium, and product. The data assembly method includes: obtaining a data query request, wherein the data query request specifies the dimensions, indicators, and query conditions of the data to be queried; if no data list corresponding to the data query request exists in the cache, querying information for each dimension from a multidimensional header table according to the data query request; querying data to be assembled from a multidimensional data table according to the information of each dimension, the indicators, and the query conditions; and mapping the data to be assembled to the corresponding dimension positions according to the order of the dimensions defined in the multidimensional header table, thereby obtaining an assembled data list. The above technical solution utilizes a multidimensional header table to store information for each dimension and a multidimensional data table to store the data to be assembled. Then, based on the dimensions, indicators, and query conditions specified in the data query request, it can automatically match the corresponding dimensions and query and map the data to be assembled to obtain an assembled data list. This achieves dynamic and real-time adjustment of data dimensions without requiring system restarts or upgrades or database structure redesign, improving the flexibility and efficiency of multidimensional data assembly. Attached Figure Description

[0023] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0024] Figure 1 A flowchart illustrating a data assembly method provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of the structure of a data assembly apparatus provided in an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present application are shown in the drawings, not the entire structure.

[0028] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0029] It should be noted that the concepts of "first" and "second" mentioned in the embodiments of this application are only used to distinguish different devices, modules, units or other objects, and are not used to limit the order or interdependence of the functions performed by these devices, modules, units or other objects.

[0030] Furthermore, the embodiments and features described in this application may be combined with each other, unless otherwise specified.

[0031] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0032] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the relevant content of the solution.

[0033] Figure 1 This is a flowchart illustrating a data assembly method provided in this application embodiment. This embodiment is applicable to situations where data is assembled according to dimensions, such as assembling data in a database or assembling data in real time during webpage operation. The purpose of assembly is to store or display data in the form of a data list according to corresponding dimensions. Specifically, this data assembly method can be executed by a data assembly device, which can be implemented through software and / or hardware and integrated into an electronic device. Electronic devices include, but are not limited to, devices with data processing capabilities such as computers, smartphones, host computers, or servers.

[0034] like Figure 1 As shown, the method specifically includes the following steps:

[0035] S110. Obtain a data query request, wherein the data query request is used to specify the dimensions, indicators and query conditions of the data to be queried;

[0036] For example, a user-inputted data query request can be obtained through a front-end interface. The data query request explicitly specifies the dimensions to be queried or displayed, such as "time," "region," and "product category," which can be represented as a dimension list. The data query request can also specify the metrics to be queried or displayed, such as "total sales revenue" and "average order quantity," which can be represented as a metric list. The data query request can also specify the data query conditions (or filtering conditions), such as "time range: Q1 2024" or "region: East China." After receiving the data query request, the back-end can parse and validate the parameters to ensure that the requested dimensions and metrics are defined and valid in the system.

[0037] Optionally, based on the data query request, the cache can be searched to see if there is a list of identical or similar assembled data. If so, the assembly result can be returned directly to avoid unnecessary duplicate queries and calculations.

[0038] S120. If the data list corresponding to the data query request does not exist in the cache, query the information of each dimension from the multidimensional header table according to the data query request.

[0039] For example, if no identical or similar assembled data list is found in the cache, the current assembly process begins. Specifically, based on the data query request, information for each dimension is retrieved from the multidimensional header table. The multidimensional header table stores information for different headers, which are typically text located at the top or left of the table and describe the dimensions of the data in the table. Information for each dimension can include at least one of the following: a unique identifier (dimension_ID), a name, a data type (Type, such as int / float / string / date), a unit, and a possible description. This ensures that the meaning and format of each field are correctly understood during subsequent data assembly.

[0040] S130. Based on the information of each dimension, the indicators, and the query conditions, query the data to be assembled from the multidimensional data table;

[0041] For example, based on the information, metrics, and query conditions specified in the data query request for each dimension, the data that needs to be assembled can be located from a multidimensional data table. A multidimensional data table can be used to store the data to be assembled; the data that satisfies the content specified in the data query request is the data that needs to be processed during this assembly process. During the process of locating the data to be assembled, it can be aggregated, sorted, and / or formatted according to its format (such as JAVA or EVA) to give it clearer relationships and a more standardized format.

[0042] S140. According to the order of the dimensions defined in the multidimensional header table, map the data to be assembled to the corresponding dimension positions to obtain the assembled data list.

[0043] For example, the data to be assembled, retrieved from a multidimensional data table, is mapped to the corresponding dimensions in a specific dimensional order. For instance, data from dimension A is mapped to the corresponding rows or columns, then data from dimension B is mapped to the corresponding rows or columns, and so on, resulting in the assembled data list. The order of the dimensions can refer to the order defined by the user in the multidimensional table header. For example, if the defined dimension order is "time, region, product category, sales revenue," then the data in the final output list will follow this order.

[0044] Optionally, the data to be assembled can be rearranged and formatted during the mapping process. Furthermore, the assembled data list can be converted into a JSON format that is easy for the front-end to parse and display. This JSON object contains not only the calculated metric values ​​but also intermediate process data or metadata used for front-end display. The data to be assembled, along with these metric values ​​and metadata, can be encapsulated into a structured data object containing complete header information (i.e., all dimensions and attributes specified in the data query request) and the assembled data list after aggregation, sorting, and formatting.

[0045] This application provides a multi-dimensional data processing system based on dynamic configuration and efficient storage structure, which can solve technical problems such as the inability to flexibly adjust data dimensions during operation, the inability of the storage structure to adapt to dynamically changing dimensions, low data assembly efficiency, and imperfect front-end and back-end interaction processes. The overall architecture of the system can be divided into a front-end interaction layer, a back-end processing layer, and a data storage layer. The front-end interaction layer allows users to drag and drop and edit the dimensions and meanings of multidimensional data through a visual interface, thereby initiating data query requests in real time. The front-end interaction layer can send data query requests to the back-end processing layer via an Application Programming Interface (API), where the back-end processing layer responds and processes them in real time. In the back-end processing layer, the multidimensional data header saver (through a multidimensional header table) is responsible for receiving and storing new dimension definitions, the multidimensional data saver (through a multidimensional data table) handles the updating and storage of data content, and the data correspondent maintains the mapping relationship between dimensions and data. Furthermore, it can preprocess user-defined dimension definitions and submitted data, such as standardization, data validation, data cleaning, data normalization, and dimensionality reduction, ensuring that dynamic adjustments to dimensions take effect in real time and data display is updated synchronously. This eliminates the need for system restarts or upgrades or database structure redesign, greatly simplifying the user operation process.

[0046] The data assembly algorithm of this application embodiment can quickly and accurately extract and aggregate the data required by the user from unstructured or semi-structured data sources in a dynamic multidimensional data environment, and transform this data into a data list in a format easy to display on the front end. This method extracts the required dimension information and data to be assembled from multidimensional header tables and multidimensional data tables, and performs corresponding calculations in conjunction with specified indicators. It can efficiently query, aggregate, and map data in formats such as JSON or EAV, ultimately generating the Data object required by the front end, containing a complete header and data list. Based on this, it optimizes the data extraction path, reduces redundant operations, and implements intelligent aggregation and mapping mechanisms. It does not require system restarts or upgrades or database structure redesign, significantly improving the efficiency of multidimensional data query and assembly. While achieving dynamic adjustment of data dimensions and real-time effect, it also features low server resource consumption, no need for additional middleware maintenance, low learning curve, and the ability for users to complete data assembly without writing SQL.

[0047] In one embodiment, the process of querying data to be assembled from a multidimensional data table includes: for data in JSON format, querying the data to be assembled from the multidimensional data table according to a JSON path expression.

[0048] For example, for JSON formatted data, it is possible to quickly filter the data using JSON path expressions in the database or at the application level, thereby identifying the smallest subset of data directly related to the user's request as quickly as possible, laying the foundation for subsequent aggregation operations.

[0049] For example, for JSON format, you can leverage database support for JSON types (such as MongoDB's aggregation pipeline, PostgreSQL's JSONB operator, etc.), or use an efficient JSON parsing library to process it at the application layer. For instance, for the JSON record {"Time": "2024-01", "Region": "East China", "Product Category": "Mobile Phone", "Sales Amount": 10000}, you can directly retrieve the value using the key name "Sales Amount". <c114>10000, then accumulate according to the indicator requirements (such as summation).

[0050] In one embodiment, the process of querying data to be assembled from a multidimensional data table includes: for data in Entity-Attribute-Value (EAV) format, querying the data to be assembled from the multidimensional data table based on the query conditions and the entity name and attribute name in the Structured Query Language (SQL) query.

[0051] For example, for EAV format data, entity names (Entity_ID) and attribute names (Attribute) can be used for exact matching and range filtering, and the dataset can be further narrowed down by combining query conditions (such as time range and region code).

[0052] For example, EAV format data is typically stored in a table with three columns: "Entity-Attribute-Value" or a similar format. Records belonging to the same entity can be filtered out based on the Entity_ID, and then the corresponding Value can be extracted based on the requested dimension (i.e., attribute name).

[0053] In one embodiment, the process of querying the data to be assembled from the multidimensional data table further includes: aggregating the data to be assembled.

[0054] In this embodiment, the data to be assembled, retrieved from a multidimensional data table, is aggregated. For example, if a user requests the sum of "sales," the algorithm will execute aggregation logic similar to the following: total_sales = sum(sales_i). It iterates through the filtered data records and calculates the specified fields according to predefined calculation logic or formulas in the specified indicators, such as summation (SUM), average (AVG), number counting (COUNT), maximum value (MAX), minimum value (MIN), etc. The indicators specified in the data query request can be represented by an indicator list. This list not only stores calculation formulas but can also contain more complex calculation logic expressions (e.g., calculating gross profit margin might require "(sales - cost) / sales").

[0055] Optionally, multiple aggregation methods are supported during the aggregation process to handle complex computational logic. Various optimization strategies can be employed during aggregation: for example, for some aggregation operations, database-specific aggregation functions (such as SQL's SUM and AVG) can be used to push some calculations down to the database level, reducing the amount of data transmitted to the application server; furthermore, for non-relational structures like JSON or EAV, efficient traversal and aggregation mechanisms can be designed, and considering data type consistency, necessary data type conversions and validations can be performed before aggregation.

[0056] For example, for aggregations in JSON format, you can use the database's JSON functions (such as sonb_path_query_array) to extract values ​​from a specific path for calculation, or parse the JSON data into an in-memory structure and then iterate and aggregate it.

[0057] For example, for aggregations in EAV format, you can first group by Entity_ID, and then aggregate the relevant Attribute values ​​under each Entity_ID. Since EAV data is inherently sparse, missing attributes (i.e., attributes with NULL values ​​or that do not appear) will be affected during aggregation. For example, when calculating "total sales," the algorithm will find all records with "attribute name" as "sales" and sum their values. For more complex aggregations, such as calculating "average sales," the algorithm needs to count the number of Entity_IDs that meet the conditions, sum the values ​​of all "sales," and finally perform a division operation. Optionally, this process can be efficiently implemented by constructing multi-level loops or using aggregate functions such as GROUP BY and SUM in the database.

[0058] In one embodiment, the method further includes:

[0059] S1110. Generate a cache key based on the data query request;

[0060] S1120. Based on the cache key, search the cache for a list of valid data corresponding to the cache key in the cache;

[0061] S1130. If the data list corresponding to the cache key is not found, it is determined that there is no data list in the cache corresponding to the data query request.

[0062] This embodiment provides a caching mechanism and performance optimization method, in which an intelligent caching mechanism is introduced. After generating a Data object from the assembled data list, a unique cache key can be generated based on the content of the corresponding data query request (such as dimensions, metrics, and query conditions). When a data query request is received, the system first searches the cache. If no matching key exists, or if a Data object with the same key exists but has expired or become invalid, it is determined that there is no data list corresponding to the data query request in the cache, and operations such as querying the data to be assembled, aggregation, rearrangement, and formatting are then performed.

[0063] Optionally, this caching mechanism can set a reasonable expiration time or adopt an eviction policy such as Least Recently Used (LRU). It can also support a cache invalidation mechanism, such as automatically invalidating the relevant cache when the underlying data changes, ensuring real-time cache updates and improving the reliability and timeliness of data assembly.

[0064] Optionally, multi-threading or asynchronous processing techniques can be used to pre-calculate or preload potentially related data to be assembled or data lists as soon as a user initiates a query, thereby further shortening the waiting time for front-end users and improving system response speed.

[0065] In one embodiment, the method further includes: S150, if a data list corresponding to the cache key is found, then the valid data list is read and the valid data list is used as the assembled data list.

[0066] For example, when a data query request is received, the system can first search the cache. If a Data object with the same key and not expired exists, it can be read directly from the cache and returned, skipping time-consuming steps such as data location, aggregation calculation, and format conversion.

[0067] The data assembly algorithm in this application adopts a three-layer structure of "multidimensional header table + multidimensional data table (supporting variable structures such as JSON / EAV) + indicator list" to store dynamic multidimensional data. It can flexibly handle user-defined dimension combinations and indicator requests. When dealing with dynamic multidimensional data, it provides fast, accurate, and flexible data query and display services without the need for pre-defined complex query templates or Cube structures. It is highly adaptable and can respond in real-time to changes in dimension combinations, significantly reducing the time required for data assembly. The performance improvement is particularly noticeable when processing high-dimensional, large-scale JSON or EAV format data. This method encapsulates aggregation and mapping logic within the system. Front-end users do not need to write any SQL or complex query statements; they can obtain the required data simply by configuring through a visual interface, greatly reducing the barrier to entry and difficulty of use, improving the convenience of dynamic dimensional data assembly, and avoiding errors from manual coding. This method has good scalability and can easily integrate new aggregation functions or calculation logic to support new business analysis needs that may arise in the future. This method can efficiently process both JSON and EAV, two mainstream variable-structure data formats, meeting the data storage preferences and needs of different scenarios. Furthermore, it employs different internal processing strategies for JSON and EAV data to maximize efficiency. This method does not rely on external middleware, reducing system complexity and maintenance costs, and consumes fewer resources. It is lightweight and easy to use, making it suitable even for resource-constrained deployment environments.

[0068] This method can be applied to the construction of various website systems that require flexible display of multi-dimensional data, such as data dashboards / dashboards: such as leadership dashboards and business analysis dashboards, where indicators and dimensions often change with business needs; configurable reporting systems: where users can dynamically adjust the dimensional structure of reports according to their own needs; e-commerce platform data analysis: where dimensions such as product attributes and user tags may be frequently adjusted; and IoT data analysis platforms: where the indicators and dimensions reported by devices may be diverse and changeable.

[0069] Figure 2 This is a schematic diagram of a data assembly apparatus provided in an embodiment of this application. Figure 2 As shown, the data assembly apparatus provided in this embodiment includes:

[0070] The request acquisition module 210 is used to acquire a data query request, wherein the data query request is used to specify the dimensions, indicators and query conditions of the query data;

[0071] The dimension query module 220 is used to query information of each dimension from the multidimensional header table according to the data query request when there is no data list corresponding to the data query request in the cache.

[0072] Data query module 230 is used to query the data to be assembled from the multidimensional data table based on the information of each dimension, the indicators and the query conditions;

[0073] The data assembly module 240 is used to map the data to be assembled to the corresponding dimension positions according to the order of the dimensions defined in the multidimensional header table, so as to obtain the assembled data list.

[0074] This device uses a multidimensional header table to store information for each dimension and a multidimensional data table to store the data to be assembled. Then, based on the dimensions, indicators, and query conditions specified in the data query request, it can automatically match the corresponding dimensions and query and map the data to be assembled to obtain the assembled data list. This enables dynamic and real-time adjustment of data dimensions without the need to restart or upgrade the system or redesign the database structure, thus improving the flexibility and efficiency of multidimensional data assembly.

[0075] Based on any of the above embodiments, the data query module 230 is specifically used to query the data to be assembled from the multidimensional data table according to the JSON path expression for JSON format data.

[0076] Based on any of the above embodiments, the data query module 230 is specifically used to query the data to be assembled from the multidimensional data table for EAV format data according to the query conditions and the entity name and attribute name in the structured query language SQL query.

[0077] Based on any of the above embodiments, the data query module 230 is also used for the process of querying the data to be assembled from the multidimensional data table, including: aggregating the data to be assembled.

[0078] Based on any of the above embodiments, the device further includes:

[0079] The generation module is used to generate cache keys based on the data query request;

[0080] The lookup module is used to search for a list of valid data corresponding to the cache key in the cache.

[0081] The determination module is used to determine that if the data list corresponding to the cache key is not found, the data list corresponding to the data query request does not exist in the cache.

[0082] Based on any of the above embodiments, the device further includes: a reading module, configured to read the valid data list if a data list corresponding to the cache key is found, and use the valid data list as the assembled data list.

[0083] The data assembly apparatus provided in this application embodiment can be used to execute the data assembly method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0084] Figure 3 A schematic diagram of an electronic device 10, which can be used to implement embodiments of this application, is shown. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, user equipment, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0085] like Figure 3 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0086] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks and wireless networks.

[0087] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above.

[0088] In some embodiments, the methods described above can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the methods of any of the embodiments described above by any other suitable means (e.g., by means of firmware).

[0089] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0091] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0092] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device 10, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device 10. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0093] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0094] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0095] This application also provides a computer program product, including a computer program and / or instructions, which, when executed by a processor, implement the data assembly method as described in any of the above embodiments.

[0096] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0097] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data assembly method, characterized in that, include: Obtain a data query request, which is used to specify the dimensions, metrics, and query conditions of the data to be queried; If the data list corresponding to the data query request does not exist in the cache, information for each dimension is queried from the multidimensional header table according to the data query request; Based on the information of each dimension, the indicators, and the query conditions, query the data to be assembled from the multidimensional data table; Based on the order of the dimensions defined in the multidimensional header table, the data to be assembled is mapped to the corresponding dimension positions to obtain the assembled data list.

2. The method according to claim 1, characterized in that, The process of retrieving data to be assembled from a multidimensional data table includes: For JSON formatted data, retrieve the data to be assembled from the multidimensional data table based on the JSON path expression.

3. The method according to claim 1, characterized in that, The process of retrieving data to be assembled from a multidimensional data table includes: For entity attribute values ​​in EAV format, the data to be assembled is retrieved from the multidimensional data table based on the query conditions and the entity name and attribute name in the structured query language SQL query.

4. The method according to claim 2 or 3, characterized in that, The process of querying the data to be assembled from the multidimensional data table also includes: aggregating the data to be assembled.

5. The method according to claim 1, characterized in that, Also includes: Generate a cache key based on the data query request; Based on the cache key, search the cache for a list of valid data corresponding to the cache key; If the data list corresponding to the cache key is not found, it is determined that there is no data list in the cache corresponding to the data query request.

6. The method according to claim 5, characterized in that, Also includes: If the data list corresponding to the cache key is found, the valid data list is read and used as the assembled data list.

7. A data assembly apparatus, characterized in that, include: The request acquisition module is used to acquire data query requests, which specify the dimensions, indicators, and query conditions of the data to be queried. The dimension query module is used to query information for each dimension from the multidimensional header table according to the data query request if the data list corresponding to the data query request does not exist in the cache. The data query module is used to query the data to be assembled from the multidimensional data table based on the information of each dimension, the indicators, and the query conditions. The data assembly module is used to map the data to be assembled to the corresponding dimension positions according to the order of the dimensions defined in the multidimensional header table, so as to obtain the assembled data list.

8. An electronic device, characterized in that, include: At least one processor; A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data assembly method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the data assembly method as described in any one of claims 1-6.

10. A computer program product comprising a computer program and / or instructions, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the data assembly method as described in any one of claims 1-6.

Images