Data processing method and device, computer device and storage medium
By establishing the association between the identifier of the table to be created and the dimension information, a virtual table to be populated is generated and data is obtained from the dimension data source. This solves the problem of high database resource consumption caused by traditional data table templates and achieves efficient data table management and resource saving.
Patent Information
- Application Number
- CN202211161525.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-23
- Publication Date
- 2026-02-10
- Estimated Expiration
- 2042-09-23
AI Technical Summary
The diversity of traditional data table templates leads to the creation of a large number of physical data tables in the database, resulting in management and maintenance pressure and high resource consumption.
By generating the association between the identifier of the table to be created and the dimension information, a virtual table to be populated is established, and the population data is obtained from the dimension data source, thus avoiding the storage of physical tables in the database.
It saves database resources, improves the efficiency of data table creation and management, and reduces the storage requirements of physical tables.
Smart Images

Figure CN115495452B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer application technology, and in particular to a data processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology
[0002] Data in databases is typically stored in the form of tables. Traditional tables, due to their different structures, require corresponding physical table templates to store related data. For example, the header structures of tables like "Personnel" and "Organization" differ. Therefore, it is necessary to pre-create physical table templates that may be used in various application scenarios, and then generate the physical data tables required for the application scenarios by storing data using the created physical table templates. However, due to the diverse nature of application scenarios, a large number of data table templates are created, generating a large number of physical data tables. As a result, a large number of physical data tables in the database puts a lot of pressure on database management and maintenance, leading to problems such as high database resource consumption. Summary of the Invention
[0003] Therefore, it is necessary to provide a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can save database resources to address the above-mentioned technical problems.
[0004] Firstly, this application provides a data processing method. The method includes:
[0005] Get the data table creation request. The data table creation request carries the dimension information of the table to be created. The dimension information of the table to be created includes the name of each dimension and the source of the dimension data corresponding to each dimension name.
[0006] Based on the data table creation request, generate the identifier of the table to be created and establish the association between the identifier of the table to be created and the dimension information of the table to be created;
[0007] A virtual table to be populated is created based on the names of each dimension in the dimension information of the table to be created and the identifier of the table to be created. The virtual table to be populated is then returned to the requesting end corresponding to the data table creation request.
[0008] Obtain the population request for the virtual table to be populated. The population request carries the name of the dimension to be populated and the identifier of the table to be created.
[0009] Based on the fill request, find the dimension information of the table to be created associated with the table to be created according to the relationship, obtain the data source of the dimension to be filled corresponding to the dimension name to be filled from the dimension information of the table to be created, and obtain the dimension data to be filled based on the data source of the dimension to be filled;
[0010] The dimension data to be populated is returned to the requesting end so that the requesting end can write the dimension data to be populated into the item corresponding to the dimension name to be populated in the virtual table to be populated.
[0011] Secondly, this application also provides a data processing apparatus. The apparatus includes:
[0012] The request retrieval module is used to retrieve data table creation requests. The data table creation request carries the dimension information of the table to be created, which includes the name of each dimension and the source of the dimension data corresponding to each dimension name.
[0013] The association module is used to generate a table identifier to be created based on the data table creation request, and to establish the association relationship between the table identifier and the dimension information of the table to be created.
[0014] The table generation module is used to create a virtual table to be populated based on the names of each dimension in the dimension information of the table to be created and the identifier of the table to be created, and return the virtual table to be populated to the request end corresponding to the data table creation request.
[0015] The module to be populated is used to obtain the population request of the virtual table to be populated. The population request carries the name of the dimension to be populated and the identifier of the table to be created.
[0016] The data acquisition module is used to search for the dimension information of the table to be created that is associated with the table to be created based on the filling request and the relationship, to obtain the data source of the dimension to be filled corresponding to the dimension name to be filled from the dimension information of the table to be created, and to obtain the dimension data to be filled based on the data source of the dimension to be filled.
[0017] The populate module is used to return the dimension data to be populated to the requesting end, so that the requesting end can write the dimension data to be populated into the item corresponding to the dimension name to be populated in the virtual table to be populated.
[0018] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:
[0019] Get the data table creation request. The data table creation request carries the dimension information of the table to be created. The dimension information of the table to be created includes the name of each dimension and the source of the dimension data corresponding to each dimension name.
[0020] Based on the data table creation request, generate the identifier of the table to be created and establish the association between the identifier of the table to be created and the dimension information of the table to be created;
[0021] A virtual table to be populated is created based on the names of each dimension in the dimension information of the table to be created and the identifier of the table to be created. The virtual table to be populated is then returned to the requesting end corresponding to the data table creation request.
[0022] Obtain the population request for the virtual table to be populated. The population request carries the name of the dimension to be populated and the identifier of the table to be created.
[0023] Based on the fill request, find the dimension information of the table to be created associated with the table to be created according to the relationship, obtain the data source of the dimension to be filled corresponding to the dimension name to be filled from the dimension information of the table to be created, and obtain the dimension data to be filled based on the data source of the dimension to be filled;
[0024] The dimension data to be populated is returned to the requesting end so that the requesting end can write the dimension data to be populated into the item corresponding to the dimension name to be populated in the virtual table to be populated.
[0025] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:
[0026] Get the data table creation request. The data table creation request carries the dimension information of the table to be created. The dimension information of the table to be created includes the name of each dimension and the source of the dimension data corresponding to each dimension name.
[0027] Based on the data table creation request, generate the identifier of the table to be created and establish the association between the identifier of the table to be created and the dimension information of the table to be created;
[0028] A virtual table to be populated is created based on the names of each dimension in the dimension information of the table to be created and the identifier of the table to be created. The virtual table to be populated is then returned to the requesting end corresponding to the data table creation request.
[0029] Obtain the population request for the virtual table to be populated. The population request carries the name of the dimension to be populated and the identifier of the table to be created.
[0030] Based on the fill request, find the dimension information of the table to be created associated with the table to be created according to the relationship, obtain the data source of the dimension to be filled corresponding to the dimension name to be filled from the dimension information of the table to be created, and obtain the dimension data to be filled based on the data source of the dimension to be filled;
[0031] The dimension data to be populated is returned to the requesting end so that the requesting end can write the dimension data to be populated into the item corresponding to the dimension name to be populated in the virtual table to be populated.
[0032] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:
[0033] Get the data table creation request. The data table creation request carries the dimension information of the table to be created. The dimension information of the table to be created includes the name of each dimension and the source of the dimension data corresponding to each dimension name.
[0034] Based on the data table creation request, generate the identifier of the table to be created and establish the association between the identifier of the table to be created and the dimension information of the table to be created;
[0035] A virtual table to be populated is created based on the names of each dimension in the dimension information of the table to be created and the identifier of the table to be created. The virtual table to be populated is then returned to the requesting end corresponding to the data table creation request.
[0036] Obtain the population request for the virtual table to be populated. The population request carries the name of the dimension to be populated and the identifier of the table to be created.
[0037] Based on the fill request, find the dimension information of the table to be created associated with the table to be created according to the relationship, obtain the data source of the dimension to be filled corresponding to the dimension name to be filled from the dimension information of the table to be created, and obtain the dimension data to be filled based on the data source of the dimension to be filled;
[0038] The dimension data to be populated is returned to the requesting end so that the requesting end can write the dimension data to be populated into the item corresponding to the dimension name to be populated in the virtual table to be populated.
[0039] The aforementioned data processing method, apparatus, computer equipment, storage medium, and computer program product acquire a data creation request sent by the requesting end. This request carries dimension information for a table to be created. Based on the data creation request, a table identifier is generated, and an association is established between the table identifier and the dimension information. Using the table identifier and the dimension names within the dimension information, a virtual table to be populated (required by the requesting end) is created, and this virtual table is returned to the requesting end. The method also acquires a population request for the virtual table from the requesting end. Using the dimension names and table identifier in the population request, the source of the dimension data corresponding to the dimension name can be quickly determined based on the association between the table identifier and the dimension information. The data is then retrieved based on the data source. Finally, the dimension data is returned to the requesting end so that the requesting end can write the dimension data into the corresponding fields of the dimension names in the virtual table. Since the virtual table to be populated, generated based on the dimension information of the table to be created, is not stored in the database, and the population data in the virtual table is obtained from the dimension data source, a large number of physical tables can be avoided in the server database, thus saving database resources. Attached Figure Description
[0040] Figure 1 This is a diagram illustrating the application environment of a data processing method in one embodiment.
[0041] Figure 2 This is a flowchart illustrating a data processing method in one embodiment;
[0042] Figure 3 This is a schematic diagram of the creation process of a virtual table to be populated in one embodiment;
[0043] Figure 4 This is a flowchart illustrating a data processing method in one embodiment;
[0044] Figure 5 This is a flowchart illustrating the data processing method in another embodiment;
[0045] Figure 6 This is a structural block diagram of a data processing device in one embodiment;
[0046] Figure 7 This is an internal structural diagram of a computer device in one embodiment;
[0047] Figure 8 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0049] The data processing method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server. Server 104 receives a data table creation request sent by terminal 102. The data table creation request carries dimension information of the table to be created, including the name of each dimension and the data source corresponding to each dimension name. Server 104 generates a table identifier based on the data table creation request and establishes an association between the table identifier and the dimension information. Server 104 creates a virtual table to be populated based on the dimension names and the table identifier in the dimension information and returns the virtual table to be populated to the requesting end corresponding to the data table creation request. Server 104 receives a population request for the virtual table to be populated, which carries the dimension name to be populated and the table identifier. Server 104 searches for the dimension information associated with the table identifier according to the association based on the population request, obtains the data source of the dimension name to be populated from the dimension information, and obtains the dimension data to be populated based on the data source. Server 104 returns the dimension data to be populated to the requesting end so that the requesting end can write the dimension data to be populated in the field corresponding to the dimension name in the virtual table. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0050] In one embodiment, such as Figure 2 As shown, a data processing method is provided, which can be applied to... Figure 1 Taking a server as an example, it can be understood that this method can also be applied to a terminal, and also to a system that includes both a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:
[0051] Step 202: Obtain the data table creation request. The data table creation request carries the dimension information of the table to be created. The dimension information of the table to be created includes the name of each dimension and the source of the dimension data corresponding to each dimension name.
[0052] In this context, a data table refers to a table in a database used to store data. The dimension information of the table to be created refers to the dimension information corresponding to the data table to be created. Dimension information refers to the relevant information about the data dimensions in the data table to be created, including the dimension name and the source of the dimension data corresponding to that name. A dimension is a description representing a data attribute in a data table. A dimension name is the attribute name of the data. Dimension data refers to the data corresponding to different attributes; it can be a dataset, and different dimension names correspond to different dimension data. The dimension data source refers to the source information of the dimension data, used to obtain the dimension data; the dimension data source can be the address where the dimension data is retrieved.
[0053] Specifically, the server receives the data table creation request sent by the requesting client and retrieves the dimension information of the table to be created carried in the request. This dimension information includes the dimension name and the source of the dimension data. The server can pre-set a dimension information list to store at least two candidate dimensions and their corresponding identifiers. The dimension information can be obtained by the server sending each candidate dimension to the requesting client based on the dimension information list. The user can then select the desired dimension from the candidate dimensions through the requesting client. Alternatively, the user can input the name of the table to be created through the requesting client, and the server will generate a data table creation request based on the dimension information and the name of the table to be created, and send it to the server.
[0054] Step 204: Generate a table identifier to be created based on the data table creation request, and establish the association between the table identifier to be created and the dimension information of the table to be created.
[0055] The table identifier to be created refers to the unique identifier corresponding to the data table to be created.
[0056] Specifically, the server generates a table identifier corresponding to the table to be created based on the data table creation request. Then, it writes the generated table identifier to a pre-defined table identifier record table. This record table includes at least one table identifier and the corresponding data table name. Alternatively, the server can generate a table identifier based on the data table name in the creation request, and then establish an association between the table identifier and the table's dimension information. This association indicates the corresponding dimension information for the table in the dimension information list.
[0057] Step 206: Based on the dimension names and the identifier of the table to be created in the dimension information of the table to be created, a virtual table to be populated is created, and the virtual table to be populated is returned to the requesting end corresponding to the data table creation request.
[0058] Among them, the virtual table to be populated refers to the data table to be populated generated based on the dimension information of the table to be created.
[0059] Specifically, the server determines the table structure of the data table to be created based on the names of each dimension in the dimension information and the identifier of the data table to be created. Then, the server generates a virtual table to be populated based on the table structure and sends the virtual table to the client corresponding to the data table creation request for display, so that the user can populate the data through the client.
[0060] Step 208: Obtain the fill request for the virtual table to be filled. The fill request carries the name of the dimension to be filled and the identifier of the table to be created.
[0061] Step 210: Based on the fill request, find the dimension information of the table to be created associated with the table to be created identifier according to the relationship, obtain the data source of the dimension to be filled corresponding to the dimension name to be filled from the dimension information of the table to be created, and obtain the dimension data to be filled based on the data source of the dimension to be filled.
[0062] Here, "fill request" refers to the data retrieval request generated when filling the virtual table to be filled. "Dimension name to be filled" refers to the dimension name corresponding to the item to be filled in the virtual table. "Dimension data to be filled" refers to the dimension data corresponding to the dimension name to be filled.
[0063] Specifically, the server obtains the population request from the requesting client for the virtual table to be populated. The population request includes the identifier of the virtual table to be created and the name of the dimension to be populated corresponding to the item to be populated in the virtual table. Based on the association between the identifier and the dimension information of the virtual table to be created, the server searches the dimension information list for the dimension information associated with the identifier. It then obtains the source of the dimension data corresponding to the name to be populated within the dimension information. The source of the dimension data can be the address of the dimension data to be populated. The server then retrieves the dimension data corresponding to the name to be populated based on the source of the dimension data.
[0064] Step 212: Return the dimension data to be populated to the requesting end so that the requesting end can write the dimension data to be populated into the item corresponding to the dimension name to be populated in the virtual table to be populated.
[0065] Specifically, the server returns the dimension data to be populated to the requesting client, which then writes the dimension data into the corresponding fields in the virtual table. The dimension information for the table to be created also includes the dimension type, indicating the data type of the data to be populated corresponding to the dimension name. Dimension types include text and non-text types. The server generates the virtual table to be populated based on the dimension information and sends it to the requesting client. If the requesting client detects that the dimension type of a field to be populated in the virtual table is non-text, it generates a population request based on the corresponding dimension name and the table identifier, sends it to the server, and retrieves the dimension data returned by the server, writing it into the corresponding field. If the requesting client detects that the dimension type of a field to be populated in the virtual table is text, it directly retrieves the user-inputted text data and writes it into the corresponding field in the virtual table. Finally, the requesting client returns all the populated data from the virtual table to the server.
[0066] In the above data processing method, a data creation request sent by the requesting end is obtained. This request carries the dimension information of the table to be created. A table identifier is generated based on the data creation request, and an association is established between the table identifier and the dimension information. Using the table identifier and the dimension names in the dimension information, the virtual table to be populated required by the requesting end can be created, and this virtual table is returned to the requesting end. A population request for the virtual table to be populated is also obtained from the requesting end. Using the dimension names and table identifier in the population request, the source of the dimension data corresponding to the dimension name to be populated can be quickly determined based on the association between the table identifier and the dimension information. The dimension data corresponding to the dimension name to be populated is then obtained based on the data source. Finally, the dimension data is returned to the requesting end so that the requesting end can write the dimension data into the fields corresponding to the dimension names in the virtual table. Since the virtual table to be populated generated based on the dimension information is not stored in the database, and the population data in the virtual table is obtained from the dimension data source, a large number of physical tables can be avoided in the server database, thus saving database resources.
[0067] In one embodiment, before obtaining a data table creation request in step 202, the data table creation request carries dimension information of the table to be created. This dimension information includes the name of each dimension and the data source corresponding to each dimension name.
[0068] Get the current dimension creation request sent by the requesting end. The current dimension creation request carries the current dimension creation information. Generate the current dimension information based on the current dimension creation information.
[0069] Obtain historical dimension information, and based on the current dimension information and historical dimension information, obtain at least one candidate dimension information. Send at least one candidate dimension information to the requesting end so that the requesting end can determine the dimension information of the table to be created from at least one candidate dimension information, and generate a data table creation request based on the dimension information of the table to be created.
[0070] Here, "Current Dimension Creation Request" refers to the dimension creation request at the current moment, used to generate dimension information. "Current Dimension Creation Information" refers to the creation information required to create dimension information. "Current Dimension Information" refers to dimension information newly created at the current moment. "Historical Dimension Information" refers to dimension information created at historical moments.
[0071] Specifically, the server retrieves the current dimension creation information from the current dimension creation request sent by the requesting client, and generates current dimension information based on this information. The current dimension information can be one or multiple. Then, the server retrieves historical dimension information and uses it along with the current dimension information as candidate dimension information, resulting in multiple candidate dimension information. The server sends each candidate dimension information to the requesting client, and the user uses the requesting client to determine the table dimension information to be created from among these candidate dimension information, and then generates a data table creation request based on that candidate table dimension information.
[0072] In one specific embodiment, after receiving a request to create current dimension information, the server can generate current dimension information based on the current dimension creation information and write the current dimension information into a pre-set dimension information list. The dimension information list includes historical dimension information and current dimension information. The dimension information list also includes the dimension information identifier, dimension name, dimension data source, and dimension type corresponding to each dimension.
[0073] The server receives a data table creation request from the client, which includes the data table name. Based on this request, the server generates a table identifier to be created and writes the table name and identifier to a pre-defined table identifier record table. This record table includes the table identifier, name, and creation time for each data table. The server then verifies the client's permissions for the data table creation request. It selects the current dimension created by the client and historical dimensions that satisfy the client's permissions from the dimension information list as candidate dimensions and returns them to the client. The client then determines the table to be created from these candidate dimensions and sends it to the server.
[0074] In this embodiment, by generating current dimension information based on current dimension creation information, and returning the current dimension information and historical dimension information as candidate dimension information to the requesting end to determine the dimension information of the table to be created, it is possible for users to customize dimension information and generate a virtual table to be filled based on the dimension information of the table to be created, thereby improving the efficiency of data table creation.
[0075] In one embodiment, the method further includes:
[0076] Get the fill requests corresponding to each item in the virtual table to be filled, and get the dimension data to be filled for each item based on the fill requests and relationships.
[0077] The data of the dimension to be filled for each item to be filled is returned to the requesting end, so that the requesting end can determine the target filling data for each item to be filled from the data of the dimension to be filled for each item to be filled, and write the corresponding target filling data into each item to be filled in the virtual table to be filled, thus obtaining the target virtual table.
[0078] The target data to be filled refers to the data written into the fields to be filled in the virtual table to be filled. The target virtual table is the data table after all fields to be filled in the virtual table to be filled have been filled with data.
[0079] Specifically, the requesting end responds to the user's click on an item in the virtual table to be populated by generating a population request for that item and sending the population request to the server. The server receives the population request sent by the requesting end and obtains the dimension name to be populated and the table identifier to be created corresponding to the item based on the population request.
[0080] The server can pre-configure a dimension relationship table containing table identifiers and dimension information. This table stores the identifiers of each table to be created and the corresponding dimension information identifiers. The server can look up the dimension information identifiers corresponding to the table to be created in the dimension relationship table and determine the corresponding dimension information from the dimension information list. The server then retrieves the data source for the dimension name to be filled from the dimension information corresponding to the table to be created, thus obtaining the data source for the dimension to be filled item. Finally, the server retrieves the dimension data corresponding to the item based on its data source and returns it to the requesting client.
[0081] The requesting client displays the dimension data to be populated, which can be a dataset. Responding to the user's selection of the dimension data, the requesting client determines the target data corresponding to the item to be populated within the dimension data. The requesting client then displays the target data in the corresponding item to be populated and returns the target data to the server. The server iterates through the population requests sent by the client for each item to be populated, sends the dimension data to be populated for each item to the client, and retrieves the target data returned by the client for each item to be populated, thus obtaining the target virtual table.
[0082] In this embodiment, by finding the dimension data to be filled corresponding to the item to be filled based on the fill request, and enabling the user to quickly determine the target fill data corresponding to the item to be filled through the request end, the security and authenticity of the data are guaranteed, and the data processing efficiency of the data table is improved.
[0083] In one embodiment, after returning the dimension data to be filled corresponding to each item to the requesting end, so that the requesting end can determine the target filling data corresponding to each item in the dimension data to be filled, and write the corresponding target filling data into each item to be filled in the virtual table to be filled, and obtain the target virtual table, the method further includes:
[0084] Retrieve the target data to be filled for each item in the target virtual table returned by the requester, and the dimension name corresponding to each item to be filled.
[0085] Based on the target filling data corresponding to each item to be filled, determine the row filling dataset corresponding to each row in the target virtual table, and write the row filling dataset corresponding to each row in the target virtual table into the preset row data record table according to the dimension name corresponding to each item to be filled.
[0086] When a request is received from the client to query row-fill data for the target virtual table, the target row-fill dataset is determined in the preset row data record table based on the query instruction, and the target row-fill dataset is returned to the client.
[0087] Here, the row filling dataset refers to the filling dataset generated from the target filling data corresponding to each row in the target virtual table. The preset row data record table refers to the pre-set record table that stores the filling datasets for each row. The row filling data query command refers to the command to query the data of a specific row in the target virtual table.
[0088] Specifically, the server retrieves the target data to be filled for each item in the target virtual table returned by the requesting client, along with the corresponding dimension names. This means the server divides the target data into rows based on the row identifiers carried by each row, resulting in individual row-level data. Then, based on the dimension names, the server performs feature-based segmentation, resulting in row-level data corresponding to feature dimensions and non-feature dimensions. Feature dimensions and non-feature dimensions refer to two pre-defined dimension categories. For example, if the target virtual table has dimension names like "Personnel," "Subject," "Customer," and "Amount," then pre-defined dimension names like "Personnel" and "Subject" are feature dimensions; while pre-defined dimension names like "Customer" and "Amount" are non-feature dimensions. The row-level data corresponding to feature dimensions can be used as a query index for querying row-level data corresponding to non-feature dimensions; that is, row-level data corresponding to non-feature dimensions can be found by using the row-level data corresponding to feature dimensions.
[0089] The server performs format conversion and storage on each feature dimension name and its corresponding row-filled data. The same format conversion and storage is performed on each non-feature dimension name and its corresponding row-filled data. Format conversion may involve serialization using JSON (Lightweight Text Data Interchange) format to obtain the serialized values corresponding to the feature dimensions (i.e., the feature row-filled dataset) and the serialized values corresponding to the non-feature dimensions (i.e., the non-feature row-filled dataset). The server uses both the feature and non-feature row-filled datasets as row-filled datasets, resulting in separate row-filled datasets for each row. The server then stores the feature and non-feature row-filled datasets from each row's corresponding row-filled dataset into a preset row data record table. This preset row data record table includes the identifier of the table to be created corresponding to the row-filled dataset. The preset row data record table can store at least one identifier of the table to be created and the row-filled datasets for each row corresponding to that identifier.
[0090] The server receives a row-filling data query instruction from the requesting client for the target virtual table. This instruction includes query data corresponding to all feature dimension names within the feature dimensions. The query data corresponding to each feature dimension name can be fill data. The server searches for the corresponding feature row-filling dataset in a pre-defined row data record table based on the query data, and then retrieves the corresponding non-feature row-filling dataset from the same table. The server sends the found non-feature row-filling dataset corresponding to the feature row-filling dataset to the requesting client, enabling the client to deserialize both datasets to obtain the dimension names and corresponding row-filling data for each dimension name in the target virtual table. Alternatively, the server can send the found non-feature row-filling dataset corresponding to the non-dimensional features to the requesting client for deserialization to obtain the corresponding row-filling data for each non-feature dimension name. For example, the dimension names in the target virtual table might be "Personnel," "Subject," "Customer," and "Amount," and the fill data for a row in the target virtual table might be "Small B," "Subject A1," "Customer 1," and "Amount 1." If the feature dimension names are "Personnel" and "Subject", and the non-feature dimension names are "Customer" and "Amount", and the query data in the row fill data query instruction is "Personnel = Xiao B" and "Subject = Subject A1", then the server will find the data corresponding to the non-feature dimension names in the preset row data record table based on the query data, which will be "Customer 1" and "Amount 1". The server will then return "Customer 1" and "Amount 1" to the requesting client for display.
[0091] In this embodiment, a row filling dataset is generated based on the filling data corresponding to each filling item. The row filling dataset includes a feature row filling dataset and a non-feature row filling dataset. The feature row filling dataset and the non-feature row filling dataset are stored in a preset row data record table. The preset row data record table can be used to quickly and accurately query a certain row of filling data in the target virtual table, thereby improving the efficiency and accuracy of data query.
[0092] In one embodiment, the dimension name includes the query index dimension name; the preset row data record table includes the dataset identifier corresponding to the populated dataset; after returning the dimension data to be populated corresponding to each item to the requesting end, so that the requesting end can determine the target populated data corresponding to each item in the dimension data to be populated corresponding to each item, and write the corresponding target populated data into each item of the virtual table to be populated, and obtain the target virtual table, the method further includes:
[0093] Get the index population data and the corresponding query index dimension name of each index population data in the target virtual table returned by the requester, and determine the dataset identifier corresponding to each index population data in the preset row data record table based on each index population data.
[0094] The data is written into the preset index filling data query data table based on the table identifier to be created, the dataset identifier corresponding to each index filling data, the query index dimension name corresponding to each index filling data, and each index filling data.
[0095] When a requesting client receives a data population query instruction for the target virtual table, the target index population data and the target dataset identifier corresponding to the target index population data are determined from the preset index population data query data table based on the data population query instruction.
[0096] Retrieve the row-filled dataset corresponding to the target dataset identifier from the preset row data record table, and return the row-filled dataset corresponding to the target dataset identifier to the requesting end.
[0097] Here, "dataset identifier" refers to the identifier corresponding to the row filling dataset in the preset row data record table. "Query index dimension name" refers to the feature dimension name. "Index filling data" refers to the filling data corresponding to the query index dimension name. "Preset index filling data query table" refers to the pre-set record table storing index filling data.
[0098] Specifically, the server retrieves the index-filled data and the corresponding query index dimension names from the target virtual table returned by the requesting client; that is, each feature dimension name and its corresponding fill data in the target virtual table. The server searches the preset row data record table for the feature row fill dataset containing each index fill data and obtains the dataset identifier corresponding to that dataset. Alternatively, the server can retrieve the dataset identifiers and their corresponding feature row fill datasets from the preset row data record table, and then split these datasets to obtain the index fill data and their corresponding query index dimension names. Finally, the server writes the table identifier to be created, the dataset identifiers corresponding to each index fill data, the query index dimension names corresponding to each index fill data, and the index fill data into the preset index fill data query data table.
[0099] The server sends a data population query instruction to the requesting client for the target virtual table. This instruction includes query data corresponding to some feature dimension names, indicating that the user selects a subset of feature dimension names from all feature dimension names for a custom dimension combination query. Based on the query data corresponding to the subset of feature dimension names, the server searches for the target index population data and the corresponding target dataset identifier in the preset index population data query table. Based on the target dataset identifier, the server determines the corresponding row population dataset in the preset row data record table and retrieves the non-feature row population dataset from the row population dataset corresponding to the target dataset identifier. There can be one or more non-feature row population datasets. The server returns the non-feature row population dataset corresponding to the target dataset identifier to the requesting client for display. For example, if the dimension names in the target virtual table are "Personnel," "Subject," "Customer," and "Amount," the corresponding population data for each dimension name in the target virtual table could be: "Small B," "Subject A1," "Customer 1," "Amount 1"; "Small C," "Subject A2," "Customer 2," "Amount 2"; "Small D," "Subject B1," "Customer 3," "Amount 3." If the feature dimension names are "Personnel" and "Subject", and the non-feature dimension names are "Customer" and "Amount", and the query data in the row fill data query instruction is "Personnel = Xiao" ("Xiao" represents the surname of the person) and "Subject = Subject A", then the server will find the data corresponding to the non-feature dimension names in the preset row data record table based on the query data as "Customer 1", "Amount 1" and "Customer 2", "Amount 2". The server will then return "Customer 1", "Amount 1" and "Customer 2", "Amount 2" to the requesting client for display.
[0100] In this embodiment, by storing the query index dimension name and the dataset identifier corresponding to the index-filled data in the target virtual table into a preset index-filled data query table, user-defined queries can be enabled. The preset index-filled data query table can quickly and accurately query partial rows of fill data in the target virtual table, thereby improving the efficiency and accuracy of data query.
[0101] In one embodiment, the data table creation request includes at least two; the method further includes:
[0102] Obtain at least two data table creation requests, and based on the at least two data table creation requests, obtain at least two dimension information of the table to be created. The dimension information of the table to be created includes the name of each dimension and the source of the dimension data corresponding to each dimension name.
[0103] Generate at least two table identifiers to be created based on at least two data table creation requests, and establish a target association relationship between at least two table identifiers to be created and the corresponding dimension information of the table to be created;
[0104] Based on the dimension names and identifiers of at least two tables to be created, at least two virtual tables to be populated are created, and the at least two virtual tables to be populated are returned to the requesting end corresponding to the at least two data table creation requests.
[0105] Retrieve the target population request, which carries the identifier of the target table to be created and the name of the target dimension.
[0106] Based on the target population request, the target dimension information of the target table to be created is found according to the target association relationship. The target dimension data source corresponding to the target dimension name is obtained from the target dimension information of the target table to be created. The target dimension data is obtained based on the target dimension data source.
[0107] The target dimension data is returned to the target request end corresponding to the target population request, so that the target request end can write the target dimension data into the virtual table to be populated corresponding to the target table to be created identifier.
[0108] Here, "target association relationship" refers to the correspondence between each table identifier to be created and its corresponding dimension information. "Target population request" refers to a population request sent from the target requesting end, which is one of at least two requesting ends. "Target table dimension information" refers to the dimension information corresponding to each target table identifier.
[0109] Specifically, the server obtains at least two data table creation requests and retrieves the dimension information of the tables to be created from each of the at least two data table creation requests. The server generates corresponding identifiers for the tables to be created based on the at least two data table creation requests, obtaining at least two identifiers, and stores these identifiers in a preset data table identifier record table. The server retrieves the corresponding dimension information identifiers from the dimension information list based on the dimension names in the dimension information of each table to be created, and then stores the corresponding dimension information identifiers for each table to be created identifier in a dimension relationship table, establishing a target association between each table to be created identifier and its corresponding dimension information.
[0110] The server creates virtual tables to be populated for each table to be created based on each table identifier and the corresponding dimension names, and returns each virtual table to be populated to the requesting client corresponding to each data table creation request.
[0111] The server receives the target population requests sent by each requesting client, and retrieves the target table identifier and target dimension name from these requests. Based on the target table identifier, the server searches for the corresponding target table dimension information in the dimension relationship table, and then retrieves the dimension information corresponding to the target dimension name from that information. The server then obtains the target dimension data source from the dimension information corresponding to the target dimension name and retrieves the target dimension data based on that source. Finally, the server returns the target dimension data to the requesting client corresponding to the target population request. The requesting client determines a population data entry from the target dimension data, writes this entry into the corresponding population item of the virtual table to be created, and sends the population data back to the server.
[0112] In one specific embodiment, such as Figure 3 The diagram illustrates a process for creating a virtual table to be populated. The server establishes a dimension set, or dimension information list, based on system data and external data. System data represents data pre-stored in the server's database, such as information on all company employees or company organization information. External data represents data not stored in the server's database, which can be data from external databases provided by the user. The server encapsulates the system data and external data according to data attributes to obtain at least one dimension, and generates candidate dimension information by obtaining the source of each dimension. Responding to a table creation request from at least one requesting client, the server sends the candidate dimension information to each requesting client, determining the dimension information for each table to be created through each requesting client. Then, the server uses a dynamic virtualization module to perform virtualization processing based on the at least one candidate table dimension information to obtain at least one virtual table to be populated. The server can generate a template for a data table to be populated based on the virtual table. Users can download this template for offline population, and then import the populated data tables in batches to generate the target virtual table on the server. The dynamic virtualization module is a module that dynamically configures the creation of virtual tables to be populated by obtaining the user's candidate table dimension information. Dynamic means that the dimension information of the table to be created obtained by the server is not fixed, but is selected by the user configuration. Virtualization refers to the process of determining the table structure corresponding to the virtual table to be populated based on the dimension information of the table to be created, and creating the virtual table to be populated based on the table structure.
[0113] In one specific embodiment, such as Figure 4 As shown, a flowchart of a data processing procedure is provided. Figure 4-a indicates the core view, which includes codes, names, source dimensions, target dimensions, an edit function bar, and populated virtual tables. Codes can be identifiers for the table to be created, names represent the virtual table name, the system to which the virtual table belongs represents the business system, source dimensions represent feature dimensions, and target dimensions represent non-feature dimensions. The core view is used for creating, editing, and displaying solutions. A solution represents a virtual table formed by combining dimension information.
[0114] Figure 4 -b indicates dimension-related information, which is introduced from data inside and outside the system to define the relevant information of the dimension: dimension code, dimension name, dimension hierarchy, dimension type, dimension source, type field and type value.
[0115] Figure 4 -c indicates a list of dimension information, which generates a list of dimension information using dimension-related information.
[0116] Figure 4 The `-d` option specifies the query method for the virtual table, which retrieves data corresponding to the target dimension based on the source dimension. Query methods include exact queries and planar queries. An exact query retrieves one record based on all source dimensions defined in the virtual table. A planar query retrieves multiple records based on a subset of source dimensions defined in the virtual table.
[0117] Figure 4 The `-e` option indicates a data import template. It generates a downloadable data table template based on the virtual table to be filled. This template can be an Excel file. After users fill in the data in the template, the completed data table is imported in batches.
[0118] In one specific embodiment, such as Figure 5 As shown, a flowchart of a data processing procedure is provided. Figure 5 -a indicates a list of dimension information, including primary key, dimension code, dimension name, dimension source, dimension type, type field, type value, and system hierarchy. The primary key stores the dimension information identifier. Dimension types include system fields and text fields. System fields indicate that the data to be filled for the corresponding dimension type can be obtained from the dimension data source, while text fields indicate that the data to be filled for the corresponding dimension type can be entered as editable text. Figure 5 -a represents a list of all dimensional information corresponding to system 01. Different systems correspond to different lists of dimensional information.
[0119] Figure 5The `-b` option indicates the mapping scheme table, which is the table containing records identifying the table to be created. This table includes the primary key, scheme code, scheme name, system affiliation, and creation time. The primary key stores the identifier of the table to be created. When a user creates a new virtual table, a new mapping scheme record is added to the mapping scheme table. Figure 5 -b indicates the two mapping schemes corresponding to system 01, namely the mapping scheme corresponding to primary key 21 and the mapping scheme corresponding to primary key 22.
[0120] Figure 5 The `-c` option indicates a mapping scheme-dimension relationship table, storing the association between the identifier of the table to be created and the dimension information of the table to be created. The mapping scheme-dimension relationship table includes a primary key, scheme identifier, dimension identifier, feature identifier, and sequence number. The scheme identifier stores the identifier of the table to be created. The dimension identifier stores the dimension information identifier. The feature identifier indicates whether it is a feature dimension. Figure 5 -c indicates that the mapping scheme with primary key 21 has dimensions corresponding to primary keys 1-7 in the dimension information list. Among them, the dimensions corresponding to primary keys 1-3 are feature dimensions, and the dimensions corresponding to primary keys 3-7 are non-feature dimensions.
[0121] Figure 5 The `-d` option indicates a mapping relationship data table, i.e., a row data record table, used to store the fill data corresponding to the non-feature dimensions that are mapped to the fill data corresponding to the feature dimensions. The mapping relationship represents the mapping relationship between feature dimensions and non-feature dimensions. The mapping relationship data table includes a primary key, scheme identifier, feature values, feature dimension serialization, and non-feature dimension serialization. The primary key stores the data identifier. The scheme identifier stores the identifier of the table to be created. The feature value stores the value obtained by encrypting the feature dimension serialization value using CRC32 (Cyclic Redundancy Check), used for precise lookup of the mapping relationship. The feature dimension stores the serialized value of the fill item corresponding to each feature dimension name. The non-feature dimension serialization stores the serialized value of the fill item corresponding to each non-feature dimension name. Figure 5 -d indicates the serialized values of the feature dimension and the non-feature dimension in the first to fifth rows (primary keys 1001-1005) of the virtual table with the identifier 21 to be created.
[0122] The mapping relationship data table is used for precise queries. The server can receive the row-filling data query instruction sent by the requesting client, serialize and encrypt the query data of all feature dimensions in the row-filling data query instruction to obtain the query value. Then, the server matches the query value with the feature values in the mapping relationship data table, obtains the serialized value of the corresponding non-feature dimension, and returns it to the requesting client.
[0123] Figure 5The `-e` option indicates the mapping relationship feature dimension table, which is the record table populated with indexed data. The mapping relationship feature dimension table includes the primary key, scheme identifier, mapping identifier, dimension code, and dimension value. The scheme identifier stores the identifier of the table to be created. The mapping identifier stores the dataset identifier. Figure 5 -e indicates that the fill data for each feature dimension name in the first row (primary key 1001) and the fill data for each feature dimension name in the second row (primary key 1002) of the virtual table to be created are specified.
[0124] In one specific embodiment, the server receives the current dimension creation request sent by the requesting client, generates current dimension information based on the current dimension creation information in the current dimension creation request, and stores the current dimension information in the dimension information list. When the server receives a data table creation request, it adds a mapping scheme record corresponding to the data table creation request to the mapping scheme table according to the data table creation. Then, the server sends the dimension information of each dimension to the requesting client according to the dimension information list and receives the dimension information of the table to be created returned by the requesting client. The server adds an association record between the identifier of the table to be created and each dimension information identifier in the mapping scheme-dimensional relationship table according to the dimension information identifier and the identifier of the table to be created. Then, the server generates a virtual table to be populated according to each dimension name and the identifier of the table to be created in the dimension information of the table to be created, and returns the virtual table to be populated to the requesting client corresponding to the data table creation request.
[0125] The server obtains the fill requests from the client for each item in the virtual table to be filled, and retrieves the corresponding dimension data to be filled based on the fill requests. It then returns this dimension data to the client. Next, the server obtains the target fill data and dimension names for each item from the client, and generates serialized feature dimension values and non-feature dimension values for each row based on these values. Finally, the server writes these serialized feature dimension values to the mapping relationship data table. Finally, the server writes the fill data corresponding to each feature dimension name for each row in the virtual table into the mapping relationship feature dimension table.
[0126] In this embodiment, the dimension information list, mapping scheme table, mapping scheme-dimensional relationship table, mapping relationship data table, and mapping relationship feature dimension table are used as base tables. Virtual tables can be generated in real time for users based on the information recorded in these base tables. When a user wants to create a new table, they only need to add a set of related records to each base table, rather than actually creating a new physical table in the database. Furthermore, when a table needs to be deleted, only the relevant data in the base tables needs to be deleted. Therefore, a large number of virtual tables can be constructed using these fixed base tables, thus saving database resources.
[0127] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0128] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.
[0129] In one embodiment, such as Figure 6 As shown, a data processing apparatus 600 is provided, including: a request acquisition module 602, an association module 604, a table generation module 606, a fill module 608, a data acquisition module 610, and a fill module 612, wherein:
[0130] The request acquisition module 602 is used to acquire a data table creation request. The data table creation request carries the dimension information of the table to be created. The dimension information of the table to be created includes the name of each dimension and the source of the dimension data corresponding to each dimension name.
[0131] The association module 604 is used to generate a table identifier to be created based on the data table creation request, and to establish the association relationship between the table identifier to be created and the dimension information of the table to be created.
[0132] Table generation module 606 is used to create a virtual table to be populated based on the names of each dimension in the dimension information of the table to be created and the identifier of the table to be created, and return the virtual table to be populated to the request end corresponding to the data table creation request.
[0133] The module 608 to be populated is used to obtain the population request of the virtual table to be populated. The population request carries the name of the dimension to be populated and the identifier of the table to be created.
[0134] The data acquisition module 610 is used to search for the dimension information of the table to be created associated with the table identifier according to the relationship based on the fill request, obtain the data source of the dimension to be filled corresponding to the dimension name to be filled from the dimension information of the table to be created, and obtain the dimension data to be filled based on the data source of the dimension to be filled.
[0135] The population module 612 is used to return the dimension data to be populated to the requesting end, so that the requesting end can write the dimension data to be populated into the item corresponding to the dimension name to be populated in the virtual table to be populated.
[0136] In one embodiment, the data processing apparatus 600 further includes:
[0137] The dimension creation unit includes: obtaining the current dimension creation request sent by the requesting end, the current dimension creation request carrying the current dimension creation information, generating current dimension information based on the current dimension creation information; obtaining historical dimension information, obtaining at least one candidate dimension information based on the current dimension information and the historical dimension information, sending at least one candidate dimension information to the requesting end, so that the requesting end can determine the table dimension information to be created from at least one candidate dimension information, and generating a data table creation request based on the table dimension information to be created.
[0138] In one embodiment, the data processing apparatus 600 further includes:
[0139] The data population unit includes: obtaining the population request corresponding to each item to be populated in the virtual table to be populated; obtaining the dimension data to be populated corresponding to each item to be populated based on the population request and the relationship between each item to be populated; returning the dimension data to be populated corresponding to each item to be populated to the requesting end, so that the requesting end can determine the target population data corresponding to each item to be populated in the dimension data to be populated corresponding to each item to be populated, and write the corresponding target population data into each item to be populated in the virtual table to be populated, thereby obtaining the target virtual table.
[0140] In one embodiment, the data processing apparatus 600 further includes:
[0141] The row data query unit includes: obtaining the target data to be filled for each item to be filled and the dimension name corresponding to each item to be filled in the target virtual table returned by the requesting end; determining the row filling dataset corresponding to each row in the target virtual table based on the target data to be filled for each item to be filled; writing the row filling dataset corresponding to each row in the target virtual table into a preset row data record table according to the dimension name corresponding to each item to be filled; when a row filling data query instruction for the target virtual table is obtained from the requesting end, determining the target row filling dataset in the preset row data record table based on the row filling data query instruction, and returning the target row filling dataset to the requesting end.
[0142] In one embodiment, the data processing apparatus 600 further includes:
[0143] The data query unit includes: obtaining the index-filled data and the corresponding query index dimension names of each index-filled data in the target virtual table returned by the requesting end; determining the dataset identifier corresponding to each index-filled data in a preset row data record table based on the index-filled data; writing the data into a preset index-filled data query data table based on the table identifier to be created, the dataset identifier corresponding to each index-filled data, the query index dimension names corresponding to each index-filled data, and the index-filled data; when a data-filled data query instruction for the target virtual table is obtained from the requesting end, determining the target index-filled data and the target dataset identifier corresponding to the target index-filled data from the preset index-filled data query data table based on the data-filled data query instruction; obtaining the row-filled dataset corresponding to the target dataset identifier in the preset row data record table, and returning the row-filled dataset corresponding to the target dataset identifier to the requesting end.
[0144] In one embodiment, the data processing apparatus 600 further includes:
[0145] The table creation unit includes: obtaining at least two data table creation requests; obtaining at least two table dimension information to be created based on the at least two data table creation requests, the table dimension information including the name of each dimension and the data source corresponding to each dimension name; generating at least two table identifiers to be created based on the at least two data table creation requests, and establishing a target association relationship between the at least two table identifiers and the corresponding table dimension information; creating at least two virtual tables to be populated based on the dimension names and the at least two table identifiers in the at least two table dimension information, and returning the at least two virtual tables to be populated to the requesting end corresponding to the at least two data table creation requests; obtaining a target population request, the target population request carrying the target table identifier and the target dimension name; searching for the target table dimension information associated with the target table identifier according to the target association relationship based on the target population request, obtaining the target dimension data source corresponding to the target dimension name from the target table dimension information, and obtaining the target dimension data based on the target dimension data source; and returning the target dimension data to the target requesting end corresponding to the target population request, so that the target requesting end writes the target dimension data into the virtual table to be populated corresponding to the target table identifier.
[0146] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0147] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores information such as table dimension information to be created. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method.
[0148] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a data processing method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.
[0149] Those skilled in the art will understand that Figure 7-8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0150] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.
[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.
[0152] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0154] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0155] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0156] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A data processing method, characterized in that, The method includes: Obtain a data table creation request, the data table creation request carrying dimension information of the table to be created, the dimension information of the table to be created including the name of each dimension and the source of the dimension data corresponding to each dimension name; Based on the data table creation request, a table identifier to be created is generated, and the association between the table identifier to be created and the dimension information of the table to be created is established. A virtual table to be populated is created based on the names of each dimension in the dimension information of the table to be created and the identifier of the table to be created. The virtual table to be populated is then returned to the requesting end corresponding to the data table creation request. Obtain a fill request for the virtual table to be filled, the fill request carrying the name of the dimension to be filled and the identifier of the table to be created; Based on the fill request, the dimension information of the table to be created associated with the table to be created is found according to the association relationship. The data source of the dimension to be filled corresponding to the dimension name to be filled is obtained from the dimension information of the table to be created. The dimension data to be filled is obtained based on the data source of the dimension to be filled. The dimension data to be filled is returned to the requesting end, so that the requesting end writes the dimension data to be filled into the item corresponding to the dimension name to be filled in the virtual table to be filled.
2. The method according to claim 1, characterized in that, Before obtaining the data table creation request, which carries dimension information of the table to be created, including the name of each dimension and the corresponding data source of the dimension, the following is also included: Obtain the current dimension creation request sent by the requesting end, the current dimension creation request carries current dimension creation information, and generate current dimension information based on the current dimension creation information; Obtain historical dimension information, and based on the current dimension information and the historical dimension information, obtain at least one candidate dimension information. Send the at least one candidate dimension information to the requesting end so that the requesting end can determine the dimension information of the table to be created from the at least one candidate dimension information, and generate the data table creation request based on the dimension information of the table to be created.
3. The method according to claim 1, characterized in that, The method further includes: Obtain the fill request corresponding to each item in the virtual table to be filled, and obtain the dimension data to be filled corresponding to each item based on the fill request corresponding to each item and the association relationship; The dimension data to be filled corresponding to each item to be filled is returned to the requesting end, so that the requesting end can determine the target filling data corresponding to each item to be filled from the dimension data to be filled corresponding to each item to be filled, and write the corresponding target filling data into each item to be filled in the virtual table to be filled, thereby obtaining the target virtual table.
4. The method according to claim 3, characterized in that, After returning the dimension data corresponding to each item to be filled to the requesting end, so that the requesting end can determine the target filling data corresponding to each item to be filled from the dimension data corresponding to each item to be filled, and write the corresponding target filling data into each item to be filled in the virtual table to be filled, and obtain the target virtual table, the process further includes: Obtain the target data to be filled for each item to be filled and the dimension name to be filled for each item to be filled in the target virtual table returned by the requesting end; Based on the target filling data corresponding to each item to be filled, determine the row filling dataset corresponding to each row in the target virtual table, and write the row filling dataset corresponding to each row in the target virtual table into a preset row data record table according to the dimension name corresponding to each item to be filled. When the requesting end receives a row-fill data query instruction for the target virtual table, the target row-fill dataset is determined in the preset row data record table based on the row-fill data query instruction, and the target row-fill dataset is returned to the requesting end.
5. The method according to claim 4, characterized in that, The dimension name includes the query index dimension name; the preset row data record table includes the dataset identifier corresponding to the populated dataset; After returning the dimension data corresponding to each item to be filled to the requesting end, so that the requesting end can determine the target filling data corresponding to each item to be filled from the dimension data corresponding to each item to be filled, and write the corresponding target filling data into each item to be filled in the virtual table to be filled, and obtain the target virtual table, the process further includes: Obtain the index population data and the query index dimension name corresponding to each index population data in the target virtual table returned by the requesting end, and determine the dataset identifier corresponding to each index population data in the preset row data record table based on each index population data; The table identifier to be created, the dataset identifier corresponding to each index fill data, the query index dimension name corresponding to each index fill data, and the index fill data are written into the preset index fill data query data table. When a requesting client receives a data population query instruction for the target virtual table, the target index population data and the target dataset identifier corresponding to the target index population data are determined from the preset index population data query data table based on the data population query instruction. Obtain the row-filled dataset corresponding to the target dataset identifier from the preset row data record table, and return the row-filled dataset corresponding to the target dataset identifier to the requesting end.
6. The method according to claim 1, characterized in that, The data table creation request includes at least two; the method further includes: Obtain the at least two data table creation requests, and based on the at least two data table creation requests, obtain at least two dimension information of tables to be created, wherein the dimension information of tables to be created includes the name of each dimension and the source of dimension data corresponding to each dimension name; Based on the at least two data table creation requests, at least two table identifiers to be created are generated, and a target association relationship is established between the at least two table identifiers to be created and the corresponding table dimension information to be created. Based on the dimension names in the dimension information of the at least two tables to be created and the identifiers of the at least two tables to be created, at least two virtual tables to be populated are established, and the at least two virtual tables to be populated are returned to the requesting end corresponding to the at least two data table creation requests respectively. Obtain the target population request, which carries the target table identifier to be created and the target dimension name; Based on the target filling request, the target dimension information of the target table to be created associated with the target table identifier is searched according to the target association relationship. The target dimension data source corresponding to the target dimension name is obtained from the target dimension information of the target table to be created. The target dimension data is obtained based on the target dimension data source. The target dimension data is returned to the target request end corresponding to the target fill request, so that the target request end writes the target dimension data into the virtual table to be filled corresponding to the target table to be created identifier.
7. A data processing apparatus, characterized in that, The device includes: The request acquisition module is used to acquire a data table creation request. The data table creation request carries dimension information of the table to be created. The dimension information of the table to be created includes the name of each dimension and the source of the dimension data corresponding to each dimension name. The association module is used to generate a table identifier to be created based on the data table creation request, and to establish an association between the table identifier to be created and the dimension information of the table to be created. The table generation module is used to create a virtual table to be populated based on the names of each dimension in the dimension information of the table to be created and the identifier of the table to be created, and return the virtual table to be populated to the request end corresponding to the data table creation request. The module to be filled is used to obtain a fill request for the virtual table to be filled, wherein the fill request carries the name of the dimension to be filled and the identifier of the table to be created; The data acquisition module is used to search for the dimension information of the table to be created associated with the table to be created identifier according to the association relationship based on the fill request, obtain the data source of the dimension to be filled corresponding to the dimension name to be filled from the dimension information of the table to be created, and obtain the dimension data to be filled based on the data source of the dimension to be filled. The population module is used to return the dimension data to be populated to the requesting end, so that the requesting end can write the dimension data to be populated into the item corresponding to the dimension name to be populated in the virtual table to be populated.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Materializing expressions within in-memory virtual column units to accelerate analytic queries
CN107851123A
Virtual expression driving method and device, electronic equipment and storage medium
CN114092611A