Method and apparatus for processing business data based on online form
By integrating an online spreadsheet module into the enterprise system, providing an Excel interface and functions, online processing of cross-system data is achieved, solving the problems of slow data processing speed and low security in existing technologies, and improving the speed and security of data processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 蒲惠智造科技股份有限公司
- Filing Date
- 2025-12-05
- Publication Date
- 2026-04-17
AI Technical Summary
In existing technologies, enterprises face complex operations when using big data or business intelligence tools for cross-system data processing, and offline processing methods suffer from data leakage and consistency issues, resulting in slow data processing speed and low security.
This paper provides a business data processing method based on online spreadsheets. By integrating an online spreadsheet module with the same user interface and functions as Excel, it enables online data processing and secure storage, ensuring data consistency and security.
It improved the speed and efficiency of business data processing, ensured data security and consistency, prevented data leakage and tampering, and enhanced the overall quality and reliability of data processing.
Smart Images

Figure CN121257500B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data management technology, and in particular to a business data processing method and apparatus based on online forms. Background Technology
[0002] In modern enterprise operations, collaborative data processing is crucial for decision-making. For example, a company might need to provide its customers with inspection reports for products and raw materials. These reports require integrating data from multiple subsystem modules, including production inspection records, production order information, sales order information, and purchase order information. This cross-system data processing requirement necessitates that companies be able to quickly, accurately, and securely combine data from different sources and generate reports that conform to a specific format.
[0003] In related technologies, enterprises typically employ two main approaches to handle such data integration needs. One approach is to use big data or business intelligence (BI) tools. While these tools are powerful, they are difficult for ordinary business personnel to master quickly, slowing down data processing. The other approach is to export business data to local Excel spreadsheets for offline processing. Although this method is flexible, the data may be accessed by unauthorized personnel during transmission and storage, posing a risk of data leakage. Furthermore, it cannot guarantee data consistency, reducing the accuracy and security of data processing. Summary of the Invention
[0004] This application provides a business data processing method and apparatus based on online forms. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general description, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.
[0005] In a first aspect, embodiments of this application provide a business data processing method based on an online form, the method comprising:
[0006] Upon receiving a data processing request submitted by a user in the business system, the system calls the preset online form module and initializes an online form file. The preset online form module is integrated into the business system and provides the same user interface and functions as Excel.
[0007] Based on data processing requirements and pre-defined standardized data interfaces, raw business data from multiple subsystem modules are obtained from various business databases in the background.
[0008] The raw business data of each subsystem module is loaded into an online spreadsheet file to obtain visualized spreadsheet data for display.
[0009] Responding to user commands based on the user interface and functions for displaying visual table data, the system processes the visual table data and generates target business data that meets data processing requirements.
[0010] Optionally, the operation command is generated by the user triggering any of the multiple operation functions provided by Excel. The multiple operation functions must support at least the following: fill, modify, copy, paste, data filter, match, aggregate, and formula calculation.
[0011] Processing visualized tabular data, including:
[0012] Semantic segmentation of operation instructions is performed to identify the functional category to which the operation instructions belong and the scope of the user's interaction with the visualized table data;
[0013] Based on the function library provided by Excel, determine the target function corresponding to the function category;
[0014] By using an objective function, the scoped data is processed to obtain visualized tabular data.
[0015] Optionally, after generating the target business data that meets the data processing requirements, the process may also include:
[0016] Responding to data output commands triggered by the user interface;
[0017] When the data output command instructs the target business data to be exported as a file in a preset format, the preset format converter is invoked;
[0018] The target business data is converted using a format converter to obtain a data file that conforms to the preset format.
[0019] Store the data files according to the preset storage path; or,
[0020] When the data output command instructs the target business data to be printed, the printing scenario is determined; based on the printing scenario, the printing operation is performed on the target business data.
[0021] Optionally, the raw business data of each subsystem module can be loaded into an online spreadsheet file to obtain visualized table data for display, including:
[0022] The raw business data of each subsystem module is preprocessed and standardized to obtain the standard business data of each subsystem module;
[0023] Based on data processing requirements, the standard business data of each subsystem module is divided into multiple data blocks;
[0024] For each data block, assign the target worksheet and starting cell coordinates in the online spreadsheet file, and generate a data block-coordinate mapping table;
[0025] By using a data block-coordinate mapping table, multiple data blocks are written one by one into an online table file to obtain visualized table data for display.
[0026] Optionally, based on data processing requirements, the standard business data of each subsystem module can be split into multiple data blocks, including:
[0027] The data processing requirements are input into a pre-trained grouping field recognition model, which is obtained by machine learning based on multiple historical data processing requirements submitted by users in the business system.
[0028] Output the explicit and implicit grouping fields corresponding to the data processing requirements;
[0029] The explicit and implicit grouping fields are arranged and combined to obtain the split dimension combination matrix. Each row of the split dimension combination matrix represents a split dimension combination.
[0030] Multiple data traversal scripts are created, using each row element of the split dimension combination matrix as the independent variable for the preset query conditions;
[0031] In the standard business data of each subsystem module, execute each data traversal script to split the standard business data of each subsystem module into multiple data blocks.
[0032] Optionally, a pre-trained grouping field recognition model is generated by following these steps:
[0033] Collect multiple historical data processing requests submitted by users in the business system;
[0034] Determine the explicit historical grouping field and implicit historical grouping field corresponding to each historical data processing requirement, and use them as the grouping label for each historical data processing requirement;
[0035] Using the group labels corresponding to each historical data processing requirement, data annotation is performed on each historical data processing requirement to obtain model training samples;
[0036] A grouping field recognition model is created using a neural network;
[0037] Input the training samples of the model into the grouping field recognition model to perform machine learning, and output the model loss value;
[0038] If the model loss value reaches its minimum, a pre-trained grouped field recognition model is obtained; or if the model loss value does not reach its minimum, the step of inputting the model training samples into the grouped field recognition model for machine learning continues until the model loss value reaches its minimum.
[0039] Optionally, the grouping field recognition model includes a transformation layer, a hidden layer, an output layer, and a loss function. The model training samples include each historical data processing requirement and the grouping label corresponding to each historical data processing requirement.
[0040] The training samples are input into the grouped field recognition model for machine learning, and the model loss value is output, including:
[0041] The input layer converts each historical data processing requirement into a word embedding vector;
[0042] The input layer converts the grouping labels corresponding to each historical data processing requirement into one-hot encoding;
[0043] The hidden layer obtains the hidden features of the word embedding vector based on the word embedding vector and the preset activation function;
[0044] The output layer obtains the probability distribution for each historical data processing requirement based on the hidden features of the word embedding vectors and the preset Softmax function;
[0045] Substitute the one-hot encoding corresponding to each historical data processing requirement and the probability distribution of each historical data processing requirement into the loss function to obtain the loss value for each historical data processing requirement.
[0046] The average loss value for each historical data processing requirement is calculated to obtain and output the model loss value; the model loss function expression for the grouping field recognition model is as follows:
[0047]
[0048] in, This represents the model loss value. This represents the total number of historical data processing needs. It is the first The loss value for historical data processing needs. It is the first The historical data processing requirement belongs to the category The probability distribution, It is the first The one-hot encoding corresponding to each historical data processing requirement, where the grouping label for each historical data processing requirement is a category. hour, Otherwise, it is 0. This represents the total number of categories in the grouping field.
[0049] Optionally, based on data processing requirements and pre-defined standardized data interfaces, raw business data from multiple subsystem modules can be obtained from various backend business databases, including:
[0050] Analyze the data processing requirements to obtain the relevant business subsystems to be matched, the names of the data objects to be matched, and the key dimension filtering conditions to be matched;
[0051] Read the preset standardized data interface registry. The preset standardized data interface registry stores the mapping relationship between tuples and metadata. The tuple consists of the business subsystem and the data object name. The metadata is used to provide a standardized interface description.
[0052] Encapsulate the business subsystem to be matched and the name of the data object to be matched into a target tuple;
[0053] From the mapping relationship, obtain the target metadata corresponding to the target tuple;
[0054] Based on the target metadata, preset standardized data interfaces, and filtering conditions for key dimensions to be matched, the system retrieves the original business data of multiple subsystem modules from the various business databases in the background.
[0055] Optionally, based on the target metadata, preset standardized data interfaces, and filtering conditions for key dimensions to be matched, the original business data of multiple subsystem modules can be obtained from the backend business databases, including:
[0056] Parse the target metadata to obtain a list of interface instances corresponding to the business subsystem to be matched;
[0057] From the preset standardized data interfaces, obtain the standardized data interfaces corresponding to each interface instance in the interface instance list, and obtain the set of standardized data interfaces corresponding to the business subsystem to be matched;
[0058] Encapsulate the business subsystem to be matched and the standardized data interface set corresponding to the business subsystem to be matched into a calling blueprint of a Map collection structure;
[0059] The blueprint is combined with the filtering conditions of the key dimensions to be matched into multiple data acquisition tasks that can be executed concurrently, and each data acquisition task corresponds to an interface instance;
[0060] Based on the backend business databases, multiple data acquisition tasks are executed to obtain the original business data of the business subsystems to be matched.
[0061] The original business data of each business subsystem to be matched is used as the original business data of multiple subsystem modules.
[0062] Secondly, embodiments of this application provide a business data processing apparatus based on an online form, the apparatus comprising:
[0063] The calling module is used to call the preset online form module and initialize an online form file when a user submits a data processing request in the business system. The preset online form module is integrated into the business system and provides the same operation interface and functions as Excel.
[0064] The data acquisition module is used to acquire raw business data from multiple subsystem modules in the background business databases according to data processing requirements and preset standardized data interfaces.
[0065] The data loading module is used to load the raw business data of each subsystem module into an online spreadsheet file to obtain visualized spreadsheet data for display.
[0066] The collaborative results generation module is used to respond to user operation commands based on the operation interface and functions for the displayed visual table data, process the visual table data, and generate target business data that meets the data processing requirements.
[0067] The technical solutions provided in this application embodiment may include the following beneficial effects:
[0068] In this embodiment, on the one hand, by integrating a pre-defined online spreadsheet module with the same interface and functions as Excel into the business system, business personnel can directly operate business data online based on the familiar Excel interface, thereby significantly improving the speed and efficiency of business data processing. On the other hand, with the online spreadsheet module integrated into the business system, all user operations are performed in a secure network environment provided by the business system, ensuring the security and consistency of business data, preventing business data leakage and tampering, and thus improving the overall quality and reliability of business data processing.
[0069] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0070] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0071] Figure 1 This is a schematic diagram of a business data processing method based on an online table provided in an embodiment of this application;
[0072] Figure 2 This is a UI diagram of a data processing requirement submission provided in an embodiment of this application;
[0073] Figure 3 This is a flowchart illustrating a process for acquiring raw business data, as provided in an embodiment of this application.
[0074] Figure 4 This is a schematic flowchart illustrating a visual table data generation process provided in an embodiment of this application;
[0075] Figure 5 This is a schematic flowchart illustrating a visual tabular data processing procedure provided in an embodiment of this application;
[0076] Figure 6 This is a schematic diagram of the method flow for training a grouped field recognition model according to an embodiment of this application;
[0077] Figure 7 This is a schematic diagram of the model structure of a grouping field recognition model provided in an embodiment of this application;
[0078] Figure 8 This is a schematic diagram of the structure of a business data processing device based on an online form provided in an embodiment of this application;
[0079] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0080] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them.
[0081] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.
[0082] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0083] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.
[0084] Currently, enterprises typically employ two main approaches to handle such data integration needs. One approach is to use big data or business intelligence (BI) tools. The other approach is to export business data to local Excel spreadsheets for offline processing.
[0085] The applicant of this application recognizes that while BI tools are powerful, ordinary business personnel find them difficult to master quickly, thus slowing down data processing. Offline processing methods, while flexible, may allow unauthorized access to data during transmission and storage, posing a risk of data breaches and failing to guarantee data consistency, thereby reducing the accuracy and security of data processing.
[0086] To address the aforementioned problems, this application provides a business data processing method and apparatus based on online spreadsheets, thereby resolving the issues present in the related technologies. In the embodiments of this application, on one hand, by integrating a preset online spreadsheet module with the same interface and functions as Excel into the business system, business personnel can directly operate business data online using the familiar Excel interface, significantly improving the speed and efficiency of business data processing. On the other hand, with the online spreadsheet module integrated into the business system, all user operations are performed within the secure network environment provided by the business system, ensuring the security and consistency of business data, preventing data leakage and tampering, and thus improving the overall quality and reliability of business data processing. Exemplary embodiments are described in detail below.
[0087] The following will be combined with the appendix Figure 1 - Appendix Figure 7 This application provides a detailed description of the business data processing method based on online tables, as provided in its embodiments. This method can be implemented using a computer program and can run on a business data processing device based on an online table using a von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application.
[0088] Please see Figure 1This document provides a flowchart illustrating a business data processing method based on an online table, as described in an embodiment of this application. Figure 1 As shown, the method in this application embodiment includes the following steps:
[0089] S101, upon receiving a data processing request submitted by a user in the business system, calls the preset online form module and initializes an online form file. The preset online form module is integrated into the business system and provides the same operation interface and functions as Excel.
[0090] The business system is an integrated software system used to manage the company's daily business operations, such as order processing, customer relationship management, and inventory management. Data processing requirements are specific data processing tasks submitted by users within the business system, such as label printing, data query and statistics, and business document generation. The pre-installed online spreadsheet module is integrated into the business system, providing an Excel-like interface and functions for processing and displaying data. The interface and functions offer a user interface and operational features similar to Excel, such as data input, editing, formula calculation, and data filtering. For example, users can use the SUM function to calculate sums and filter specific data within the spreadsheet.
[0091] In some embodiments of this application, a user submits a data processing request to a business system, such as generating a monthly sales report. The business system receives and parses the request to determine the scope and type of data to be processed. The business system then invokes an integrated online spreadsheet module to prepare for data processing. The online spreadsheet module initializes a new online spreadsheet file, sets default worksheet and cell formats, and provides an online interface and functionality similar to Excel within the business system.
[0092] For example Figure 2 As shown, Xiaoming submits a data processing request in the data processing module of his company's business system (such as an ERP system), stating "Generate this month's sales report." After Xiaoming clicks the "Start Draw Table" button, the business system receives the data processing request and automatically calls the integrated online spreadsheet module. The online spreadsheet module initializes a new online spreadsheet file, which provides the same interface and functionality as Excel. For example, the worksheet in this file contains header rows such as "Order Number," "Customer Name," "Sales Amount," and "Sales Date."
[0093] S102, based on data processing requirements and preset standardized data interfaces, obtain raw business data from multiple subsystem modules in the background business databases;
[0094] The pre-defined standardized data interfaces are pre-defined interfaces within the business system used for data interaction and access. These interfaces are APIs (Application Programming Interfaces), enabling different subsystem modules to exchange data in a standardized manner. The backend business databases are databases that store business data. These databases can be relational (e.g., MySQL, Oracle) or non-relational (e.g., MongoDB, Redis). They store all the data required for the business system to operate, such as order data, customer data, and inventory data. Multiple subsystem modules are different functional modules within the business system, each responsible for handling specific business logic and data. For example, an e-commerce system might include an order module (processing order-related data), an inventory module (managing product inventory), and a customer relationship management module (managing customer information). Raw business data is unprocessed business data directly retrieved from the database.
[0095] In some embodiments of this application, the specific process of obtaining raw business data of multiple subsystem modules from various business databases in the background, based on data processing requirements and preset standardized data interfaces, includes: parsing data processing requirements to obtain the business subsystems to be matched, the names of data objects to be matched, and the key dimension filtering conditions to be matched; reading the preset standardized data interface registry, which stores the mapping relationship between tuples and metadata, where each tuple consists of a business subsystem and a data object name, and the metadata is used to provide a standardized interface description; encapsulating the business subsystems to be matched and the names of data objects to be matched into target tuples; obtaining the target metadata corresponding to the target tuples from the mapping relationship; and obtaining the raw business data of multiple subsystem modules from various business databases in the background based on the target metadata, the preset standardized data interface, and the key dimension filtering conditions to be matched.
[0096] The business subsystems to be matched are relatively independent software systems or functional modules within an enterprise, each responsible for specific business operations. Examples include Enterprise Resource Planning (ERP) systems and Supply Chain Management (SCM) systems. The data object name to be matched is the name of the specific data entity to be retrieved from the business subsystem. Examples include sales order tables, user information tables, and product catalog tables. The key dimension filtering conditions to be matched are used to filter and locate the required data, equivalent to the WHERE clause in an SQL query. Examples include time range (date BETWEEN '2024-01-01' AND '2024-03-31'), region (region = 'East China'), and status (order status = 'Completed'). The pre-defined standardized data interface registry is a mapping configuration center, existing as a configuration file or a small database. A tuple is a combination of "business subsystem" and "data object name," such as (CRM, user information). This tuple serves as the unique key for lookup in the registry. Metadata describes the key attributes and access methods of the target data object, including: interface description (the endpoint URL of the standardized API for accessing the data object, the calling method (GET / POST), request parameter format, etc.), data structure (the format of the returned data (e.g., JSON, XML), field definitions, data types), and connection information (the required data source connection string, authentication method (e.g., API Key, OAuth token)). A predefined standardized data interface is a set of predefined, unified data access specifications. Its purpose is to eliminate differences in data format and transmission protocols between different business subsystems (e.g., CRM, ERP).
[0097] For example, a company has two main business subsystems: CRM (Customer Relationship Management) and ERP (Enterprise Resource Planning). Sales staff need a "detailed list of all completed VIP customer orders for the first quarter of 2024." The data processing request submitted by the sales staff is "Retrieve details of all completed VIP customer orders for the first quarter of 2024." By parsing this data processing request, the following can be obtained:
[0098] Business subsystems to be matched: [CRM (for obtaining VIP customer information), ERP (for obtaining order information)];
[0099] Name of the data object to be matched: [Customer Master Data, Sales Order Table];
[0100] Key dimensions to be matched: Order status = 'Completed' AND Order date BETWEEN '2024-01-01' AND '2024-03-31' AND Customer level = 'VIP'.
[0101] At this point, the system encapsulates two binary tuples: (CRM, customer master data) and (ERP, sales order table). The registry is queried to obtain the corresponding metadata. Finally, using the target metadata, preset standardized data interfaces, and filtering conditions for key dimensions to be matched, the raw business data from multiple subsystem modules is retrieved from the backend business databases.
[0102] Specifically, the process of obtaining raw business data from multiple subsystem modules in the backend business databases based on target metadata, preset standardized data interfaces, and key dimension filtering conditions to be matched includes: parsing the target metadata to obtain a list of interface instances corresponding to the business subsystems to be matched; obtaining the standardized data interfaces corresponding to each interface instance in the interface instance list from the preset standardized data interfaces to obtain a set of standardized data interfaces corresponding to the business subsystems to be matched; encapsulating the business subsystems to be matched and the set of standardized data interfaces corresponding to the business subsystems to be matched into a call blueprint of a Map collection structure; combining the call blueprint with the key dimension filtering conditions to be matched into multiple data acquisition tasks that can be executed concurrently, with each data acquisition task corresponding to an interface instance; executing multiple data acquisition tasks based on the backend business databases to obtain the raw business data of the business subsystems to be matched; and using the raw business data of each business subsystem to be matched as the raw business data of multiple subsystem modules.
[0103] The interface instance list is a collection of specific data interfaces available for invocation within a business subsystem. For example, a CRM subsystem might provide multiple instances such as "Customer Basic Information Interface" and "Customer Order List Interface." The invocation blueprint is a data structure (a Map collection) that records which specific data interfaces correspond to which business subsystem.
[0104] For example, parsing metadata determines that a customer list needs to be retrieved from the CRM subsystem and an order list from the ERP subsystem. Next, the corresponding standardized interfaces are assembled to form an interface set. The system encapsulation call blueprint is {CRM:[getVipCustomers], ERP: [getOrderDetails]}. Then, two tasks that can be executed concurrently are created:
[0105] Task 1 (CRM-oriented): Execute the getVipCustomers interface, passing in the parameters region='East China' and level='VIP'.
[0106] Task 2 (for ERP): Execute the getOrderDetails interface, passing in the parameters region='East China', orderDate=['2024-01-01', '2024-03-31'].
[0107] The system simultaneously launches Task 1 and Task 2. Task 1 retrieves basic information about VIP customers from the CRM database, while Task 2 retrieves order data based on specified criteria from the ERP database. Upon completion of both tasks, the system will obtain the two parts of raw business data: basic information and order details.
[0108] For example Figure 3 As shown, Figure 3 This application provides a flowchart illustrating a raw business data acquisition process. The system first receives data processing requests from users, such as specific queries, report generation requests, or other types of data processing tasks. Key parameters are extracted from the parsed requests, including the business subsystem to be matched, the name of the data object to be matched, and the key dimension filtering conditions to be matched. A pre-defined registry is read, containing the mapping relationship between business subsystems and data object names and metadata. This metadata provides standardized interface descriptions to guide how to access and process data. The extracted key parameters are encapsulated into a target tuple, consisting of the business subsystem and the data object name. The encapsulated target tuple is used to query the mapping relationship and obtain the corresponding target metadata. The obtained target metadata is parsed to extract a list of interface instances corresponding to the business subsystem to be matched. The standardized data interface corresponding to each instance in the pre-defined standardized data interface list is obtained, forming a set of standardized data interfaces corresponding to the business subsystem to be matched. The business subsystem to be matched and the corresponding set of standardized data interfaces are encapsulated into a call blueprint of a Map collection structure. By combining the blueprint with the key dimension filtering conditions to be matched, multiple data acquisition tasks can be created that can be executed concurrently, with each task corresponding to an interface instance. All created data acquisition tasks are executed concurrently to retrieve raw business data from multiple subsystem modules in the backend business databases.
[0109] S103: Load the original business data of each subsystem module into an online table file to obtain visualized table data for display;
[0110] In some embodiments of this application, the specific process of loading the original business data of each subsystem module into an online spreadsheet file to obtain visualized table data for display includes: preprocessing and standardizing the original business data of each subsystem module to obtain standard business data for each subsystem module; dividing the standard business data of each subsystem module into multiple data blocks according to data processing requirements; assigning a target worksheet and starting cell coordinates in the online spreadsheet file to each data block to generate a data block-coordinate mapping table; and writing multiple data blocks one by one into the online spreadsheet file through the data block-coordinate mapping table to obtain visualized table data for display.
[0111] Among them, a data block is a smaller data set obtained by splitting standard business data, and each set contains a part of the data. The target worksheet is a specific worksheet in an online spreadsheet file for storing a specific data block. The starting cell coordinates are the positions where the data block starts to be written in the target worksheet, such as cell A1 in the worksheet. The data block - coordinate mapping table is used to record the mapping relationship between each data block and its corresponding position in the online spreadsheet file. For example, for a table, it lists which cell in which worksheet each data block should be written to.
[0112] In this application, data preprocessing includes handling missing values, handling outliers, and handling duplicate values. For example, identifying null values or NULL values. The strategies include: filling with specific values (such as "unknown", "0"), filling with statistical values (such as mean, median), or directly deleting entire records with severe gaps. Identifying and handling data that does not conform to logical or statistical rules. For example, age is -1 or 200. The strategies include: correcting according to business rules (such as setting to boundary values) or directly eliminating. Identifying and deleting completely duplicate data records to ensure data uniqueness.
[0113] In this application, standardization includes date / time formatting, unit unification, and code unification. For example, converting various date formats (such as 20231027, 2023 - 10 - 27, 27 / 10 / 2023) into a standard format (such as YYYY - MM - DD). Unifying the data into standard measurement units. For example, unifying "ten thousand yuan" and "yuan" into "yuan"; unifying "kilogram" and "gram" into "gram". Standardizing classification information. For example, unifying "male", "M", "Male" into "male"; unifying "Beijing", "Beijing City", "BeiJing" into "Beijing City".
[0114] In one possible implementation, the user requirement is to "view sales performance by region and quarter". Then, "region" and "quarter" are the clear grouping fields. Using the identified grouping fields as the splitting dimensions, the complete standard business data is sliced into multiple subsets using database queries (such as querying each group of data after GROUP BY in SQL) or in - memory calculations (such as groupby in Pandas). For example, after splitting by "region" and "quarter", a list of data blocks similar to [East China - Q1 data, East China - Q2 data, South China - Q1 data,...] will be obtained. Determine a starting write position for each data block, represented by Excel coordinates, such as A1, D10, A20. Create an internal mapping table, usually a dictionary or Map structure. Traverse the mapping table and write the data in the corresponding data block in tabular form (writing the header first and then writing the data row by row) starting from this starting coordinate to obtain visual spreadsheet data for display.
[0115] In some embodiments of this application, the specific process of splitting the standard business data of each subsystem module into multiple data blocks according to data processing requirements includes: inputting the data processing requirements into a pre-trained grouping field recognition model, which is obtained by machine learning based on multiple historical data processing requirements submitted by users in the business system; outputting the explicit grouping fields and implicit grouping fields corresponding to the data processing requirements; arranging and combining the explicit grouping fields and implicit grouping fields to obtain a splitting dimension combination matrix, where each row of the splitting dimension combination matrix represents a splitting dimension combination; creating multiple data traversal scripts using each row of the splitting dimension combination matrix as the independent variable of the preset query conditions; and executing each data traversal script in the standard business data of each subsystem module to split the standard business data of each subsystem module into multiple data blocks.
[0116] The grouping field identification model is a machine learning model used to identify grouping fields in data processing requirements—that is, which fields should be used to group the data. Explicit grouping fields are fields explicitly specified in the data processing requirements for grouping, such as "region" and "quarter." Implicit grouping fields are other fields identified by the model that may be helpful for data analysis, besides the explicit grouping fields. The split dimension combination matrix is a matrix consisting of all possible combinations of explicit and implicit grouping fields, with each row representing a possible grouping method.
[0117] In one possible implementation, the user inputs data processing requirements, such as "view sales performance by region and quarter." These requirements are then fed into a pre-trained grouping field recognition model. The model outputs explicit grouping fields (e.g., "region" and "quarter") and possible implicit grouping fields (e.g., "product type"). The explicit and implicit grouping fields are then combined to generate a matrix of all possible grouping methods. Multiple data traversal scripts are created based on each row of the splitting dimension combination matrix. Each data traversal script is executed on standard business data, splitting the data according to the specified grouping fields. After splitting, multiple data blocks are obtained, each containing a subset of data. The splitting dimension combination matrix could be, for example, [region, quarter], [region, product type], or [quarter, product type].
[0118] For example Figure 4 As shown, Figure 4This application provides a schematic flowchart of a visual table data generation process. The system first receives raw data from different business systems, such as order processing, inventory management, and customer relationship management. The received raw data is cleaned and formatted to ensure consistency and accuracy. The user inputs specific data processing requirements, such as viewing sales performance by region and quarter. A pre-trained model is used to identify key grouping fields in the data processing requirements, such as region and quarter. These fields will be used for data grouping and splitting. The model outputs the identified grouping fields. Based on the identified grouping fields, a splitting dimension combination matrix is generated. Each row of this matrix represents a possible data splitting method. Based on the splitting dimension combination matrix, multiple data traversal scripts are created. These scripts will be used to extract specific data blocks from standard business data. The created data traversal scripts are run to extract multiple data blocks from the standard business data. After executing the scripts, multiple data blocks split according to different dimensions are obtained, such as sales data split by region and quarter. A target worksheet and starting cell coordinates in an online table are assigned to each data block, generating a mapping table. Based on the mapping table, each data block is written to a specified location in the online table file. Ultimately, all data blocks are correctly written into the online table, allowing users to visually view and analyze the data.
[0119] In some embodiments of this application, the specific process of generating a pre-trained grouping field recognition model includes: collecting multiple historical data processing requests submitted by historical users in the business system; determining the historical explicit grouping field and historical implicit grouping field corresponding to each historical data processing request, as the grouping label corresponding to each historical data processing request; using the grouping label corresponding to each historical data processing request to annotate each historical data processing request to obtain model training samples; creating a grouping field recognition model using a neural network; inputting the model training samples into the grouping field recognition model for machine learning, and outputting the model loss value; obtaining the pre-trained grouping field recognition model when the model loss value reaches its minimum; or continuing to execute the step of inputting the model training samples into the grouping field recognition model for machine learning when the model loss value does not reach its minimum, until the model loss value reaches its minimum.
[0120] S104 responds to user commands based on the user interface and functions to process the displayed visual table data and generate target business data that meets data processing requirements.
[0121] The operation command is generated by the user triggering any of the multiple operation functions provided by Excel. These multiple operation functions must support at least the following: fill, modify, copy, paste, filter, match, aggregate, and calculate formulas.
[0122] In some embodiments of this application, the specific process of processing visualized table data includes: semantically decomposing operation instructions to identify the function category to which the operation instructions belong and the scope data of the user's view of the visualized table data; determining the target function corresponding to the function category based on the function library provided by Excel; and using the target function to process the scope data to obtain the visualized table data.
[0123] Among these, function category refers to the function type identified after semantic decomposition of the operation command, such as data filtering and formula calculation. Scope data is the range of data affected by the operation command specified by the user in the visual table, such as the selected cell range. The Excel function library is a series of built-in functions provided by Excel for performing specific calculations and data processing tasks, such as SUM, AVERAGE, and VLOOKUP.
[0124] In one possible implementation, the user selects the sales data in column C and clicks the "AutoSum" button in the "Formulas" menu. The system recognizes the user's command as "Calculate Average," the function category as "Formula Calculation," and the scope data as the sales data in column C. Based on the function category "Formula Calculation," the system selects the target function "AVERAGE" from Excel's function library. The system uses the AVERAGE function to calculate the average of the selected range in column C. The system displays the calculated average in the cells next to the selected range, updating the visual table data.
[0125] Furthermore, after obtaining the target business data for the final data processing requirements, the final objective can be achieved based on this target business data. Specifically, this includes: responding to a data output command triggered by the user through the operation interface; when the data output command instructs the target business data to be exported as a file in a preset format, calling a preset format converter; converting the target business data to a format using the format converter to obtain a data file that conforms to the preset format; storing the data file according to a preset storage path; or, when the data output command instructs the target business data to be printed, determining the printing scenario; and performing a printing operation on the target business data based on the printing scenario.
[0126] For example Figure 5 As shown, Figure 5This application provides a flowchart illustrating a process for processing visualized tabular data. For an online spreadsheet file, the system receives and identifies user commands such as filling, modifying, copying, and pasting. The system breaks down the user's commands to identify their functional category and scope. Based on the identified functional category, the system selects the appropriate target function from its function library. The system uses the selected target function to process the data within its scope. The processed data is then updated in the visualized spreadsheet. The system can continue to respond to user commands and perform further operations on the processed visualized spreadsheet data. Finally, the system generates target business data that meets the user's needs based on the processed data. When the user triggers a data output command, the system responds. If the user selects to export a file, the system calls a format converter to convert the data to a preset format. The system uses the format converter to convert the data to the user-specified file format. The converted data file is stored in a specified location. If the user selects to print, the system performs a print operation.
[0127] In this embodiment, on the one hand, by integrating a pre-defined online spreadsheet module with the same interface and functions as Excel into the business system, business personnel can directly operate business data online based on the familiar Excel interface, thereby significantly improving the speed and efficiency of business data processing. On the other hand, with the online spreadsheet module integrated into the business system, all user operations are performed in a secure network environment provided by the business system, ensuring the security and consistency of business data, preventing business data leakage and tampering, and thus improving the overall quality and reliability of business data processing.
[0128] Please see Figure 6 This is a flowchart illustrating a grouping field recognition model training method provided in an embodiment of this application. Figure 6 As shown, the method in this application embodiment may include the following steps:
[0129] S201, collect multiple historical data processing requests submitted by users in the business system;
[0130] S202, determine the explicit historical grouping field and implicit historical grouping field corresponding to each historical data processing requirement, and use them as the grouping label for each historical data processing requirement;
[0131] S203, using the group labels corresponding to each historical data processing requirement, data annotation is performed on each historical data processing requirement to obtain model training samples;
[0132] S204, a grouping field recognition model is created using a neural network;
[0133] S205, Input the model training samples into the grouping field recognition model to perform machine learning, and output the model loss value;
[0134] Among them, for example Figure 7 As shown, the grouping field recognition model includes a transformation layer, a hidden layer, an output layer, and a loss function. The model training samples include each historical data processing requirement and the corresponding grouping label for each historical data processing requirement.
[0135] In some embodiments of this application, the specific process of inputting training samples into a grouped field recognition model for machine learning and outputting the model loss value includes: the input layer converting each historical data processing requirement into a word embedding vector; the input layer converting the grouping label corresponding to each historical data processing requirement into a one-hot encoding; the hidden layer obtaining the hidden features of the word embedding vector based on the word embedding vector and a preset activation function; the output layer obtaining the probability distribution of each historical data processing requirement based on the hidden features of the word embedding vector and a preset Softmax function; substituting the one-hot encoding corresponding to each historical data processing requirement and the probability distribution of each historical data processing requirement into the loss function to obtain the loss value of each historical data processing requirement; and averaging the loss values of each historical data processing requirement to obtain and output the model loss value; wherein, the model loss function expression of the grouped field recognition model is:
[0136]
[0137] in, This represents the model loss value. This represents the total number of historical data processing needs. It is the first The loss value for historical data processing needs. It is the first The historical data processing requirement belongs to the category The probability distribution, It is the first The one-hot encoding corresponding to each historical data processing requirement, where the grouping label for each historical data processing requirement is a category. hour, Otherwise, it is 0. This represents the total number of categories in the grouping field.
[0138] The preset activation function (such as ReLU) processes the linear transformation and outputs the hidden features. ,in The weight matrix, For bias.
[0139] The probability distribution is output through the Softmax function. If the total number of categories in the grouping field is (If the fields include "department", "time", "region", etc.), then the output layer dimension is... The predicted probability is:
[0140] in The weight matrix, For bias
[0141] in, It is a length of The vector.
[0142] S206, if the model loss value reaches the minimum, obtain the pre-trained grouping field recognition model; or if the model loss value does not reach the minimum, continue to execute the step of inputting the model training samples into the grouping field recognition model for machine learning until the model loss value reaches the minimum.
[0143] In this embodiment, by automatically converting massive historical demands into high-quality training samples, and using both explicit and implicit fields as labels, the neural network can accurately capture users' explicit intent and uncover potential analytical dimensions while minimizing the loss function. This results in a grouping field recognition model with strong generalization capabilities, capable of recommending comprehensive and accurate grouping dimensions in seconds when faced with new demands. This significantly reduces manual configuration costs and improves data splitting efficiency and insight depth.
[0144] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.
[0145] Please see Figure 8 This illustration shows a schematic diagram of the structure of a business data processing apparatus based on an online form, provided in an exemplary embodiment of this application. This online form-based business data processing apparatus can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The apparatus 1 includes a calling module 10, a data acquisition module 20, a data loading module 30, and a collaborative result generation module 40.
[0146] Module 10 is used to call the preset online form module and initialize an online form file when a user submits a data processing request in the business system. The preset online form module is integrated into the business system and provides the same operation interface and functions as Excel.
[0147] The data acquisition module 20 is used to acquire raw business data from multiple subsystem modules in the background business databases according to data processing requirements and preset standardized data interfaces.
[0148] The data loading module 30 is used to load the original business data of each subsystem module into an online table file to obtain visualized table data for display.
[0149] The collaborative result generation module 40 is used to respond to user operation instructions on the displayed visual table data based on the operation interface and functions, process the visual table data, and generate target business data that meets the data processing requirements.
[0150] It should be noted that the business data processing device based on online tables provided in the above embodiments is only illustrated by the division of the above functional modules when executing the business data processing method based on online tables. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the business data processing device based on online tables provided in the above embodiments and the business data processing method embodiments based on online tables belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.
[0151] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0152] In this embodiment, on the one hand, by integrating a pre-defined online spreadsheet module with the same interface and functions as Excel into the business system, business personnel can directly operate business data online based on the familiar Excel interface, thereby significantly improving the speed and efficiency of business data processing. On the other hand, with the online spreadsheet module integrated into the business system, all user operations are performed in a secure network environment provided by the business system, ensuring the security and consistency of business data, preventing business data leakage and tampering, and thus improving the overall quality and reliability of business data processing.
[0153] This application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the business data processing method based on online tables provided in the above-described method embodiments.
[0154] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the online table-based business data processing methods of the various method embodiments described above.
[0155] Please see Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9As shown, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.
[0156] The communication bus 1002 is used to realize the connection and communication between these components.
[0157] The user interface 1003 may include a display screen and a camera. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.
[0158] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).
[0159] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the electronic device 1000 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip, without being integrated into the processor 1001.
[0160] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage system located remotely from the aforementioned processor 1001. Figure 9 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a business data processing application based on online forms.
[0161] exist Figure 9 In the illustrated electronic device 1000, the user interface 1003 is mainly used to provide an input interface for the user and to obtain the user's input data; while the processor 1001 can be used to call the online table-based business data processing application stored in the memory 1005, and specifically perform the following operations:
[0162] Upon receiving a data processing request submitted by a user in the business system, the system calls the preset online form module and initializes an online form file. The preset online form module is integrated into the business system and provides the same user interface and functions as Excel.
[0163] Based on data processing requirements and pre-defined standardized data interfaces, raw business data from multiple subsystem modules are obtained from various business databases in the background.
[0164] The raw business data of each subsystem module is loaded into an online spreadsheet file to obtain visualized spreadsheet data for display.
[0165] Responding to user commands based on the user interface and functions for displaying visual table data, the system processes the visual table data and generates target business data that meets data processing requirements.
[0166] In one embodiment, when the processor 1001 processes the visualized tabular data, it specifically performs the following operations:
[0167] Semantic segmentation of operation instructions is performed to identify the functional category to which the operation instructions belong and the scope of the user's interaction with the visualized table data;
[0168] Based on the function library provided by Excel, determine the target function corresponding to the function category;
[0169] By using an objective function, the scoped data is processed to obtain visualized tabular data.
[0170] In one embodiment, after generating target business data that meets data processing requirements, the processor 1001 also performs the following operations:
[0171] Responding to data output commands triggered by the user interface;
[0172] When the data output command instructs the target business data to be exported as a file in a preset format, the preset format converter is invoked;
[0173] The target business data is converted using a format converter to obtain a data file that conforms to the preset format.
[0174] Store the data files according to the preset storage path; or,
[0175] When the data output command instructs the target business data to be printed, the printing scenario is determined; based on the printing scenario, the printing operation is performed on the target business data.
[0176] In one embodiment, when the processor 1001 loads the raw business data of each subsystem module into an online table file to obtain visualized table data for display, it specifically performs the following operations:
[0177] The raw business data of each subsystem module is preprocessed and standardized to obtain the standard business data of each subsystem module;
[0178] Based on data processing requirements, the standard business data of each subsystem module is divided into multiple data blocks;
[0179] For each data block, assign the target worksheet and starting cell coordinates in the online spreadsheet file, and generate a data block-coordinate mapping table;
[0180] By using a data block-coordinate mapping table, multiple data blocks are written one by one into an online table file to obtain visualized table data for display.
[0181] In one embodiment, when the processor 1001 splits the standard business data of each subsystem module into multiple data blocks according to data processing requirements, it specifically performs the following operations:
[0182] The data processing requirements are input into a pre-trained grouping field recognition model, which is obtained by machine learning based on multiple historical data processing requirements submitted by users in the business system.
[0183] Output the explicit and implicit grouping fields corresponding to the data processing requirements;
[0184] The explicit and implicit grouping fields are arranged and combined to obtain the split dimension combination matrix. Each row of the split dimension combination matrix represents a split dimension combination.
[0185] Multiple data traversal scripts are created, using each row element of the split dimension combination matrix as the independent variable for the preset query conditions;
[0186] In the standard business data of each subsystem module, execute each data traversal script to split the standard business data of each subsystem module into multiple data blocks.
[0187] In one embodiment, when the processor 1001 executes the generation of a pre-trained grouped field recognition model, it specifically performs the following operations:
[0188] Collect multiple historical data processing requests submitted by users in the business system;
[0189] Determine the explicit historical grouping field and implicit historical grouping field corresponding to each historical data processing requirement, and use them as the grouping label for each historical data processing requirement;
[0190] Using the group labels corresponding to each historical data processing requirement, data annotation is performed on each historical data processing requirement to obtain model training samples;
[0191] A grouping field recognition model is created using a neural network;
[0192] Input the training samples of the model into the grouping field recognition model to perform machine learning, and output the model loss value;
[0193] If the model loss value reaches its minimum, a pre-trained grouped field recognition model is obtained; or if the model loss value does not reach its minimum, the step of inputting the model training samples into the grouped field recognition model for machine learning continues until the model loss value reaches its minimum.
[0194] In one embodiment, when processor 1001 performs machine learning by inputting model training samples into a grouping field recognition model and outputting the model loss value, it specifically performs the following operations:
[0195] The input layer converts each historical data processing requirement into a word embedding vector;
[0196] The input layer converts the grouping labels corresponding to each historical data processing requirement into one-hot encoding;
[0197] The hidden layer obtains the hidden features of the word embedding vector based on the word embedding vector and the preset activation function;
[0198] The output layer obtains the probability distribution for each historical data processing requirement based on the hidden features of the word embedding vectors and the preset Softmax function;
[0199] Substitute the one-hot encoding corresponding to each historical data processing requirement and the probability distribution of each historical data processing requirement into the loss function to obtain the loss value for each historical data processing requirement.
[0200] The average loss value for each historical data processing requirement is calculated to obtain and output the model loss value.
[0201] In one embodiment, when the processor 1001 retrieves raw business data from multiple subsystem modules in the background business databases according to data processing requirements and preset standardized data interfaces, it specifically performs the following operations:
[0202] Analyze the data processing requirements to obtain the relevant business subsystems to be matched, the names of the data objects to be matched, and the key dimension filtering conditions to be matched;
[0203] Read the preset standardized data interface registry. The preset standardized data interface registry stores the mapping relationship between tuples and metadata. The tuple consists of the business subsystem and the data object name. The metadata is used to provide a standardized interface description.
[0204] Encapsulate the business subsystem to be matched and the name of the data object to be matched into a target tuple;
[0205] From the mapping relationship, obtain the target metadata corresponding to the target tuple;
[0206] Based on the target metadata, preset standardized data interfaces, and filtering conditions for key dimensions to be matched, the system retrieves the original business data of multiple subsystem modules from the various business databases in the background.
[0207] In one embodiment, when the processor 1001 retrieves raw business data from multiple subsystem modules in the background business databases based on target metadata, preset standardized data interfaces, and filtering conditions for key dimensions to be matched, it specifically performs the following operations:
[0208] Parse the target metadata to obtain a list of interface instances corresponding to the business subsystem to be matched;
[0209] From the preset standardized data interfaces, obtain the standardized data interfaces corresponding to each interface instance in the interface instance list, and obtain the set of standardized data interfaces corresponding to the business subsystem to be matched;
[0210] Encapsulate the business subsystem to be matched and the standardized data interface set corresponding to the business subsystem to be matched into a calling blueprint of a Map collection structure;
[0211] The blueprint is combined with the filtering conditions of the key dimensions to be matched into multiple data acquisition tasks that can be executed concurrently, and each data acquisition task corresponds to an interface instance;
[0212] Based on the backend business databases, multiple data acquisition tasks are executed to obtain the original business data of the business subsystems to be matched.
[0213] The original business data of each business subsystem to be matched is used as the original business data of multiple subsystem modules.
[0214] In this embodiment, on the one hand, by integrating a pre-defined online spreadsheet module with the same interface and functions as Excel into the business system, business personnel can directly operate business data online based on the familiar Excel interface, thereby significantly improving the speed and efficiency of business data processing. On the other hand, with the online spreadsheet module integrated into the business system, all user operations are performed in a secure network environment provided by the business system, ensuring the security and consistency of business data, preventing business data leakage and tampering, and thus improving the overall quality and reliability of business data processing.
[0215] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for processing business data based on online forms can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program for processing business data based on online forms can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.
[0216] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.
Claims
1. An online form-based business data processing method, characterized by, The method includes: Upon receiving a data processing request submitted by a user in the business system, a preset online form module is invoked and an online form file is initialized. The preset online form module is integrated into the business system and provides the same user interface and functions as Excel. The data processing requirements are analyzed to obtain the relevant business subsystems to be matched, the names of the data objects to be matched, and the key dimension filtering conditions to be matched. A preset standardized data interface registry is read, which stores a mapping relationship between tuples and metadata. Each tuple consists of a business subsystem and a data object name, and the metadata provides a standardized interface description. The business subsystems to be matched and the names of the data objects to be matched are encapsulated into target tuples. The target metadata corresponding to the target tuples is obtained from the mapping relationship. Based on the target metadata, the preset standardized data interface, and the key dimension filtering conditions to be matched, the original business data of multiple subsystem modules are obtained from the various business databases in the background. The raw business data of each subsystem module is preprocessed and standardized to obtain standard business data for each subsystem module. Based on the data processing requirements, the standard business data of each subsystem module is divided into multiple data blocks. For each data block, a target worksheet and starting cell coordinates are assigned to the online spreadsheet file to generate a data block-coordinate mapping table. The multiple data blocks are then written into the online spreadsheet file one by one using the data block-coordinate mapping table to obtain visualized table data for display. Specifically, according to the data processing requirements, the standard business data of each subsystem module is divided into multiple data blocks, including: The data processing requirements are input into a pre-trained grouping field recognition model, which is obtained through machine learning based on multiple historical data processing requirements submitted by users in the business system. The model outputs explicit and implicit grouping fields corresponding to the data processing requirements. These explicit and implicit grouping fields are then combined to obtain a splitting dimension combination matrix, where each row represents a splitting dimension combination. Multiple data traversal scripts are created using each row of the splitting dimension combination matrix as the independent variable for preset query conditions. Each data traversal script is executed within the standard business data of each subsystem module to split the standard business data of each subsystem module into multiple data blocks. The grouping field recognition model includes a transformation layer, a hidden layer, an output layer, and a loss function. The model training samples include each historical data processing requirement and the corresponding grouping label for each historical data processing requirement. In response to the user's operation instructions on the displayed visual table data based on the operation interface and functions, the visual table data is processed to generate target business data that meets the data processing requirements; the operation instructions are generated by the user triggering any one of the multiple operation functions provided by Excel, and the multiple operation functions support at least filling, modifying, copying, pasting, data filtering, matching, aggregation, and formula calculation respectively.
2. The method of claim 1, wherein, The processing of the visualized table data includes: The operation instructions are semantically segmented to identify the functional category to which the operation instructions belong and the scope data of the user's interaction with the visualized table data; Based on the function library provided by Excel, determine the target function corresponding to the function category; The target function is used to process the scope data to obtain visualized tabular data.
3. The method of claim 1, wherein, After generating the target business data that meets the data processing requirements, the process further includes: In response to the data output command triggered by the user based on the operation interface; When the data output instruction instructs the target business data to be exported as a file in a preset format, the preset format converter is invoked; The target business data is converted into a data file that conforms to the preset format using the format converter. The data file is stored according to a preset storage path; or... When the data output instruction indicates that the target business data should be printed, a printing scenario is determined; based on the printing scenario, a printing operation is performed on the target business data.
4. The method of claim 1, wherein, Generate a pre-trained grouping field recognition model by following these steps: Collect multiple historical data processing requests submitted by users in the business system; Determine the explicit historical grouping field and implicit historical grouping field corresponding to each historical data processing requirement, and use them as the grouping label for each historical data processing requirement; Using the grouping labels corresponding to each historical data processing requirement, data annotation is performed on each historical data processing requirement to obtain model training samples; A grouping field recognition model is created using a neural network; The training samples of the model are input into the grouping field recognition model to perform machine learning, and the model loss value is output. If the model loss value reaches its minimum, a pre-trained grouping field recognition model is obtained; or if the model loss value does not reach its minimum, the step of inputting the model training samples into the grouping field recognition model for machine learning continues until the model loss value reaches its minimum.
5. The method of claim 4, wherein, The step of inputting the training samples of the model into the grouping field recognition model for machine learning and outputting the model loss value includes: The transformation layer converts each historical data processing requirement into a word embedding vector. The conversion layer converts the grouping labels corresponding to each historical data processing requirement into one-hot encoding; The hidden layer obtains the hidden features of the word embedding vector based on the word embedding vector and a preset activation function; The output layer obtains the probability distribution of each historical data processing requirement based on the hidden features of the word embedding vector and the preset Softmax function; Substitute the one-hot encoding corresponding to each historical data processing requirement and the probability distribution of each historical data processing requirement into the loss function to obtain the loss value of each historical data processing requirement; The average of the loss values for each historical data processing requirement is calculated to obtain and output the model loss value; wherein, the model loss function expression for the grouping field identification model is: in, This represents the model loss value. This represents the total number of historical data processing needs. It is the first The loss value for historical data processing needs. It is the first The historical data processing requirement belongs to the category The probability distribution, It is the first The one-hot encoding corresponding to each historical data processing requirement, where the grouping label corresponding to each historical data processing requirement is a category. hour, Otherwise, it is 0. This represents the total number of categories in the grouping field.
6. The method according to claim 1, characterized in that, The process involves retrieving raw business data from multiple subsystem modules in the background business databases based on the target metadata, preset standardized data interfaces, and filtering conditions for key dimensions to be matched. Parse the target metadata to obtain a list of interface instances corresponding to the business subsystem to be matched; From the preset standardized data interfaces, obtain the standardized data interfaces corresponding to each interface instance in the interface instance list, and obtain the set of standardized data interfaces corresponding to the business subsystem to be matched; The standardized data interface set corresponding to the business subsystem to be matched is encapsulated into a call blueprint of a Map collection structure; The blueprint is combined with the filtering conditions of the key dimensions to be matched into multiple data acquisition tasks that can be executed concurrently, and each data acquisition task corresponds to an interface instance; Based on the aforementioned background business databases, execute the multiple data acquisition tasks to obtain the original business data of the business subsystem to be matched; The original business data of each of the business subsystems to be matched is used as the original business data of multiple subsystem modules.
7. A business data processing apparatus based on an online table implemented using the method according to any one of claims 1-6, characterized in that, The device includes: The calling module is used to call the preset online form module and initialize an online form file when a user submits a data processing request in the business system. The preset online form module is integrated into the business system and provides the same operation interface and functions as Excel. The data acquisition module is used to acquire raw business data from multiple subsystem modules in the background business databases according to the data processing requirements and the preset standardized data interface. The data loading module is used to load the original business data of each subsystem module into the online table file to obtain visualized table data for display. The collaborative result generation module is used to respond to the user's operation instructions on the displayed visual table data based on the operation interface and functions, process the visual table data, and generate target business data that meets the data processing requirements.
Citation Information
Patent Citations
Implementation method and device for accessing online table to third-party data
CN118536489A
Intelligent data query method and device based on large model and medium
CN120296126A