Data processing method and device, computer equipment and storage medium

Abstract

The invention relates to a data processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: displaying a data format template of a candidate interface; wherein the data format template is used for defining object format data; in response to a selection operation on the data format templates, screening out a target template from the data format templates, screening out target data from the data source according to the target template, and generating and displaying an initial data table according to the target data; according to each data format template of the initial data table, target data corresponding to each data format template in the initial data table is verified, and a verification result of the initial data table is obtained and displayed; and in response to an editing operation on the data format template in the initial data table, performing corresponding editing processing on the corresponding data format template in the initial data table and the target data to obtain a target data table in an object format. By adopting the method, the data processing efficiency and flexibility can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, and in particular, to a data processing method and device, computer equipment, a storage medium, and a computer program product. BACKGROUND

[0002] With the continuous development of big data technology, technology based on high-frequency data export has gradually become an important support for core business scenarios. This technology can quickly and accurately process and analyze massive data, meeting the diverse data application needs in different business environments. However, the current data display processing technology mainly relies on hard-coded custom interfaces and static Excel templates, which limits its flexibility and scalability to some extent. The development efficiency of hard-coded interfaces is low, and it is difficult to quickly respond to changing business needs. Static Excel templates are inefficient and prone to data format mismatches when processing complex data structures and large amounts of data, which affects the accuracy and reliability of data processing. Therefore, as the data size continues to grow, the existing technology has been unable to meet the urgent needs of businesses for diversified and multi-level data processing. SUMMARY

[0003] Therefore, it is necessary to provide a data processing method, device, computer equipment, computer-readable storage medium, and computer program product to solve the above technical problems.

[0004] In a first aspect, the present application provides a data processing method. The method comprises:

[0005] displaying a data format template of a candidate interface; wherein the data format template is used to define object format data; the data format template is obtained based on the metadata conversion of the candidate interface;

[0006] In response to a selection operation on the data format template, a target template is filtered from the data format template, target data is filtered from a data source according to the target template, and an initial data table is generated and displayed according to the target data;

[0007] According to the data format template of the initial data table, the target data corresponding to each data format template in the initial data table is verified, and the verification result of the initial data table is obtained and displayed;

[0008] In response to an editing operation on the data format template in the initial data table, the corresponding data format template and target data in the initial data table are subjected to corresponding editing processing, and the target data table of the object format is obtained.

[0009] In one of the embodiments, the target template is filtered from the data format templates in response to the selection operation on the data format templates, the target data is filtered from the data source according to the target template, and the initial data table is generated and displayed according to the target data, including:

[0010] In response to the selection operation on the target set from the preset candidate template sets, the initial template corresponding to the target set is acquired; each of the candidate template sets includes a plurality of preset data format templates;

[0011] According to the initial template, the initial data is filtered from the data source, and the set data table is generated and displayed according to the initial data;

[0012] In response to the selection operation on the data format templates, the target template is filtered from the data format templates, the target data is filtered from the data source according to the target template, and the target data is merged into the set data table to obtain the initial data table.

[0013] In one of the embodiments, the target data is filtered from the data source, and the initial data table is generated and displayed according to the target data, including:

[0014] According to the target template, the data identifier corresponding to the target data is obtained;

[0015] An association relationship between the target template and the data source is established;

[0016] The target data corresponding to the target template is filtered from the data source;

[0017] According to the target data and the data identifier, the initial data table is generated.

[0018] In one of the embodiments, after the target data corresponding to each of the data format templates in the initial data table is verified according to each of the data format templates in the initial data table, the verification result of the initial data table is obtained and displayed, further including:

[0019] In the case that the verification result is failed, the verification failure type is identified;

[0020] According to the verification failure type, the data format template modification strategy is generated, and the verification failure type and the data format template modification strategy are displayed.

[0021] In one of the embodiments, the data format templates of the candidate interface are displayed, including:

[0022] The historical operation of the target account is acquired;

[0023] According to historical operations of the target account, the data format templates of the candidate interfaces are prioritized respectively;

[0024] According to the prioritization of the data format templates, the data format templates of the candidate interfaces are displayed.

[0025] In one of the embodiments, the method further comprises:

[0026] Converting data in the target data table into object format data; wherein the data in the target data table comprises target data and a data format template;

[0027] Storing the object format data corresponding to the data in the target data table into a target address.

[0028] In a second aspect, the application further provides a data processing device. The device comprises:

[0029] A template display module for displaying data format templates of candidate interfaces; wherein the data format templates are used for defining object format data; the data format templates are obtained based on metadata of the candidate interfaces;

[0030] A data generation module for, in response to a selection operation on the data format templates, filtering a target template from the data format templates, filtering target data from a data source according to the target template, and generating and displaying an initial data table according to the target data;

[0031] A data verification module for, according to each data format template of the initial data table, verifying target data corresponding to each data format template in the initial data table respectively, obtaining and displaying a verification result of the initial data table;

[0032] A data editing module for, in response to an editing operation on a data format template in the initial data table, performing corresponding editing processing on the corresponding data format template and target data in the initial data table, obtaining the object format target data table.

[0033] In one of the embodiments, the data generation module comprises:

[0034] A set selection sub-module for, in response to a selection operation on a target set from preset candidate template sets, obtaining corresponding initial templates in the target set; wherein each candidate template set comprises a plurality of preset data format templates;

[0035] A data filtering sub-module for filtering initial data from a data source according to the initial templates, and generating and displaying a set data table according to the initial data;

[0036] The data merging submodule is configured to, in response to a selection operation on the data format template, filter a target template from the data format template, filter target data from a data source according to the target template, and merge the target data into the collection data table to obtain an initial data table.

[0037] In one of the embodiments, the data generation module comprises:

[0038] The identification acquisition submodule is configured to acquire a data identification corresponding to the target data according to the target template.

[0039] The data correlation submodule is configured to establish a correlation between the target template and the data source.

[0040] The data filtering submodule is configured to filter the target data corresponding to the target template from the data source.

[0041] The data table generation submodule is configured to generate an initial data table according to the target data and the data identification.

[0042] In one of the embodiments, the device further comprises:

[0043] The type identification module is configured to identify a type of the failed verification in the case that the verification result is a failure.

[0044] The policy generation module is configured to generate a data format template modification policy according to the type of the failed verification, and display the type of the failed verification and the data format template modification policy.

[0045] In one of the embodiments, the template display module comprises:

[0046] The data acquisition submodule is configured to acquire historical operations of a target account.

[0047] The template sorting submodule is configured to sort the data format templates of the candidate interfaces in priority according to the historical operations of the target account.

[0048] The template display submodule is configured to display the data format templates of the candidate interfaces based on the priority sorting of the data format templates.

[0049] In one of the embodiments, the device further comprises:

[0050] The data conversion module is configured to convert data in the target data table into object format data, wherein the data in the target data table comprises target data and a data format template.

[0051] The data storage module is configured to store the object format data corresponding to the data in the target data table to a target address.

[0052] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the data processing method according to any one of the embodiments of the present application when executing the computer program.

[0053] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the data processing method according to any one of the embodiments of the present application.

[0054] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program, and the computer program is executed by a processor to implement the data processing method according to any one of the embodiments of the present application.

[0055] The data processing method, device, computer device, storage medium and computer program product described above can display and select the data format templates of the candidate interfaces, filter the target data from the data source to generate an initial data table, and perform verification and editing processing on the initial data table, thereby realizing the display, filtering, data verification and editing processing of the data format templates. The method breaks the limitations of traditional hard-coded custom interfaces and static Excel templates, improves the flexibility and expansibility of data processing, quickly responds to changes in business requirements, efficiently processes complex data structures and large amounts of data, reduces data format mismatching and other problems, thereby improving the accuracy and reliability of data processing, and provides an effective solution for business data processing needs of various levels. Moreover, when editing the data format templates in the initial data table, the data format templates of the original candidate interfaces are not affected, the stability and flexibility of system data processing are ensured, and the maintainability, expansibility and data security of the system are also enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 FIG. 1 is a diagram of an application environment of the data processing method in an embodiment;

[0057] Figure 2 FIG. 2 is a flowchart of the data processing method in an embodiment;

[0058] Figure 3 FIG. 3 is a diagram of displaying each candidate template set in an embodiment;

[0059] Figure 4 FIG. 4 is a diagram of displaying the data format templates of the candidate interfaces in an embodiment;

[0060] Figure 5This is a schematic diagram illustrating the editing of a data format template in an initial data table in one embodiment;

[0061] Figure 6 This is a structural block diagram of a data processing device in one embodiment;

[0062] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] The data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 displays data format templates for candidate interfaces on terminal 102, and the user selects a target template through terminal 102. Server 104 filters the target templates and target data, and uses the data format templates to verify the number of target users, obtaining the verification result, which is then displayed to the user through terminal 102. Based on the verification result, the user edits the data format templates in the initial data packet through terminal 102. Server 104 edits the data format templates and target data in the initial data table to obtain the target data table. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0065] In one embodiment, such as Figure 2 As shown, a data processing method is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0066] Step S102: Display the data format template of the candidate interface; wherein, the data format template is used to define object format data; the data format template is obtained by converting the metadata of the candidate interface.

[0067] The candidate interfaces can include various data interfaces inside the system, external data source interfaces, and the like. Different candidate interfaces have different metadata, which contains information about the structure, type, relationship, and limitation conditions of the data involved in the interface. Through conversion processing of the metadata, a data format template suitable for defining object format data can be obtained. The data format template can include JSON Schema and the like (a language for describing JSON data format, used to define the type and format constraints of JSON data). The object format data can include JSON (JavaScript Object Notation) data and the like.

[0068] In an exemplary embodiment, the metadata of the candidate interface can also include routing data of the candidate interface, and the like. One candidate interface can include single or multiple metadata, and all the metadata of one candidate interface are collectively converted to obtain the data format template. When the data format template of the candidate interface is displayed, the displayed template can be grouped or classified according to the usage frequency of the interface, the importance of the data, and the like, so as to facilitate the user to find the required template more quickly. At the same time, for some commonly used templates, a shortcut selection method can be provided, such as setting a list of commonly used templates, and the like.

[0069] In step S104, in response to the selection operation on the data format template, a target template is filtered from the data format template. According to the target template, target data is filtered from the data source, and an initial data table is generated and displayed according to the target data.

[0070] The selection operation on the data format template can include user selection of the candidate interface by a mouse or the like, so as to obtain the target template. The target template can include JSON schema and the like. The data source can include various databases, file systems, network interfaces, and the like. The initial data table can include data identifiers and target data. The data identifiers, the target template, and the identifiers stored in the database can be different. For example, the data identifiers and the target template are named as “age”, and the identifier in the database can be stored as “age” and the like.

[0071] In an embodiment, in the initial data table, all data in the data source can be stored in the initial data table, including data that does not conform to the data format template. Then, the initial data table is verified by using the data format template, and the like.

[0072] In an example embodiment, a user can select data format templates of candidate interfaces through data, so as to obtain a target template, and filter data conforming to the target template from a data source. For example, the target template can include "class (greater than 3 and less than 6)", "name", "score (greater than 90)", and the like. The data source can include classes, names, genders, scores, and the like of all students in a school; and students with classes greater than 3 and less than 6 and scores greater than 90 are filtered from the data source to obtain corresponding classes, names, and scores. An initial data table is generated, with table headers being classes, names, and scores, and table data being classes, names, and scores of students with classes greater than 3 and less than 6 and scores greater than 90. Meanwhile, after the initial data table is generated, the table can be further adjusted according to user operations, such as adjusting column width, row height, font style, and the like, to meet different display requirements and the like.

[0073] In step S106, target data corresponding to each data format template in the initial data table is verified according to each data format template of the initial data table, and verification results of the initial data table are obtained and displayed.

[0074] The data format template includes data types, data constraints, and the like; the target data can be verified by using the data format template, so as to determine whether the target data meets requirements of the data format template and the like. The verification result can include data type errors, or failure to meet data constraints, and the like.

[0075] In an example embodiment, the target data in the initial data table can be verified according to data types and data constraints of the data format template. If the verification is successful, the target data can be continuously processed, such as being used for data analysis, report generation, and the like. If the verification fails, a step of identifying a verification failure type according to the verification result is entered. For example, when the data format template specifies that the target data is an integer type, and the actual filtered target data contains characters, a verification failure type of data type mismatch is generated. The system generates a corresponding data format template modification strategy for different verification failure types. For example, for the data type mismatch problem, the modification strategy can be data type conversion of the data source, or adjustment of the data format template to adapt to the existing data type. Through a series of operations, the accuracy and reliability of data processing can be effectively improved, and the needs of businesses for diversified and multi-level data processing can be better met.

[0076] In step S108, in response to an editing operation of the data format template in the initial data table, corresponding editing processing is performed on the data format template and the target data in the initial data table, to obtain a target data table of an object format.

[0077] The editing operation on the data format template in the initial data table can include modification of the data format template by the user, and the editing operation can include modification of the identification, constraint condition, etc. of the data format template. For example, the original data format template identification "today's data" can be modified to "yesterday's data"; the original data format template requires "greater than 0" to be modified to "greater than 0 and less than 6"; the original data type is modified from integer type to character type, etc. It can be understood that after the data format template is edited, the corresponding target data also needs to be modified, etc. After the data format template and the target data are modified, the target data table is obtained, etc.

[0078] In an exemplary embodiment, the editing operation on the data format template can include adding, deleting, modifying, etc., and specifically can include adding a data format template, for example, adding "name-gender-score" from "name-gender"; or adding a constraint condition "cannot be empty" to "name"; deleting a data format template can include deleting the score in "name-gender-score" to obtain "name-gender"; or deleting a constraint condition, etc.; modifying a data format template, etc.

[0079] In an exemplary embodiment, the user can edit the data format template in the initial data table through the terminal, such as modifying the constraint condition "greater than 10" of a certain data format template to "greater than 15". After the server receives the editing operation, the corresponding data format template and target data in the initial data table are modified accordingly. For data in the target data that originally satisfies "greater than 10" but does not satisfy "greater than 15", corresponding screening and adjustment are performed, and finally the target data table that meets the new data format template is obtained. Subsequently, the server can store the target data table to the target address, so as to facilitate subsequent data use and management, such as data analysis, report generation, etc. business scenarios, improve the efficiency and accuracy of data processing, and better meet the diversified needs of businesses.

[0080] In an example embodiment, editing the data format template in the initial data table can include editing the data format template in the initial data table, but not affecting the data format template of the original candidate interface, i.e., the data format template of the original candidate interface remains in its original state and can continue to be used for subsequent other data processing procedures. In this way, when the template is adjusted for a specific data processing task, other businesses in the system that may rely on the original template will not be disturbed. For example, in a large enterprise data processing system, there are multiple business departments using the same set of data format templates of the candidate interface, but each department may have different personalized data processing needs. When a department edits the data format template in the initial data table it uses, the business processes of other departments will not be affected by this editing operation and can still process data according to the original template, ensuring the stability and flexibility of the entire system's data processing. At the same time, this design also makes the data processing process more modular, facilitating system maintenance and expansion. When the system needs to introduce new businesses or data processing needs, a new initial data table can be quickly generated based on the data format template of the original candidate interface and personalized template editing can be performed without worrying about damaging existing businesses. In terms of data security, since the data format template of the original candidate interface is not affected, the risk of system data processing chaos due to misoperation or malicious tampering is also reduced.

[0081] In the above data processing method, the data format template of the candidate interface is displayed and selected, and the target data is filtered from the data source to generate the initial data table, and the initial data table is also verified and edited, realizing the display, filtering, data verification and editing of the data format template. This method breaks the limitations of traditional hard-coded custom interfaces and static Excel templates, improving the flexibility and expandability of data processing. It can quickly respond to changes in business needs, efficiently process complex data structures and large amounts of data, reduce data format mismatching and other problems, thereby improving the accuracy and reliability of data processing, providing an effective solution for business data processing needs of various levels. And when editing the data format template in the initial data table, the data format template of the original candidate interface is not affected, ensuring the stability and flexibility of the system data processing, and enhancing the maintainability, expandability and data security of the system.

[0082] In one embodiment, in response to the selection operation of the data format template, the target template is filtered from the data format template, the target data is filtered from the data source according to the target template, and the initial data table is generated and displayed according to the target data, including:

[0083] In response to a selection operation of a target set from the preset candidate template sets, initial templates corresponding to the target set are obtained.

[0084] Each candidate template set can include a plurality of data format templates, which can be classified and preset according to different business requirements, data types or use scenarios. For example, one candidate template set can be a template set for financial data, which includes various data format templates related to financial statements, budgets, cost accounting, etc. Another candidate template set can be a template set for human resource data, which includes data format templates for employee information, attendance records, salary and welfare, etc. The user can select a target set from the preset candidate template sets according to his actual needs. When the user completes the selection operation of the target set, the system obtains the initial templates corresponding to the target set. These initial templates will serve as the basis for subsequent target data screening and initial data table generation. Then, the system screens the target data that meets the template requirements from the data source according to the initial templates, and integrates the target data to generate an initial data table for display to the user, so as to facilitate subsequent data processing and analysis operations of the user.

[0085] In an exemplary embodiment, the preset candidate template sets can be displayed in the form of a list or a tree structure on the terminal interface, facilitating the user to intuitively view and select. The user selects a target set on the terminal through mouse clicking, keyboard inputting and other operations. For example, in an enterprise-level data processing system, after the user logs in to the system, he sees a plurality of candidate template sets classified by business department and data type on the operation interface, such as a sales data template set of the sales department, a production data template set of the production department, etc. The user selects the sales data template set of the sales department as the target set according to the business requirements of his department. After receiving the selection operation of the target set by the user, the system quickly obtains the initial templates corresponding to the target set from the storage area. Then, the system screens the data in the data source in detail according to the data format requirements in the initial templates. For example, the initial templates require that the sales date in the sales data must be within a certain time period, and the sales amount must reach a certain value, etc. The system screens the target data that meets these conditions from the sales database of the enterprise as the data source. Finally, the system arranges and formats the target data screened to generate a clear and standardized initial data table, and displays it on the terminal interface for the user to further view and analyze, such as sales performance statistics, market trend analysis, etc.

[0086] According to the initial templates, initial data is screened from the data source, and a set data table is generated and displayed according to the initial data.

[0087] The set data table can include a data format template corresponding to the target set and data corresponding to the data format template. The data of the set data table can be data processed by aggregation, summarization, etc. For example, for a sales data template set, the set data table can display summary data such as total sales and sales quantity in different time periods. When generating the set data table, the system will process the initial data filtered from the data source according to the data format requirements of the initial template. For example, if the initial template requires sales amount to be counted by quarter, the system will group the initial data by quarter and calculate the total sales amount of each quarter, etc.

[0088] In response to the selection operation on the data format template, the target template is filtered from the data format template, the target data is filtered from the data source according to the target template, and the target data is merged into the set data table to obtain the initial data table.

[0089] The selection operation on the data format template can be that the user selects the target template corresponding to the needs from the data format templates of the candidate interface based on the needs after viewing the set data table. After selecting the target template, the target data corresponding to the target template is filtered from the data source in real time, the target data is merged into the set data table, and the initial data table is obtained, etc. Specifically, the selection operation on the data format template can be realized by dragging, for example, the terminal side displays the set data table, and the other side displays the data format templates of the candidate interface. The data format templates of the candidate interface can be dragged into the set data table to add the data format templates to the set data table, etc. After adding the data format templates to the set data table, the target data can be filtered from the data source and displayed synchronously, etc. For example, the set data table includes “name-gender” and corresponding data. After dragging the score into the set data table, the table header of the set data table becomes “name-gender-score”, and the corresponding “score” data is displayed in the set data table, etc.

[0090] In this embodiment, the preset candidate template sets are classified and displayed, which facilitates the user to select the target set as needed and further obtain the initial template. The initial data is filtered from the data source to generate the set data table, and then the target data is added according to the selection operation of the user on the data format template, which not only meets the diversified data processing needs of the user, but also effectively improves the flexibility and accuracy of data processing. The user can easily select the appropriate template from different candidate template sets according to the actual business scenario, and filter and integrate the data in the data source. This way can quickly adapt to changes in business needs and efficiently process complex data structures and large amounts of data.

[0091] In one embodiment, the target data is filtered from the data source, and the initial data table is generated and displayed according to the target data, including:

[0092] According to the target template, the data identifier corresponding to the target data is obtained.

[0093] The data identifier can include the name, number, type, and other information of the data, and is used to uniquely identify each piece of data in the data source. Specifically, the data format template and the data identifier have a corresponding association relationship. The identifier in the data format template can be the identifier stored in the database, and the identifier corresponding to the target data can include the displayed data identifier, for example, the data format template stores "name", "age", etc. The data identifier corresponding to the target data is "name", "age", etc.

[0094] An association relationship between the target template and the data source is established.

[0095] The target data corresponding to the target template is filtered out from the data source.

[0096] The association relationship between the target template and the data source can include obtaining the target data source in response to the selection operation of the candidate data source, and establishing the association relationship between the target template and the target data source. For example, the target template is "student-score", the candidate data source can include "first monthly exam score", "second monthly exam score", "midterm exam score", etc., and the user can select the target data source, such as "midterm exam score". After the system receives the selection operation, the association relationship between the target template and the "midterm exam score" target data source is established. Based on this association relationship, the system can filter out the target data corresponding to the target template from the "midterm exam score" data source, etc.

[0097] According to the target data and the data identifier, an initial data table is generated.

[0098] The generation of the initial data table is a process of organizing and arranging the filtered target data according to the data identifier. The system will classify and arrange the target data according to the data identifier to ensure that the structure of the initial data table is clear and logical. For example, when the data identifier is "name", "age", and "score", the system will classify the corresponding "name", "age", and "score" data in the target data into the corresponding columns to form a neat and standardized table. The generated initial data table will be displayed on the user interface, which is convenient for users to view and further operate. Users can perform data verification, editing, etc. on the initial data table according to actual needs to meet the diversified requirements of business data processing. At the same time, the generated initial data table also provides a basis for subsequent data processing and analysis, which can better support business decision-making and development.

[0099] In this embodiment, the association between the target template and the data source is established by obtaining the data identifier according to the target template, the target data can be accurately screened out, and the initial data table can be generated by reasonably organizing and arranging the target data according to the data identifier. This way can make the data screening more accurate and ensure that the screened data meets the requirements of the target template. At the same time, the data is organized and arranged according to the data identifier, which ensures that the structure of the initial data table is clear, and facilitates user viewing and operation. In addition, the generated initial data table provides a good foundation for subsequent data verification, editing, analysis and other operations, further improves the overall efficiency and quality of data processing, and better meets the various needs of business for data processing, providing strong data support for the stable development and decision-making of business.

[0100] In one embodiment, according to the data format templates of the initial data table, the target data corresponding to each data format template in the initial data table is verified respectively, and after obtaining and displaying the verification result of the initial data table, it further includes:

[0101] In the case of a failed verification result, the verification failure type is identified.

[0102] Among them, all data in the data source can be stored in the initial data table, including data that does not meet the data format template. Then the initial data table is verified using the data format template. The verification result failure can include data format mismatch, data missing, data duplication, data exceeding the range, etc. In the case of verification failure, the target data of the verification failure can be identified, for example, by bold, underline, background color, etc. At the same time, based on each verification failure target data in the data format template, the verification failure type is identified, etc.

[0103] According to the verification failure type, generate a data format template modification strategy, and display the verification failure type and the data format template modification strategy.

[0104] Among them, the data format template modification strategy can be generated based on the verification failure type, for example, when the verification failure type is data format mismatch, the strategy can be to modify the format requirement in the data format template to match the data format in the data source; if it is data missing, a default value or a prompt for the user to supplement the missing data can be set in the template; for data duplication, a deduplication rule can be added; for data exceeding the range, the range limit in the template can be adjusted. Displaying the verification failure type and the modification strategy can help users clearly understand the data problem, and modify the data format template according to the strategy, so that the initial data table passes the verification, improves the accuracy and reliability of data processing. After modification, the initial data table can be verified again to ensure that the data meets the requirements, providing high-quality data support for subsequent data processing and business decision-making, etc.

[0105] In this embodiment, in the case of a failed check, the type of failed check is identified, and a data format template modification strategy is generated, thereby achieving effective processing of data check problems. By specifying the type of failed check, the problem with the data can be accurately located, and the targeted data format template modification strategy provides a feasible way to solve these problems.

[0106] In one embodiment, the data format templates of the candidate interface are displayed, including:

[0107] The historical operations of the target account are obtained.

[0108] The historical operations can include user selection operations on data format templates, filtering operations on data sources, editing operations on data tables, and the like. By obtaining the historical operations of the target account, the system can understand the user's past data processing habits and preferences. For example, if the user often selects a set of financial data templates and focuses on certain specific data fields in these templates, the system can provide template options that better meet the user's needs when displaying the data format templates of the candidate interface based on these historical operation records.

[0109] According to the historical operations of the target account, the data format templates of each candidate interface are prioritized.

[0110] In an exemplary embodiment, the preferences and usage habits of the target account for data format templates in historical operations can be analyzed, such as the types of templates the user often selects, the editing methods of commonly used templates, and the like. Based on these preferences and usage habits, each data format template is assigned a corresponding priority weight. For example, if the user frequently uses a certain type of sales data template for sales performance statistics, the priority weight of this type of template will be higher. Then, the data format templates of each candidate interface are sorted in order of priority from high to low. After sorting, the sorted data format templates of the candidate interface are displayed to the user. This display method can be in the form of a list, with high-priority templates at the front to facilitate the user to quickly find the templates they may need; or in a tree structure, with high-priority templates placed in a more visible position. In this way, the user's historical operations can be used to provide more personalized data format templates that better meet the user's needs, improve the user's efficiency in selecting appropriate templates, further optimize the user's data processing experience, and enable the user to more conveniently carry out data processing and analysis work, and better meet business needs.

[0111] Based on the priority ranking of each data format template, the data format templates of the candidate interface are displayed.

[0112] In this embodiment, by obtaining the historical operations of the target account and prioritizing the data format templates of the candidate interfaces, the user's data processing preferences and habits can be accurately grasped. Displaying the sorted templates to the user greatly improves the efficiency of finding and selecting the appropriate template, reducing the time cost of filtering among numerous templates. This not only optimizes data processing, but also makes the entire data processing process more efficient and convenient, better adapts to diverse business needs, and further improves the practicality of the data processing system.

[0113] In an embodiment, the method further comprises:

[0114] Converting the data in the target data table into object format data; wherein the data in the target data table includes target data and a data format template.

[0115] Storing the object format data corresponding to the data in the target data table to a target address.

[0116] The object format data can include JSON data, XML data, etc. Converting the data in the target data table into object format data facilitates data transmission and interaction between different systems or modules. After storage in the target address, these data can be conveniently called and used by other programs or systems. For example, in an enterprise-level data processing system, the object format data stored in the target address can be used by a data analysis module for in-depth data analysis, or by a report generation module for generating various reports. In addition, storing data in object format can also improve data maintainability and scalability, facilitating subsequent data modification, updating, etc. When business requirements change, the stored object format data can be easily adjusted and optimized to meet new business requirements.

[0117] In an exemplary embodiment, converting the data in the target data table into object format data can use a special data conversion tool or write a custom conversion program. For example, using the JSON library in Python can convert the data in the target data table into JSON format data. During the conversion process, the integrity and accuracy of the data must be ensured to avoid data loss or errors. At the same time, the selection of the target address is also important, and the security, reliability of storage and the convenience of data access should be considered. Local disk, network storage device or cloud storage service, etc. can be selected as the target address. After storage, in order to facilitate data management and searching, corresponding indexes or metadata can also be established to record relevant information of the data, such as data source, generation time, data type, etc. In this way, when the data is needed, the required data can be quickly located and obtained, further improving the efficiency of data processing and use.

[0118] In this embodiment, by converting the data in the target data table into object format data and storing it to the target address, efficient transmission and interaction of data between different systems or modules is realized, and the maintainability and scalability of the data are improved. This processing mode makes the data better adapt to the needs of different business scenarios and plays an important role in enterprise-level data processing environment.

[0119] In an exemplary embodiment, the data processing method can further include showing each candidate template set to the user as shown in Figure 3 The user can select a candidate template set by clicking it, show each data format template included in the candidate template set by clicking the current configuration, change each data format template included in the candidate template set by clicking the configuration, etc. A candidate template set can also be created and configured by the add icon below each candidate data template set. It can be understood that any candidate template can not be selected, and the target template can be directly selected from the data format templates of each candidate interface. In the case of selecting a target set or not selecting a candidate template set, the data format template diagram can be shown to the user as shown in Figure 4 The left side is the initial data table, and the right side is the data format template. The target template can be selected from the data format templates on the right side and added to the initial data table. After adding a new data format template to the initial data table, a target data corresponding to the added data format template will also be generated. The right side of the initial data table contains an add icon. By responding to the add icon, an empty data format template can be added to the initial data table, and a target template can be further selected from the right side and added to the empty data format template, etc. The left side of the data format template on the right side is the data identifier in the database, and the right side is the data identifier added to the initial data table, etc. In addition, the user can select the data identifier in the initial data table to constrain the data in the corresponding column. As shown in Figure 5 The cell name, cell width, default value (the default value can be empty, etc.), data type, and data range can be modified. It can be understood that when the data in the corresponding column is constrained, only the constraint in the corresponding column in the initial data table is changed, and the constraint in the data format template of the candidate interface is not changed, etc.

[0120] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately executed with other steps or steps or stages in other steps.

[0121] Based on the same inventive concept, the embodiments of the present application also provide a data processing apparatus for implementing the above-mentioned data processing method. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, so the specific limitations in one or more data processing apparatus embodiments provided below can refer to the limitations of the data processing method described above, which will not be repeated here.

[0122] In one embodiment, as shown in Figure 6 A data processing apparatus 400 is provided, comprising a template display module 401, a data generation module 402, a data verification module 403 and a data editing module 404, wherein:

[0123] The template display module is configured to display a data format template of a candidate interface; wherein the data format template is used to define object format data; the data format template is converted based on the metadata of the candidate interface;

[0124] The data generation module is configured to, in response to a selection operation on the data format template, filter a target template from the data format template, filter target data from a data source according to the target template, and generate and display an initial data table according to the target data;

[0125] The data verification module is configured to verify the target data corresponding to each data format template in the initial data table according to each data format template of the initial data table, and obtain and display a verification result of the initial data table;

[0126] The data editing module is configured to, in response to an editing operation on the data format template in the initial data table, perform corresponding editing processing on the corresponding data format template and the target data in the initial data table, and obtain an object format target data table.

[0127] In one embodiment, the data generation module comprises:

[0128] The collection selecting submodule is configured to, in response to a selection operation of a target collection from a plurality of preset candidate template collections, acquire an initial template corresponding to the target collection; each candidate template collection includes a plurality of preset data format templates.

[0129] The data filtering submodule is configured to filter initial data from the data source according to the initial template, and generate and display a collection data table according to the initial data.

[0130] The data merging submodule is configured to, in response to a selection operation of a data format template, filter a target template from the data format template, filter target data from the data source according to the target template, and merge the target data into the collection data table to obtain an initial data table.

[0131] In one of the embodiments, the data generation module comprises:

[0132] The identification acquisition submodule is configured to acquire a data identification corresponding to the target data according to the target template.

[0133] The data association submodule is configured to establish an association relationship between the target template and the data source.

[0134] The data filtering submodule is configured to filter target data corresponding to the target template from the data source.

[0135] The data table generation submodule is configured to generate an initial data table according to the target data and the data identification.

[0136] In one of the embodiments, the device further comprises:

[0137] The type identification module is configured to identify a type of the verification failure in the case that the verification result is a failure.

[0138] The policy generation module is configured to generate a data format template modification policy according to the type of the verification failure, and display the type of the verification failure and the data format template modification policy.

[0139] In one of the embodiments, the template display module comprises:

[0140] The data acquisition submodule is configured to acquire historical operations of the target account.

[0141] The template sorting submodule is configured to sort the data format templates of the candidate interfaces in priority according to the historical operations of the target account.

[0142] The template display submodule is configured to display the data format templates of the candidate interfaces based on the priority sorting of the data format templates.

[0143] In one of the embodiments, the device further comprises:

[0144] a data conversion module, configured to convert data in a target data table into object format data; wherein the data in the target data table comprises target data and a data format template;

[0145] a data storage module, configured to store the object format data corresponding to the data in the target data table to a target address.

[0146] The modules in the data editing module device can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be called and executed by the processor to perform operations corresponding to the modules.

[0147] In one embodiment, a computer device, which can be a terminal, has an internal structure as shown in Figure 7 The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program is executed by the processor to implement a data processing method. The display unit of the computer device is configured to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0148] Those skilled in the art can understand that Figure 7 The structure shown in the above description is only a block diagram of part of the structure related to the scheme of the present application, and does not limit the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of the country and region.

[0150] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to a memory, database or other medium used in the embodiments provided by the present application can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (Read-Only Memory, ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive memory (ReRAM), a magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), a ferroelectric memory (Ferroelectric Random Access Memory, FRAM), a phase change memory (Phase Change Memory, PCM), a graphene memory, etc. The volatile memory can include a random access memory (Random Access Memory, RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0151] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

[0152] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A data processing method, characterized by, The method comprises: displaying data format templates of candidate interfaces; wherein the data format templates are used to define object format data; the data format templates are converted based on metadata of the candidate interfaces; in response to a selection operation on the data format templates, filtering target templates from the data format templates, filtering target data from a data source according to the target templates, and generating and displaying an initial data table according to the target data; according to each data format template of the initial data table, respectively verifying target data corresponding to each data format template in the initial data table to obtain and display a verification result of the initial data table; in response to an editing operation on a data format template in the initial data table, performing corresponding editing processing on the corresponding data format template and target data in the initial data table to obtain a target data table in the object format.

2. The method of claim 1, wherein, In response to a selection operation on the data format templates, filtering target templates from the data format templates, filtering target data from a data source according to the target templates, and generating and displaying an initial data table according to the target data, comprising: in response to a selection operation on a target set from a plurality of preset candidate template sets, obtaining an initial template corresponding to the target set; wherein each candidate template set comprises a plurality of preset data format templates; filtering initial data from a data source according to the initial template, and generating and displaying a set data table according to the initial data; in response to a selection operation on the data format templates, filtering target templates from the data format templates, filtering target data from a data source according to the target templates, and merging the target data into the set data table to obtain an initial data table.

3. The method of claim 1, wherein, Filtering target data from a data source and generating and displaying an initial data table according to the target data, comprising: obtaining a data identifier corresponding to the target data according to the target template; establishing an association relationship between the target template and the data source; filtering target data corresponding to the target template from the data source; generating an initial data table according to the target data and the data identifier.

4. The method of claim 1, wherein, After the respective target data corresponding to each data format template in the initial data table is verified according to each data format template of the initial data table to obtain and display the verification result of the initial data table, the method further comprises: in the case that the verification result is failed, identifying a verification failure type according to the verification result; generating a data format template modification strategy according to the verification failure type, and displaying the verification failure type and the data format template modification strategy.

5. The method of claim 1, wherein, The method further comprises: converting data in the target data table into object format data; wherein the data in the target data table comprises target data and a data format template; according to the historical operations of the target account, respectively prioritizing the data format templates of each candidate interface; based on the priority of each data format template, displaying the data format templates of the candidate interfaces.

6. The method of claim 1, wherein, The method further comprises: converting data in the target data table into object format data; wherein the data in the target data table comprises target data and a data format template; Corresponding object format data of data in the target data table is stored to a target address.

7. A data processing apparatus, characterized by, The device comprises: A template display module is configured to display a data format template of a candidate interface, wherein the data format template is used to define object format data, and the data format template is converted based on metadata of the candidate interface. A data generation module is configured to, in response to a selection operation on the data format template, filter a target template from the data format template, filter target data from a data source according to the target template, and generate and display an initial data table according to the target data. A data verification module is configured to, according to each data format template of the initial data table, verify target data corresponding to each data format template in the initial data table respectively, and obtain and display a verification result of the initial data table. A data editing module is configured to, in response to an editing operation on a data format template in the initial data table, perform corresponding editing processing on the corresponding data format template and target data in the initial data table, and obtain the target data table in the object format.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images