Data processing method, device and equipment and computer readable storage medium

Abstract

The embodiment of the invention provides a data processing method and device, equipment and a computer readable storage medium. The method comprises the steps that in response to a setting instruction acting on a data query interface, query information is determined; when the query information indicates that the to-be-queried data table is a first wide table, in response to a query instruction acting on the data query interface, querying in the first wide table to obtain target data corresponding to the query information; wherein the number of the first service fields in the first wide table is equal to the maximum value in the number of the first attribute fields corresponding to the event fields. According to the embodiment of the invention, the data redundancy of the wide table model can be reduced, and the data query efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer processing, and relates to but is not limited to data processing methods, devices, equipment and computer-readable storage media. Background Art

[0002] In the current context of rapid iteration and development of the Internet, product iterations are getting faster and faster, operational activities are becoming more intensive, and cross-business data analysis is becoming more numerous and complex. Therefore, the problems of traditional data warehouses are constantly highlighted.

[0003] At present, in order to solve the problems of large scale and complex query of traditional data warehouses, wide table modeling is usually used for data processing. However, while wide table models improve data usability and query performance, with the continuous iterative development of business, large amounts of data will lead to data redundancy in wide table models, thus interfering with the problem of dimensional explosion, and further affecting data query efficiency. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, device, and computer-readable storage medium, which can reduce data redundancy in a wide table model and improve data query efficiency.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0007] In response to a setting instruction acting on the data query interface, determining query information;

[0008] When the query information indicates that the data table to be queried is the first wide table, in response to the query instruction acting on the data query interface, a query is performed in the first wide table to obtain the target data corresponding to the query information; wherein the number of first business fields in the first wide table is equal to the maximum value of the number of first attribute fields corresponding to each event field.

[0009] In a second aspect, an embodiment of the present application provides a data processing device, including a determination unit and a query unit; wherein:

[0010] The determining unit is used to determine the query information in response to the setting instruction acting on the data query interface;

[0011] The query unit is used to query in the first wide table in response to a query instruction acting on the data query interface when the query information indicates that the data table to be queried is a first wide table, so as to obtain the target data corresponding to the query information; wherein the number of first business fields in the first wide table is equal to the maximum value of the number of first attribute fields corresponding to each event field.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the data processing method as described above.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, and when the computer program is executed by at least one processor, the data processing method as described above is implemented.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer program instructions, which enable a computer to execute the above-mentioned data processing method.

[0015] In a sixth aspect, an embodiment of the present application provides a computer program which, when executed on a computer, enables the computer to perform the above-mentioned data processing method.

[0016] The embodiments of the present application provide a data processing method, apparatus, device and computer-readable storage medium, which may include: first, determining query information in response to a setting instruction acting on a data query interface; then, when the query information indicates that the data table to be queried is a first wide table, in response to a query instruction acting on the data query interface, querying in the first wide table to obtain the target data corresponding to the query information. Among them, the number of first business fields in the first wide table is equal to the maximum value of the number of first attribute fields corresponding to each event field. Since the first attribute fields between different event fields do not conflict with each other, different event fields can use the same first business field without affecting the data. In this way, by sharing a first business field in the first wide table with the first attribute fields of different event fields, the dimension of the wide table can be effectively reduced, thereby reducing the data redundancy of the wide table, effectively solving the problem of dimensionality explosion, and thus improving the query efficiency of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0019] Figure 1 A schematic diagram of the structure of an optional traditional data warehouse model provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the structure of an optional wide table data warehouse model provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of an optional data processing process provided in an embodiment of the present application Figure 1 ;

[0022] Figure 4A A schematic diagram of an optional data query interface provided in an embodiment of the present application Figure 1 ;

[0023] Figure 4B A schematic diagram of an optional data query interface provided in an embodiment of the present application Figure 2 ;

[0024] Figure 4C A schematic diagram of an optional data query interface provided in an embodiment of the present application Figure 3 ;

[0025] Figure 4D Schematic diagram 4 of an optional data query interface provided in an embodiment of the present application;

[0026] Figure 4E A schematic diagram of an optional data query interface provided in an embodiment of the present application Figure 5 ;

[0027] Figure 4F A schematic diagram of an optional data query interface provided in an embodiment of the present application Figure 6 ;

[0028] Figure 4G A schematic diagram of an optional data query interface provided in an embodiment of the present application Figure 7;

[0029] Figure 4H Schematic diagram 8 of an optional data query interface provided in an embodiment of the present application;

[0030] Fig. 4I A schematic diagram of an optional data query interface provided in an embodiment of the present application Figure 9 ;

[0031] Figure 5 A schematic diagram of an optional data processing process provided in an embodiment of the present application Figure 2 ;

[0032] Figure 6 A schematic diagram of an optional management platform interface provided in an embodiment of the present application;

[0033] Figure 7 A schematic diagram of an optional import setting pop-up window provided in an embodiment of the present application;

[0034] Fig. 8A A schematic diagram of an optional data viewing interface provided in an embodiment of the present application Figure 1 ;

[0035] Figure 8B A schematic diagram of an optional data viewing interface provided in an embodiment of the present application Figure 2 ;

[0036] Fig. 9 A schematic diagram of an optional data processing process provided in an embodiment of the present application Figure 3 ;

[0037] Fig.10 A schematic diagram of the structure of an optional wide table model provided in an embodiment of the present application;

[0038] Fig.11 A schematic diagram of the composition structure of an optional data processing device provided in an embodiment of the present application;

[0039] Fig.12 A schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0042] In the following description, reference is made to “some embodiments”, “this embodiment”, “embodiments of the present application” and examples, etc., which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0044] In the embodiments of the present application, the term "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, object A and / or object B may represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0045] In the context of the rapid iteration and development of the current Internet, product iteration is getting faster and faster, while operational activities are becoming more intensive. Cross-business data analysis is becoming more numerous and complex, which leads to the continuous emergence of problems in traditional data warehouses, including the following:

[0046] 1) Due to the hierarchical construction of traditional data warehouses, the scale of data tables is getting larger and larger;

[0047] 2) Due to the large number of data tables, users may encounter inconsistent definitions and complex queries when using table calculation indicators;

[0048] 3) Since traditional data warehouse construction relies on data research and development, the development cycle is long and cannot keep up with the rapidly iterating business.

[0049] Wide table modeling can solve some of the problems faced by traditional data warehouses to a certain extent, especially when facing the high-speed iteration of the Internet and complex data analysis needs.

[0050] Specifically, wide table modeling is a data modeling method that aims to merge multiple source data tables into a data table with a larger width for easier query and analysis. It is often used in the field of data warehouse and business intelligence to support complex data analysis and reporting needs. In wide table modeling, a common practice is to extract relevant dimension data and measure data from different source tables and combine them into a large fact table. The fact table contains the keys of multiple dimensions and the corresponding measure data. The dimension data describes the characteristics and attributes of the data, while the measure data represents quantifiable business indicators such as sales, quantity, profit, etc. The advantage of wide table modeling is that it simplifies the complexity of data query and analysis. By integrating data into a wide table, users can directly perform query and analysis operations on the wide table without having to care about the source of the data and the associated logic. This can improve query performance and user experience and reduce the workload of development and maintenance. Wide table modeling can also provide more flexibility and scalability. In a wide table, new dimensions and measures can be added as needed to meet changing business needs. At the same time, wide tables can be optimized through technologies such as indexing and partitioning to improve query performance and data access efficiency.

[0051] In general, wide table modeling is mainly to streamline dozens of data warehouse tables under a single business theme into one, which is very suitable for fast-iteration data-driven businesses and is an important trend in the current data field. Its following advantages can greatly solve the problems of traditional data warehouses:

[0052] 1) Reduce data warehouse stratification and significantly reduce the number of tables;

[0053] 2) The caliber is unified, which is easier to understand and use;

[0054] 3) Avoid the large amount of redundancy between layers in traditional data warehouses, saving storage costs.

[0055] In summary, wide table modeling is a data modeling method that integrates multiple source data tables into a large fact table, aiming to simplify the complexity of data query and analysis and improve performance and flexibility. It is widely used in the fields of data warehouse and business intelligence, providing users with convenient and efficient data access and analysis capabilities.

[0056] However, while the wide table model improves data usability and query performance, it also brings some challenges. According to tests, a wide table with 1,500 columns is 14% to 22% slower than a wide table with 1,000 columns, and a wide table with 2,000 columns is 43% to 67% slower than a wide table with 1,000 columns. Related technologies can clean up useless attributes through methods such as attribute saturation, but in the long run there will be huge maintenance costs and even business losses. Therefore, a wide table dimensionality reduction method that can solve most of the current dimensionality explosion problems in the industry is particularly important.

[0057] Figure 1 A schematic diagram of the structure of an optional traditional data warehouse model provided in the embodiment of the present application is shown in FIG. Figure 1 As shown in the figure, the traditional data warehouse model is also called the classic dimensional model. The classic dimensional model can include logs, data warehouse layers, and report queries from bottom to top. The different data layers in the data warehouse layer represent different data storage layers and data processing stages. Common data layers can include the following parts:

[0058] 1) ADS (Application Data Store): ADS is the application data storage layer, also known as the application data layer. It is used to store data that has been processed and cleaned, usually data for specific business applications. The data in ADS has been converted into a format and structure suitable for use by specific applications;

[0059] 2) DWS (Data Warehouse Store): DWS is the data warehouse storage layer, also known as the data warehouse layer. It is the most important layer in the data warehouse, used to store data after ETL (extraction, transformation and loading), usually organized by subject. The data in DWS has been integrated, cleaned and consolidated to support complex analysis and query operations;

[0060] 3) DWD (Data Warehouse Detail): DWD is the data warehouse detail layer, also known as the data warehouse detail layer. It is the next layer of the DWS layer and stores more detailed and original data. DWD retains more data details, including historical changes, incremental updates, etc. It is usually used to support data tracing, data investigation, and troubleshooting.

[0061] 4) ODS (Operational Data Store): ODS is the operational data storage layer, also known as the operational data layer. It is used to store operational data that has been preliminarily processed, usually raw data extracted from multiple source systems. The data in ODS can be used for real-time business operations and monitoring, and can also be used as a data source for data warehouses.

[0062] The above data levels and table divisions are mainly to meet different data processing needs, from raw data to processed data to data for specific applications, forming a hierarchical structure to support data integration, consolidation, cleaning and analysis operations. Data tables at different levels may differ in data structure, granularity and processing methods to meet different levels of data processing and usage needs. It should be noted that Figure 1 Field 1 to field n are preset fields, and dimension 1 to dimension 6 are preset dimensions.

[0063] Figure 2 A schematic diagram of the structure of an optional wide table data warehouse model provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the wide table data warehouse model is also called the large wide table model. The large wide table model can include: logs, wide tables, report queries, etc. from bottom to top. Among them, the wide table can include: public attributes, business attributes, and business indicators, etc. Figure 2 Row 1, row 2, and row 3 are row labels, and c1, c2, ... are column labels.

[0064] In related technologies, data dimensionality reduction can be processed in two ways: wide table dimensionality reduction and feature dimensionality reduction. The main differences between the two are as follows:

[0065] 1) Different scenarios:

[0066] For feature dimensionality reduction: In model training, when the number of features increases to a certain critical point, further increase will lead to worse performance of the classifier, resulting in "dimensionality disaster". Feature dimensionality reduction can solve the problem of poor model performance.

[0067] For wide table dimensionality reduction: In data warehouse construction and usage scenarios, multiple tables in the same business field are usually merged into a wide table to solve problems such as too many tables, redundancy between data warehouse layers, and storage costs.

[0068] 2) Different methods:

[0069] For feature dimensionality reduction: usually, whether data will be lost is not considered. Among them, the filter feature dimensionality reduction method mainly explores the characteristics of the feature itself, the relationship between features and target values, such as variance selection method (low variance feature filtering), correlation coefficient, etc.; the embedded feature dimensionality reduction method mainly focuses on the automatic selection of features by the algorithm (the relationship between features and target values), such as decision trees, (information entropy, information gain), regularization (L1, L2), deep learning (convolution, etc.).

[0070] For wide table dimensionality reduction: it is necessary to ensure that the wide table dimensionality reduction does not lose data. For example, for the original wide table, the redundant attributes are merged, such as trigger_from and trigger_type, which both refer to the trigger method, and multiple attributes are merged into one attribute; in addition, for the newly added data, the existing attributes in the wide table need to be selected.

[0071] At present, while wide table models improve data usability and query performance, with the continuous iterative development of business, a large number of table fields will lead to data redundancy in wide table models, thereby interfering with the problem of dimensionality explosion, and further affecting data query efficiency.

[0072] In order to solve the above problems, the data processing method provided in the embodiment of the present application is executed by a data processing device and an electronic device, wherein the data processing device can be stored in the electronic device in the form of a software function model, the data processing device can also be integrated in the electronic device as a hardware function module, and the data processing device can also be combined with the electronic device in software and hardware to implement the data processing method. The present application does not impose any limitations on this.

[0073] In the embodiment of the present application, the electronic device may be a terminal device with a display screen, and the terminal device may be a mobile phone, a tablet personal computer (TPC), a media player, a smart TV, a laptop computer (LC), a personal digital assistant (PDA), a personal computer (PC), a camera, a camcorder, a smart watch, a wearable device (WD) or an autonomous driving vehicle, etc., and the embodiment of the present application is not limited to this.

[0074] The embodiment of the present application provides a data processing method, the basic idea of ​​which is: first, the electronic device determines the query information in response to the setting instruction acting on the data query interface; then, when the query information indicates that the data table to be queried is the first wide table, the electronic device responds to the query instruction acting on the data query interface, and performs a query in the first wide table to obtain the target data corresponding to the query information. Among them, the number of first business fields in the first wide table is equal to the maximum value of the number of first attribute fields corresponding to each event field. Since the first attribute fields between different event fields do not conflict with each other, different first event fields can use the same first business field without affecting the data. In this way, by sharing the first business field in the first wide table with the attribute fields of different event fields, the dimension of the wide table can be effectively reduced, and the data redundancy of the wide table can be reduced, so that the problem of dimensional explosion can be effectively solved, thereby improving the query efficiency of the data.

[0075] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0076] Figure 3 A schematic diagram of an optional drive communication method provided in an embodiment of the present application Figure 1 ,like Figure 3 As shown, including S101 to S102:

[0077] S101. Determine query information in response to a setting instruction acting on a data query interface.

[0078] In the embodiment of the present application, the electronic device has a display screen. The electronic device may be a mobile phone, a tablet computer, a laptop computer or other device with a display screen. The embodiment of the present application does not limit the type of electronic device, and the specific type may be selected according to the actual application scenario.

[0079] In an embodiment of the present application, the data query interface is used to determine query information by setting instructions.

[0080] In the embodiment of the present application, the shape of the display area of ​​the data query interface can be a rectangle, square, diamond, trapezoid, rounded rectangle or clipped rectangle, etc. The embodiment of the present application does not limit the shape, size and specific position of the display area of ​​the data query interface.

[0081] In the embodiment of the present application, the data query interface can also be presented on the display screen in the form of a floating window, a shorthand window, etc. The embodiment of the present application does not impose any limitation on the presentation method of the data query interface.

[0082] In an embodiment of the present application, the setting instructions may include single-click, double-click, long press, preset actions, touch screen and other interaction methods, which are not limited in the embodiment of the present application.

[0083] In the embodiment of the present application, the data query interface includes a series of setting controls and tool bars, etc. The above controls and tool bars respond to corresponding setting instructions and can determine corresponding query information.

[0084] In the embodiment of the present application, the query information is used to indicate the specific information that the electronic device needs to query in the data table to be queried. In other words, the query information is used to retrieve and operate the data in the database. The query information can be a specific syntax or statement to query the database and perform operations such as inserting, updating, deleting, and querying the data.

[0085] Exemplarily, the query information may be at least one of the following: SQL language, MySQL language, Oracle SQL language, PostgreSQL language, NoSQL language, etc. The present application does not impose any limitation on the language form of the query information.

[0086] In some embodiments of the present application, the data query interface includes: a first display area and a second display area, wherein the second display area is used to display a logic statement corresponding to a setting instruction acting on the first display area.

[0087] It should be noted that the first display area and the second display area can be displayed in any area in the data query interface. For the convenience of viewing and processing, any one of the first display area and the second display area can also be presented in the form of a floating window through interactive methods such as dragging, clicking, and preset gestures. This application does not impose any restrictions on this.

[0088] As an implementable manner, in some embodiments of the present application, in response to the setting instruction acting on the data query interface, determining the implementation of the query information in S101 may include:

[0089] In response to a setting instruction acting on at least one setting control in the first display area, query information is determined, and a logic statement corresponding to the setting instruction is synchronously displayed in the second display area; the logic statement corresponds to the query information.

[0090] In the embodiments of the present application, Figure 4A A schematic diagram of an optional data query interface provided in an embodiment of the present application Figure 1 ,like Figure 4A As shown, the data query interface includes a first display area and a second display area. The first display area includes multiple setting controls, and the second display area is used to present the logical statement corresponding to the setting instruction. The first display area includes: an event setting area and a property setting area, wherein the event setting area is used to determine the setting information of the event through the setting instruction, and the property setting area is used to determine the setting information of the property through the setting instruction. According to the setting information of the event and the setting information of the property, the final query information can be determined.

[0091] In the event setting area of ​​the embodiment of the present application, the default query indicator (event field) can be displayed, for example, the total number of clicks on all buried points. Figure 4A As shown, by clicking Figure 4A Click the "Expand Indicator" button in the dialog box to set detailed events / indicators. Figure 4B As shown, it shows the setting controls after clicking the "Expand Indicator" button. By clicking the drop-down menu corresponding to "All Buried Point Clicks" and the drop-down menu corresponding to "Total Number of Times", you can further set the event / indicator.

[0092] It should be noted that in the process of setting event-related information, the embodiment of the present application is exemplified by clicking, but in actual application scenarios, it is not limited to clicking, but can also include double-clicking, touching, preset advantages, remote operation and the like. The embodiment of the present application does not make any limitation to this, and the specific selection can be made according to the actual application scenario.

[0093] For the sake of explanation, here we use Figure 4B Take the drop-down menu corresponding to "Total Times" as an example. Figure 4C As shown, in response to a trigger operation (such as a click) acting on the drop-down menu, a corresponding drop-down menu is displayed in the data query interface. The drop-down menu can display multiple attributes or indicator names, such as "total number of times", "number of users", "average number of times", etc. By clicking to select an operation, the corresponding parameters can be set.

[0094] In the property setting area of ​​the embodiment of the present application, specific grouping property parameters can be set. Figure 4D As shown, click Figure 4D The drop-down menu of the grouping condition can display multiple attribute names. It should be noted that the number of attributes displayed in the drop-down menu can be consistent with the number of business fields in the data table to be queried.

[0095] Exemplarily, when the data table to be queried is the first wide table, assuming that the first business field in the first wide table includes: attribute1, attribute2, and attribute3, the attributes displayed in the drop-down menu corresponding to the grouping condition may include 3: attribute1, attribute2, and attribute3. When the data table to be queried is the second wide table, since the number of second business fields in the second wide table is the sum of the first attribute fields corresponding to each event field, the attributes displayed in the drop-down menu corresponding to the grouping condition may include: the public attributes corresponding to the target event field (i.e., the second attribute field) and the custom attributes corresponding to the target event field (i.e., the first attribute field).

[0096] In the embodiment of the present application, in the process of determining the query information, the logic statement corresponding to the setting instruction is synchronously displayed in the second display area; the logic statement corresponds to the query information. Figure 4D As shown, the setting instruction corresponding to the change logic statement is synchronously presented in the second display area. For example: in the first display interface, the event / indicator information is set to the "total number of clicks on all buried points", and the grouping condition is set to "attribute1", then the following logic statement is synchronously presented in the second display area:

[0097] select count(*)

[0098] from the first wide table

[0099] where embedding point / index = 'all embedding point clicks'

[0100] group by attribute1

[0101] In the embodiment of the present application, the first wide table is a wide table after dimensionality reduction.

[0102] In an embodiment of the present application, the presentation mode of the logic statement can be set in the second display area, such as SQL, MySQL, etc. Here, the common setting mode of the existing front-end page can be referred to. For example, in response to the trigger operation (such as click) of the logic statement setting control in the second display area, the corresponding logic statement setting interface is presented, and at least one option of the logic statement presentation mode is displayed in the logic statement setting interface, such as: SQL, MySQL, etc. After selecting one of the options of the logic statement presentation mode, in response to the trigger operation of the control determined in the logic statement setting interface, the presentation mode of the logic statement in the second display area is successfully set.

[0103] It can be understood that by synchronously displaying the logical statements corresponding to the setting instructions acting in the first display area in the second display area, it is convenient for users to view, improve the visualization ability of the data query interface, and facilitate users to confirm the accuracy of the query information settings.

[0104] In the embodiment of the present application, the first display area may also include multiple other setting controls or toolbars. For example, by clicking the "+ Add Filter" control, three text boxes may be displayed, and the three text boxes are used to set the filter conditions. Figure 4E As shown, by clicking the first text box, you can display the corresponding drop-down menu, and by selecting the corresponding parameter value, you can determine the filtering conditions (such as selecting the name of the advertising campaign). Figure 4F As shown, by clicking the second text box, the corresponding drop-down menu can be displayed, and the filter condition can be determined by selecting the corresponding parameter value (for example, selecting equal to). Figure 4G As shown in the figure, by clicking the third text box, the corresponding drop-down menu can be displayed, and the filtering conditions (such as selecting a game) can be determined by selecting the corresponding parameter value. Figure 4H As shown, by clicking the "+ Add Filter" control in the global filter, you can set multiple global filter conditions, and the relationship between multiple global filter conditions can be and or or.

[0105] It should be noted that the setting controls shown in the embodiment of the present application are merely examples. In actual application scenarios, the first display area may include other setting controls and toolbars, and the embodiment of the present application does not impose any limitations on this.

[0106] As another achievable method, in some embodiments of the present application, Figure 4A-4D Taking the data query interface shown as an example, the setting instruction includes: an edit instruction and a write instruction. In S101, in response to the setting instruction acting on the data query interface, the implementation of the query information is determined, and the following may also be included: S1011 to S1013:

[0107] S1011. In response to an editing instruction acting on the second display area, the second display area is set to a write-allowed state.

[0108] In the embodiments of the present application, Figure 4A As shown, the electronic device sets the second display area to a write-allowed state in response to an editing instruction acting on an editing control in the second display area.

[0109] Exemplarily, by clicking the "Edit SQL" control in the second display area, an input cursor is displayed in the editing area in the second display area, thereby setting the second display area to a write-allowed state.

[0110] It should be noted that in the embodiment of the present application, the text editing area in the second display area can also be awakened by triggering methods such as single-clicking, double-clicking, touching, and preset gestures, and one of the signs for awakening the text editing area can be displaying the input cursor.

[0111] S1012. In response to a write instruction acting on the second display area, present a logic statement corresponding to the write instruction in the second display area.

[0112] In the embodiment of the present application, after the second display area is set to a write-allowed state, the electronic device responds to the write instruction acting on the second display area and presents a logical statement corresponding to the write instruction in the second display area.

[0113] In an embodiment of the present application, the electronic device may receive a write instruction through a virtual keyboard or a physical keyboard, and the electronic device may also receive a write instruction through a touch screen. The embodiment of the present application does not limit the method for receiving the write instruction.

[0114] In the embodiment of the present application, the logical statement may be a SQL statement.

[0115] S1013. Parse the logical statement to determine the query information.

[0116] In the embodiment of the present application, after receiving the logical statement corresponding to the write instruction, the electronic device parses the logical statement to determine the query information.

[0117] In an embodiment of the present application, the logical statement can also be written into the text editing area in the second display area by a one-click import method. The specific method may include: the electronic device displays a text import pop-up window in response to the text import control acting in the second display area; in the text import pop-up window, a text selection control is displayed, and at least one candidate text is displayed in response to a trigger operation acting on the text selection control; in response to a trigger operation of determining the import control in the text import pop-up window, the query statement in the selected target text is imported into the text editing area.

[0118] It can be understood that in the data query interface, a first display area and a second display area are to be set, the first display area is used to determine the query information by clicking, and the second display area is used to display the logical statement corresponding to the query information determined by the first display area, and the second display area also supports manual editing or one-key import to write the logical statement to determine the query information. In this way, the query information determination process can be simplified, thereby improving the efficiency of writing the query information.

[0119] S102. When the query information indicates that the data table to be queried is the first wide table, in response to the query instruction acting on the data query interface, a query is performed in the first wide table to obtain target data corresponding to the query information; wherein the number of first business fields in the first wide table is equal to the maximum value of the number of first attribute fields corresponding to each event field.

[0120] In the embodiment of the present application, the event field is also called an indicator / point, or an event.

[0121] In the embodiment of the present application, the first attribute field is a unique field of each event field. In other words, the attribute fields corresponding to each event field in the first wide table are different.

[0122] In the embodiment of the present application, the first wide table is a wide table obtained by performing dimensionality reduction processing on the second wide table.

[0123] In an embodiment of the present application, the second wide table is subjected to dimensionality reduction processing according to the number of first attribute fields corresponding to each event field in the second wide table to obtain a first wide table, so that at least one first attribute field in the first wide table that has a mapping relationship with each first business field corresponds to a different event field.

[0124] Exemplarily, assuming that the first attribute fields in the first wide table that establish a mapping relationship with the first business field attribute1 include A1, B1, and C1, the first attribute fields A1, B1, and C1 correspond to three different event fields respectively: the first attribute field A1 corresponds to event field A, the first attribute field B1 corresponds to event field B, and the first attribute field C1 corresponds to event field C. In this way, by having the first attribute fields of different event fields share the same first business field, wide table dimensionality reduction is achieved, thereby improving the query efficiency of the wide table.

[0125] In some embodiments of the present application, when the query information indicates that the data table to be queried is a second wide table, in response to a query instruction acting on the data query interface, a query is performed in the second wide table to obtain target data corresponding to the query information; wherein, at least one first attribute field corresponding to each event field in the second wide table each occupies a second business field.

[0126] In the embodiment of the present application, at least one first attribute field corresponding to each event field in the second wide table occupies one second business field. In other words, the first attribute field and the second business field in the second wide table are in one-to-one correspondence. That is to say, the number of second business fields in the second wide table is equal to the sum of the number of first attribute fields of each event field in the second wide table.

[0127] In some embodiments of the present application, the number of at least one first business field in the first wide table is equal to the maximum value of the number of first attribute fields corresponding to each event field.

[0128] In an embodiment of the present application, at least one first attribute field in the first wide table that has a mapping relationship with each first business field corresponds to a different event field. Exemplarily, it is assumed that the first wide table includes event field A, event field B, and event field C. Event field A includes first attribute field A1, first attribute field A2, and first attribute field A3; event field B includes first attribute field B1 and first attribute field B2; event field C includes first attribute field C1 and first attribute field C2. At this time, the first attribute field A1, first attribute field B1, and first attribute field C1 that have a mapping with the first business field attribute1 share the business field attribute2; the first attribute field A3 shares the business field attribute3.

[0129] It can be seen that the number of the first business fields is equal to the maximum number of the first attribute fields included in each event field in the first wide table.

[0130] In the embodiment of the present application, the number of first business fields in the first wide table = MAX{the number of first attribute fields of event field 1, the number of first attribute fields of event field 2, the number of first attribute fields of event field 3, ..., the number of first attribute fields of event field n}. Exemplarily, assuming that the number of first attribute fields of event field 1 in the first wide table is 3, the number of first attribute fields of event field 2 is 2, and the number of first attribute fields of event field 3 is 2, then the number of first business fields is 3.

[0131] In the embodiment of the present application, the number of second business fields in the second wide table = SUM{the number of first attribute fields of event field 1, the number of first attribute fields of event field 2, the number of first attribute fields of event field 3, ..., the number of first attribute fields of event field n}. Exemplarily, assuming that the number of first attribute fields of event field 1 in the second wide table is 3, the number of first attribute fields of event field 2 is 2, and the number of first attribute fields of event field 3 is 2, the number of first business fields is 3+2+2=7.

[0132] It can be seen that compared with the second wide table, the number of first business fields in the first wide table is significantly smaller than that of the second business fields. Therefore, by adopting the first wide table, data redundancy of the wide table can be effectively reduced, thereby effectively solving the problem of dimensionality explosion.

[0133] An embodiment of the present application provides a data processing method, first, the electronic device determines the query information in response to the setting instruction acting on the data query interface; then, when the query information indicates that the data table to be queried is the first wide table, the electronic device responds to the query instruction acting on the data query interface, and performs a query in the first wide table to obtain the target data corresponding to the query information. Since the attribute fields between different event fields do not conflict with each other, different event fields can use the same business field without affecting the data. In this way, by sharing the business field in the first wide table with the attribute fields of different event fields, the dimension of the wide table can be effectively reduced, thereby reducing the data redundancy of the wide table, which can effectively solve the problem of dimensionality explosion and improve the query efficiency of the data.

[0134] In some embodiments of the present application, the first wide table also includes a second attribute field; the second attribute field is an attribute field common to each event field in the first wide table; the sum of the number of second attribute fields and the number of business fields is the dimension of the first wide table.

[0135] In the embodiment of the present application, the second attribute field is an attribute field common to each event field in the first wide table and the second wide table. The second attribute field can also be understood as a common attribute field between each event field.

[0136] In the embodiment of the present application, the dimensions of the first wide table and the second wide table are related to the number of second attribute fields and the number of business fields.

[0137] In an embodiment of the present application, for the first wide table, the dimension of the first wide table = the number of second attribute fields + the number of first business fields = the number of second attribute fields + MAX{the number of first attribute fields of event field 1, the number of first attribute fields of event field 2, the number of first attribute fields of event field 3, ..., the number of first attribute fields of event field n}.

[0138] In an embodiment of the present application, for the second wide table, the dimension of the second wide table = the number of second attribute fields + the number of second business fields = the number of second attribute fields + SUM {the number of first attribute fields of event field 1, the number of first attribute fields of event field 2, the number of first attribute fields of event field 3, ..., the number of first attribute fields of event field n}.

[0139] It can be seen that, since at least one first attribute field in the first wide table that has a mapping relationship with each first business field corresponds to a different event field, the dimension of the first wide table is significantly smaller than that of the second wide table. Therefore, by adopting the first wide table, the data redundancy of the wide table can be effectively reduced, and the overall sparsity of the wide table can be improved, thereby effectively solving the problem of dimensionality explosion.

[0140] In some embodiments of the present application, in S102, when the query information indicates that the data table to be queried is the first wide table, in response to the query instruction acting on the data query interface, a query is performed in the first wide table, and the implementation after obtaining the target data corresponding to the query information may also include: displaying the target data in the data query interface.

[0141] In the embodiment of the present application, the data query interface further includes: a third display area, the third display area is used to display the target data according to a preset presentation method. The preset presentation method may include at least one of the following: a table presentation method, an image presentation method, a text presentation method, etc.

[0142] For example, Fig. 4I As shown, the electronic device displays the target data in the third display area of ​​the data query interface in an image presentation manner. Fig. 4I In the example, the meaning of the horizontal and vertical axes needs to be explained. The target data is presented in the form of a line graph. Of course, you can also choose different image presentation methods, such as: cylinder chart, pie chart, bar chart, etc. You can make corresponding settings through the data presentation control in the third display area.

[0143] It can be understood that by presenting the target data corresponding to the query information in the data query interface, the visualization performance of the wide table query can be improved, and the user interaction capability can be further improved.

[0144] In some embodiments of the present application, the query information includes at least: a target event field and a target attribute field; and at least one first attribute field corresponding to the target event field includes the target attribute field.

[0145] Exemplarily, the target event field is event A, and the at least one first attribute field corresponding to event A includes: A1, A2, and A3, and the target attribute field is A1.

[0146] In some embodiments of the present application, in response to a query instruction acting on a data query interface, performing a query in the first wide table in S102 to obtain target data corresponding to the query information may include S1021 to S1022:

[0147] S1021. In response to a query instruction acting on a data query interface, determine, in at least one first business field in a first wide table, a target business field where a target attribute field corresponding to a target event field is located.

[0148] In the embodiment of the present application, the query instruction is used to instruct the electronic device to perform a query operation in the data table to be queried according to the query information.

[0149] In the embodiment of the present application, Figures 4A to 4I Taking the data query interface shown as an example, when the query information indicates that the data table to be queried is the first wide table, by clicking the "Run" control in the first display area or the "Run" control in the second display area, the target business field where the target attribute field corresponding to the target event field is located in at least one first business field in the first wide table of the electronic device is determined.

[0150] Exemplarily, at least one first business field in the first wide table includes: attribute1, attribute2, attribute3, the target event field is A1, and the electronic device determines that the target business field where the target event field A1 is located is attribute1 among the three first business fields attribute1, attribute2, attribute3.

[0151] S1022. According to the target event field and the target business field, determine in the first wide table the target data corresponding to the target event field having a mapping relationship with the target business field, and complete the query in the first wide table.

[0152] In an embodiment of the present application, after the electronic device determines the target business field where the target attribute field corresponding to the target event field is located, the electronic device determines the target data corresponding to the target event field having a mapping relationship with the target business field in the first wide table based on the target event field and the target business field, and completes the query in the first wide table.

[0153] Exemplarily, the electronic device searches the first wide table for target data A1 corresponding to the target event field A having a mapping relationship with the target business field attribute1, where the target data is data corresponding to the target attribute field.

[0154] It can be understood that, on the one hand, since the first attribute fields between different event fields in the first wide table do not conflict with each other, different event fields can use the same business field without affecting the data. In this way, by sharing the business field in the first wide table with the attribute fields of different event fields, the dimension of the wide table can be effectively reduced, thereby reducing the data redundancy of the wide table, which can effectively solve the problem of dimensional explosion and improve the query efficiency of the data. On the other hand, since at least one first attribute field in the first wide table that has a mapping relationship with each first business field corresponds to different event fields, while effectively improving data query and data writing performance, the storage and computing costs can be effectively reduced.

[0155] In some embodiments of the present application, based on Figure 3 ,like Figure 5 As shown, the implementation before determining the query information in response to the setting instruction acting on the data query interface in S101 may include S201 to S203:

[0156] S201. In response to a system management instruction, display a management platform interface.

[0157] In an embodiment of the present application, the electronic device presents a management platform interface in a display interface in response to a system management instruction.

[0158] In the embodiment of the present application, the system management instruction can be a click, a double click, a preset gesture, a touch, a remote operation, etc., and the embodiment of the present application does not impose any limitation on this.

[0159] Figure 6 A schematic diagram of an optional management platform interface provided in an embodiment of the present application, such as Figure 6 As shown, by clicking the "Management Platform" control, the management platform interface is displayed. Figure 6 The various metadata information of the event is displayed in the dotted box. The management platform interface also includes various operation controls, such as: "Edit" control, "Status Setting" control, "Share" control, "Event Analysis" control, etc.

[0160] It should be noted that the operation controls displayed in the above management platform interface are only examples. In actual application scenarios, other operation controls may also be included, and the embodiments of the present application do not impose any limitations on this.

[0161] S202: In response to a registration instruction acting on the management platform interface, metadata information of at least one event field is displayed on the management platform interface; the at least one event field includes a target event field.

[0162] In the embodiment of the present application, the registration instruction can be a click, a double click, a preset gesture, a touch, a remote operation, etc., and the embodiment of the present application does not impose any limitation on this.

[0163] In some embodiments of the present application, in response to a registration instruction acting on the management platform interface in S202, the implementation of displaying metadata information of at least one event field on the management platform interface may include S2021 to S2022:

[0164] S2021. In response to a triggering operation on an import control in the management platform interface, an import settings pop-up window is presented in the management platform interface.

[0165] In the embodiment of the present application, Figure 6 As an example of the management platform interface shown in FIG. 1 , the electronic device responds to the import control ( Figure 6 A trigger operation (such as clicking) is performed (shown as "batch import" in the figure), and an import setting pop-up window is displayed in the management platform interface.

[0166] In an embodiment of the present application, the import setting pop-up window is used to set metadata information corresponding to at least one event field in the management platform interface.

[0167] S2022. In response to a confirmed import instruction in the import setting pop-up window, metadata information of at least one event field in the metadata description table indicated by the confirmed import instruction is displayed on the management platform interface.

[0168] Figure 7 A schematic diagram of an optional import setting pop-up window provided in an embodiment of the present application, such as Figure 7 As shown, the electronic device responds to the trigger operation of the "Upload File" control in the import setting pop-up window, and displays the metadata information of at least one event field in the metadata description table indicated by the import instruction on the management platform interface. It should be noted that the trigger operation of the above-mentioned "Upload File" control is to determine the import instruction.

[0169] In the embodiment of the present application, the metadata description table is a pre-set table that records metadata information of event fields.

[0170] In some embodiments of the present application, the method further includes S301 to S302:

[0171] S301. In response to a template download instruction in an import setting pop-up window, obtain a metadata description template indicated by the template download instruction.

[0172] In the embodiment of the present application, the electronic device responds to the template download instruction (such as clicking on the template download instruction in the import setting pop-up window) Figure 7 ), obtain the metadata description template indicated by the template download instruction.

[0173] In the embodiment of the present application, the metadata description template is a pre-set table.

[0174] S302: Determine a metadata description table based on the metadata description template.

[0175] In the embodiment of the present application, the metadata information of the event field is written into the metadata description template to obtain a metadata description table.

[0176] In some embodiments of the present application, the metadata information includes at least one of the following: event classification information ( Figure 6 "Category" in the event name information ( Figure 6 "Event name", "Event Chinese name" in the event), event notes ( Figure 6 Event priority information ( Figure 6 "Priority" in the event information ( Figure 6 In the figure, it is shown as "belonging project" and "belonging data table").

[0177] In the embodiment of the present application, the event belonging information may be the project information to which the event belongs.

[0178] It should be noted that the metadata information displayed in the above management platform interface is only an example. In actual application scenarios, other metadata information may also be included, and the embodiments of the present application do not impose any limitations on this.

[0179] S203: In response to a trigger operation on an analysis control corresponding to a target event field, jump from the management platform interface to a data query interface.

[0180] In the embodiment of the present application, the electronic device responds to the analysis control ( Figure 6 The triggering operation (such as clicking) of the management platform interface is shown as "Event Analysis" in the figure, and the page is redirected to the following page: Figure 4A The data query interface shown.

[0181] In some embodiments of the present application, in response to the registration instruction acting on the management platform interface in S202, the implementation after displaying metadata information of at least one event field on the management platform interface may also include S204 to S205:

[0182] S204: In response to a triggering operation on a target event field in the management platform interface, jump from the management platform interface to a data viewing interface, and display attribute description information of at least one attribute field of the target event field in the data viewing interface.

[0183] In the embodiment of the present application, Figure 6 As an example, in response to the target event field (with Figure 6 In the example, a trigger operation (such as a click) of the target event field is performed, and the data viewing interface is jumped from the management platform interface to the data viewing interface, and the attribute description information of at least one attribute field of the target event field is displayed in the data viewing interface.

[0184] In the embodiment of the present application, the data viewing interface can be presented in a jump manner, a small window, a floating window, etc. The embodiment of the present application is not limited to this, and the specific presentation method can be selected according to the actual application scenario.

[0185] Fig. 8A A schematic diagram of an optional data viewing interface provided in an embodiment of the present application Figure 1 ,like Fig. 8A As shown, the target event field is "Event 1", and its attribute fields include: connection session ID, error code, connection duration, device serial number, etc.

[0186] In the embodiment of the present application, the attribute description information includes at least one of the following: attribute name ( Fig. 8A "Attribute display name", "Attribute key name" and attribute description information ( Fig. 8A "Attribute value examples or descriptions" in the table), attribute update time information, business field information ( Fig. 8A In the figure, it is shown as “the corresponding data table field”) etc.

[0187] S205 . In response to a triggering operation on a query control in the data viewing interface, jump from the data viewing interface to the data query interface.

[0188] In the embodiment of the present application, Fig. 8A Taking the data viewing interface shown in the figure as an example, the electronic device responds to the query control ( Fig. 8A A trigger operation (such as clicking) (shown as "data query" in the figure) jumps from the data viewing interface to the data query interface.

[0189] In the embodiment of the present application, Fig. 8A Taking the data viewing interface shown in the figure as an example, the electronic device responds to the editing control ( Fig. 8AThe trigger operation (such as clicking) of the target event field in the data viewing interface is set to the edit state. Figure 8B As shown, you can set the attribute description information of the target event field, such as "attribute display name", "attribute value type", "event Chinese name", etc.

[0190] In some embodiments of the present application, the method further includes S401 to S402:

[0191] S401. In response to a wide table building instruction, display a wide table building interface.

[0192] In an embodiment of the present application, the electronic device displays a wide table construction interface in response to a wide table construction instruction, wherein the wide table construction interface is used to construct a first wide table.

[0193] In the embodiment of the present application, the wide table construction instruction can be a single click, a double click, a preset gesture, a remote control, etc., and the present application does not impose any limitation on this.

[0194] S402: In response to a construction instruction acting on a wide table construction interface, determine a first wide table according to a target data source indicated by the construction instruction.

[0195] In some embodiments of the present application, in response to a construction instruction acting on a wide table construction interface, determining the implementation of the first wide table according to a target data source indicated by the construction instruction in S402 may include S4021 to S4023:

[0196] S4021. In response to a construction instruction acting on a wide table construction interface, determine a first attribute field of each event field in a target data source indicated by the construction instruction.

[0197] In an embodiment of the present application, after receiving a construction instruction acting on a wide table construction interface, the electronic device determines the first attribute field of each event field in the target data source indicated by the construction instruction. For example, it is determined that the first attribute field of event field A includes: A1, A2, A3; the first attribute field of event field B includes: B1, B2; the first attribute field of event field C includes: C1, C2.

[0198] S4022: Determine the maximum value of the number of first attribute fields in each event field as the first number of at least one first business field.

[0199] In the embodiment of the present application, the electronic device uses the maximum value of the number of first attribute fields in each event field as the first number of the first service field of the first wide table.

[0200] For example, assuming that the number of first attribute fields of event field A is 3, the number of first attribute fields of event field B is 2, and the number of first attribute fields of event field C is 2, the maximum value (3) of the first attribute field is used as the first number of the first business field. That is, the first business field includes: attribute1, attribute2, attribute3.

[0201] S4023. For each first attribute field in each event field in the target data source, establish a mapping relationship between each first attribute field and any first business field in the first number of first business fields, thereby obtaining a first wide table.

[0202] In the embodiment of the present application, the electronic device establishes a mapping relationship between each first attribute field and any first service field in the first number of first service fields, thereby obtaining a first wide table.

[0203] Exemplarily, for the first number (for example 3: attribute1, attribute2, attribute3) of the first business fields, for the first attribute field of event field A, a mapping relationship is established between A1 and attribute1, a mapping relationship is established between A2 and attribute2, and a mapping relationship is established between A3 and attribute3; for the first attribute field of event field B, a mapping relationship is established between B1 and attribute1, and a mapping relationship is established between B2 and attribute2; for the first attribute field of event field C, a mapping relationship is established between C1 and attribute1, and a mapping relationship is established between C2 and attribute2.

[0204] It is understandable that in the embodiment of the present application, since the first attribute fields between different event fields in the first wide table do not conflict with each other, different event fields can use the same business field without affecting the data. In this way, by sharing the business field in the first wide table with the attribute fields of different event fields, the dimension of the wide table can be effectively reduced, thereby reducing the data redundancy of the wide table, which can effectively solve the problem of dimensional explosion and improve the query efficiency of data.

[0205] In addition, in an embodiment of the present application, since the attribute fields of different event fields in the first wide table share the business fields in the first wide table, that is, at least one first attribute field in the first wide table that has a mapping relationship with each first business field corresponds to different event fields, while effectively improving data query and data writing performance, the storage and computing costs can be effectively reduced.

[0206] In some embodiments of the present application, the method further includes S501 to S502:

[0207] S501. In response to a wide table conversion instruction, display a wide table conversion interface.

[0208] In an embodiment of the present application, the electronic device displays a wide table conversion interface in response to a wide table conversion instruction, wherein the wide table conversion interface is used to convert the second wide table into the first wide table.

[0209] In the embodiment of the present application, the wide table conversion instruction can be a single click, a double click, a preset gesture, a remote control, etc., and the present application does not impose any limitation on this.

[0210] S502 : In response to a conversion instruction on the wide table conversion interface, perform dimensionality reduction processing on the second wide table to obtain a first wide table.

[0211] In some embodiments of the present application, in response to the conversion instruction acting on the wide table conversion interface, the second wide table is subjected to dimensionality reduction processing to obtain the implementation of the first wide table, which may include S5021 to S5023:

[0212] S5021. In response to a conversion instruction acting on the wide table conversion interface, determine a second number of first attribute fields in each event field in the second wide table.

[0213] In the embodiment of the present application, the electronic device determines the number of first attribute fields in each event field in the second wide table in response to the conversion instruction acting on the wide table conversion interface. For example, since the first attribute field of event field A in the second wide table includes: A1, A2, A3; the first attribute field of event field B includes: B1, B2; the first attribute field of event field C includes: C1, C2. Therefore, the electronic device can determine that the number of first attribute fields of event field A in the second wide table is 3, the number of first attribute fields of event field B is 2, and the number of first attribute fields of event field C is 2.

[0214] S5022: Use the maximum value of the number of first attribute fields in each event field in the second wide table as the number of at least one first business field in the first wide table.

[0215] Exemplarily, since the number of first attribute fields of event field A in the second wide table is 3, the number of first attribute fields of event field B is 2, and the number of first attribute fields of event field C is 2, the electronic device uses the maximum number of first attribute fields (3) as the number of at least one first business field in the first wide table.

[0216] S5023. For each first attribute field in each event field in the second wide table, establish a mapping relationship between each first attribute field and any first business field in the second number of first business fields, thereby completing the dimensionality reduction processing of the second wide table, thereby obtaining the first wide table.

[0217] Exemplarily, the second number (for example, 3) of first business fields include: attribute1, attribute2, attribute3. For the first attribute field of event field A, a mapping relationship is established between A1 and attribute1, a mapping relationship is established between A2 and attribute2, and a mapping relationship is established between A3 and attribute3; for the first attribute field of event field B, a mapping relationship is established between B1 and attribute1, and a mapping relationship is established between B2 and attribute2; for the first attribute field of event field C, a mapping relationship is established between C1 and attribute1, and a mapping relationship is established between C2 and attribute2, so as to form a final first wide table, thereby realizing dimensionality reduction processing of the second wide table.

[0218] It is understandable that in the embodiment of the present application, by performing dimensionality reduction processing on the second wide table, the first attribute fields between different event fields in the first wide table after dimensionality reduction do not conflict with each other. Therefore, for the first wide table, different event fields can use the same business field without affecting the data. In this way, the dimension of the wide table can be effectively reduced, thereby reducing the data redundancy of the wide table, thereby effectively solving the problem of dimensionality explosion and improving the query efficiency of the data.

[0219] Fig. 9 A schematic diagram of an optional data processing process provided in an embodiment of the present application Figure 3 ,like Fig. 9 As shown, the data processing method may include S601 to S610:

[0220] S601, in response to a wide table conversion instruction, displaying a wide table conversion interface; in response to a conversion instruction acting on the wide table conversion interface, performing dimensionality reduction processing on the second wide table to obtain a first wide table;

[0221] S602, in response to a wide table construction instruction, displaying a wide table construction interface; in response to a construction instruction acting on the wide table construction interface, determining a first wide table according to a target data source indicated by the construction instruction;

[0222] It should be noted that S601 and S602 are parallel solutions, and either S601 or S602 can be executed.

[0223] S603, in response to the system management instruction, displaying the management platform interface;

[0224] S604, in response to the registration instruction acting on the management platform interface, displaying metadata information of at least one event field on the management platform interface;

[0225] S605, in response to a triggering operation on an analysis control corresponding to a target event field, jumping from the management platform interface to a data query interface;

[0226] S606: In response to a trigger operation acting on a target event field in the management platform interface, jump from the management platform interface to a data viewing interface, and display attribute description information of at least one attribute field of the target event field in the data viewing interface;

[0227] S607, in response to a triggering operation on a query control in the data viewing interface, jumping from the data viewing interface to the data query interface;

[0228] It should be noted that SS605 and S606-S607 are parallel solutions, and SS605 and S606-S607 can be executed either one.

[0229] S608, determining query information in response to a setting instruction acting on the data query interface;

[0230] S609: When the query information indicates that the data table to be queried is the first wide table, in response to the query instruction acting on the data query interface, query is performed in the first wide table to obtain target data corresponding to the query information;

[0231] S610: When the query information indicates that the data table to be queried is the second wide table, in response to a query instruction acting on the data query interface, query is performed in the second wide table to obtain target data corresponding to the query information.

[0232] It should be noted that S609 and S610 are parallel solutions, and either S609 or S610 can be executed.

[0233] The embodiment of the present application provides a data processing method, because at least one first attribute field in the first wide table that has a mapping relationship with each first business field corresponds to different event fields. The attribute fields between different event fields in the first wide table do not conflict with each other, so different event fields can use the same business field without affecting the data. In this way, by sharing the business field in the first wide table with the attribute fields of different event fields, the dimension of the wide table can be effectively reduced, thereby reducing the data redundancy of the wide table, which can effectively solve the problem of dimensionality explosion, thereby improving the query efficiency of the data.

[0234] The data processing method in the embodiments of the present application is explained below with some specific embodiments.

[0235] (I) Description of wide table dimensionality reduction measurement indicators

[0236] Measuring the dimensionality reduction of wide tables can include the following two indicators:

[0237] 1) Attribute reuse rate

[0238] Attribute reuse rate = number of times the attribute is reused in different indicators or tracking events / number of all attributes in the large wide table.

[0239] For example, the current large wide table (the first wide table or the second wide table) has 300 fields, and a certain field is used in 60 indicators or tracking points (event fields). The reuse rate of the attribute is 20%. This indicator assesses the reuse of the attribute.

[0240] 2) Attribute data saturation

[0241] Attribute data saturation = the number of data items with values ​​for the attribute / the total number of data items in the large wide table.

[0242] For example, if the current large wide table has 10,000 rows and a certain field has values ​​in all 1,000 rows, then the saturation of the attribute data is 10%. This indicator can be used to consider the overall sparsity of the large wide table.

[0243] (II) Wide table dimensionality reduction method

[0244] 1) Traditional wide table (equivalent to the second wide table):

[0245] The number of traditional wide table dimensions = the number of common attributes + SUM{the number of custom attributes of tracking point / indicator 1 (A1+A2+A3), the number of custom attributes of tracking point / indicator 2 (B1+B2), the number of custom attributes of tracking point / indicator 3 (C1+C2), ..., the number of custom attributes of tracking point / indicator n}, that is, as long as there is a newly added attribute, if the newly added attribute does not exist in the initial wide table, a column for the attribute will be added to the initial wide table.

[0246] In the traditional wide table design, as long as a new tracking point / indicator (equivalent to the event field) is added, if the custom attribute (equivalent to the first attribute field) of the new tracking point / indicator does not exist in the original wide table, a new attribute value needs to be added to the wide table (that is, a new column). Therefore, with the addition and change of business, the dimension of the wide table shows a rapid and uncontrollable growth trend.

[0247] Table 1 is an optional traditional wide table provided in an embodiment of the present application. As shown in Table 1, since the custom attributes of A, B, and C points / indicators are all different, each custom attribute needs to be retained in the wide table, which results in very sparse data in the wide table, wasting storage space and reducing data query efficiency. It should be noted that indicator 1 in Table 1 is the attribute reuse rate, and indicator 2 is the attribute data saturation. A1, A2, and A3 are the first attribute fields (i.e., custom attributes) of event field A; B1 and B2 are the first attribute fields of event field B; C1 and C2 are the first attribute fields of event field C, and public attributes 1, public attributes 2, and public attributes 3 are the second attribute fields.

[0248] Table 1

[0249]

[0250] 2) Wide table after dimensionality reduction (equivalent to the first wide table):

[0251] The number of dimensions in the new wide table = the number of public attributes + MAX{the number of custom attributes of tracking point / indicator 1, the number of custom attributes of tracking point / indicator 2, the number of custom attributes of tracking point / indicator 3, ..., the number of custom attributes of tracking point / indicator n}. The design of the new wide table does not determine whether the business meaning of the custom attributes of the tracking point / indicator is covered in the original large wide table. It only needs to consider whether the number of attributes of the tracking point / indicator is less than or equal to the number of dimensions in the new wide table. If the number of attributes of the tracking point / indicator is greater than the number of dimensions in the new wide table, add new attributes to the new wide table. It should be noted that the dimension attribute n in the new wide table does not have any business meaning and only represents the nth custom attribute in the data.

[0252] Table 2 is an optional wide table after dimensionality reduction provided in an embodiment of the present application. As shown in Table 2, event / indicator A has 3 custom attributes, and its custom attributes in the wide table are attribute1, attribute2, and attribute3 respectively; event / indicator B has 2 custom attributes, and its custom attributes in the wide table are attribute1 and attribute2 respectively. The reason why event / indicator A and event / indicator B can share attribute1 and attribute2 fields in the wide table is that there is no conflict in using attribute1 and attribute2 in these two events / indicators respectively, because different events / indicators will not appear in the same data, so it will not affect data query.

[0253] Table 2

[0254]

[0255] Through the calculation of wide table measurement indicators, it is found that after using the wide table dimensionality reduction method, the attribute reuse rate and attribute data saturation of the wide table will be greatly improved, as can be seen by comparing Table 1 and Table 2. Therefore, this method can avoid other disadvantages of wide tables (limited query performance due to too many dimensions and waste of storage space due to too sparse data tables) while retaining the advantages of wide tables.

[0256] Fig.10 A schematic diagram of the structure of an optional wide table model provided in an embodiment of the present application, such as Fig.10 As shown, the wide table model includes, from top to bottom: a data query unit 10, a wide table unit 20, a management platform metadata unit 30 and a log unit 40. Fig.10 The wide table model shown is applicable to the first wide table (dimensionality-reduced wide table) and the second wide table (traditional wide table).

[0257] For the data query unit 10, for example, assuming that the total usage time of XX application in the wide table is queried, for the second wide table, its logical language can be expressed as: select sum (usage time) from second wide table where event name = usage application; for the first wide table, its logical language can be expressed as: select sum (usage time) from first wide table where event name = usage application.

[0258] For the wide table unit 20, illustratively, for the second wide table, the storage form of the wide table can be expressed as:

[0259] Event name, sn, time, application name, usage duration, video playback, film and television type

[0260] Use application, 2, 20230703, application 1, 300, [null value], [null value]

[0261] Use application,3,20230704,Apply2,500,[NULL],[NULL]

[0262] Video playback, 1, 20230705, [null value], [null value], Movie 1, TV series

[0263] Use application,4,20230706,Application 1,400,[NULL],[NULL]

[0264] Video playback, 4, 20230707, [null value], [null value], Movie 2, Movie

[0265] For the first wide table, the storage form of the wide table can be expressed as:

[0266] Event name, sn, time, attribute1, attribute2

[0267] Use application, 2, 20230703, application 1, 300

[0268] Use application,3,20230704,application 2,500

[0269] Video playback, 1, 20230705, Movie 1, TV series

[0270] Use application,4,20230706,application1,400

[0271] Video playback, 4, 20230707, Movie 2, Movie

[0272] For the management platform metadata unit 30, exemplarily, for the first wide table, in the "Application Use" event field, the "Application Name" attribute field corresponds to the "attribute1" business field, and the "Usage Time" attribute field corresponds to the "attribute2" business field; in the "Video Playback" event field, the "Movie and Television Name" attribute field corresponds to the "attribute1" business field, and the "Movie and Television Type" attribute field corresponds to the "attribute2" business field.

[0273] For the log unit 40, illustratively, its storage form can be expressed as:

[0274] Event name, sn, data details

[0275] Use application, 2, {"time":"20230703","application name":"application 1","usage duration":300}

[0276] Use application, 3, {"time":"20230704","application name":"application 2","usage duration":500}

[0277] Video playback, 1, {"Time":"20230705","Movie Name":"Movie 1","Movie Type":"TV Series}

[0278] Use application, 4, {"time":"20230706","application name":"application 1","usage duration":400}

[0279] Video playback, 4, {"Time":"20230707","Movie Name":"Movie 2","Movie Type":Movie}

[0280] 3) Dimensionality reduction effect of wide tables

[0281] The process of performance testing the first wide table and the second wide table is described below.

[0282] A. Test data: Use 1000 columns, 1500 columns, and 2000 columns to import the same data, with 100 million data items, namely table1, table2, and table3.

[0283] B. Test method: query table1, table2, and table3 respectively, using three data levels: full data (10 million records), 50% data (5 million records), and 25% data (2.5 million records).

[0284] C. Test steps:

[0285] Step 1: Obtain the data source. Exemplarily, its logical language can be expressed as:

[0286] create table test.test_1

[0287] as select*from olap.test_table_1

[0288] where udt = '20230420';

[0289] Step 2: Create a test table table1 with 1,000 columns. For example, its logical language can be expressed as:

[0290] create table test.test_col1000(

[0291] col1 string,

[0292] col2 string,

[0293] col3 string,

[0294] col4 string, ...

[0296] col1000 string)using iceberg

[0297] comment 'Testing a thousand columns'

[0298] partitioned by(

[0299] `udt` string comment 'Partition field, by day' )

[0301] TBLPROPERTIES('write.metadata.delete-after-commit.enabled'='true','write.metadata.previous-versions-max'='10','write.spark.fanout.enabled'='true'

[0302] ,'protect_mode'='no_drop' ) ;

[0305] Step 3: Write 2.5 million records to table 1. For example, the logical language can be expressed as:

[0306] insert overwrite table test.test_col1000 partition(udt)

[0307] select

[0308] label_level_4

[0309] ,sn

[0310] ,category

[0311] ,type

[0312] ,open_id

[0313] ,id

[0314] ,ui_major_ver

[0315] ,ui_minor_ver ...

[0317] ,error_code

[0318] ,'20230410'as udt

[0319] from test.test_liam_col_data_source

[0320] TABLESAMPLE(2500000ROWS);

[0321] Step 4: When the cluster is idle, submit the performance test SQL in the SparkSQL client and record the query duration. For example, its logical language can be expressed as:

[0322] select

[0323] col985

[0324] ,count(1)

[0325] ,count(distinct col2)

[0326] from test.test_liam_col1000

[0327] where udt = '20230410'

[0328] group by col985

[0329] Step 5. Repeat steps 1 to 4 to test the performance of 1000 columns, 1500 columns, and 2000 columns on 2.5 million, 5 million, and 10 million data respectively; illustratively, the query method is: query 1 that is common to the three tables.

[0330] The query results are as follows:

[0331] ①. Query efficiency

[0332] Table 3 is a statistical table of optional query time provided in an embodiment of the present application, and Table 4 is a statistical table of optional query efficiency provided in an embodiment of the present application. It can be seen that in terms of data query efficiency, 1000 columns are 38%-60% faster than 1500 columns, and 1000 columns are 70%-110% faster than 2000 columns.

[0333] Table 3

[0334] table1 (1000 columns) table2 (1500 columns) table3 (2000 columns) 25 million 18.261sec - - 10 million 15.367sec 21.4sec 27.153sec 5 million 12.778sec 20.474sec 26.696sec 2.5 million 13.878sec 19.164sec 23.527sec

[0335] Table 4

[0336] table1 (1000 columns) VS table2 (1500 columns) table1 (1000 columns) VS table3 (2000 columns) 10 million 39.26％ 76.70％ 5 million 60.23％ 108.92％ 2.5 million 38.09％ 69.53％

[0337] ② Writing efficiency

[0338] Table 5 is a statistical table of optional write time provided in an embodiment of the present application, and Table 6 is a statistical table of optional write efficiency provided in an embodiment of the present application. It can be seen that in terms of data writing efficiency, 1000 columns are 26%-142% faster in writing than 1500 columns, and 1000 columns are 57%-104% faster in querying than 2000 columns.

[0339] Table 5

[0340] table1 (1000 columns) table2 (1500 columns) table3 (2000 columns) 25 million 45.5min - - 10 million 26.3min 37.7 min 46min 5 million 15.2min 36.8 min 31min 2.5 million 11.3min 14.3min 17.8min

[0341] Table 6

[0342] table1 (1000 columns) vs table2 (1500 columns) table1 (1000 columns) vs table3 (2000 columns) 10 million 43.35％ 74.90％ 5 million 142.11％ 103.95％ 2.5 million 26.55％ 57.52％

[0343] (III) Wide table data query method

[0344] Step 1: Register the public attributes, custom attributes, and data tables (equivalent to metadata information) of the tracking points / indicators on the management platform (equivalent to the management platform interface);

[0345] Step 2: View the metadata of the tracking point / indicator (equivalent to the target event field) in the management system. Click a tracking point / indicator in the management platform interface to query the metadata details of the tracking point / indicator (equivalent to metadata information). You can edit / modify the attribute corresponding to the custom attribute in the wide table (equivalent to attribute field information).

[0346] Step 3. Click the "Data Query" button (equivalent to the query control) to jump to the tracking point / indicator data query system (equivalent to the data query interface) to query the data. You can configure the desired data in a point-to-point manner, and the corresponding SQL statement will be generated when you click.

[0347] In the above process, click the "Data Query" button (equivalent to the query control) to jump to the data query system. The user is unaware of the entire data query process. Through point-and-click interaction, first select a certain point / indicator, and then choose to group by a certain attribute (Chinese name). The backend will query the attribute n of the point / indicator in the wide table after dimensionality reduction according to the attribute (Chinese name) selected by the frontend. After splicing the corresponding SQL, query the wide table after dimensionality reduction to obtain data, and click the "Run" button (equivalent to the query command) to obtain data.

[0348] Step 4: Support running after custom editing in the SQL editing area (equivalent to the second display area). Pseudo code example:

[0349]

[0350]

[0351] The beneficial effects of the data processing method provided in the embodiment of the present application are as follows:

[0352] 1. The embodiment of the present application uses events as the minimum granularity and distinguishes different events through the event type field; when writing data into a wide table, the business fields corresponding to different events in attribute1, attribute2, etc. are recorded to maintain the mapping relationship;

[0353] 2. In all cases involving reading the wide table, the table fields such as attribute1 and attribute2 are mapped back to business fields of different event types at the event granularity, for example: attribute1→application name, attribute2→usage duration;

[0354] 3. According to business needs, the embodiment of the present application reads all events required for this analysis from the wide table, and associates multiple events (including operations such as union and join) according to corresponding scenarios to generate a data set;

[0355] 4. The embodiment of the present application performs analysis operations on the generated data set, outputs results, and performs visualization.

[0356] The core innovations of the data processing method provided in the embodiments of the present application are as follows:

[0357] 1. The present application embodiment defines the indicators for wide table evaluation: attribute reuse rate and attribute data saturation.

[0358] 2. The implementation results of the embodiments of the present application show that the method has a significant effect on improving data query and data writing performance, improving efficiency while saving storage and computing costs for enterprises. In terms of data query efficiency, 1000 columns are 38%-60% faster than 1500 columns, and 1000 columns are 70%-110% faster than 2000 columns; in terms of data writing efficiency, 1000 columns are 26%-142% faster than 1500 columns, and 1000 columns are 57%-104% faster than 2000 columns.

[0359] 3. The embodiments of the present application have many usage scenarios, a wide range of applications, and low implementation difficulty. The embodiments of the present application can solve the query performance problem of large and wide tables, not limited to embedded points and indicators. As long as it is a data table in the same business domain, this method can be used to solve the performance problem caused by multi-table joint analysis.

[0360] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the present application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be arbitrarily combined, as long as they do not violate the idea of ​​the present application, they should also be regarded as the contents disclosed in the present application. For another example, under the premise of no conflict, the various embodiments and / or the technical features in the various embodiments described in the present application can be arbitrarily combined with the prior art, and the technical solution obtained after the combination should also fall within the protection scope of the present application.

[0361] It should be understood that in the various method embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0362] Based on the same inventive concept as the above embodiments, Fig.11 A schematic diagram of the composition structure of an optional data processing device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the data processing device 70 includes a determination unit 701, a query unit 702 and a display unit 703; wherein,

[0363] The determining unit 701 is used to determine the query information in response to the setting instruction acting on the data query interface;

[0364] The query unit 702 is used to query in the first wide table in response to a query instruction acting on the data query interface when the query information indicates that the data table to be queried is a first wide table, so as to obtain the target data corresponding to the query information; wherein the number of first business fields in the first wide table is equal to the maximum value of the number of first attribute fields corresponding to each event field.

[0365] In some embodiments of the present application, the query information includes at least: a target event field and a target attribute field; the at least one first attribute field corresponding to the target event field includes the target attribute field; the query unit 702 is further used to respond to the query instruction acting on the data query interface, and determine the target business field where the target attribute field corresponding to the target event field is located in at least one first business field in the first wide table; according to the target event field and the target business field, determine the target data corresponding to the target event field having a mapping relationship with the target business field in the first wide table, and complete the query in the first wide table. .

[0366] In some embodiments of the present application, the determination unit 701 is also used to display a management platform interface in response to a system management instruction; to display metadata information of at least one event field on the management platform interface in response to a registration instruction acting on the management platform interface; the at least one event field includes the target event field; and to jump from the management platform interface to the data query interface in response to a trigger operation of an analysis control corresponding to the target event field.

[0367] In some embodiments of the present application, the determination unit 701 is also used to jump from the management platform interface to the data viewing interface in response to a trigger operation acting on the target event field in the management platform interface, and display attribute description information of at least one attribute field of the target event field in the data viewing interface; and jump from the data viewing interface to the data query interface in response to a trigger operation acting on a query control in the data viewing interface.

[0368] In some embodiments of the present application, the determination unit 701 is also used to present an import settings pop-up window in the management platform interface in response to a trigger operation of an import control in the management platform interface; in response to a determination import instruction in the import settings pop-up window, the metadata information of the at least one event field in the metadata description table indicated by the determination import instruction is displayed on the management platform interface.

[0369] In some embodiments of the present application, the determination unit 701 is further used to respond to a template download instruction in the import setting pop-up window, obtain the metadata description template indicated by the template download instruction; and determine the metadata description table based on the metadata description template.

[0370] In some embodiments of the present application, the metadata information includes at least one of the following: event classification information, event name information, event note information, event priority information, and event belonging information.

[0371] In some embodiments of the present application, the data query interface includes: a first display area and a second display area; the determination unit 701 is also used to determine the query information in response to a setting instruction acting on at least one setting control in the first display area, and synchronously display a logical statement corresponding to the setting instruction in the second display area; the logical statement corresponds to the query information.

[0372] In some embodiments of the present application, the setting instructions include: editing instructions and writing instructions; the data query interface includes: a second display area; the determination unit 701 is also used to set the second display area to a write-allowed state in response to the editing instructions acting on the second display area; in response to the write instructions acting on the second display area, present the logical statement corresponding to the write instruction in the second display area; and parse the logical statement to determine the query information.

[0373] In some embodiments of the present application, the determination unit 701 is also used to, when the query information indicates that the data table to be queried is a second wide table, respond to a query instruction acting on the data query interface, to query in the second wide table to obtain target data corresponding to the query information; wherein, at least one first attribute field corresponding to each event field in the second wide table each occupies a second business field.

[0374] In some embodiments of the present application, the determination unit 701 is further used to display a wide table construction interface in response to a wide table construction instruction; in response to a construction instruction acting on the wide table construction interface, determine the first wide table according to a target data source indicated by the construction instruction.

[0375] In some embodiments of the present application, the determination unit 701 is further used to determine, in response to a construction instruction acting on the wide table construction interface, the first attribute field of each event field in the target data source indicated by the construction instruction; determine the maximum value of the number of the first attribute fields in each event field as the first number of the at least one first business field; and for each first attribute field in each event field in the target data source, establish a mapping relationship between each first attribute field and any one of the first business fields of the first number, thereby obtaining the first wide table.

[0376] In some embodiments of the present application, the determination unit 701 is further used to display a wide table conversion interface in response to a wide table conversion instruction; in response to a conversion instruction acting on the wide table conversion interface, perform dimensionality reduction processing on the second wide table to obtain the first wide table.

[0377] In some embodiments of the present application, the determination unit 701 is also used to determine the number of first attribute fields in each event field in the second wide table in response to a conversion instruction acting on the wide table conversion interface; take the maximum value of the number of first attribute fields in each event field in the second wide table as the second number of at least one first business field in the first wide table; for each first attribute field in each event field in the second wide table, establish a mapping relationship between each first attribute field and any one of the second number of first business fields, thereby completing the dimensionality reduction processing of the second wide table, thereby obtaining the first wide table.

[0378] In some embodiments of the present application, the display unit 703 is used to display the target data in the data query interface.

[0379] In some embodiments of the present application, the first wide table also includes a second attribute field; the second attribute field is an attribute field common to each event field in the first wide table; the sum of the number of the second attribute fields and the number of the business fields is the dimension of the first wide table.

[0380] Those skilled in the art should understand that the relevant description of the above-mentioned data processing device in the embodiment of the present application can be understood by referring to the relevant description of the data processing method in the embodiment of the present application.

[0381] Fig.12 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. Fig.12 As shown, the electronic device 80 includes a processor 81 and a memory 82. The memory 82 can store a computer program, and the processor 81 can call and run the computer program from the memory 82 to implement the method in the embodiment of the present application.

[0382] The memory 82 may be a separate device independent of the processor 81 , or may be integrated into the processor 81 .

[0383] In some embodiments of the present application, Fig.12 As shown, the electronic device 80 may further include a transceiver 83, and the processor 81 may control the transceiver 83 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices.

[0384] The transceiver 83 may include a transmitter and a receiver. The transceiver 83 may further include an antenna, and the number of the antennas may be one or more.

[0385] It can be understood that the processor of the embodiment of the present application may be an integrated circuit chip with information processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0386] It can also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0387] It can also be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application can also be static random access memory (Static RAM, SRAM), dynamic random access memory (DynamicRAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Dynch Link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0388] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0389] In some embodiments of the present application, the computer-readable storage medium can be applied to the server in the embodiments of the present application, and when the computer program is executed by at least one processor, it implements the corresponding processes implemented by the server in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0390] In some embodiments of the present application, the computer-readable storage medium can be applied to the graphics processor in the embodiments of the present application, and when the computer program is executed by at least one processor, the corresponding processes implemented by the graphics processor in the various methods of the embodiments of the present application are implemented. For the sake of brevity, they are not repeated here.

[0391] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0392] In some embodiments of the present application, the computer program product can be applied to the server in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the server in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0393] In some embodiments of the present application, the computer program product can be applied to the graphics processor in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the graphics processor in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0394] The embodiment of the present application also provides a computer program.

[0395] In some embodiments of the present application, the computer program can be applied to the server in the embodiments of the present application. When the computer program runs on the computer, the computer executes the corresponding processes implemented by the server in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0396] In some embodiments of the present application, the computer program can be applied to the graphics processor in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the graphics processor in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0397] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0398] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0399] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0400] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0401] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0402] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0403] It should be noted that, in this application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0404] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0405] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0406] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0407] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0408] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The method comprises: In response to a setting instruction acting on the data query interface, determining query information; When the query information indicates that the data table to be queried is the first wide table, in response to the query instruction acting on the data query interface, a query is performed in the first wide table to obtain the target data corresponding to the query information; wherein the number of first business fields in the first wide table is equal to the maximum value of the number of first attribute fields corresponding to each event field.

2. The method according to claim 1, characterized in that The query information at least includes: a target event field and a target attribute field; the first attribute field corresponding to the target event field includes the target attribute field; The step of querying the first wide table in response to the query instruction on the data query interface to obtain target data corresponding to the query information includes: In response to a query instruction acting on the data query interface, determining, in at least one first business field in the first wide table, a target business field where the target attribute field corresponding to the target event field is located; According to the target event field and the target business field, the target data corresponding to the target event field having a mapping relationship with the target business field is determined in the first wide table, and the query in the first wide table is completed.

3. The method according to claim 1 or 2, characterized in that: Before determining the query information in response to the setting instruction acting on the data query interface, the method further includes: In response to the system management instruction, display the management platform interface; In response to a registration instruction acting on the management platform interface, metadata information of at least one event field is displayed on the management platform interface; the at least one event field includes a target event field; In response to a triggering operation acting on the analysis control corresponding to the target event field, the interface is jumped from the management platform interface to the data query interface.

4. The method according to claim 3, characterized in that The method further comprises: In response to a triggering operation acting on the target event field in the management platform interface, jumping from the management platform interface to a data viewing interface, and displaying attribute description information of at least one attribute field of the target event field in the data viewing interface; In response to a triggering operation acting on a query control in the data viewing interface, the user jumps from the data viewing interface to the data query interface.

5. The method according to claim 3, characterized in that: The step of displaying metadata information of at least one event field on the management platform interface in response to a registration instruction acting on the management platform interface includes: In response to a triggering operation acting on an import control in the management platform interface, presenting an import settings pop-up window in the management platform interface; In response to a confirmed import instruction in the import setting pop-up window, the metadata information of the at least one event field in the metadata description table indicated by the confirmed import instruction is displayed on the management platform interface.

6. The method according to claim 5, characterized in that The method further comprises: In response to a template download instruction in the import setting pop-up window, obtaining a metadata description template indicated by the template download instruction; Based on the metadata description template, the metadata description table is determined.

7. The method according to any one of claims 1 to 6, characterized in that: The data query interface includes: a first display area and a second display area; the determining of query information in response to the setting instruction acting on the data query interface includes: In response to a setting instruction acting on at least one setting control in the first display area, the query information is determined, and a logic statement corresponding to the setting instruction is synchronously displayed in the second display area; the logic statement corresponds to the query information.

8. The method according to any one of claims 1 to 6, characterized in that: The data query interface includes: a second display area; the setting instructions include: an edit instruction and a write instruction; The step of determining the query information in response to the setting instruction acting on the data query interface includes: In response to the editing instruction acting on the second display area, setting the second display area to a write-allowed state; In response to the write instruction acting on the second display area, presenting a logic statement corresponding to the write instruction in the second display area; The logical statement is parsed to determine the query information.

9. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: In response to the wide table building instruction, displaying a wide table building interface; In response to a construction instruction acting on the wide table construction interface, determining the first wide table according to a target data source indicated by the construction instruction; The step of determining the first wide table in response to a construction instruction acting on the wide table construction interface and according to a target data source indicated by the construction instruction includes: In response to a construction instruction acting on the wide table construction interface, determining a first attribute field of each event field in the target data source indicated by the construction instruction; Determine the maximum value of the number of the first attribute fields in each event field as the first number of at least one first business field; For each first attribute field in each event field in the target data source, a mapping relationship is established between each first attribute field and any first business field in the first number of first business fields, thereby obtaining the first wide table.

10. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: In response to the wide table conversion instruction, displaying a wide table conversion interface; In response to a conversion instruction acting on the wide table conversion interface, performing dimensionality reduction processing on the second wide table to obtain the first wide table; The step of performing dimensionality reduction processing on the second wide table to obtain the first wide table in response to the conversion instruction acting on the wide table conversion interface includes: In response to a conversion instruction acting on the wide table conversion interface, determining the number of first attribute fields in each event field in the second wide table; taking the maximum value of the number of first attribute fields in each event field in the second wide table as the second number of at least one first business field in the first wide table; For each first attribute field in each event field in the second wide table, a mapping relationship is established between each first attribute field and any first business field in the second number of first business fields, thereby completing the dimensionality reduction processing of the second wide table and obtaining the first wide table.

11. The method according to any one of claims 1 to 10, characterized in that: The first wide table also includes a second attribute field; the second attribute field is an attribute field common to each event field in the first wide table; the sum of the number of the second attribute fields and the number of the first business fields is the dimension of the first wide table.

12. A data processing device, characterized in that: It includes a determination unit and a query unit; wherein, The determining unit is used to determine the query information in response to the setting instruction acting on the data query interface; The query unit is used to query in the first wide table in response to a query instruction acting on the data query interface when the query information indicates that the data table to be queried is a first wide table, so as to obtain the target data corresponding to the query information; wherein the number of first business fields in the first wide table is equal to the maximum value of the number of first attribute fields corresponding to each event field.

13. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the data processing method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer storage medium stores a computer program, and when the computer program is executed by at least one processor, the data processing method according to any one of claims 1 to 11 is implemented.

Images