Data processing method, electronic equipment and medium

Abstract

The invention discloses a data processing method, electronic equipment and a medium, and relates to the technical field of data center asset management and computers. According to the method, the expansion table is dynamically created under the drive of the metadata, and under the condition that a database table structure is not transformed on a large scale, the expansion attribute is recorded, the maintenance cost is reduced, and the storage efficiency is improved; the expansion table is directly created according to metadata information, metadata provides the capability of dynamically defining a data structure, a system is allowed to dynamically generate a table structure and a field according to actual requirements, storage expansion attributes are flexibly achieved, and the situation that the utilization rate of storage space is low or insufficient is avoided; by splicing the query statements instead of modifying the original query statements and logics, the query efficiency of the extended attributes is improved; through the cooperation of splicing the query statement and intercepting the operation of the target query interface before executing the data query statement, the decoupling with the existing service is realized, and the influence on the existing service is avoided.

Description

Technical Field

[0001] The present invention relates to the fields of data center asset management and computer technology, and in particular, to a data processing method, an electronic device, and a medium. Background Art

[0002] With the continuous expansion of the scale of data centers, enterprises' demand for refined management of informatization assets (such as devices like servers and switches) is increasing day by day. In practical applications, different customers or different business scenarios may need to record some unique attribute information.

[0003] In related technologies, the method of updating the MySQL database table is adopted to record the extended attribute information. Modifying the database table structure to store the extended attributes results in high maintenance costs. Moreover, when the extended fields are stored in the secondary table, their field lengths are already fixed and cannot be dynamically adjusted according to actual needs, leading to low storage space utilization and poor flexibility. In related technologies, the method of creating reserved blank fields is also used to record the extended attribute information, but the reserved fields are usually set with fixed lengths, also resulting in low storage space utilization and poor flexibility. In addition, in order to be able to read the extended attributes of assets, the original query statements and logic are modified in related technologies, resulting in low query efficiency of the extended attributes.

[0004] Therefore, it can be seen that how to flexibly and quickly implement the extension of asset attributes and improve the query efficiency of extended attributes is a technical problem that needs to be urgently solved by those skilled in the art. Summary of the Invention

[0005] The present invention provides a data processing method, an electronic device, and a medium to at least solve the problems of poor flexibility and low efficiency in implementing the extension of asset attributes and low query efficiency of extended attributes in related technologies.

[0006] The present invention provides a data processing method, including: obtaining metadata information and creating an extended table according to the metadata information; wherein, the metadata information is used to represent the extended attribute information of the devices in the data center; intercepting a target query interface before executing a data query statement to obtain the device for which the extended attribute is to be queried; splicing the data query statement used to represent the query of the extended attribute, and using the spliced data query statement as a new data query statement; executing the new data query statement to obtain the extended attribute information of the device for which the extended attribute is to be queried from the extended table.

[0007] The beneficial effects of the present invention are as follows. First, in this method, since the extended table is established based on metadata information, when the metadata information changes, correspondingly, the extended table will also change. That is, the present invention realizes the dynamic creation of the extended table under the drive of metadata, enabling the recording of extended attributes without large-scale transformation of the database table structure, reducing the system maintenance cost, and being able to quickly respond to changes in customer or business requirements, thereby improving the efficiency of storing extended attributes. Second, compared with the method of setting fixed-length fields as extended fields, in the method provided by the present invention, the extended table is directly created according to the metadata information. The metadata provides the ability to dynamically define the data structure, allowing the system to dynamically generate the table structure and fields according to actual needs, flexibly realizing the storage of extended attributes, and without the need to pre-fix the number and type of fields, avoiding the occurrence of low or insufficient storage space utilization. Third, when reading the extended attributes, by splicing the query statement instead of modifying the original query statement and logic, the query efficiency of the extended attributes is improved. In addition, through the cooperation of splicing the query statement and intercepting the target query interface operation before executing the data query statement, the decoupling from the existing business is achieved, avoiding the impact on the existing business when querying data.

[0008] The present invention also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above data processing methods when executing the computer program.

[0009] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program realizes the steps of any of the above data processing methods when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0011] Figure 1 It is a schematic diagram of a MySQL database table provided by an embodiment of the present invention;

[0012] Figure 2 It is a schematic diagram of a scenario of a data processing method provided by an embodiment of the present invention;

[0013] Figure 3 It is a schematic diagram of a device for dynamically expanding asset attribute fields driven by metadata provided by an embodiment of the present invention;

[0014] Figure 4Flowchart of a data processing method provided by an embodiment of the present invention;

[0015] Figure 5 Schematic diagram of a database table structure provided by an embodiment of the present invention;

[0016] Figure 6 Partial schematic diagram of a metadata structure provided by an embodiment of the present invention;

[0017] Figure 7 Schematic diagram of an extended table generated according to metadata provided by an embodiment of the present invention;

[0018] Figure 8 Flowchart of a processing method of a structured query language statement interceptor provided by an embodiment of the present invention. Detailed implementation manners

[0019] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0020] It should be noted that in the description of the present invention, the terms "include", "comprise" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present invention are used to distinguish similar objects and not to describe a specific order or sequence.

[0021] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0022] Figure 1 Schematic diagram of a MySQL database table provided by an embodiment of the present invention, as Figure 1As shown in the figure, the MySQL table adopts the main and sub-table method. Among them, the main table stores the initially defined fields, and the sub-table changes the columns into rows; the main table includes two fields, namely the main table identifier (id) and whether there are extended fields; whether there are extended fields is used to identify whether the main table has extended fields; the sub-table includes four fields, namely id, and the field name, field type, and field value of the extended field. The main and sub-tables are associated through the id of the main table. It should be noted that: the database table describes the columns included in the database table, so it shows the content of each column in the database table. The extension of asset attributes usually requires modifying the database table structure. This method has a long development cycle, high maintenance costs, and is difficult to quickly respond to changes in customer or business requirements. Moreover, the values of the extended fields are stored in the sub-table, and their field lengths are fixed during initialization and cannot be dynamically adjusted according to actual needs, resulting in low storage space utilization and insufficient flexibility.

[0023] In addition, when creating the asset table, reserved blank fields are extracted. For example, reserved fields are preset in advance in the asset table, and the mapping relationship between the reserved fields and the extended field names is maintained when creating the extended fields. The number of reserved fields is fixed, and there may be a problem that when the number of extended fields exceeds the number of reserved fields, further extension is not possible. Moreover, the setting of reserved fields is not flexible enough. Usually, it is set to a fixed length. Even if the actual data length is short, it will occupy the storage space of the fixed length, resulting in low storage efficiency.

[0024] When querying the extended attributes, by modifying the original query statement and logic, the query efficiency of the extended attributes is low.

[0025] Therefore, in order to flexibly and quickly implement the extension of asset attributes and improve the query efficiency of extended attributes, the present invention provides a new data processing method. Figure 2 It is a schematic diagram of the scenario of a data processing method provided by an embodiment of the present invention. As Figure 2 shown, the front end and the back end interact. The back end establishes an extended attribute table, the front end queries the extended attributes, and the back end updates the extended table according to the information sent by the front end to update the extended table.

[0026] In order to facilitate those skilled in the art to better understand the entire processing process of the data processing method of the present invention, the following provides a dynamic extension device for asset attribute fields based on metadata driving. Figure 3 It is a schematic diagram of a dynamic extension device for asset attribute fields based on metadata driving provided by an embodiment of the present invention. As Figure 3As shown in the figure, the device includes: a metadata management module, an asset data storage module, an asset data query module, an attribute update module, a front-end display and interaction module, and a Structured Query Language (SQL) statement optimizer. Among them, the metadata management module is used to maintain and manage the metadata of the extended attribute information, and drives the asset data storage module to manage the dynamic creation and management of the extended tables and extended fields of various types of asset attributes; the asset data query module uses an SQL interceptor to implement the dynamic query of extended attributes and the unified encapsulation and processing of data; the front-end display and interaction module implements the retrieval of extended attributes and the dynamic rendering and interaction processing of attribute data; the attribute update module implements the dynamic update of the extended attribute values; the SQL optimizer monitors and quantitatively analyzes the query efficiency of extended field filtering, and dynamically creates indexes.

[0027] Combined with Figure 3 the metadata-driven dynamic extension device of asset attribute fields in [reference], the embodiments of the data processing method provided by the present invention will be specifically described below.

[0028] Figure 4 As shown in Figure 4 the flowchart of a data processing method provided by an embodiment of the present invention, the method includes:

[0029] S10: Obtain metadata information and create an extended table according to the metadata information; wherein, the metadata information is used to represent the extended attribute information of the devices in the data center.

[0030] S11: Intercept the target query interface before executing the data query statement to obtain the device for which the extended attribute is to be queried.

[0031] S12: Concatenate the data query statement used to represent the query of the extended attribute, and use the concatenated data query statement as the new data query statement.

[0032] S13: Execute the new data query statement to obtain the extended attribute information of the device for which the extended attribute is to be queried from the extended table.

[0033] The metadata information is not limited. For example, the metadata information at least includes attribute information, field information, device type information, and information used to characterize the characteristics of the attribute information when presented on the front end. Specifically, the metadata information includes attribute name, field name, asset type, field type, field length, whether it is required, whether it is presented in a list, whether sorting is supported, whether filtering is supported, and uniqueness.

[0034] The extended table belongs to the database table, and the database table also includes a main table and a sub-table; the main table includes the device type and the common attribute fields of various devices; the sub-table includes the unique attribute fields of various devices; the extended table, the main table, and the sub-table are associated through the unique identifier of the main table. Figure 5A schematic diagram of a database table structure provided by an embodiment of the present invention is as follows Figure 5 As shown, the database table structure includes an asset master table, sub-tables of various types of assets, a metadata table, and extended tables of various types of assets. The asset master table includes an identifier (id) and an asset identifier (asset id). The asset type and common attribute fields of other various types of assets are recorded in the asset master table. Unique attribute fields of other various types of assets are recorded in the sub-tables of various types of assets. The metadata table includes asset type (asset_type), attribute name (display_name), field name (table_filed), field type (data_type), field length (length), whether it is required (required), whether it is presented in the list (list_display), whether sorting is supported (sorted), whether screening is supported (screening), and uniqueness (unique).

[0035] When obtaining extended attributes from a MySQL database table, since the extended fields are stored in the sub-table, the main table and the sub-table need to be associated during the query, and the extended fields are "converted into rows" through a JOIN statement to restore a complete record. In the scenario where multiple extended fields need to be queried, the Structured Query Language statement (SQL) may involve multiple levels of nesting or multiple JOIN operations, resulting in a long query statement, low execution efficiency, and difficulty in maintenance. When both the extended fields and the field values are stored in the sub-table, each extended field occupies one row of data, resulting in complex logic and a significant reduction in query performance for paging, sorting, or filtering operations on the extended fields. Therefore, in order to improve the query efficiency, creating an extended table according to the metadata information in the present invention includes:

[0036] Obtain multiple extended attribute information of the same device recorded in the metadata information;

[0037] Store multiple extended attribute information belonging to the same device in the same row to form an extended table.

[0038] Figure 6 A partial schematic diagram of a metadata structure provided by an embodiment of the present invention is as follows Figure 6 As shown, the attribute name includes grouping and business attribution, and the field names include extended field a and extended field b. Figure 7 A schematic diagram of an extended table generated according to the metadata provided by an embodiment of the present invention is as follows Figure 7 As shown, all extended attributes of Asset 1 are in the same row, and all extended attributes of Asset 2 are in the same row.

[0039] In the method provided by the present invention, multiple extended attributes of the same asset are placed on the same line, and the SQL statement does not need to involve multiple nestings or multiple joins, improving the query efficiency.

[0040] The establishment of the extended table is achieved through the metadata management module. In implementation, the process of the metadata management module establishing the extended table includes:

[0041] (1) Define the metadata model. Define a metadata model for describing the extended information of asset attributes, which includes the following content:

[0042] a) Attribute name: The name of the attribute input by the user, used for presentation on the front-end page;

[0043] b) Field name: The field name of the attribute in the extended table, automatically generated by the backend according to the rules;

[0044] c) Asset type: The asset type to which the attribute belongs, identifying which asset type's extended attribute this attribute is, such as server for server, switch for switch, etc.;

[0045] d) Field type: The data type of the attribute, such as string, integer, date, etc.;

[0046] e) Field length: The maximum length of the attribute;

[0047] f) Whether it is required: Whether the attribute can be null and whether it is required during editing;

[0048] g) Whether to be presented in the list: Identifying whether this attribute information is to be presented in the asset list;

[0049] h) Whether sorting is supported: Identifying whether this attribute information can be sorted during list presentation;

[0050] i) Whether filtering is supported: Identifying whether this attribute information can be filtered and queried during list presentation:

[0051] j) Uniqueness: The user selects whether uniqueness verification is required.

[0052] (2) Design the metadata table structure, as Figure 5 shown in the metadata table in the database table structure, including the following fields: primary key id, asset type, attribute name, field name, field type, field length, whether it is required, whether to be presented in the list, whether sorting is supported, whether filtering is supported, whether it is unique.

[0053] (3) Formulate the field name naming rule.

[0054] On the page, the user only needs to select or fill in the asset type, attribute name, field type, field length, whether it is required, whether it is presented in the list, whether sorting is supported, whether filtering is supported, and whether it is unique. The field name is automatically generated by the backend according to the rules, and the generation rules are as follows:

[0055] {asset_type}_ext{count};

[0056] Among them, asset_type represents the asset type, and count is the number of extended attributes of this type of asset in the current metadata table. For example, if a property information of a server asset is custom extended and the count is 0, the generated field name is server_ext0.

[0057] (4) The addition, deletion, modification, and query of metadata are implemented through interfaces, supporting operations of creating, reading, updating, and deleting metadata.

[0058] (5) Metadata change notification. When metadata changes, other modules (such as the asset data storage module) are made aware of the metadata change by publishing a metadata change event so as to perform corresponding operations.

[0059] The metadata management module defines the extended attribute metadata model and the metadata table structure; implements the addition, deletion, modification, and query of extended attribute metadata; publishes metadata change events. That is, it defines the extended attribute metadata model and the metadata table, which are used to manage and maintain extended attribute metadata information and change notifications. Through the metadata management module, the metadata of extended attribute information is maintained and managed, driving the asset data storage module to realize the dynamic creation and management of extended tables and extended fields of various types of asset attributes.

[0060] In practice, the metadata may change. Therefore, it is necessary to update the fields in the extended table. Specifically, after obtaining the metadata information and before creating the extended table according to the metadata information, it also includes:

[0061] If information indicating the addition of metadata is received, obtain the devices recorded in the information of the added metadata;

[0062] Judge whether the extended table corresponding to the device recorded in the metadata addition information exists;

[0063] If so, enter the step of intercepting the target query interface before executing the data query statement to obtain the device of the extended attribute to be queried;

[0064] If not, enter the step of creating an extended table according to the metadata information;

[0065] After obtaining the metadata information and creating the extended table according to the metadata information, it also includes:

[0066] Receive information for characterizing metadata deletion or update;

[0067] Delete or update the extended fields of the extended table according to the information for characterizing metadata deletion or update.

[0068] In this method, the update of the extended table is implemented according to the information of metadata addition, deletion or update.

[0069] The above process describes the establishment of an extended table according to metadata. After the extended table is established, there is usually a need for the front end to query the extended attribute information of the device. In order to facilitate the front end to read the extended attribute information, before executing a new data query statement to obtain the extended attribute information of the device with the extended attribute to be queried from the extended table, it further includes:

[0070] Create a field for characterizing the extended attribute set;

[0071] After executing a new data query statement to obtain the extended attribute information of the device with the extended attribute to be queried from the extended table, it further includes:

[0072] Store the extended attribute information of the device with the extended attribute to be queried in the field for characterizing the extended attribute set.

[0073] In this method, the field for characterizing the extended attribute set is set in advance, making the organization and storage of data more standardized and orderly; when the user obtains the extended attribute, the extended attribute information can be found by locating these fields, improving the query efficiency.

[0074] The above process of implementing the update of the extended table according to the metadata information is implemented by the asset data storage module. In implementation, the process of the asset data storage module implementing the update of the extended table includes:

[0075] (1) The asset main table and the asset sub-table, the table structure design of which is as Figure 5 shown in the asset main table and various types of asset sub-tables in the database table structure.

[0076] (2) If a metadata addition message is received, dynamically create an extended table and extended attributes.

[0077] a. Judge whether the extended table of this asset type exists. If it exists, go to the next step; otherwise, create an extended table. The naming rule of the extended table is:

[0078] asset_{asset_type}_ext;

[0079] Among them, asset_type represents the asset type. For example, the extended table name of the server is asset_server_ext, and the extended table name of the switch is asset_switch_ext. The extended table structure is attached Figure 3 Extended tables of each asset type in the database table structure;

[0080] (3)If a metadata deletion message is received, the extended fields of the corresponding extended table are deleted;

[0081] (4)If a metadata update message is received, the extended fields of the corresponding extended table are updated, such as setting the field length value, etc.

[0082] The asset data storage module receives the messages published by the metadata management module, dynamically creates, updates, or deletes extended tables and extended fields, and realizes the dynamic management of the extended tables and attribute fields of various types of asset attributes.

[0083] In order to query extended attributes, in the related art, the original query statements and logics are modified, resulting in low query efficiency. Therefore, in the present invention, in order to improve the query efficiency, the target query interface is intercepted before executing the data query statement to obtain the device for which the extended attributes are to be queried; the data query statement for representing the query of extended attributes is concatenated, and the concatenated data query statement is used as the new data query statement; after executing the new data query statement, the extended attribute information of the device for which the extended attributes are to be queried can be obtained from the extended table.

[0084] In practice, there may be no extended attribute information of some devices in the metadata information. Therefore, even if the extended table is queried, the extended attribute information cannot be obtained. Therefore, before concatenating the data query statement for representing the query of extended attributes and using the concatenated data query statement as the new data query statement, it further includes:

[0085] Judging whether the metadata information contains the extended attribute information of the device for which the extended attributes are to be queried;

[0086] If not, end;

[0087] If so, enter the step of concatenating the data query statement for representing the query of extended attributes and using the concatenated data query statement as the new data query statement.

[0088] Specifically, concatenating the data query statement for representing the query of extended attributes and using the concatenated data query statement as the new data query statement includes at least one of the following methods:

[0089] Method 1: Obtain the selection clause and obtain the extended attribute information of the device for which the extended attributes are to be queried recorded in the metadata information;

[0090] According to the extended attribute information of the device with the extended attribute to be queried recorded in the metadata information, splice the extended attribute fields to be queried and displayed into the selection clause to obtain a new data query statement;

[0091] Method 2: Obtain the from clause and the statement for characterizing the associated extended table;

[0092] Splice the statement for characterizing the associated extended table in the from clause to obtain a new data query statement;

[0093] Method 3: Obtain the where clause, filtering conditions, and query parameters;

[0094] Splice the where clause according to the filtering conditions and query parameters to obtain a new data query statement.

[0095] In this method, by splicing the data query statement for characterizing the query of extended attributes, after executing the new data query statement, the found extended attribute information meets the requirements and is more complete.

[0096] After executing the new data query statement, the found fields may not belong to the extended attribute fields. To ensure that the found results belong to the extended attribute information, before storing the extended attribute information of the device with the extended attribute to be queried in the field for characterizing the extended attribute set after executing the new data query statement, it further includes:

[0097] Obtain the result set obtained after executing the new data query statement;

[0098] Traverse the result set and check whether each field is an extended attribute field;

[0099] If so, enter the step of storing the extended attribute information of the device with the extended attribute to be queried in the field for characterizing the extended attribute set.

[0100] In this method, by traversing the result set and checking whether each field is an extended attribute field, only when it is determined that all fields are extended attribute fields, the extended attribute information of the device with the extended attribute to be queried is stored in the field for characterizing the extended attribute set, ensuring that the content stored in the field for characterizing the extended attribute set is all extended attribute fields.

[0101] After executing the new data query statement, in addition to querying the extended attribute information, in practice, information other than the extended attribute information may also be queried. To obtain complete query information, executing the new data query statement to obtain the extended attribute information of the device with the extended attribute to be queried from the extended table includes: executing the new data query statement to obtain a data object containing the extended attribute information of the device with the extended attribute to be queried from the database table.

[0102] The query process is specifically implemented through the asset data query module. In implementation, the query process of the asset data query module specifically includes:

[0103] (1) Modify the asset data model. In the original data model returned to the front end, add the extendedAttributes field, which is an extended attribute set and its type is List <extendedattributes>, where the extendedAttribute class contains metadata information such as attribute name, field name, whether sorting is supported, etc., and attribute values;

[0104] (2) Implementation of a Structured Query Language statement interceptor (i.e., SQL interceptor).

[0105] The SQL interceptor is used for dynamic splicing of SQL statements for extended attribute queries and data encapsulation. Since both the attribute extension table and extended fields are created dynamically, the traditional way of writing static SQL can no longer meet the queries of dynamically extended attributes. Therefore, an SQL interceptor is created to implement the query of extended attributes and the encapsulation of the returned result object. Figure 8 The flowchart of a processing method for a Structured Query Language statement interceptor provided by an embodiment of the present invention is as Figure 8 shown, and the method includes:

[0106] S14: Create an interceptor and configure the interface method to be intercepted;

[0107] S15: Dynamically modify and splice the query statement based on parameters before the execution of the Structured Query Language statement;

[0108] S16: Intercept the execution result after the execution of the Structured Query Language statement and obtain the result set;

[0109] S17: Traverse the result set and store the extended attribute metadata information and attribute values into the extended attribute set of the asset data model object.

[0110] In implementation, the Structured Query Language statement interceptor specifically executes the following steps:

[0111] a. Create and register an interceptor, and configure the query interface method to be intercepted and processed;

[0112] b. Intercept the specified query interface before SQL execution and dynamically modify the SQL query statement. The steps are as follows:

[0113] Query the extended attribute information in the metadata table according to the asset type. If it does not exist, directly skip the interceptor without performing any operations. Otherwise, proceed to the next step;

[0114] Dynamically splice the extended attribute fields that need to be queried and displayed into the selection clause (i.e., the SELECT clause) according to the definition in the queried metadata;

[0115] Dynamically associate the attribute extension table and dynamically splice the statement for associating the extension table in the from clause (i.e., the FROM clause);

[0116] Dynamically splice the conditional clause (i.e., the WHERE clause) according to the filtering conditions and query parameters.

[0117] c. Intercept the execution result of the dynamically constructed SQL statement to obtain the result set;

[0118] d. Traverse the result set and check whether each field is an extended attribute field. If so, store the attribute value and the metadata information of the corresponding extended field in the extended attribute set in the asset data model;

[0119] f. After the processing is completed, return the asset data object including the extended attribute information.

[0120] The asset data query module adds an extended attribute set attribute to the asset data model; creates an SQL interceptor to implement the dynamic query and unified data processing encapsulation of the extended attribute data and information.

[0121] To improve the query efficiency of the extended attributes, after concatenating the data query statement used to represent the query of the extended attributes and using the concatenated data query statement as the new data query statement, it further includes:

[0122] Store the new data query statements corresponding to different device types;

[0123] Use the stored new data query statements corresponding to different device types as the sample data query statements;

[0124] Train a neural network model using the device type and the sample data query statements to obtain the trained neural network model;

[0125] After determining the device with the new extended attribute to be queried, obtain the current device type;

[0126] Input the current device type into the trained neural network model;

[0127] Output the new data query statement corresponding to the current device type through the trained neural network model.

[0128] There is no limitation on the selected neural network model, which is determined according to the actual situation.

[0129] In this method, by training the neural network model using the device type and the sample data query statements, after obtaining the device with the new extended attribute to be queried, the new data query statement corresponding to this device can be directly output, improving the efficiency of obtaining the data query statement, and thus improving the query efficiency of the extended attributes.

[0130] The backend executes the data processing method described above. Although the extended attribute information is queried in the above process, in order to facilitate the front-end to view the queried extended attribute information, the extended attribute information needs to be presented on the page. In the process of presenting the extended attribute information to the front-end, in order to make the presented extended attribute information meet the user's needs, after executing a new data query statement to obtain the extended attribute information of the device with the extended attribute to be queried from the extended table, it further includes:

[0131] Obtain the asset list filtering conditions of the device with the extended attribute to be queried;

[0132] Render the extended attribute columns according to the asset list filtering conditions;

[0133] Traverse the fields used to represent the extended attribute set;

[0134] Display the attribute names and attribute values in the extended attributes on the front-end page.

[0135] Specifically, obtaining the asset list filtering conditions of the device with the extended attribute to be queried includes:

[0136] Obtain the metadata information of the device with the extended attribute to be queried;

[0137] Obtain the information used to represent the characteristics of the attribute information when presented on the front-end;

[0138] Determine the asset list filtering conditions of the device with the extended attribute to be queried according to the information used to represent the characteristics of the attribute information when presented on the front-end; wherein, the characteristics of the attribute information when presented on the front-end at least include one of the following: supporting or not supporting filtering dynamic rendering query conditions, supporting or not supporting presenting in the list, and supporting or not supporting sorting.

[0139] After presenting the extended attributes on the page, the user may have a need to update the extended attributes. To meet the user's need to update the extended attributes, the data processing method further includes:

[0140] Pre-create a new extended attribute set field; wherein, the new extended attribute set field is used to store the field names and attribute values of the extended attributes to be modified;

[0141] Receive the information of the parameters used to represent the device attribute update sent by the front-end;

[0142] Determine the extended attribute set to be modified according to the information of the parameters used to represent the device attribute update;

[0143] Execute a query statement used to represent the extended attribute update according to the new extended attribute set field, the information of the parameters used to represent the device attribute update, and the extended attribute set to be modified, so as to complete the update operation of the extended attributes.

[0144] The process of displaying extended attributes is implemented through the front-end display and interaction module. In implementation, the process of the front-end display and interaction module implementing the display of extended attributes specifically includes:

[0145] (1)Dynamic rendering and display of the asset list filtering conditions and the table.

[0146] a. Obtain the metadata list of the extended attributes of assets of this type according to the asset category;

[0147] b. Traverse the metadata list;

[0148] c. Dynamically render the query conditions according to whether the metadata information supports filtering;

[0149] d. Dynamically render the extended attribute columns in the asset list table according to whether the metadata information supports presentation in the list and whether it supports sorting;

[0150] e. Based on the set query conditions, the asset list data can be obtained through the asset data query module, and the basic asset attributes and dynamic extended attribute data are dynamically filled.

[0151] (2)Dynamic display of extended attributes in the asset detailed information: Obtain the asset data object through the asset data query module, traverse the extended attribute set in the data object, and dynamically present the attribute names and attribute values in the extended attributes on the front-end page.

[0152] (3)Provide an editing function for asset attributes (including dynamically presented extended attributes), and interact with the extended attribute update module to achieve dynamic update of extended attributes.

[0153] Through the front-end display and interaction module, the dynamic rendering and presentation of the asset data list filtering conditions and the asset extended attribute data are realized; the dynamic display, presentation and interaction of the extended attribute information and data are realized.

[0154] In order to achieve accurate update of extended attributes, before executing the query statement for characterizing extended attribute update according to the newly added extended attribute set fields, the information of the parameters for characterizing device attribute update, and the extended attribute set to be modified, it also includes:

[0155] Query the metadata information of the extended attributes according to the attribute field names of the extended attribute set to be modified;

[0156] Perform parameter verification on the extended attribute set to be modified according to the metadata information; among them, the parameter verification at least includes: verifying the data type and / or uniqueness detection;

[0157] In the case of passing the verification, enter the step of executing the query statement for characterizing extended attribute update according to the newly added extended attribute set fields, the information of the parameters for characterizing device attribute update, and the extended attribute set to be modified.

[0158] The update of the extended attributes is achieved through this method, and parameter verification is performed according to the metadata information, improving the accuracy and security of the extended attribute update.

[0159] The process of updating the extended attributes is implemented through the attribute update module. In implementation, the specific steps of the attribute update module for updating the extended attributes include:

[0160] (1) Modify the original update parameter model, add an extended attribute set field, whose type is Map<String, Object>, and is used to store the extended attribute field names and attribute values to be modified;

[0161] (2) Receive the asset attribute update parameters submitted by the front-end display and interaction module, and obtain the extended attribute set to be modified;

[0162] (3) Traverse the extended attribute set, query the metadata information of the extended attribute according to the attribute field name, and perform parameter verification according to the metadata information, such as data type, uniqueness detection, etc.;

[0163] (4) Dynamically construct the extended attribute update SQL;

[0164] (5) Execute the update SQL to complete the update operation of the attribute value.

[0165] The attribute update module receives the update request, dynamically constructs and generates the SQL statement, completes the update of the extended attribute value, and realizes the dynamic update operation of the extended attributes of various types of assets.

[0166] The above process completes the query of the extended attribute information. However, when the query efficiency is low, it may affect the user experience. Therefore, to improve the query efficiency, the data processing method further includes:

[0167] Obtain the query efficiency when querying through the new data query statement;

[0168] In the case where it is detected that the query efficiency is lower than the query threshold, obtain the queried extended fields and create indexes for the queried extended fields.

[0169] There is no limit to the query threshold, which is determined according to the actual situation. For the case of low query efficiency, by creating indexes, the efficiency of subsequent queries to the extended fields is improved, thus improving the user experience.

[0170] To accurately determine the query efficiency, obtaining the query efficiency when querying through the new data query statement includes:

[0171] Obtain query metrics; among them, the query metrics at least include query execution time, the number of rows scanned during query execution, and the number of result rows returned by the query;

[0172] Obtain the product result obtained by multiplying the query execution time in the query metrics by the number of rows scanned during query execution.

[0173] Obtain the result obtained by dividing the number of rows returned by the query by the product result.

[0174] Use the result as the query efficiency when querying through the new data query statement.

[0175] In this method, the query efficiency is quantitatively calculated, realizing an accurate analysis of the query efficiency.

[0176] Determine that the query efficiency is achieved through the SQL optimizer. In implementation, the SQL optimizer determines the query efficiency including the following steps:

[0177] (1) When the SQL interceptor in the asset data query module executes the filtering query of a certain extended field, obtain the following query metrics:

[0178] Query Execution Time (QET): The execution time of a certain query.

[0179] Rows Scanned (RS): The number of rows scanned during the execution of a certain query.

[0180] Rows Returned (RR): The number of rows returned by a certain query.

[0181] (2) Calculate the query efficiency score QES through the following formula, and based on this, quantitatively analyze the efficiency of the query. The lower the score, the worse the query efficiency and the more optimization is needed:

[0182] ;

[0183] RR / RS represents the ratio of the number of rows returned to the number of rows scanned, reflecting the selectivity of the query.

[0184] 1 / QET represents the reciprocal of the execution time, reflecting the query speed.

[0185] (3) Configure a scoring threshold (for example, 0.1). When it is detected that QES is lower than the preset threshold, dynamically create an index for this extended field to optimize the retrieval query efficiency of the extended field.

[0186] Obtain the three metrics of execution time, number of rows scanned, and number of rows returned during the filtering query of the extended attribute through the SQL optimizer, and calculate the quantitative query efficiency; when the query efficiency is lower than the threshold, dynamically create an index, realizing the monitoring analysis and dynamic optimization of the filtering query efficiency of the extended attribute.

[0187] Through the data processing method provided by the present invention, the following is achieved:

[0188] 1) Improve the flexibility of asset management: Through the metadata-driven method, extension tables and extension attributes can be dynamically created, facilitating the addition, modification, and deletion of asset extension attribute fields without large-scale modification of the existing database table structure, reducing the system maintenance cost;

[0189] 2) Optimize queries: Through the dynamic creation of extension tables and extension attributes, an efficient join query between the asset table and the extension table is achieved. By means of an SQL interceptor, the query statement is dynamically concatenated and the result of the extension attribute information is encapsulated, reducing the intrusion of the original business code; and through the monitoring and analysis module, the query performance of the extension attribute fields is monitored in real time, and indexes are dynamically added, which can effectively improve the efficiency of filtering and paging queries and enhance the user experience;

[0190] 3) Achieve front-end and back-end collaboration: The front-end display module can perform dynamic rendering and interactive processing based on the extension attribute set data returned by the back-end, realizing seamless docking of front-end and back-end data, and improving the overall performance and maintainability of the system.

[0191] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation manner.

[0192] The embodiment of the present invention also provides a data processing device, including:

[0193] A first acquisition module, configured to acquire metadata information and create an extension table according to the metadata information; wherein, the metadata information is used to represent the extension attribute information of the devices in the data center;

[0194] An interception module, configured to intercept a target query interface before executing a data query statement to obtain the device for which the extension attribute is to be queried;

[0195] A concatenation module, configured to concatenate a data query statement used to represent the query of the extension attribute, and use the concatenated data query statement as a new data query statement;

[0196] A first execution module, configured to execute the new data query statement to obtain the extension attribute information of the device for which the extension attribute is to be queried from the extension table.

[0197] The data processing device includes a first creation module, configured to create an extension table according to the metadata information;

[0198] The first creation module specifically includes:

[0199] A second acquisition module, configured to acquire multiple extension attribute information of the same device recorded in the metadata information;

[0200] A storage module for storing multiple pieces of extended attribute information belonging to the same device in the same row to form an extended table.

[0201] In some embodiments, the first execution module is specifically configured to execute a new data query statement to obtain a data object containing the extended attribute information of the device with the to-be-query extended attribute from a database table.

[0202] In some embodiments, the data processing device further includes:

[0203] A third acquisition module for acquiring the device recorded in the information indicating the addition of metadata if the information indicating the addition of metadata is received;

[0204] A first judgment module for judging whether the extended table corresponding to the device recorded in the metadata addition information exists; if so, triggering an interception module; if not, triggering a first creation module;

[0205] The data processing device further includes:

[0206] A first reception module for receiving information indicating the deletion or update of metadata;

[0207] A deletion or update module for deleting or updating the extended fields of the extended table according to the information indicating the deletion or update of metadata.

[0208] In some embodiments, the data processing device further includes:

[0209] A second creation module for creating a field for representing an extended attribute set;

[0210] The data processing device further includes:

[0211] A storage module for storing the extended attribute information of the device with the to-be-query extended attribute in the field for representing the extended attribute set.

[0212] In some embodiments, the data processing device further includes:

[0213] A second judgment module for judging whether the metadata information contains the extended attribute information of the device with the to-be-query extended attribute; if not, ending; if so, triggering a splicing module.

[0214] The splicing module specifically includes:

[0215] A fourth acquisition module for acquiring a selection clause and acquiring the extended attribute information of the device with the to-be-query extended attribute recorded in the metadata information;

[0216] The first splicing sub-module is used to splice the extended attribute fields to be queried and displayed into the selection clause according to the extended attribute information of the device with the extended attributes to be queried recorded in the metadata information, so as to obtain a new data query statement;

[0217] The splicing module specifically includes:

[0218] The fifth acquisition module is used to acquire the from clause and the statement for characterizing the associated extended table;

[0219] The second splicing sub-module is used to splice the statement for characterizing the associated extended table in the from clause to obtain a new data query statement;

[0220] The splicing module specifically includes:

[0221] The sixth acquisition module is used to acquire the where clause, the filtering conditions, and the query parameters;

[0222] The third splicing sub-module is used to splice the where clause according to the filtering conditions and the query parameters to obtain a new data query statement.

[0223] In some embodiments, the data processing device further includes:

[0224] The seventh acquisition module is used to acquire the result set obtained after executing the new data query statement;

[0225] The checking module is used to traverse the result set and check whether each field is an extended attribute field; if the checking result is that each field is an extended attribute field, the storage module is triggered.

[0226] In some embodiments, the data processing device further includes:

[0227] The eighth acquisition module is used to acquire the asset list filtering conditions of the device with the extended attributes to be queried;

[0228] The rendering module is used to render the extended attribute columns according to the asset list filtering conditions;

[0229] The traversing module is used to traverse the fields for characterizing the extended attribute set;

[0230] The display module is used to display the attribute names and attribute values in the extended attributes on the front-end page.

[0231] In some embodiments, the eighth acquisition module specifically includes:

[0232] The first acquisition sub-module is used to acquire the metadata information of the device with the extended attributes to be queried;

[0233] The second acquisition sub-module is used to acquire the information for characterizing the characteristics of the attribute information when presented on the front end;

[0234] A first determination module, configured to determine a screening condition for an asset list of a device for which an extended attribute is to be queried according to information on characteristics of the attribute information when presented on the front end; wherein, the characteristics of the attribute information when presented on the front end include at least one of the following: supporting or not supporting screening dynamic rendering query conditions, supporting or not supporting presentation in a list, and supporting or not supporting sorting.

[0235] In some embodiments, the data processing device further includes:

[0236] A third creation module, configured to pre-create a new extended attribute set field; wherein, the new extended attribute set field is used to store the names and attribute values of the extended attribute fields to be modified;

[0237] A second receiving module, configured to receive information on parameters sent from the front end for characterizing device attribute updates;

[0238] A second determination module, configured to determine an extended attribute set to be modified according to the information on parameters for characterizing device attribute updates;

[0239] A second execution module, configured to execute a query statement for characterizing extended attribute updates according to the new extended attribute set field, the information on parameters for characterizing device attribute updates, and the extended attribute set to be modified, so as to complete the update operation of the extended attribute.

[0240] In some embodiments, the data processing device further includes:

[0241] A query module, configured to query metadata information of the extended attribute according to the attribute field name of the extended attribute set to be modified;

[0242] A verification module, configured to perform parameter verification on the extended attribute set to be modified according to the metadata information; wherein, the parameter verification includes at least: verifying the data type and / or performing uniqueness detection; in the case where the verification passes, triggering the second execution module.

[0243] In some embodiments, the data processing device further includes:

[0244] A ninth acquisition module, configured to acquire the query efficiency when querying through a new data query statement;

[0245] A tenth acquisition module, configured to acquire the extended fields queried and create indexes for the extended fields queried in the case where the detected query efficiency is lower than the query threshold.

[0246] The ninth acquisition module specifically includes:

[0247] A third acquisition sub-module, configured to acquire query metrics; wherein, the query metrics include at least query execution time, the number of rows scanned during query execution, and the number of result rows returned by the query;

[0248] A fourth acquisition sub-module, configured to acquire a product result obtained by multiplying the query execution time in the query metrics by the number of rows scanned during query execution;

[0249] A fifth acquisition sub-module, configured to acquire a result obtained by dividing the number of result rows returned by the query by the product result;

[0250] As a module, configured to use the result as the query efficiency when querying through a new data query statement.

[0251] For the descriptions of the features in the embodiments corresponding to the data processing device, reference may be made to the relevant descriptions of the embodiments corresponding to the data processing method, which will not be elaborated here one by one.

[0252] An embodiment of the present invention further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above data processing method embodiments.

[0253] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any one of the above data processing method embodiments when running.

[0254] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as a USB flash drive, a read-only memory (ROM for short), a random access memory (RAM for short), a mobile hard disk, a magnetic disk, or an optical disc that can store a computer program.

[0255] An embodiment of the present invention further provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any one of the above data processing method embodiments.

[0256] An embodiment of the present invention further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any one of the above data processing method embodiments.

[0257] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0258] The above has introduced in detail a data processing method, an electronic device, and a medium provided by the present invention. Specific examples are used herein to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.< / extendedattributes>

Claims

1. A data processing method, characterized in that, Including: Obtain metadata information and create an extended table according to the metadata information; wherein, the metadata information is used to characterize the extended attribute information of the devices in the data center; Intercept the target query interface before executing the data query statement to obtain the devices for which the extended attributes are to be queried; Concatenate the data query statement for characterizing the query of extended attributes, and use the concatenated data query statement as the new data query statement; Execute the new data query statement to obtain the extended attribute information of the devices for which the extended attributes are to be queried from the extended table.

2. The data processing method according to claim 1, wherein The metadata information at least includes attribute information, field information, device type information, and information for characterizing the characteristics of the attribute information when presented at the front end; Creating an extended table according to the metadata information includes: Obtain multiple extended attribute information records of the same device in the metadata information; Store the multiple extended attribute information belonging to the same device in the same row to form an extended table.

3. The data processing method according to claim 2, characterized in that The extended table belongs to a database table, and the database table also includes a main table and a sub-table; the main table includes device types and common attribute fields for various devices; the sub-table includes unique attribute fields for various devices; the extended table, the main table, and the sub-table are associated through the unique identifier of the main table; The executing the new data query statement to obtain the extended attribute information of the devices for which the extended attributes are to be queried from the extended table includes: Execute the new data query statement to obtain a data object containing the extended attribute information of the devices for which the extended attributes are to be queried from the database table.

4. The data processing method according to claim 2, wherein After obtaining the metadata information and before creating an extended table according to the metadata information, it further includes: If receiving information for characterizing the addition of metadata, obtain the devices recorded in the information for the addition of metadata; Judge whether the extended table corresponding to the devices recorded in the metadata addition information exists; If so, enter the step of intercepting the target query interface before executing the data query statement to obtain the devices for which the extended attributes are to be queried; If not, enter the step of creating an extended table according to the metadata information; After obtaining the metadata information and creating an extended table according to the metadata information, it further includes: Receive information for characterizing the deletion or update of metadata; Delete or update the extended fields of the extended table according to the information for characterizing the deletion or update of metadata.

5. The data processing method according to claim 3, wherein Before the executing the new data query statement to obtain the extended attribute information of the devices for which the extended attributes are to be queried from the extended table, it further includes: Create a field for characterizing the extended attribute set; After executing the new data query statement to obtain the extended attribute information of the devices for which the extended attributes are to be queried from the extended table, it further includes: Store the extended attribute information of the devices for which the extended attributes are to be queried in the field for characterizing the extended attribute set.

6. The data processing method according to claim 5, wherein Before the concatenating the data query statement for characterizing the query of extended attributes and using the concatenated data query statement as the new data query statement, it further includes: Judge whether the metadata information contains the extended attribute information of the devices for which the extended attributes are to be queried; If not, end; If so, enter the step of splicing the data query statement for characterizing the query extension attribute, and using the spliced data query statement as the new data query statement; The splicing of the data query statement for characterizing the query extension attribute and using the spliced data query statement as the new data query statement at least includes: Obtain the selection clause, and obtain the extension attribute information of the device with the to-be-query-extended attribute recorded in the metadata information; according to the extension attribute information of the device with the to-be-query-extended attribute recorded in the metadata information, splice the extension attribute fields to be queried and displayed into the selection clause to obtain the new data query statement; And / or, obtain the from clause and the statement for characterizing the associated extension table; splice the statement for characterizing the associated extension table in the from clause to obtain the new data query statement; And / or, obtain the where clause, the filtering condition and the query parameter; splice the where clause according to the filtering condition and the query parameter to obtain the new data query statement.

7. The data processing method according to claim 5 or 6, characterized in that, Before storing the extension attribute information of the device with the to-be-query-extended attribute in the field for characterizing the extension attribute set after executing the new data query statement, it further includes: Obtain the result set obtained after executing the new data query statement; Traverse the result set and check whether each field is an extension attribute field; If so, enter the step of storing the extension attribute information of the device with the to-be-query-extended attribute in the field for characterizing the extension attribute set.

8. The data processing method according to claim 6, wherein After executing the new data query statement to obtain the extension attribute information of the device with the to-be-query-extended attribute from the extension table, it further includes: Obtain the asset list filtering condition of the device with the to-be-query-extended attribute; Render the extension attribute column according to the asset list filtering condition; Traverse the field for characterizing the extension attribute set; Display the attribute name and attribute value in the extension attribute on the front-end page.

9. The data processing method according to claim 8, wherein The obtaining of the asset list filtering condition of the device with the to-be-query-extended attribute includes: Obtain the metadata information of the device with the to-be-query-extended attribute; Obtain the information for characterizing the characteristics of the attribute information when presented on the front end; Determine the asset list filtering condition of the device with the to-be-query-extended attribute according to the information for characterizing the characteristics of the attribute information when presented on the front end; wherein, the characteristics of the attribute information when presented on the front end at least include one of the following: supporting or not supporting filtering dynamic rendering query conditions, supporting or not supporting presenting in the list, and supporting or not supporting sorting.

10. The data processing method according to claim 8, characterized in that, It further includes: Pre-create a new extension attribute set field; wherein, the new extension attribute set field is used to store the field name and attribute value of the extension attribute to be modified; Receive the information of the parameter for characterizing the device attribute update sent by the front end; Determine the extension attribute set to be modified according to the information of the parameter for characterizing the device attribute update; Execute the query statement for characterizing the extension attribute update according to the new extension attribute set field, the information of the parameter for characterizing the device attribute update and the extension attribute set to be modified to complete the update operation of the extension attribute.

11. The data processing method according to claim 10, characterized in that, Before executing the query statement for characterizing the extended attribute update based on the newly added extended attribute set fields, the information of the parameters for characterizing the device attribute update, and the extended attribute set to be modified, it further includes: Query the metadata information of the extended attributes according to the attribute field names of the extended attribute set to be modified; Perform parameter verification on the extended attribute set to be modified according to the metadata information; wherein, the parameter verification at least includes: verifying the data type and / or detecting uniqueness; In the case where the verification passes, enter the step of executing the query statement for characterizing the extended attribute update based on the newly added extended attribute set fields, the information of the parameters for characterizing the device attribute update, and the extended attribute set to be modified.

12. The data processing method according to claim 1, characterized in that, It further includes: Obtain the query efficiency when querying through the new data query statement; In the case where it is detected that the query efficiency is lower than the query threshold, obtain the extended fields queried and create indexes for the extended fields queried.

13. The data processing method according to claim 12, characterized in that, The obtaining the query efficiency when querying through the new data query statement includes: Obtain query metrics; wherein, the query metrics at least include the query execution time, the number of rows scanned during query execution, and the number of result rows returned by the query; Obtain the product result obtained by multiplying the query execution time in the query metrics by the number of rows scanned during query execution; Obtain the result obtained by dividing the number of result rows returned by the query by the product result; Use the result as the query efficiency when querying through the new data query statement.

14. An electronic device, characterized in that, It includes: A memory for storing a computer program; A processor for implementing the steps of the data processing method according to any one of claims 1 to 13 when executing the computer program.

15. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the data processing method according to any one of claims 1 to 13 when executed by a processor.

Images