Script table processing method and device, equipment and medium

Abstract

The invention discloses a script table processing method, which can be applied to the financial field and the medical field, and comprises the following steps: determining a to-be-analyzed operation script; acquiring script attributes of the operation script, and determining a write-in table of the operation script according to the script attributes, the write-in table being a table of write-in data of the operation script; script keywords of the operation script are obtained, a source table of the operation script is determined according to the script keywords, and the source table is a table of data obtained by the operation script; and determining a target reference relationship network between the write-in table and the source table according to the script statements corresponding to the write-in table and the source table. According to the method, the script attribute and the script keyword of the operation script are determined, and the write-in table and the source table corresponding to the operation script are determined, so that the reference relation between the write-in table and the source table is established, accurate analysis of the operation script is achieved, the relation between the tables is combed into structured data which is easy to recognize and can be analyzed, the operation and maintenance efficiency is improved, and the user experience is improved. And the data risk is reduced.

Description

Technical Field

[0001] This invention relates to the fields of data processing, financial technology, and healthcare, and particularly to a script table processing method, apparatus, device, and medium. Background Technology

[0002] With the development of digital technology and the accelerated advancement of digital transformation, the scale of enterprise data assets is growing exponentially. Therefore, when maintaining large amounts of data, the existence of massive script files, complex relationship networks, and large-scale data warehouse references makes it impossible to accurately obtain table-level dependency networks, which is not conducive to the maintenance of script files and data between various tables. Summary of the Invention

[0003] This invention provides a script table processing method, apparatus, device, and medium to solve the problem that it is impossible to accurately obtain the table-level dependency network, which in turn leads to the lack of script files and low maintenance efficiency between table data.

[0004] Firstly, this disclosure provides a script table processing method, including: Identify the operation script to be parsed; Get the script attributes of the operation script, and determine the write table of the operation script based on the script attributes. The write table is the table where the operation script writes data. Obtain the script keywords of the operation script, and determine the source table of the operation script based on the script keywords. The source table is the table from which the operation script obtains data. Based on the script statements corresponding to the write table and the source table, determine the target reference network between the write table and the source table.

[0005] Secondly, this disclosure provides a script table processing apparatus, including: The script determination module is used to determine the operation script to be parsed; The attribute determination module is used to obtain the script attributes of the operation script, and determine the write table of the operation script based on the script attributes. The write table is the table in which the operation script writes data. The keyword determination module is used to obtain the script keywords of the operation script. Based on the script keywords, the source table of the operation script is determined. The source table is the table from which the operation script obtains data. The relationship establishment module is used to determine the target reference network between the write table and the source table based on the script statements corresponding to the write table and the source table.

[0006] Thirdly, this disclosure provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned script table processing method. Fourthly, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described script table processing method.

[0007] The above-mentioned script table processing method, device, equipment, and medium implement the following scheme: The operation script to be parsed is determined; the script attributes of the operation script are obtained; based on the script attributes, the write table of the operation script is determined, which is the table for writing data into the operation script; the script keywords of the operation script are obtained; based on the script keywords, the source table of the operation script is determined, which is the table for acquiring data from the operation script; and based on the script statements corresponding to the write table and the source table, the target reference relationship network between the write table and the source table is determined. This method, by determining the script attributes and script keywords of the operation script, determines the corresponding write table and source table for the operation script, thereby establishing the reference relationship between the write table and the source table to achieve accurate parsing of the operation script. It organizes the relationships between tables into easily identifiable and analyzable structured data, which helps improve operational efficiency and reduce data risks. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart of a script table processing method according to an embodiment of the present invention; Figure 2 This is another flowchart of a script table processing method in one embodiment of the present invention; Figure 3 This is another flowchart of a script table processing method in one embodiment of the present invention; Figure 4 This is another flowchart of a script table processing method in one embodiment of the present invention; Figure 5 This is another flowchart of a script table processing method in one embodiment of the present invention; Figure 6 This is another flowchart of a script table processing method in one embodiment of the present invention; Figure 7 This is a schematic block diagram of a script table processing device according to an embodiment of the present invention; Figure 8 This is a schematic block diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0010] The script table processing method provided in this embodiment of the invention can be applied to, for example, Figure 1 In application environments, including such Figure 1 The diagram illustrates a client and server. The client and server communicate over a network to determine the target reference network for each script to be parsed. The client, also known as the user terminal, is the program that provides local services to the client, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0011] In one embodiment, such as Figure 2 As shown, a script table processing method is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included: S101, Determine the operation script to be parsed; S102, Obtain the script attributes of the operation script, and determine the write table of the operation script based on the script attributes; S103, Obtain the script keywords of the operation script, and determine the source table of the operation script based on the script keywords; S104. Based on the script statements corresponding to the write table and the source table, determine the target reference relationship network between the write table and the source table.

[0012] The operation script can be any length and size. This disclosure does not limit the specific script type, such as HIVE scripts, Spark scripts, etc.

[0013] The script attributes can include the data writing method of the operation script, the first header character of the data table corresponding to the operation script, etc.

[0014] The write table is the table in which the operation script writes data; in other words, the write table is the table generated when the operation script is executed to store data.

[0015] The source table is the table from which the operation script obtains data; in other words, the source table is the table to which the data written to the table belongs when the operation script is executed.

[0016] It should be understood that when the operation script is executed, the operation script can retrieve data from the source table and write the retrieved data into the generated write table.

[0017] It should be understood that the concepts of the writing table and the source table are relative. Taking an operation script containing multiple script statements as an example, if the operation script writes data from data table 1 to data table 2 in script statement 1, and writes some data from data table 2 to data table 3 in script statement 2, then in script statement 1, data table 1 is the source table and data table 2 is the writing table, while in script statement 2, data table 2 is the source table and data table 3 is the writing table.

[0018] In particular, the same data table can be both its own write table and its own source table. For example, if script statement 1 implements the repeated writing of some data from data table 1 to data table 1, then data table 1 is both its own write table and its own source table.

[0019] The operation script may contain one or more script statements, that is, the operation script is composed of script statements, which are used to implement the functions of the operation script.

[0020] As an example, in step S101, the operation script to be parsed can be obtained by extracting data from the library storing the operation script. For example, taking the operation script as a HIVE script, the script in the HIVE database can be extracted, and the extracted HIVE script can be identified as the operation script to be parsed.

[0021] As an example, in step S102, the script attributes of the operation script can be obtained by identifying the attributes of the operation script code, and then the write table in the table corresponding to the operation script can be determined based on the identified script attributes.

[0022] For example, if an operation script contains script statement 1 and script statement 2, where script statement 1 corresponds to table 1 and script statement 2 corresponds to table 2, then by identifying the code of script statement 1, it can be determined that the script attribute of script statement 1 includes a write attribute (i.e., script statement 1 is used to write data into table 1), while script statement 2 does not include a write attribute. Therefore, it can be determined that table 1 is a write table and table 2 is not a write table.

[0023] As an example, in step S103, keywords in the code of the operation script can be obtained by keyword recognition, and the source table in the table corresponding to the operation script can be determined based on the identified keywords.

[0024] For example, by performing keyword recognition on the operation script, it can be determined that the operation script contains keyword 1 and keyword 2. Among them, the identified keyword 1 is "from", which means that keyword 1 is used to indicate the data source. The table corresponding to keyword 1 is the source table.

[0025] As an example, in step S104, the corresponding write table and source table in the same script statement can be determined based on the script statements corresponding to the write table and the source table, thereby determining the correspondence between the write table and the source table; or, when multiple script statements perform the same write operation across statements, the write table and source table corresponding to multiple script statements can be determined, thereby determining the correspondence between the write table and the source table; then, based on the determined correspondence, a target reference relationship network between the write table and the source table is established.

[0026] For example, if script statement 1 corresponds to write table 1 and source table 1, then the target reference network is determined to be the correspondence between write table 1 and source table 1; for example, if script statement 1 and script statement 2 jointly perform a write operation, and script statement 1 and script statement 2 correspond to write table 1, source table 1 and source table 2, then the target reference network is determined to be the correspondence between write table 1, source table 1 and source table 2.

[0027] It should be understood that this disclosure does not limit the execution order of steps S102 and S103. Step S102 can be executed first, followed by step S103; step S103 can be executed first, followed by step S102; or steps S102 and S103 can be executed simultaneously.

[0028] In this example, the following steps are taken: First, the operation script to be parsed is identified. Then, the script attributes of the operation script are obtained. Based on these attributes, the write table (the table where the operation script writes data) is determined. Next, the script keywords of the operation script are obtained. Based on these keywords, the source table (the table where the operation script retrieves data) is determined. Finally, based on the script statements corresponding to the write and source tables, the target reference network between the write and source tables is established. This method determines the write and source tables by identifying the script attributes and keywords of the operation script, thereby establishing the reference relationship between them. This enables precise parsing of the operation script, transforming the relationships between tables into easily identifiable and analyzable structured data, which helps improve operational efficiency and reduce data risks.

[0029] like Figure 3 As shown, step S102, which involves obtaining the script attributes of the operation script, includes: S201, Extract features from the operation script to determine the writing method of the operation script; S202, Identify the data table corresponding to the operation script and determine the first header character of the data table; S203, the write method and the first header character are determined to be script attributes.

[0030] The features of the operation script obtained by feature extraction can be specific code, statement structure, etc., which are not limited in this disclosure.

[0031] The operation script can be written in a persistent or temporary manner. Persistent writing means that the data in the write table generated after writing can be stored for a long time or eventually saved to physical storage. Temporary writing means that the data in the write table generated after writing may be deleted or transferred to other write tables. It should be understood that the write table generated by temporary writing can be retained or deleted, and this disclosure does not limit it.

[0032] The first header character can be the name of the data table corresponding to the operation script.

[0033] As an example, in step S201, the method of feature extraction of the operation script is not limited. For example, regular expression matching can be used to determine the code contained in the operation script, thereby realizing feature extraction of the operation script; another example is to use a lexical analyzer to determine the syntactic structure of the script statements in the operation script, thereby realizing feature extraction of the operation script; yet another example is to use the corresponding recognition tool to realize feature extraction of the operation script based on the statement type in the operation script (for example, when the operation script consists of SQL statements, the sqlparse tool can be used to parse the SQL statements and determine the characteristic code contained in the SQL statements to realize feature extraction).

[0034] As an example, in step S201, based on the features of the operation script extracted from the features, the code or statement structure used to perform the write operation in the operation script is determined, and then the identified code or statement structure is compared with the pre-set write mapping table to determine whether the write method of the operation script is persistent write or temporary write.

[0035] For example, by extracting features from the manipulation script and determining that the statement structure of the script is "insert intotable" and "insert overwrite table", the write method corresponding to the above two structures can be determined to be persistent write based on the pre-defined write mapping table. For example, by extracting features from the manipulation script and determining that the script contains the code "with as", the write method corresponding to this code can be determined to be temporary write based on the pre-defined write mapping table.

[0036] In other words, the aforementioned pre-defined write mapping table can contain various write methods corresponding to features such as code and statement structure. Therefore, after extracting the features of the operation script, the write method of the operation script can be determined by comparing the extracted features with the pre-defined write mapping table.

[0037] As an example, in step S202, the identification method in step S201 can be used to identify the data table of the operation script and determine the first header character of the data table, that is, to identify the script statement of the operation script and determine the first header character in the operation script used to indicate the table name.

[0038] For example, when the operation script consists of SQL statements, the sqlparse tool can be used to parse the SQL statements and identify the character "dwd_order" that indicates the table name in the operation script. Then, "dwd_order" can be identified as the first table header character.

[0039] As an example, in step S203, the writing method and the first header character can be directly determined as script attributes to achieve attribute extraction of the operation script.

[0040] As an example, in step S102, when the writing method is persistent writing and the first header character is used to indicate the data warehouse, the data table is determined to be a target type write table. According to the big data development specification table name, the first header character indicating the data warehouse can be a name that starts with dim, dwd, dws, sub, or agg.

[0041] Among them, the target type write table refers to the write table that represents the final result when the operation script is executed. The target type write table is usually stored in physical storage.

[0042] For example, when it is determined that the operation script is written via insert into table, insertoverwrite table, or create table as, and the first header character is dim, dwd, dws, sub, or agg, the corresponding data table is determined to be the target type write table.

[0043] As an example, in step S102, when the write mode is persistent write and the first header character is used to indicate transitional storage, the data table is determined to be an intermediate write table.

[0044] Among them, intermediate write tables refer to write tables generated during the execution of operation scripts that can be stored for a long time or stored in physical storage.

[0045] For example, when it is determined that the operation script is written via insert into table, insertoverwrite table, or create table as, and the first header character is mid, the corresponding data table is determined to be an intermediate type write table.

[0046] As an example, in step S102, when the writing method is temporary writing and the writing method can be executed across multiple statements in the operation script, the data table is determined to be the first temporary writing table.

[0047] The first temporary write table can be a data table generated during the execution of the operation script for temporary storage. Its writing method is different from that of the second temporary write table. After the operation script is completed, the first temporary write table can be deleted or retained, depending on the application scenario of the first temporary write table.

[0048] For example, when the write method is determined to be temporary write, and the write method is to write via create temporarytable as, the corresponding data table can be determined to be the first temporary write table.

[0049] As an example, in step S102, when the writing method is temporary writing and the writing method cannot be executed across statements in multiple statements of the operation script, the data table is determined to be a second temporary writing table.

[0050] The second temporary write table can be a data table generated during the execution of the operation script for temporary storage. After the operation script is completed, the second temporary write table can be deleted or retained, depending on the application scenario of the second temporary write table.

[0051] For example, when the write method is determined to be temporary write, and the write method is written via with as, the corresponding data table can be determined to be the second temporary write table.

[0052] In the above embodiments, by classifying the written table, the granularity of the target reference relationship network can be further subdivided, which improves the convenience of operation and maintenance of the operation script.

[0053] like Figure 4 As shown, step S103, which involves obtaining the script keywords for the operation script, includes: S301, Perform syntax analysis on the operation script to determine the smallest syntactic unit of the operation script; S302, Based on the smallest syntactic unit, determine the script keywords through regular expression matching.

[0054] The smallest syntactic unit can be an indivisible element (e.g., a single code, a code phrase, etc.) obtained by performing syntactic analysis on the script of the operation script, such as the lexical unit (Token) in the HIVE operation script.

[0055] Script keywords can be the smallest syntactic unit used to indicate the source of data.

[0056] As an example, in step S301, the operation script can be parsed according to the statement type used by the operation script and the corresponding syntax specification to determine the smallest syntactic unit of the operation script; or the smallest syntactic unit of the operation script can be determined by using the corresponding identification tool (for example, when the operation script is composed of SQL statements, the sqlparse tool can be used to parse the SQL statements to determine the smallest syntactic unit contained in the SQL statements).

[0057] For example, taking a Hive script as the operation script, the operation script can be analyzed using the syntax rules of Hive SQL to determine the code and code phrases such as from, join, select, and left join contained in the operation script as the smallest syntactic unit.

[0058] As an example, in step S302, the smallest syntactic unit can be matched using a pre-defined regular expression in the regular expression matching to determine the keyword that matches the regular expression.

[0059] For example, using the predefined regular expression \b(select|from|where)\b (where \b represents a word boundary), the smallest syntactic unit is matched to identify the three keywords select, from, and where within that unit.

[0060] like Figure 5 As shown, step S103, which involves determining the source table of the operation script based on the script keywords, includes: S401, Determine the code block corresponding to the script keyword based on the script keyword; S402, Based on the code block, determine the second header character contained in the code block; S403, determine the source table based on the second header character.

[0061] A code block can be used to indicate a complete operation or instruction content of an operation script (e.g., writing method, data writing process, writing target, etc.), and the code block can contain one or more code characters.

[0062] The second header character is the table name contained in the code block.

[0063] As an example, in step S401, one or more code characters covered by the complete operation or instruction content corresponding to the script keyword can be determined according to the script keyword, thereby determining the one or more code characters as a code block, so as to facilitate the splitting of the operation script, enabling the determination of multiple source tables at the same time, and improving the efficiency of source table determination.

[0064] For example, if the defined script keyword is "from", then the code "from dim_user a" (indicating that the data source is dim_user a) can be defined as a code block.

[0065] As an example, in step S402, the code characters contained in the segmented code block can be identified, and the table name (i.e., the second table header character) contained in the code block can be located according to the meaning of the script keywords.

[0066] It should be understood that since script keywords can be the smallest syntactic unit used to indicate the source of data, the code phrase following the script keyword in a code block is usually the second header character. The second header character can be determined from the code block based on this, but it can also be determined in other ways, such as by using a corresponding character detector. This disclosure does not limit this.

[0067] For example, if the defined code block is "from dim_user a" and the script keyword is "from", then the second header character can be determined to be "dim_user a".

[0068] As an example, in step S403, the data table corresponding to the second header character can be determined based on the second header character, thereby identifying the corresponding data table as the source table.

[0069] For example, if the second table header character is dim_user a, then the actual data table corresponding to dim_user a can be identified as the source table.

[0070] like Figure 6 As shown, step S104, which involves determining the source table of the operation script based on the script keywords, includes: S501, perform statement recognition on the operation script to determine the script statements corresponding to the write table and the source table; S502 defines the write table and source table within the same statement as a set of table reference relationships; S503 generates a target reference relationship network based on table reference relationships.

[0071] As an example, in step S501, multiple script statements in the operation script can be identified, thereby enabling the segmentation of multiple script statements to determine the write table and source table corresponding to each script statement.

[0072] It should be understood that the above determination of the write table and source table only identifies the write table and source table contained in the entire operation script based on the corresponding characteristics. It cannot realize the correspondence between the corresponding write table and source table. Therefore, it is necessary to identify the operation script to determine the correspondence between each write table and source table.

[0073] The method of statement recognition is not limited. For example, it may be to recognize the delimiters in the operation script, or to recognize the boundaries and comments of the script statements. This disclosure does not limit this method.

[0074] As an example, in step S502, the write table and source table corresponding to the same statement can be identified as a set of corresponding table reference relationships. That is, in the same set of table reference relationships, the data source of the write table is the source table. It should be understood that the same statement can refer to a complete script statement between two delimiters, or it can be multiple script statements with the same write target (for example, when multiple script statements write data from multiple source tables to a write table through cross-statement execution, the above multiple script statements can be regarded as the same statement).

[0075] For example, if the same statement contains a single script statement, and script statement 1 contains source table 1 and write table 1, then the corresponding write table 1 of source table 1 can be identified as a set of table application relationships. If the same statement contains multiple script statements, by executing across statements, the source table 1 in script statement 1 and the source table 2 in script statement 2 can be written to the write table 1 in script statement 1, then the corresponding write table 1 of source table 1 and source table 2 can be identified as a set of table application relationships.

[0076] As an example, table references can be integrated based on all table references contained in the script statements to generate a target reference network.

[0077] Specifically, this may include: determining one or more source tables corresponding to the same write table based on table reference relationships, and generating write reference relationships for the write tables; determining one or more write tables corresponding to the same source table based on table reference relationships, and generating source reference relationships for the source tables; and determining the target reference relationship network corresponding to the operation script based on the write reference relationships and source reference relationships.

[0078] Since step S502 only organizes the correspondence between the write table and the source table in the same script statement, and there may be duplicate or overlapping correspondences in the entire operation script, it is necessary to integrate the multiple sets of table references generated in step S502 in order to facilitate the sorting out of the overall correspondence between each write table and the source table.

[0079] For example, if the table reference relationship in script statement 1 is determined to be the source table 1 corresponding to the write table 1, and the table reference relationship in script statement 2 is determined to be the source table 2 corresponding to the write table 1, then the write reference relationship of the write table 1 can be generated as the source table 1 and the source table 2 corresponding to the write table 1 based on the above table reference relationship.

[0080] For example, if the table reference relationship in script statement 1 is determined to be that the source table is written to table 1, and the table reference relationship in script statement 2 is determined to be that the source table is written to table 2, then the source table reference relationship of the source table 1 can be generated as the source table 1 corresponds to both the source table and the source table 2. In the example above, by determining the write reference relationships of each write table and the source reference relationships of the source table, the write reference relationships and source reference relationships are stored to generate the corresponding target reference relationship network.

[0081] As can be seen, in the above-mentioned scheme, considering the characteristics of the script file's complex content and intricate table relationships, this application identifies the data tables corresponding to the operation script, determines the write table in the operation script, and identifies the source table of the operation script through syntax analysis, thereby achieving accurate classification of the data tables corresponding to the operation script; furthermore, it identifies the statements in the operation script to accurately determine the correspondence between the write table and the source table, thus accurately establishing a target reference relationship network between the write table and the source table in the complex script code.

[0082] In summary, the script table processing method proposed in this disclosure can accurately and efficiently transform large batches of complex scripts with extremely long links, high customization, and length into highly available, easily identifiable, and analyzable structured data with intricate inter-table relationships into structured data that can be further calculated and analyzed. This enables its application in different R&D fields of big data, greatly facilitating project management and overall analysis, improving the efficiency and accuracy of big data system projects, and reducing data risks.

[0083] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0084] In one embodiment, a script table processing apparatus is provided, which corresponds one-to-one with the script table processing methods described in the above embodiments. For example... Figure 7 As shown, the script table processing device includes a script determination module 601, an attribute determination module 602, a keyword determination module 603, and a relationship establishment module 604. Detailed descriptions of each functional module are as follows: The script determination module 601 is used to determine the operation script to be parsed. The attribute determination module 602 is used to obtain the script attributes of the operation script and determine the write table of the operation script based on the script attributes. The write table is a table for writing data to the operation script. The keyword determination module 603 is used to obtain the script keywords of the operation script and determine the source table of the operation script based on the script keywords. The source table is the table in which the operation script obtains data. The relationship establishment module 604 is used to determine the target reference relationship network between the write table and the source table based on the script statements corresponding to the write table and the source table.

[0085] In one embodiment, the attribute determination module 602 is further configured to extract features from the operation script and determine the writing method of the operation script, including persistent writing and temporary writing. Identify the data table corresponding to the operation script and determine the first header character of the data table; The write method and the first header character are defined as script attributes.

[0086] In one embodiment, the attribute determination module 602 is further configured to determine the data table as a target type write table when the write method is persistent write and the first header character is used to indicate the data warehouse. When the write method is persistent write and the first header character is used to indicate transitional storage, the data table is determined to be an intermediate write table. When the write method is temporary write, and the write method can be executed across multiple statements in the operation script, the data table is determined to be the first temporary write table; When the write method is temporary write, and the write method cannot be executed across multiple statements in the operation script, the data table is determined to be a second temporary write table.

[0087] In one embodiment, the keyword determination module 603 is further configured to perform syntactic analysis on the operation script to determine the smallest syntactic unit of the operation script; Based on the smallest syntactic unit, the script keywords are determined through regular expression matching.

[0088] In one embodiment, the keyword determination module 603 is further configured to determine the code block corresponding to the script keyword based on the script keyword; Based on the code block, determine the second header character contained in the code block; The source table is determined based on the characters in the second header.

[0089] In one embodiment, the relationship establishment module 604 is further configured to perform statement recognition on the operation script to determine the script statements corresponding to the write table and the source table; The write table and the source table within the same statement are identified as a set of table references; Based on the table reference relationships, generate the target reference relationship network.

[0090] In one embodiment, the relationship establishment module 604 is further configured to determine one or more source tables corresponding to the same write table based on the table reference relationship, and generate a write reference relationship for the write table; Based on the table reference relationship, identify one or more write tables corresponding to the same source table, and generate the source reference relationship of the source table; Based on the write reference relationship and the source reference relationship, the target reference relationship network corresponding to the operation script is determined.

[0091] This invention provides a script table processing device, comprising: a script determination module for determining an operation script to be parsed; an attribute determination module for acquiring script attributes of the operation script and determining a write table for the operation script based on the script attributes, wherein the write table is a table for writing data into the operation script; a keyword determination module for acquiring script keywords of the operation script and determining a source table for the operation script based on the script keywords, wherein the source table is a table for acquiring data from the operation script; and a relationship establishment module for determining a target reference relationship network between the write table and the source table based on the script statements corresponding to the write table and the source table. The device proposed in this disclosure, by determining the script attributes and script keywords of the operation script, determines the corresponding write table and source table for the operation script, thereby establishing a reference relationship between the write table and the source table to achieve accurate parsing of the operation script, and organizes the relationships between tables into easily identifiable and analyzable structured data, which helps improve operational efficiency and reduce data risks.

[0092] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database is used for data processed in the script table processing method. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a script table processing method.

[0093] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described resource data processing method based on a micro-segmentation network.

[0094] In one embodiment, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described resource data processing method based on a micro-segmentation network.

[0095] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0096] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0097] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A script-based table processing method, characterized in that, include: Identify the operation script to be parsed; Obtain the script attributes of the operation script, and determine the write table of the operation script based on the script attributes. The write table is a table for writing data to the operation script. Obtain the script keywords of the operation script, and determine the source table of the operation script based on the script keywords. The source table is a table in which the operation script obtains data. Based on the script statements corresponding to the write table and the source table, determine the target reference network between the write table and the source table.

2. The method according to claim 1, characterized in that, The script attributes of the operation script are obtained, including: Feature extraction is performed on the operation script to determine the writing method of the operation script, which includes persistent writing and temporary writing; The data table corresponding to the operation script is identified, and the first header character of the data table is determined. The writing method and the first header character are determined as the script attributes.

3. The method according to claim 2, characterized in that, The step of determining the write table of the operation script based on the script attributes includes: When the writing method is persistent writing and the first header character is used to indicate the data warehouse, the data table is determined to be a target type write table; When the writing method is persistent writing and the first header character is used to indicate transitional storage, the data table is determined to be an intermediate write table. When the writing method is temporary writing, and the writing method can be executed across multiple statements in the operation script, the data table is determined to be a first temporary writing table; When the writing method is temporary writing, and the writing method cannot be executed across multiple statements in the operation script, the data table is determined to be a second temporary writing table.

4. The method according to claim 1, characterized in that, The script keywords for obtaining the operation script include: Perform syntax analysis on the operation script to determine the smallest syntactic unit of the operation script; The script keywords are determined by regular expression matching based on the smallest syntactic unit.

5. The method according to claim 1, characterized in that, Based on the script keywords, determine the source table of the operation script, including: Based on the script keywords, determine the code block corresponding to the script keywords; Based on the code block, determine the second header character contained in the code block; The source table is determined based on the second header character.

6. The method according to claim 1, characterized in that, The step of determining the target reference network between the write table and the source table based on the script statements corresponding to the write table and the source table includes: The operation script is subjected to statement recognition to determine the script statements corresponding to the write table and the source table; The write table and the source table within the same statement are identified as a set of table references; Based on the table reference relationships, the target reference relationship network is generated.

7. The method according to claim 6, characterized in that, The step of generating the target reference network based on the table reference relationships includes: Based on the table reference relationship, one or more source tables corresponding to the same write table are determined, and the write reference relationship of the write table is generated; Based on the table reference relationship, one or more write tables corresponding to the same source table are determined, and the source reference relationship of the source table is generated; Based on the written reference relationship and the source reference relationship, the target reference relationship network corresponding to the operation script is determined.

8. A script table processing device, characterized in that, include: The script determination module is used to determine the operation script to be parsed; The attribute determination module is used to obtain the script attributes of the operation script and determine the write table of the operation script based on the script attributes. The write table is a table for writing data to the operation script. The keyword determination module is used to obtain the script keywords of the operation script and determine the source table of the operation script based on the script keywords. The source table is a table in which the operation script obtains data. The relationship establishment module is used to determine the target reference relationship network between the write table and the source table based on the script statements corresponding to the write table and the source table.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the script table processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the script table processing method as described in any one of claims 1 to 7.

Images