Link relation identification method and device, equipment, medium and program product
Through the collaborative work of four agents, the link relationship of batch scripts is recognized using a large language model, which solves the problem of low link relationship recognition accuracy caused by irregular configuration files, and achieves more efficient link relationship recognition and tasks manage.
Patent Information
- Application Number
- CN202411152052.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-22
AI Technical Summary
In complex financial information technology systems, the lack of records of link relationships in the configuration files, resulting in the inability to accurately determine the link relationship of batch scripts, affecting task execution and maintenance efficiency.
Four agents are used to process the type recognition, file dependency recognition, source and target file recognition and link relationship determination of batch scripts respectively. The task execution body encapsulated by the large language model is used to use preset script pairing rules and file dependencies. As a result, accurate identification of upstream and downstream dependency scripts.
It improves the accuracy and efficiency of link relationship recognition, reduces the probability of error when a single agent deals with complex problems, simplifies the workload of each agent, and improves the automation level and management efficiency of the workflow.
Smart Images

Figure CN119002925A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence and financial technology, and more specifically to a link relationship identification method, device, equipment, medium and program product. Background Art
[0002] In complex financial information technology systems, each batch processing task (such as end-of-day reports) generally requires multiple jobs in multiple business systems to complete, and these jobs will have upstream and downstream dependencies, also known as link relationships, where a job is generally an executable script program. When the upstream job fails to execute, it often affects the execution of downstream jobs. In related technologies, the link relationship of each script is usually determined manually or by related algorithms based on the configuration file during system or software development.
[0003] In the process of implementing the concept of the present disclosure, relevant researchers found that there are at least the following problems in the relevant technology: the configuration information in the configuration file is not standardized. For example, due to later maintenance or other reasons, the configuration file lacks records of some link relationships, resulting in the inability to accurately determine the link relationship of the script. Summary of the invention
[0004] In view of the above problems, the present disclosure provides a link relationship identification method, apparatus, device, medium and program product for improving the accuracy of link relationship identification.
[0005] According to the first aspect of the present disclosure, a link relationship identification method is provided, comprising: in response to a link relationship identification request from a user, based on a first intelligent agent, performing type identification on a plurality of acquired batch scripts, and determining the script type of each of the batch scripts; based on each of the script types, using a second intelligent agent to perform file dependency identification on each of the batch scripts, and determining the file dependency result of each of the batch scripts; based on each of the script types and the file dependency results, using a third intelligent agent to perform file identification on each of the batch scripts, and determining the source file identifier and the target file identifier corresponding to each of the batch scripts; based on a fourth intelligent agent, identifying the source file identifier and the target file identifier of each of the batch scripts, and determining the link relationship formed by the plurality of batch scripts; wherein the first intelligent agent, the second intelligent agent, the third intelligent agent and the fourth intelligent agent are task execution bodies encapsulated based on a large language model.
[0006] According to an embodiment of the present disclosure, the fourth agent identifies the source file identifier and the target file identifier of each batch script to determine the link relationship consisting of multiple batch scripts, including: the fourth agent pairs the source file identifier and the target file identifier of each batch script based on a preset script pairing rule to determine the dependent scripts of each batch script, wherein the dependent scripts include upstream dependent scripts and downstream dependent scripts.
[0007] According to an embodiment of the present disclosure, the fourth agent pairs the source file identifier and the target file identifier of each batch script based on a preset script pairing rule to determine the upstream dependent script or downstream dependent script of each batch script, including: in the case where the script pairing rule represents the upstream dependent script for finding each script, the fourth agent repeatedly performs the following operations until all scripts are traversed: in the target file identifier of at least one first batch processing script, determine the target file identifier that is the same as the source file identifier of the i-th batch processing script, wherein i is greater than or equal to 1, and the first processing script represents the other batch processing scripts except the i-th batch processing script; determine the first batch processing script to which the target file identifier belongs as the upstream dependent script of the i-th batch processing script.
[0008] According to an embodiment of the present disclosure, the fourth agent pairs the source file identifier and the target file identifier of each batch script based on a preset script pairing rule to determine the upstream dependent script or downstream dependent script of each batch script, and also includes: in the case where the script pairing rule representation is used to find the downstream dependent script of each script, the fourth agent repeatedly performs the following operations until all scripts are traversed: in the source file identifier of at least one second batch script, determine the source file identifier that is the same as the target file identifier of the jth batch script, where j is greater than or equal to 1, and the second processing script represents the other batch scripts except the jth batch script; determine the second batch script to which the target file identifier belongs as the downstream dependent script of the jth batch script.
[0009] According to an embodiment of the present disclosure, the third agent is used to perform file identification on each batch script according to each script type and the file dependency result, and determine the source file identifier and the target file identifier corresponding to each batch script, including: for each batch script, the third agent performs file identification on the batch scripts whose file dependency result is file dependency according to the script type; when it is identified that there is a calling file in the batch script, the calling file is identified in a fixed format to obtain the source file identifier; when it is identified that there is a generated file in the batch script that is not called by the batch script, the generated file is identified in a fixed format to obtain the target file identifier.
[0010] According to an embodiment of the present disclosure, the above-mentioned second agent is used to identify file dependencies of each batch script according to each of the above-mentioned script types, and determine the file dependency result of each batch script, including: for each of the above-mentioned batch scripts, the above-mentioned second agent identifies the calling file and the generating file in the above-mentioned batch script based on the above-mentioned script type, and when it is identified that there is a calling file or a generating file in the above-mentioned batch script, the file dependency result of the above-mentioned batch script is determined to have file dependency; when it is identified that there is no calling file or generating file in the above-mentioned batch script, the file dependency result of the above-mentioned batch script is determined to have no file dependency.
[0011] According to an embodiment of the present disclosure, the above-mentioned first agent performs type identification on the multiple batch scripts obtained to determine the script type of each of the batch scripts, including: for each of the batch scripts obtained, the first agent performs feature extraction on the batch script to obtain a feature extraction result; based on the feature extraction result, the script type of the batch script is determined from a preset script type set.
[0012] According to an embodiment of the present disclosure, the link relationship identification method further includes: verifying the identification data outputted by each of the first intelligent agent, the second intelligent agent, the third intelligent agent and the fourth intelligent agent to obtain respective verification results; and fine-tuning the first intelligent agent, the second intelligent agent, the third intelligent agent and the fourth intelligent agent based on the respective verification results.
[0013] The second aspect of the present disclosure provides a link relationship identification device, which includes: a type determination module, which is used to respond to a link relationship identification request from a user, identify multiple batch scripts obtained based on a first agent, and determine the script type of each batch script; a dependency determination module, which is used to identify each batch script in each script type based on a second agent, and determine the file dependency result of each batch script, wherein the file dependency result includes file dependency and no file dependency; a file determination module, which is used to identify each batch script with file dependency results in each script type based on a third agent, and determine the source file identifier and target file identifier corresponding to each batch script; a relationship determination module, which is used to identify the source file identifier and the target file identifier of each batch script based on a fourth agent, and determine the link relationship composed of multiple batch scripts; wherein the first agent, the second agent, the third agent and the fourth agent are task execution bodies encapsulated based on a large language model.
[0014] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.
[0015] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.
[0016] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.
[0017] According to the embodiment of the present disclosure, the first agent, the second agent, the third agent and the fourth agent are used to respectively perform the four tasks divided by the link relationship identification process to determine the link relationship of multiple batch scripts, so that each agent can be more focused on processing simple and repetitive work in a single field. Compared with processing the link identification task by a single agent or other algorithm model, the probability of errors when a single agent processes complex problems is effectively reduced, thereby improving the overall efficiency and accuracy of the link relationship identification process. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0019] Figure 1 The application scenario diagram of the link relationship identification method according to the embodiment of the present disclosure is schematically shown;
[0020] Figure 2 A flowchart of a link relationship identification method according to an embodiment of the present disclosure is schematically shown;
[0021] Figure 3 A data flow diagram of a link relationship identification method according to an embodiment of the present disclosure is schematically shown;
[0022] Figure 4 A flowchart of a link relationship identification method according to another embodiment of the present disclosure is schematically shown;
[0023] Figure 5 A schematic diagram showing a structure of a link relationship identification device according to an embodiment of the present disclosure; and
[0024] Figure 6 A block diagram of an electronic device suitable for implementing a link relationship identification method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0026] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0027] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0028] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0029] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.
[0030] In the scenario of using personal information for automated decision-making, the methods and devices provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the results of automated decision-making; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.
[0031] It should be noted that in the embodiments of the present disclosure, certain software, components, models and other existing solutions in the industry may be mentioned, which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present disclosure, but it does not mean that the present disclosure has or will necessarily use the solution.
[0032] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.
[0033] In complex financial information technology systems, each batch task generally requires multiple jobs in multiple business systems to complete, and these jobs will have a front-to-back dependency relationship, where a job is generally an executable script program. For example: in the end-of-day report task, a job jobA in the downstream system A depends on the completion of the job jobB in the upstream system B, or depends on the completion of the file fileC in the upstream system C. Identifying the link relationship of batch tasks is helpful for analyzing the impact of upstream jobs on downstream jobs, and it is also helpful to trace the root cause jobs upward so as to carry out manual intervention in a timely manner to ensure the smooth progress of batch tasks.
[0034] However, in reality, different business systems correspond to different upstream and downstream systems. For example, file C generated by upstream system C is placed in a path. Job A of downstream system A scans dependent files from this path, but does not know which system or job the file comes from. If the upstream file fails to be generated, it is impossible to promptly determine the affected job A in the downstream.
[0035] In the related art, data processing is usually performed on the configuration information in the configuration file by manual or algorithm to determine the link relationship between batch script files in the same business system or different business systems, that is, the upstream and downstream dependencies. However, in the specific implementation process, the manual processing method consumes manpower. Although the algorithm can replace part of the manpower to identify the link relationship between batch script files, the development and maintenance process of the algorithm inevitably requires a large amount of human resources. In addition, since the configuration information in the configuration file is not standardized enough, the accuracy of the link relationship identification result is low.
[0036] An embodiment of the present disclosure provides a link relationship identification method for solving the above-mentioned problem, the method comprising: in response to a link relationship identification request from a user, based on a first intelligent agent, performing type identification on a plurality of acquired batch scripts, and determining the script type of each batch script; based on each script type, using a second intelligent agent to perform file dependency identification on each batch script, and determining the file dependency result of each batch script; based on each script type and the file dependency result, using a third intelligent agent to perform file identification on each batch script, and determining the source file identifier and target file identifier corresponding to each batch script; based on a fourth intelligent agent, identifying the source file identifier and target file identifier of each batch script, and determining the link relationship consisting of a plurality of batch scripts; wherein the first intelligent agent, the second intelligent agent, the third intelligent agent and the fourth intelligent agent are task execution bodies encapsulated based on a large language model.
[0037] Figure 1 The application scenario diagram of the link relationship identification method according to an embodiment of the present disclosure is schematically shown.
[0038] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.
[0039] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).
[0040] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0041] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.
[0042] It should be noted that the link relationship identification method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the link relationship identification device provided in the embodiment of the present disclosure can generally be set in the server 105. The link relationship identification method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the link relationship identification device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.
[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0044] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The link relationship identification method of the disclosed embodiment is described in detail.
[0045] Figure 2 The flowchart of the link relationship identification method according to the embodiment of the present disclosure is schematically shown.
[0046] like Figure 2 As shown, the link relationship identification method of this embodiment includes operations S210 to S240.
[0047] In operation S210, in response to a link relationship identification request from a user, the first agent identifies the types of the acquired batch processing scripts to determine the script type of each batch processing script.
[0048] In operation S220, according to each script type, the second agent is used to identify the file dependencies of each batch processing script to determine the file dependency results of each batch processing script.
[0049] In operation S230, a third agent is used to perform file identification on each batch processing script according to each script type and file dependency result, and a source file identifier and a target file identifier corresponding to each batch processing script are determined.
[0050] In operation S240, the fourth agent identifies the source file identifier and the target file identifier of each batch processing script to determine the link relationship of the multiple batch processing scripts.
[0051] According to an embodiment of the present disclosure, the first agent, the second agent, the third agent and the fourth agent are task execution bodies encapsulated based on large language models (LLMs). For example, the first agent, the second agent, the third agent and the fourth agent can be four different components, and each agent can include a data transmission interface for generating data output or receiving data. Each agent can call a large language model to use the large language model to process the received data based on a preset prompt to obtain a result corresponding to the function of each agent. Each agent can also store the results obtained by each agent to facilitate the client or other agents to call.
[0052] According to an embodiment of the present disclosure, a link relationship identification request is used to obtain the link relationship between input batch scripts. The link relationship identification request may include address information of a storage space storing batch scripts. A first agent obtains multiple batch scripts whose source relationship needs to be identified based on the address information, wherein the first agent is a task executor for identifying script types based on a large language model, wherein the types of batch scripts may include: Shell type, Python type, Perl type and other types. The first agent may be encapsulated with a method for obtaining batch scripts and a prompt word for prompting the large language model to identify the script type of the batch script.
[0053] According to an embodiment of the present disclosure, the second agent is constructed based on a large language model and is used to identify whether there is a file dependency in a batch script based on the script type. The file dependency result includes file dependency and no file dependency. For example, if the execution of script 2 depends on the file generated in script 1, it is considered that there is a file dependency between script 2 and script 2, and the file dependency result is file dependency.
[0054] According to an embodiment of the present disclosure, the script type and the batch script obtained by the first agent are input into the second agent, and the second agent performs file dependency identification on the batch script according to the script type of the batch script to determine the file dependency result of each batch script. The second agent may be encapsulated with prompt words for prompting the large language model to identify the file dependency result of the batch script.
[0055] According to the embodiments of the present disclosure, different script types characterize differences in the programming paradigms, syntax, etc. of batch scripts. For example, the if statement of a Python-type batch script uses a complete indentation block to define a code block, while a Shell-type batch script uses then and fi to define an if statement block. Therefore, the second agent performs differentiated processing on the batch scripts based on the script type of the batch script to adapt to the characteristics of different script types, thereby improving the accuracy and recognition efficiency of the file dependency results.
[0056] According to an embodiment of the present disclosure, the third agent is a task execution body constructed based on a large language model and used to identify the source file and target file of a batch script. The source file identifier of a batch script may include the path and file name of the file called by the batch script. Similarly, the target file identifier of a batch script may include the path and file name of the file called by other batch scripts in the batch script. For example, the target file identifier or the source file identifier may be " / a1 / a2 / a3 / a3 / transactions.txt".
[0057] For example, the script content of batch script A corresponding to job jobA may include the following statements, "BACKUP_FILE=" / path / to / remote / backup.tar.gz"", and "DECOMPRESSED_DIR=" / path / to / remote / decompressed"" are used to define the backup file and the decompressed directory. "awk ' / ^Total / {print $4}'transactions.txt > summary.txt" is used to use the text processing tool awk to perform data analysis on the decompressed file "transactions.txt", such as calculating the number of lines in a file and writing the results to the file "summary.txt". The third agent can then recognize that the source file identifier of batch script A includes: " / path / to / remote / backup.tar.gz"; the target file identifier of batch script A includes: " / path / to / remote / decompressed / summary.txt" and " / path / to / remote / decompressed / transactions.txt".
[0058] For example, the script content of batch script B corresponding to job jobB may include a shell-type statement "SOURCE_DIR=" / path / to / source / directory"", and a statement "Target_FILE=" / path / to / remote / backup.tar.gz"", which are used to define the source data directory and the backup file name. And the statement "tar -czvf $BACKUP_FILE$SOURCE_DIR" is used to create a backup file. Then the third agent can recognize that the source file identifier of batch script B includes: " / path / to / source / directory"; the target file identifier of batch script B includes: " / path / to / remote / backup.tar.gz".
[0059] According to an embodiment of the present disclosure, the third agent can identify the file dependency results of the batch script, perform file identification on the batch script with file dependency based on the script type of the batch script, and determine the source file identifier and target file identifier of each batch script. The third agent can be encapsulated with prompt words for prompting the large language model to identify the source file and target file of the batch script, and output the corresponding source file identifier and target file identifier.
[0060] According to an embodiment of the present disclosure, in the case where the second agent mistakenly identifies a batch script without file dependencies as having file dependencies, the third agent can provide a layer of protection for the second agent to improve the accuracy of the recognition result.
[0061] According to an embodiment of the present disclosure, the fourth agent is a task execution body constructed based on a large language model and used to identify file dependencies between batch scripts. For each batch script, the fourth agent determines the upstream and downstream dependencies between each batch script, that is, the link relationship between multiple batch scripts, based on the source file identifier and target file identifier of the batch script determined by the third agent and the relationship between the source file identifier and the target file identifier of other batch scripts.
[0062] According to the embodiments of the present disclosure, since the first agent, the second agent, the third agent and the fourth agent are task execution agents encapsulated based on the large language model, and since the large language model is a large model that has been trained, the large language model can be called based on the prompt words pre-encapsulated in the agent during the application process to determine the output result of each agent. In the case of inaccurate results, only fine-tuning of each agent is required, and large-scale pre-training of the large language model called by each agent is not required, which reduces the workload of relevant technical personnel in code development and maintenance.
[0063] According to the embodiment of the present disclosure, the first agent, the second agent, the third agent and the fourth agent are used to respectively perform the four tasks divided by the link relationship identification process to determine the link relationship of multiple batch scripts, so that each agent can be more focused on processing simple and repetitive work in a single field. Compared with processing the link identification task by a single agent or other algorithm model, the probability of errors when a single agent processes complex problems is effectively reduced, thereby improving the overall efficiency and accuracy of the link relationship identification process.
[0064] According to an embodiment of the present disclosure, based on the fourth agent identifying the source file identifier and the target file identifier of each batch script, the link relationship consisting of multiple batch scripts is determined, including: the fourth agent pairs the source file identifier and the target file identifier of each batch script based on a preset script pairing rule, and determines the dependent scripts of each batch script, wherein the dependent scripts include upstream dependent scripts and downstream dependent scripts.
[0065] According to an embodiment of the present disclosure, the script pairing rule is a logic or standard for determining the link relationship between batch scripts. Based on the characteristics of the source file identifier and the target file identifier, the source file identifier and the target file identifier of each batch script obtained from the third agent are paired to obtain the dependent script of each batch script, thereby determining the link relationship between multiple batch scripts.
[0066] For example, if the source file identifier of each batch script starts with "input_" and the target file identifier starts with "output_", use a regular expression to match the relationship between each batch script starting with "input_" and the scripts starting with "output_" in other batch scripts to determine the relationship between different scripts.
[0067] According to the embodiments of the present disclosure, the fourth agent is used to determine the dependency of the batch script, which improves the automation level, reliability and management efficiency of the workflow. The script pairing rules are set based on the characteristics of the source file identifier and the target file identifier. Since the script pairing rules can be set in the prompt encapsulated by the fourth agent, the staff can modify the script pairing rules by modifying the prompt without modifying the code program, which provides convenience for subsequent staff to optimize the script pairing rules.
[0068] According to an embodiment of the present disclosure, the fourth agent pairs the source file identifier and the target file identifier of each batch script based on a preset script pairing rule to determine the upstream dependent script or the downstream dependent script of each batch script, including: when the script pairing rule representation is used to find the upstream dependent script of each script, the fourth agent repeatedly performs the following operations until all scripts are traversed: in the target file identifier of at least one first batch processing script, determine the target file identifier that is the same as the source file identifier of the i-th batch processing script, where i is greater than or equal to 1, and the first processing script represents the batch processing scripts other than the i-th batch processing script; and determine the first batch processing script to which the target file identifier belongs as the upstream dependent script of the i-th batch processing script.
[0069] According to an embodiment of the present disclosure, the script pairing rule can be based on the source file identifier and the target file identifier of each batch script to determine the upstream dependent script of each batch script. The fourth agent calls the large language model, and the large language model outputs the upstream dependent script of each batch script based on the script pairing rule in the prompt.
[0070] According to an embodiment of the present disclosure, a prompt for intelligent pairing using a large language model may be, for example: "You are a batch script analysis expert. Please provide the script parsing results, including source files and target files, in the following format: source file identifier: xxx / xx / src_file1; target file identifier: yyy / yy / tgt_file2, yyy / yy / tgt_file5. According to the above data information format, any two scripts are paired, and the pairing rule is that the source file identifier of a script is consistent with the target file identifier of a script, and the returned result is that the latter is the upstream dependency of the former, and the data format is: script content 1, source file 1, upstream dependency, script content 2, target file 2; script content 1, source file 1, upstream dependency, script content 6, target file 6." Among them, the script parsing results may include the content of the batch script and the source file identifier and target file identifier corresponding to the batch script.
[0071] According to an embodiment of the present disclosure, by limiting the output format of the large language model and determining the source file identifier of each batch script and the target file identifier of the upstream dependent batch script, it is convenient to subsequently verify whether the output result is correct, thereby improving the convenience of fine-tuning the fourth intelligent agent.
[0072] According to an embodiment of the present disclosure, the fourth agent pairs the source file identifier and the target file identifier of each batch script based on a preset script pairing rule to determine the upstream dependent script or the downstream dependent script of each batch script, and also includes: when the script pairing rule representation is used to find the downstream dependent script of each script, the fourth agent repeatedly performs the following operations until all scripts are traversed: among the source file identifiers of at least one second batch script, determine the source file identifier that is the same as the target file identifier of the j-th batch script, where j is greater than or equal to 1, and the second processing script represents the batch scripts other than the j-th batch script; and determine the second batch script to which the target file identifier belongs as the downstream dependent script of the j-th batch script.
[0073] According to an embodiment of the present disclosure, according to an embodiment of the present disclosure, the script pairing rule can be based on the source file identifier and the target file identifier of each batch script to determine the downstream dependent script of each batch script. The fourth agent calls the large language model, and the large language model outputs the downstream dependent script of each batch script based on the script pairing rule in the prompt.
[0074] According to an embodiment of the present disclosure, a prompt for intelligent pairing using a large language model may be, for example: "You are a batch script analysis expert. Please parse the script according to the provided results, including source files and target files, in the following format: source file identifier: xxx / xx / src_file1; target file identifier: yyy / yy / tgt_file2, yyy / yy / tgt_file5. According to the above data information format, any two scripts are paired, and the pairing rule is that the target file identifier of a script is consistent with the source file identifier of a script, and the returned result is that the latter is the upstream dependency of the former, and the data format is: script content 1, target file 1, downstream dependency, script content 3, source file 3; script content 1, target file 1, downstream dependency, script content 5, source file 2."
[0075] For example, the third agent recognizes that the source file identifier of batch script A includes: " / path / to / remote / backup.tar.gz"; the target file identifier of batch script A includes: " / path / to / remote / decompressed / summary.txt" and " / path / to / remote / decompressed / transactions.txt", and the source file identifier of batch script B includes: " / path / to / source / directory"; the target file identifier of batch script B includes: " / path / to / remote / backup.tar.gz". The fourth agent can determine that the source file identifier of batch script A is the same as the target file identifier of batch script B by matching the target file identifier of batch script B with the source file identifier of batch script A. Then the fourth agent can output that batch script A is a downstream dependency of batch script B.
[0076] The output result of the fourth intelligent agent can be "script content B, " / path / to / remote / backup.tar.gz", downstream dependency, batch script A, " / path / to / remote / backup.tar.gz", where script content B is the code content of batch script B, script content A is the code content of batch script A, " / path / to / remote / backup.tar.gz" is the target file identifier of batch script B, "downstream dependency" indicates that the downstream dependency of batch script B is batch script A, and " / path / to / remote / backup.tar.gz" is the source file identifier of script content A.
[0077] According to an embodiment of the present disclosure, the dependency relationship between batch scripts is determined by the source file identifier and the target file identifier of the batch script. Since the source file identifier and the target file identifier are independent of the script type of the batch script, that is, independent of the coding language used by the batch script, therefore, by pre-identifying the source file identifier and the target file identifier to determine the link relationship based on the source file identifier and the target file identifier, it can be applicable to a variety of operating systems and programming environments, thereby improving the compatibility of link relationship identification.
[0078] According to an embodiment of the present disclosure, a third agent is used to perform file identification on each batch script according to each script type and file dependency result, and determine the source file identifier and target file identifier corresponding to each batch script, including: for each batch script, the third agent performs file identification on the batch scripts with file dependency results according to the script type; and when it is identified that there is a calling file in the batch script, the calling file is identified in a fixed format to obtain a source file identifier; when it is identified that there is a generated file in the batch script that is not called by the batch script, the generated file is identified in a fixed format to obtain a target file identifier.
[0079] According to an embodiment of the present disclosure, based on the file dependency result obtained by the second agent, a batch script with file dependency is determined. The code for determining the batch script with file dependency can be encapsulated in a third agent. Alternatively, the code for determining the batch script with file dependency is set outside the third agent, and is used to input the batch script with file dependency to the second agent.
[0080] According to an embodiment of the present disclosure, a third agent further identifies the file name in a batch script with file dependencies based on the script type, and determines whether the file is a calling file or a generated file. When a calling file is identified in the batch script, the calling file is identified in a fixed format to obtain a source file identification; when a generated file that is not called by the batch script is identified in the batch script, the generated file is identified in a fixed format to obtain a target file identification.
[0081] According to an embodiment of the present disclosure, when it is identified that a generated file in a batch processing script is called by the batch processing script, and the file is an intermediate file, the file is neither a source file nor a target file.
[0082] According to an embodiment of the present disclosure, the prompt for file identification using a large language model may be, for example, "You are a batch script analysis expert. Please identify the file name (including the file address) in the script for each script code based on the provided script type, script code, and whether there is file dependency, and determine whether the file belongs to a calling file (source file) or a generated file (target file). If the file is an intermediate file, this file is neither a source file nor a generated file and does not need to be listed. The return result format is: source file: xxx / xx / src_file1; target file: yyy / yy / tgt_file2, yyy / yy / tgt_file5. Only one of the two can be used, such as there is only a source file and no target file in the script. No other results need to be returned."
[0083] According to an embodiment of the present disclosure, a large language model is used to identify source files and target files in a batch script, and the identified source files or target files are output as corresponding source file identifiers or target file identifiers. The format of the source file identifier or the target file identifier is fixed, thereby reducing the recognition difficulty in the process of pairing the source file identifier or the target file identifier, reducing recognition errors caused by inconsistent formats, and improving the accuracy of link relationship recognition.
[0084] According to an embodiment of the present disclosure, according to each script type, a second agent is used to identify file dependencies for each batch script to determine the file dependency result of each batch script, including: for each batch script, the second agent identifies the calling file and the generating file in the batch script based on the script type, and when it is identified that there are calling files or generating files in the batch script, the file dependency result of the batch script is determined to have file dependency; and when it is identified that there are no calling files or generating files in the batch script, the file dependency result of the batch script is determined to have no file dependency.
[0085] According to the embodiments of the present disclosure, for scripts of different script types, it is respectively identified whether there are file dependencies in the scripts, that is, whether other files are called or new files are generated, and the execution process of the task is encapsulated into a second agent. The prompt for file dependency identification using a large language model can be, for example, "You are a script analysis expert. Please identify whether each script code file calls other files or generates new files based on the provided script type and script code. If not, return 'no file dependency', if yes, return 'file dependency'."
[0086] For example, when the second agent receives a batch script with a file type, the pre-set prompt, the script content and the script type of the batch script are input into the large language model, and the large language model uses the script type as one of the features for identifying the file dependency results of the batch script to identify the file dependency results of the batch script. For example, when there is code content for calling a file in script 3, the second agent can obtain the file dependency result of script 3 as having file dependency, where the code content can be "path 1\path 2\...\path x\file name".
[0087] According to an embodiment of the present disclosure, a second agent is used to identify whether a dependency relationship exists in a batch script, and the identification result is marked by a file dependency result, thereby distinguishing batch scripts with file dependencies from batch scripts without file dependencies. Since source files or target files only exist in batch scripts with file dependencies, determining the file dependency result of the batch script through the second agent reduces the number of batch scripts that the third agent needs to process and the complexity of the tasks that need to be performed logically, thereby improving the accuracy of the overall link relationship identification.
[0088] According to an embodiment of the present disclosure, based on the first intelligent agent, type identification is performed on multiple acquired batch scripts to determine the script type of each batch script, including: for each acquired batch script, the first intelligent agent performs feature extraction on the batch script to obtain a feature extraction result; based on the feature extraction result, the script type of the batch script is determined from a preset script type set.
[0089] According to an embodiment of the present disclosure, the preset script type set may include Shell, Python, Perl and others. The feature extraction result may include the file extension, the first line of code contains instructions specific to the language, for example, #! / bin / bash for Shell script. The feature extraction result may also include keywords that define variables and functions, for example, the def keyword in Python. The first agent extracts code features in the batch script based on the encapsulated large language model, and based on the feature extraction result, determines the script type of the batch script among Shell, Python, Perl and others.
[0090] According to an embodiment of the present disclosure, the prompt for the first agent to identify the script type may be, for example, "You are a script analysis engineer. Please classify each script code file based on the provided script code and tell me the result. Do not return other information. These categories are: Shell, Python, Perl, and others. Please select one of the categories and return the result." Among them, Shell, Python, and Perl are several widely used scripting languages.
[0091] According to an embodiment of the present disclosure, the type of batch script is identified by the first intelligent agent. Since batch scripts of different script types differ in code logic, writing methods and other aspects, the batch scripts are classified so that the second intelligent agent can identify the batch scripts in a targeted manner based on the script type, thereby improving the accuracy of link relationship identification.
[0092] Figure 3 The data flow diagram of the link relationship identification method according to an embodiment of the present disclosure is schematically shown.
[0093] like Figure 3 As shown, the first agent identifies the script type of each batch script in a plurality of batch scripts. The second agent identifies whether there is a file dependency in each batch script based on the script type identified by the first agent, and obtains a file dependency result. Thus, the third agent can identify the batch scripts with file dependency results based on the script type of each batch script to determine the source file identifier and the target file identifier of the batch script. This facilitates the fourth agent to determine the link relationship between the plurality of batch scripts based on the source file identifier and the target file identifier of each batch script.
[0094] According to an embodiment of the present disclosure, the first agent can store the script type and script of each batch script in a database, so that the second agent, the third agent, and the fourth agent can obtain corresponding data from the database. In the database, each batch script can include a primary key value and multiple attribute values corresponding thereto, and the primary key value is used to uniquely identify the batch script, for example, the primary key value can be the name or digital number of the batch script, etc. The multiple attribute values of each batch script can include the script type, file dependency results, source file identification, target file identification, etc.
[0095] According to an embodiment of the present disclosure, the second agent may include a device for periodically checking whether there are new batch scripts in the database. When a new batch script is found in the database, the script content and script type of each batch script are obtained from the database through a data transmission interface, and the script content and script type of the batch script are processed to obtain the file dependency result of each batch script. The second agent writes the file dependency result of each batch script into the database for storage.
[0096] According to an embodiment of the present disclosure, a message queue may be set between the third agent and the second agent. When the second agent processes a preset number of batch scripts, the script content, script type, and file dependency results of the batch scripts may be sent to the message queue. In addition, the second agent may also write the file dependency results of each batch script into a database to persist the file dependency results of each batch script.
[0097] According to an embodiment of the present disclosure, the third agent obtains a batch script with file dependency results through a data transmission interface. Thus, each batch script can be identified by searching for a batch script with file dependency results from the received batch scripts. The batch script with file dependency results can be processed by calling a large language model to obtain a source file identifier and a target file identifier, or a source file identifier or a target file identifier, of each batch script.
[0098] According to an embodiment of the present disclosure, the output results of each intelligent agent are stored in a database, and each intelligent agent obtains the required data from the database to realize its own function, so that when each intelligent agent performs batch processing script processing in batches, the second intelligent agent can process part of the batch processing script based on the determined script type in the database, thereby improving data processing efficiency.
[0099] According to an embodiment of the present disclosure, the fourth agent can obtain the primary key value, source file identifier and target file identifier of each batch script from the database. Based on the fourth agent identifying the source file identifier and target file identifier of each batch script, the link relationship composed of multiple batch scripts is determined. When searching for the downstream dependency of each batch script to determine the link relationship of the batch script, the fourth agent can store the target file identifier, downstream dependency, primary key value of the batch script corresponding to the downstream dependency, and source file identifier of each batch script in the database.
[0100] According to an embodiment of the present disclosure, when the fourth agent has processed all batch scripts, a topology diagram for displaying the link relationship between batch scripts can be established based on the target file identifier of each batch script, downstream dependencies, and the primary key value of the batch script corresponding to the downstream dependencies, so that operation and maintenance personnel or other staff can intuitively observe the dependency relationship between each batch script. The topology diagram can also include the source file identifier and target file identifier of each batch script, so that the staff can check the cause of the job execution failure based on the path information in the source file identifier or the target file identifier.
[0101] According to an embodiment of the present disclosure, the link relationship identification method also includes: verifying the identification data output by each of the first intelligent agent, the second intelligent agent, the third intelligent agent and the fourth intelligent agent to obtain respective verification results; and fine-tuning the first intelligent agent, the second intelligent agent, the third intelligent agent and the fourth intelligent agent based on their respective verification results.
[0102] According to an embodiment of the present disclosure, based on a known configuration file including script type, file dependency result, source file and target file, and link relationship, the identification data output by each of the first intelligent agent, the second intelligent agent, the third intelligent agent, and the fourth intelligent agent are verified respectively, wherein the identification data of the first intelligent agent is the script type of the batch script, the identification data of the second intelligent agent is the file dependency result of the batch script, the identification data of the third intelligent agent is the source file and target file of the batch script, and the identification data of the fourth intelligent agent is the link relationship of the batch script.
[0103] According to the embodiments of the present disclosure, when the verification result indicates that there are errors in the verification process, the agent with errors is fine-tuned based on the fine-tuning data marked by relevant technicians. The Low-Rank Adaptation (LoRA) method of the large language model can be used to fine-tune the agent to improve the recognition effect of the field. The peft library of Python can be used to implement fine-tuning training, and a model with 16-bit floating point precision can be used for reasoning.
[0104] According to an embodiment of the present disclosure, for the first agent, the script content and script category of the batch script can be marked according to the professional experience of the script engineer, and used as fine-tuning data for fine-tuning training. The data used for fine-tuning the first agent can include: [script content, script category]. For example, script 1, shell; script 2, perl; ... script n, python.
[0105] According to an embodiment of the present disclosure, for the second agent, data can be labeled according to the professional experience of the script engineer and used as fine-tuning data for fine-tuning training. The data used for fine-tuning the second agent may include: [script content, script category, whether it is file dependent]. For example, script 1, shell, no file dependency; script 2, perl, with file dependency; ... script n, python, with file dependency.
[0106] According to an embodiment of the present disclosure, for the third agent, data can be labeled according to the professional experience of the script engineer and used as fine-tuning data for fine-tuning training. The data used to fine-tune the third agent may include: [script content, script category, whether it is file dependent]. For example, script 1, shell, no file dependency; script 2, perl, with file dependency, {source file: a1; target file: b1, b2}; ... script n, python, with file dependency, {source file: p1, p2}.
[0107] According to an embodiment of the present disclosure, for the fourth agent, data can be labeled according to the professional experience of batch processing business experts and used as fine-tuning data for fine-tuning training. The data used to fine-tune the third agent may include: [script content 1, file 1, dependency, script content 2, file 2]. For example, script 1, source file 1, upstream dependency, script 6, target file 2; script 2, source file 2, upstream dependency, script 6, target file 2; ... script n, source file n, upstream dependency, script 2, target file 2.
[0108] According to the embodiments of the present disclosure, the same large language model can be called for each agent. Based on the fine-tuning data of each agent, different functions of the large language model are fine-tuned respectively, thereby improving the accuracy of each agent.
[0109] According to the embodiments of the present disclosure, fine-tuning training is performed for each intelligent agent separately so that the recognition effect is more suitable for the batch processing business field. Through the continuous accumulation of domain data and continuous fine-tuning of the intelligent agent, a closed-loop mechanism is formed, thereby better adapting to changes in business scenarios. Changes in business scenarios include an increase in script types, an increase in file methods, etc.
[0110] According to the embodiments of the present disclosure, through verification and fine-tuning, errors in subsequent processes caused by recognition errors can be reduced, and the accuracy of the entire workflow can be improved. The first agent, the second agent, the third agent, and the fourth agent are fine-tuned respectively to reduce the error accumulation caused by the interaction between the agents and improve the accuracy of the entire workflow.
[0111] Figure 4The flowchart of a link relationship identification method according to another embodiment of the present disclosure is schematically shown.
[0112] like Figure 4 As shown, a first agent obtains batch scripts, and the first agent identifies the script type of the obtained multiple batch scripts to determine the script type of each batch script. A second agent identifies the file dependency of each batch script after the script type identification is completed to determine the file dependency result of each batch script. A third agent identifies the source file and the target file of the batch scripts with file dependencies to obtain the source file identifier and the target file identifier of each batch script. A fourth agent identifies the link relationship of multiple batch scripts based on the source file identifier and the target file identifier of each batch script. Based on the above-mentioned link relationship identification method, the present disclosure also provides a link relationship identification device. The following will be combined with Figure 5 The device is described in detail.
[0113] Figure 5 The structure block diagram of the link relationship identification device according to the embodiment of the present disclosure is schematically shown.
[0114] like Figure 5 As shown, the link relationship identification device 500 of this embodiment includes a type determination module 510 , a dependency determination module 520 , a file determination module 530 and a relationship determination module 540 .
[0115] The type determination module 510 is used to respond to the link relationship identification request from the user, identify the multiple batch processing scripts obtained based on the first agent, and determine the script type of each batch processing script. In one embodiment, the type determination module 510 can be used to perform the operation S210 described above, which will not be repeated here.
[0116] The dependency determination module 520 is used to identify each batch script in each script type based on the second agent, and determine the file dependency result of each batch script, wherein the file dependency result includes file dependency and no file dependency. In one embodiment, the dependency determination module 520 can be used to perform the operation S220 described above, which will not be repeated here.
[0117] The file determination module 530 is used to identify each batch processing script with a file dependency result in each script type based on the third agent, and determine the source file identifier and the target file identifier corresponding to each batch processing script. In one embodiment, the file determination module 530 can be used to perform the operation S230 described above, which will not be repeated here.
[0118] The relationship determination module 540 is used to identify the source file identifier and the target file identifier of each batch processing script based on the fourth agent, and determine the link relationship of multiple batch processing scripts. In one embodiment, the relationship determination module 540 can be used to perform the operation S240 described above, which will not be repeated here.
[0119] The first agent, the second agent, the third agent and the fourth agent are task execution bodies encapsulated based on the large language model.
[0120] According to an embodiment of the present disclosure, the relationship determination module 540 includes a script determination module.
[0121] The script determination module is used by the fourth agent to pair the source file identifier and the target file identifier of each batch script based on a preset script pairing rule, and determine the dependent scripts of each batch script, wherein the dependent scripts include upstream dependent scripts and downstream dependent scripts.
[0122] According to an embodiment of the present disclosure, the script determination submodule includes: a first traversal unit.
[0123] The first traversal unit is used for, when the script pairing rule representation is used to find the upstream dependent script of each script, the fourth agent repeats the operations of the first determination subunit and the second determination subunit until all scripts are traversed.
[0124] The first determination subunit is used to determine a target file identifier that is the same as a source file identifier of an i-th batch processing script among target file identifiers of at least one first batch processing script, wherein i is greater than or equal to 1, and the first processing script represents other batch processing scripts except the i-th batch processing script.
[0125] The second determining subunit is used to determine the first batch of processing scripts to which the target file identifier belongs as the upstream dependent scripts of the i-th batch processing script.
[0126] According to an embodiment of the present disclosure, the script determination submodule further includes: a second traversal unit.
[0127] The second traversal unit is used for, when the script pairing rule representation is used to find the downstream dependent script of each script, the fourth agent repeatedly executes the operations of the third determination subunit and the fourth determination subunit until all scripts are traversed.
[0128] The third determination subunit is used to determine a source file identifier that is the same as a target file identifier of a j-th batch processing script among the source file identifiers of at least one second batch processing script, wherein j is greater than or equal to 1, and the second processing script represents other batch processing scripts except the j-th batch processing script.
[0129] The fourth determining subunit is used to determine the second batch processing script to which the target file identifier belongs as a downstream dependent script of the jth batch processing script.
[0130] According to an embodiment of the present disclosure, the file determination module 530 includes: a recognition submodule, a first identification submodule, and a second identification submodule.
[0131] The identification submodule is used to perform file identification for each batch processing script by a third intelligent agent according to the script type for the batch processing script with file dependency results.
[0132] The first identification submodule is used to identify the calling file in a fixed format to obtain a source file identification when it is identified that there is a calling file in the batch processing script.
[0133] The second identification submodule is used to identify the generated file in a fixed format to obtain a target file identification when it is identified that there is a generated file in the batch processing script that is not called by the batch processing script.
[0134] According to an embodiment of the present disclosure, the dependency determination module 520 includes: a first determination submodule and a second determination submodule.
[0135] The first determination submodule is used for, for each batch script, the second agent identifies the calling files and generating files in the batch script based on the script type, and when it is identified that there are calling files or generating files in the batch script, determines that the file dependency result of the batch script is that there is file dependency.
[0136] The second determining submodule is used to determine that the file dependency result of the batch processing script is no file dependency when it is identified that there is no calling file or generating file in the batch processing script.
[0137] According to an embodiment of the present disclosure, the type determination module 510 includes: a result determination submodule and a type determination submodule.
[0138] The result determination submodule is used for the first agent to perform feature extraction on each acquired batch processing script to obtain a feature extraction result.
[0139] The type determination submodule is used to determine the script type of the batch script from a preset script type set based on the feature extraction result.
[0140] According to an embodiment of the present disclosure, the link relationship identification device further includes: a verification module and a fine-tuning module.
[0141] The verification module is used to verify the identification data output by the first intelligent agent, the second intelligent agent, the third intelligent agent and the fourth intelligent agent respectively, and obtain respective verification results.
[0142] The fine-tuning module is used to fine-tune the first agent, the second agent, the third agent and the fourth agent respectively based on their respective verification results.
[0143] According to an embodiment of the present disclosure, any multiple modules of the type determination module 510, the dependency determination module 520, the file determination module 530, and the relationship determination module 540 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the type determination module 510, the dependency determination module 520, the file determination module 530, and the relationship determination module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the type determination module 510, the dependency determination module 520, the file determination module 530, and the relationship determination module 540 may be at least partially implemented as a computer program module, which may perform a corresponding function when executed.
[0144] Figure 6 A block diagram of an electronic device suitable for implementing a link relationship identification method according to an embodiment of the present disclosure is schematically shown.
[0145] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0146] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.
[0147] According to an embodiment of the present disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.
[0148] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0149] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.
[0150] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the link relationship identification method provided by the embodiment of the present disclosure.
[0151] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 601. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0152] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0153] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.
[0154] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
[0155] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0156] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0157] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. A link relationship identification method, characterized in that: The method comprises: In response to a link relationship identification request from a user, based on the first agent, identifying the types of the acquired batch processing scripts, determining the script type of each batch processing script; According to each of the script types, using a second agent to identify file dependencies of each of the batch processing scripts, and determining a file dependency result of each of the batch processing scripts; According to each of the script types and the file dependency results, using a third agent to perform file identification on each of the batch processing scripts, and determine a source file identifier and a target file identifier corresponding to each of the batch processing scripts; and Based on the fourth agent, the source file identifier and the target file identifier of each batch processing script are identified to determine the link relationship composed of multiple batch processing scripts; Among them, the first agent, the second agent, the third agent and the fourth agent are task execution bodies encapsulated based on a large language model.
2. The method according to claim 1, characterized in that The fourth agent identifies the source file identifier and the target file identifier of each batch processing script to determine the link relationship of the plurality of batch processing scripts, including: The fourth agent pairs the source file identifier and the target file identifier of each batch script based on a preset script pairing rule, and determines the dependent scripts of each batch script, wherein the dependent scripts include an upstream dependent script and a downstream dependent script.
3. The method according to claim 2, characterized in that The fourth agent pairs the source file identifier and the target file identifier of each batch processing script based on a preset script pairing rule, and determines an upstream dependent script or a downstream dependent script of each batch processing script, including: In the case where the script pairing rule representation is used to find the upstream dependent script of each script, the fourth agent repeatedly performs the following operations until all scripts are traversed: Determine, among the target file identifiers of at least one first batch of processing scripts, a target file identifier that is the same as the source file identifier of the i-th batch processing script, wherein i is greater than or equal to 1, and the first processing script represents the other batch processing scripts except the i-th batch processing script; and The first batch of processing scripts to which the target file identifier belongs are determined as upstream dependent scripts of the i-th batch processing script.
4. The method according to claim 2, characterized in that The fourth agent pairs the source file identifier and the target file identifier of each batch processing script based on a preset script pairing rule to determine an upstream dependent script or a downstream dependent script of each batch processing script, and further includes: In the case where the script pairing rule representation is used to find the downstream dependent script of each script, the fourth agent repeatedly performs the following operations until all scripts are traversed: Determine, among the source file identifiers of at least one second batch processing script, a source file identifier that is the same as the target file identifier of the j-th batch processing script, wherein j is greater than or equal to 1, and the second processing script represents the other batch processing scripts except the j-th batch processing script; and The second batch processing script to which the target file identifier belongs is determined as a downstream dependent script of the jth batch processing script.
5. The method according to claim 1, characterized in that The method of performing file identification on each batch processing script using a third agent according to each script type and the file dependency result, and determining a source file identifier and a target file identifier corresponding to each batch processing script, comprises: For each of the batch processing scripts, the third agent performs file identification on the batch processing scripts with file dependency results according to the script type; and When it is identified that there is a calling file in the batch processing script, the calling file is identified in a fixed format to obtain the source file identification; When it is identified that there is a generated file in the batch processing script that is not called by the batch processing script, the generated file is identified in a fixed format to obtain the target file identification.
6. The method according to claim 1, characterized in that The method of identifying the file dependency of each batch processing script using a second agent according to each script type and determining the file dependency result of each batch processing script includes: For each of the batch processing scripts, the second agent identifies the calling file and the generating file in the batch processing script based on the script type, and when it is identified that there is a calling file or a generating file in the batch processing script, determines that the file dependency result of the batch processing script is that there is a file dependency; and When it is identified that there is no calling file or generating file in the batch processing script, it is determined that the file dependency result of the batch processing script is no file dependency.
7. The method according to claim 1, characterized in that The first agent is based on which the type of the acquired multiple batch processing scripts is identified to determine the script type of each batch processing script, including: For each of the acquired batch processing scripts, the first agent performs feature extraction on the batch processing script to obtain a feature extraction result; The script type of the batch processing script is determined from a preset script type set based on the feature extraction result.
8. The method according to claim 1, characterized in that The method further comprises: Verifying the identification data outputted by the first agent, the second agent, the third agent, and the fourth agent, respectively, to obtain respective verification results; The first agent, the second agent, the third agent and the fourth agent are fine-tuned respectively based on the respective verification results.
9. A link relationship identification device, characterized in that: The device comprises: A type determination module, configured to respond to a link relationship identification request from a user, identify the acquired multiple batch processing scripts based on the first agent, and determine the script type of each batch processing script; A dependency determination module, used for identifying each batch processing script in each script type based on the second agent, and determining a file dependency result of each batch processing script, wherein the file dependency result includes a file dependency and a no file dependency; A file determination module is used to identify each batch processing script having a file dependency result in each script type based on a third agent, and determine a source file identifier and a target file identifier corresponding to each batch processing script; and A relationship determination module, used for identifying the source file identifier and the target file identifier of each batch processing script based on a fourth agent, and determining a link relationship consisting of a plurality of batch processing scripts; Among them, the first agent, the second agent, the third agent and the fourth agent are task execution bodies encapsulated based on a large language model.
10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.
11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
Citation Information
Cited By
Automatic file arrangement framework and method based on large language model and computer program product
CN119961226A
File generation method and device, equipment, storage medium and product
CN122595979A
