Data blood relationship analysis method, device and equipment based on large model and medium

Abstract

The invention provides a data blood relationship analysis method and device based on a large model, equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical fields of large models, cloud computing, big data and the like. The data consanguinity analysis method based on the large model comprises the steps of performing static analysis on to-be-analyzed content to obtain a static analysis result; generating prompt information based on the to-be-analyzed content and the static analysis result; inputting the prompt information into a large model to obtain initial blood relationship information output by the large model; and obtaining target blood relationship information based on the initial blood relationship information.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the technical fields of big models, cloud computing, big data, etc., and in particular to a data lineage analysis method, device, equipment, medium and product based on a big model. Background Art

[0002] Against the backdrop of the continuous development of big data and artificial intelligence technologies, data lineage, as an important component of data governance, can clearly describe the flow and change process of data from source to application.

[0003] How to parse data lineage is a problem that needs to be solved. Summary of the invention

[0004] The present invention provides a method, device, equipment, medium and product for data lineage analysis based on a large model.

[0005] According to one aspect of the present disclosure, a data lineage analysis method based on a large model is provided, including: statically analyzing the content to be analyzed to obtain a static analysis result; generating prompt information based on the content to be analyzed and the static analysis result; inputting the prompt information into the large model to obtain initial lineage information output by the large model; and acquiring target lineage information based on the initial lineage information.

[0006] According to another aspect of the present disclosure, a data lineage analysis device based on a large model is provided, including: a parsing module, used to statically parse the content to be analyzed to obtain a static parsing result; a generating module, used to generate prompt information based on the content to be analyzed and the static parsing result; a calling module, used to input the prompt information into the large model to obtain the initial lineage information output by the large model; and an acquiring module, used to acquire target lineage information based on the initial lineage information.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method as described in any of the above aspects.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods according to any one of the above aspects.

[0009] According to another aspect of the present disclosure, a computer script product is provided, comprising a computer script, wherein when the computer script is executed by a processor, the computer script implements any one of the methods described in any one of the above aspects.

[0010] According to the embodiments of the present disclosure, data lineage can be analyzed efficiently and accurately.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0013] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0014] Figure 2 is a schematic diagram of an implementation system for implementing an embodiment of the present disclosure;

[0015] Figure 3 is a schematic diagram of a data lineage analysis platform provided according to an embodiment of the present disclosure;

[0016] Figure 4 is a schematic diagram according to a second embodiment of the present disclosure;

[0017] Figure 5 is a schematic diagram according to a third embodiment of the present disclosure;

[0018] Figure 6 It is a schematic diagram of an electronic device used to implement the data lineage analysis method based on a large model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] Data lineage, also known as data pedigree, data origin, and data genealogy, refers to the relationship formed between data during its entire life cycle, from data generation, processing, processing, fusion, circulation to final extinction.

[0021] In related technologies, data lineage is usually obtained based on static analysis or manual reporting, which has problems such as high complexity, insufficient scalability and accuracy.

[0022] In order to parse data lineage efficiently and accurately, the present disclosure provides the following embodiments.

[0023] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. This embodiment provides a data lineage analysis method based on a large model, such as Figure 1 As shown, the method includes:

[0024] 101. Perform static analysis on the content to be analyzed to obtain a static analysis result.

[0025] 102. Generate prompt information based on the content to be parsed and the static parsing result.

[0026] 103. Input the prompt information into the large model to obtain the initial blood relationship information output by the large model.

[0027] 104. Obtain target blood relationship information based on the initial blood relationship information

[0028] The content to be analyzed refers to the content to be analyzed for data lineage.

[0029] The content to be parsed includes, for example, a Structured Query Language (SQL) script or a PySpark script.

[0030] SQL is a database query and scripting language used to access data and query, update, and manage relational database systems. PySpark is a Python API for big data processing. Python is a programming language, and API is an application programming interface (Application Programming Interface), which is a set of predefined functions that aims to provide application scripts and developers with the ability to access a set of routines based on certain software or hardware without having to access the source code or understand the details of the internal working mechanism.

[0031] The content to be parsed is used to complete specific data processing tasks. For example, the content to be parsed includes the source table of the data, the operations performed on the data (such as connection, filtering, aggregation), and the target table where the data is written after the operation.

[0032] Static parsing is a technique for analyzing scripts, scripts, or codes without executing them. It mainly understands the function and behavior of the code based on the syntax, structure, and semantic rules of the code.

[0033] Specifically, the content to be parsed can be input into a static parser, and the output is a static parsing result. A static parser is, for example, ANTLR, which is an open source syntax analyzer that can automatically generate a syntax tree based on the input and display it visually.

[0034] Static parsing results refer to the results obtained based on static parsing, such as an Abstract Syntax Tree (AST). AST is an abstract representation of the syntax structure of the source code. It represents the syntax structure of the programming language in a tree form, and each node on the tree represents a structure in the source code.

[0035] Static analysis can usually only extract the surface logic of data flow, and it is difficult to understand complex business logic or implicit relationships between fields. Therefore, relying solely on static analysis for data lineage analysis will have problems such as insufficient accuracy.

[0036] In order to improve accuracy, we can rely on the excellent performance of large models to perform data lineage analysis.

[0037] The large model refers to the Large Language Model (LLM). LLM is a hot technology in the AI ​​field in recent years. LLM is a natural language processing model based on deep learning. It has a huge number of parameters and a complex structure, which enables it to process and understand a large amount of natural language data. Through the process of pre-training and fine-tuning, the LLM model can perform well in a variety of natural language processing tasks, including but not limited to text generation, language understanding, machine translation, etc.

[0038] The large model performs corresponding operations according to the prompt information.

[0039] Prompt information is generated based on the content to be parsed and the static parsing results.

[0040] Specifically, a prompt information template can be preset, which contains static content and dynamic content. The static content is used to prompt the large model to perform data lineage analysis operations, such as "As an expert familiar with SQL semantics and data lineage tracking, you will receive the following script snippet and preliminary analysis information. Based on the script snippet and analysis information, please infer the relationship between the source, conversion and output of the fields in the code. If you encounter uncertain logic, please mark it as uncertain, and finally output the specific lineage description in JSON format"; dynamic content is content filled in according to actual conditions, such as content to be analyzed and static analysis results.

[0041] To improve accuracy, prompt information can indicate the source, transformation logic, and target field of the large model's step-by-step reasoning fields; as well as adjust the reasoning direction based on intermediate results, complete missing information, or optimize path selection.

[0042] After obtaining the prompt information, the prompt information is input into the large model, and the large model performs reasoning based on the prompt information to obtain the initial blood relationship information.

[0043] The initial lineage information may include: relationships between tables, such as the first data table -> the second data table, that is, the first data table is the source table and the second data table is the target table; relationships between fields, such as the first field of the first data table -> the second field of the second data table, that is, the first field is the source field and the second field is the target field. Fields can also be called columns; conversion relationships between fields, such as the operations from the first field to the second field mentioned above.

[0044] The target blood relationship information refers to the finally determined blood relationship information, which can be the initial blood relationship information, and the initial blood relationship information can also be adjusted to use the adjusted blood relationship information as the target blood relationship information. For example, after obtaining the initial blood relationship information obtained by the large model, the initial blood relationship information can be adjusted manually to obtain the target blood relationship information.

[0045] In this embodiment, by analyzing data lineage based on the big model, the excellent performance of the big model can be used to improve the accuracy and efficiency of data lineage analysis. In addition, prompt information is generated based on the content to be analyzed and the static analysis results, which can provide more information for the big model to refer to, further improving the accuracy of data lineage analysis.

[0046] In order to better understand the present disclosure, the application scenarios involved in the present disclosure are described as follows:

[0047] Figure 2 It is a schematic diagram of an implementation system for implementing an embodiment of the present disclosure.

[0048] like Figure 2 As shown, the system in this scenario includes:

[0049] The client 201 is used to display an interactive interface to the user, and the user can input the content to be parsed in the interface.

[0050] The data lineage analysis platform 202 is used to receive the content to be analyzed sent by the client, process the content to be analyzed, and generate prompt information (prompt) for use in the large model.

[0051] The large model 203 is used to output initial blood relationship information according to the prompt information input by the data blood relationship analysis platform.

[0052] The data lineage analysis platform 202 is also used to receive the initial lineage information fed back by the large model, and obtain the target lineage information based on the initial lineage information.

[0053] After the data kinship analysis platform obtains the initial kinship information output by the large model, it can be displayed to the user through the client. Afterwards, the user can manually adjust the initial kinship information as needed and use the adjusted kinship information as the target kinship information. Alternatively, the initial kinship information can be used as the target kinship information, or the kinship information obtained by static analysis can be obtained and used as the target kinship information.

[0054] After obtaining the target bloodline information, the data bloodline analysis platform can store the target bloodline information, provide a visualization interface, and display the target bloodline information visually. It can also provide a query interface for other systems to query the target bloodline information, and then perform analysis, data traceability, and other operations on the target bloodline information.

[0055] Figure 3 It is a schematic diagram of a data lineage analysis platform provided according to an embodiment of the present disclosure.

[0056] like Figure 3 As shown, the data lineage analysis platform includes: a static analysis module 301, a prompt information management module 302, a lineage integration module 303 and a lineage management module 304.

[0057] The static analysis module 301 is used to perform static analysis on the content to be analyzed.

[0058] Among them, the static parsing module can specifically be antlr, which can obtain the AST corresponding to the content to be parsed after parsing the content to be parsed (such as SQL script or PySpark script), which includes DataFrame reading / writing and column operations, such as SELECT, FROM, JOIN, GROUP BY, subquery and other syntax nodes.

[0059] For content that cannot be processed by static analysis, such as string concatenation, external function or library calls, user-defined functions and other complex scenarios, script fragments related to these contents can be sent to the prompt information management module to generate prompt information based on these script fragments.

[0060] In addition, static lineage information can be generated based on the AST obtained by the static parsing module. For example, table names, column names (field names), and alias relationships can be extracted from the AST. Function call relationships, operator operation relationships, etc. can also be identified. The lineage relationship generated based on this information can be called static lineage information, such as field A comes from table B.

[0061] The prompt information management module 302 is used to generate prompt information.

[0062] The prompt information management module may include: a template library and a generation module.

[0063] The template library can pre-configure various types of templates, such as "SQL lineage extraction prompt" and "UDF logic analysis prompt". Afterwards, you can select the corresponding template according to the type of content to be analyzed.

[0064] The generation module is used to generate prompt information based on templates and static parsing results.

[0065] For example, the static blood relationship information obtained from the static analysis results, as well as the relevant script fragments of the content that cannot be statically analyzed, are filled into the corresponding type template to obtain prompt information.

[0066] After the prompt information is generated, it is input into the large model (LLM), and the large model outputs the initial blood relationship information based on the prompt information.

[0067] In addition, calling parameters, such as temperature, model version, etc., may also be carried in the prompt information. Different types of content to be parsed may correspond to different calling parameters, so that different calling strategies may be implemented for different types of content to be parsed.

[0068] The initial blood relationship information can be input into the blood relationship integration module, and the blood relationship integration module can display the initial blood relationship information to the user, and the user can adjust it to obtain manually annotated blood relationship information.

[0069] For example, the prompt information can instruct the large model to mark uncertain content, such as marking it as uncertain, so that users can analyze and adjust these uncertain contents. In addition, users can also analyze and adjust the overall content based on experience.

[0070] In addition, users can also adjust the prompt information based on the initial blood relationship information fed back by the large model and continuously update the template of the prompt information.

[0071] The blood relationship integration module 303 is used to integrate various types of blood relationship information into target blood relationship information.

[0072] For example, the input of the bloodline integration module includes: initial bloodline information obtained from the large model, static bloodline information obtained from static analysis, and manually annotated bloodline information obtained from manual adjustment. The bloodline integration module unifies these bloodline information into preset formats, such as JSON and other formatting forms, and can also annotate their sources, such as from static analysis, large models or manual annotation.

[0073] The blood relationship management module 304 is used to store and / or output target blood relationship information.

[0074] After obtaining the target bloodline information, it can be stored in the bloodline management module, and then it can be used by other systems through the output interface, such as visual display through the visualization system, and third-party calls through the API.

[0075] In combination with the above application scenarios, the present disclosure also provides the following embodiments.

[0076] Figure 4 is a schematic diagram according to the second embodiment of the present disclosure. This embodiment provides a data lineage analysis method based on a large model, the method comprising:

[0077] 401. Get the content to be parsed.

[0078] The content to be parsed is, for example, a code script, such as an SQL script or a PySpark script, which usually includes operations such as reading data from a data source, converting a data frame, calling a user-defined function (UDF), and finally writing the data to a target location for persistence.

[0079] 402. Perform static analysis on the content to be analyzed to obtain a static analysis result.

[0080] Among them, static analysis can perform the following in parallel: identifying basic conversion logic and detecting dynamic logic.

[0081] Identify basic transformation logic: such as reading and writing of data frames, column operations, etc. Based on these operations, AST can be generated, and static lineage information can be obtained based on AST.

[0082] Detect dynamic logic content: For example, dynamic SQL concatenation, UDF calls, or other content that is difficult to locate statically, it can be marked for subsequent interaction with the large model (LLM).

[0083] 403. Determine whether the content to be parsed contains a preset type of content. If so, execute 404 and 408. Otherwise, execute 408 directly.

[0084] Among them, the preset type of content is, for example, dynamic logic content, such as the above-mentioned SQL splicing, UDF calling and other contents. Through the static analysis process, it can be known whether there is dynamic logic content.

[0085] In this embodiment, when the content to be analyzed contains content of a preset type, prompt information is generated, and the large model can be called to perform blood relationship analysis based on the prompt information later, so that the large model can be called simply and efficiently.

[0086] 404. Generate prompt information based on the content to be parsed and the static parsing result.

[0087] Among them, a current template corresponding to the current type of the content to be parsed can be obtained from a plurality of preset types of templates; based on the content to be parsed and the static parsing result, the current template is filled to generate the prompt information.

[0088] For example, if the content to be parsed is an SQL script, then an SQL template can be obtained, and dynamic logic content, AST, static lineage information, etc. can be filled into the SQL template to obtain prompt information.

[0089] In this embodiment, by presetting multiple types of templates, a current template of the corresponding type is obtained based on the current type of the content to be analyzed, and prompt information is generated based on the current template. Different templates can be set for different types to improve adaptability, thereby improving the accuracy of the prompt information and further improving the accuracy of blood relationship analysis.

[0090] 405. Input the prompt information into the large model to obtain the initial blood relationship information output by the large model.

[0091] 406. Determine whether manual adjustment is required. If so, execute 407; otherwise, execute 409.

[0092] 407. Obtain manually annotated blood relationship information.

[0093] After obtaining the initial blood relationship information output by the large model, the initial blood relationship information can be displayed to the target object, such as the user, and the user can determine whether to make manual adjustments according to actual needs. Alternatively, the prompt information can also instruct the large model to mark the uncertain content, and then determine whether to make manual adjustments based on the mark.

[0094] When manual adjustment is required, an adjustment interface may be displayed, through which the user can add, modify, delete, and other operations on the initial blood relationship information to obtain manually annotated blood relationship information.

[0095] In this embodiment, by displaying the initial blood relationship information for manual adjustment by the target object, the accuracy of the blood relationship information can be improved by combining the large model and manual processing methods.

[0096] 408. Obtain static blood relationship information based on the static analysis result.

[0097] For example, the static parsing result includes AST, based on which the blood relationship between related nodes can be known, and the blood relationship can be called static blood relationship information.

[0098] 409. Obtain target blood relationship information based on the initial blood relationship information, the manually annotated blood relationship information, and the static blood relationship information.

[0099] For example, the three types of blood relationship information are integrated and unified into a preset format, such as JSON format, and the unified blood relationship information is used as the target blood relationship information.

[0100] In this embodiment, the target blood relationship information is obtained based on the initial blood relationship information, the manually annotated blood relationship information and the static blood relationship information, which can improve the comprehensiveness and accuracy of the target blood relationship information.

[0101] In some embodiments, the method may further include:

[0102] 410. Store or output the target blood relationship information.

[0103] For example, after obtaining the target blood relationship information, it is stored and then output as needed, such as visual output, or called by other systems for analysis and processing.

[0104] For example, visualized lineage relationships: help data engineers or business users quickly understand the source of each field and table; lineage query and impact analysis: when an operator or field changes, the affected downstream can be quickly located; audit compliance scenarios: meet data compliance requirements and supervise the full traceability and usage scope of data.

[0105] In this embodiment, by storing or outputting the target blood relationship information, it can be used as basic data for subsequent processes to facilitate subsequent use.

[0106] Figure 5 It is a schematic diagram according to the third embodiment of the present disclosure. This embodiment provides a data lineage analysis device based on a large model. The device 500 includes: a parsing module 501, a generating module 502, a calling module 503 and an acquiring module 504.

[0107] The analysis module 501 is used to perform static analysis on the content to be analyzed to obtain the static analysis result; the generation module 502 is used to generate prompt information based on the content to be analyzed and the static analysis result; the calling module 503 is used to input the prompt information into the large model to obtain the initial blood relationship information output by the large model; the acquisition module 504 is used to obtain the target blood relationship information based on the initial blood relationship information.

[0108] In this embodiment, by analyzing data lineage based on the big model, the excellent performance of the big model can be used to improve the accuracy and efficiency of data lineage analysis. In addition, prompt information is generated based on the content to be analyzed and the static analysis results, which can provide more information for the big model to refer to, further improving the accuracy of data lineage analysis.

[0109] In some embodiments, the generating module 502 is further configured to:

[0110] From among the preset multiple types of templates, obtain a current template corresponding to the current type of the content to be parsed;

[0111] Based on the content to be parsed and the static parsing result, the current template is filled to generate the prompt information.

[0112] In this embodiment, by presetting multiple types of templates, a current template of the corresponding type is obtained based on the current type of the content to be analyzed, and prompt information is generated based on the current template. Different templates can be set for different types to improve adaptability, thereby improving the accuracy of the prompt information and further improving the accuracy of blood relationship analysis.

[0113] In some embodiments, the generating module 502 is further configured to:

[0114] If the content to be parsed includes content of a preset type, prompt information is generated based on the content to be parsed and the static parsing result.

[0115] In this embodiment, when the content to be analyzed contains content of a preset type, prompt information is generated, and the large model can be called to perform blood relationship analysis based on the prompt information later, so that the large model can be called simply and efficiently.

[0116] In some embodiments, the calling module 503 is further used to:

[0117] Acquiring manually annotated blood relationship information based on the initial blood relationship information;

[0118] Acquire static blood relationship information based on the static analysis result;

[0119] Based on the initial blood relationship information, the manually annotated blood relationship information and the static blood relationship information, target blood relationship information is acquired.

[0120] In this embodiment, the target blood relationship information is obtained based on the initial blood relationship information, the manually annotated blood relationship information and the static blood relationship information, which can improve the comprehensiveness and accuracy of the target blood relationship information.

[0121] In some embodiments, the apparatus 500 further includes:

[0122] A management module is used to store or output the target blood relationship information.

[0123] In this embodiment, by storing or outputting the target blood relationship information, it can be used as basic data for subsequent processes to facilitate subsequent use.

[0124] It can be understood that in the embodiments of the present disclosure, the same or similar contents in different embodiments can be referenced to each other.

[0125] It can be understood that the “first”, “second”, etc. in the embodiments of the present disclosure are only used for distinction and do not indicate the degree of importance, time sequence, etc.

[0126] It is understandable that unless there is any special limitation on the sequence of steps in the process, it means that the timing relationship between these steps is not limited.

[0127] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0128] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer script product.

[0129] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 600 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0130] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer script stored in a read-only memory (ROM) 602 or a computer script loaded from a storage unit 606 into a random access memory (RAM) 603. In the RAM 603, various scripts and data required for the operation of the electronic device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0131] Multiple components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0132] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as a data lineage parsing method based on a large model. For example, in some embodiments, the data lineage parsing method based on a large model may be implemented as a computer software script, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer script may be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer script is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the data lineage parsing method based on the large model described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the large model-based data lineage parsing method in any other appropriate manner (eg, by means of firmware).

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer scripts that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The script code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These script codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable task processing device, so that when the script code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The script code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0135] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a script for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0138] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer scripts that run on the corresponding computers and have a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0140] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data lineage analysis method based on a large model, comprising: Perform static analysis on the content to be analyzed to obtain a static analysis result; Generate prompt information based on the content to be analyzed and the static analysis result; Inputting the prompt information into the large model to obtain the initial blood relationship information output by the large model; Target blood relationship information is acquired based on the initial blood relationship information.

2. The method according to claim 1, wherein: The generating prompt information based on the content to be parsed and the static parsing result includes: From among the preset multiple types of templates, obtain a current template corresponding to the current type of the content to be parsed; Based on the content to be parsed and the static parsing result, the current template is filled to generate the prompt information.

3. The method according to claim 1, wherein: The generating prompt information based on the content to be parsed and the static parsing result includes: If the content to be parsed includes content of a preset type, prompt information is generated based on the content to be parsed and the static parsing result.

4. The method according to claim 1, wherein: The acquiring target blood relationship information based on the initial blood relationship information includes: Acquiring manually annotated blood relationship information based on the initial blood relationship information; Acquire static blood relationship information based on the static analysis result; Based on the initial blood relationship information, the manually annotated blood relationship information and the static blood relationship information, target blood relationship information is acquired.

5. The method according to claim 1, further comprising: The target blood relationship information is stored or outputted.

6. A data lineage analysis device based on a large model, comprising: The parsing module is used to perform static parsing on the content to be parsed to obtain a static parsing result; A generating module, used for generating prompt information based on the content to be analyzed and the static analysis result; A calling module, used for inputting the prompt information into the large model to obtain the initial blood relationship information output by the large model; An acquisition module is used to acquire target blood relationship information based on the initial blood relationship information.

7. The device according to claim 6, wherein: The generating module is further used for: From among the preset multiple types of templates, obtain a current template corresponding to the current type of the content to be parsed; Based on the content to be parsed and the static parsing result, the current template is filled to generate the prompt information.

8. The device according to claim 6, wherein: The generating module is further used for: If the content to be parsed includes content of a preset type, prompt information is generated based on the content to be parsed and the static parsing result.

9. The device according to claim 6, wherein: The calling module is further used for: Acquiring manually annotated blood relationship information based on the initial blood relationship information; Acquire static blood relationship information based on the static analysis result; Based on the initial blood relationship information, the manually annotated blood relationship information and the static blood relationship information, target blood relationship information is acquired.

10. The apparatus according to claim 6, further comprising: A management module is used to store or output the target blood relationship information.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

13. A computer script product, comprising a computer script, wherein the computer script implements the method according to any one of claims 1 to 5 when executed by a processor.

Images