Blood relationship information management method and device for multi-modal data

Abstract

The invention discloses a blood relationship information management method and device for multi-modal data. The method comprises the following steps: acquiring target multi-modal data; performing multiple rounds of processing on the target multi-modal data, and recording target metadata of the target multi-modal data in each round of processing flow; for any two processing flows, determining whether the two processing flows have a data blood relationship or not according to whether the processing result of the previous processing flow is consistent with the processing object of the post processing flow or not; and recording the target metadata of the two processing flows with the data blood relationship. According to the method and the device, the technical problem of low processing efficiency caused by lack of data blood relationship management when large-scale multi-modal data is processed in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the technical field of big data processing, and in particular to a method and device for managing bloodline information of multimodal data. Background Art

[0002] In the field of multimodal data processing, existing distributed processing frameworks usually process the multimodal data itself when processing multimodal data, without paying attention to the data change relationship during the processing process. Therefore, there is a lack of automated recording solutions for data lineage information, and each framework needs to write a separate program to process it. The operation is relatively complex and difficult to process, making it difficult to trace and analyze data lineage information. At the same time, when recording data lineage information, it is usually based on relational databases or simple log records, which cannot effectively handle the complex file name conversion relationship during multimodal data processing, especially when the data object is converted from one to multiple objects (such as video frame extraction, image segmentation, etc.).

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method and device for managing lineage information of multimodal data, so as to at least solve the technical problem that the related technology lacks management of data lineage relationships when processing large-scale multimodal data, resulting in low processing efficiency.

[0005] According to one aspect of an embodiment of the present application, a method for managing lineage information of multimodal data is provided, including: acquiring target multimodal data; performing multiple rounds of processing on the target multimodal data, and recording target metadata of the target multimodal data in each round of processing flow; for any two processing flows, determining whether there is a data lineage relationship between the two processing flows based on whether the processing result of the prior processing flow is consistent with the processing object of the subsequent processing flow; and recording the target metadata of the two processing flows that have a data lineage relationship.

[0006] Optionally, obtaining target multimodal data includes: obtaining initial multimodal data to be processed from a data source, wherein the data type of the multimodal data includes at least one of the following: pictures, audio, video, documents; performing a preprocessing operation on the initial multimodal data to obtain target multimodal data, wherein the preprocessing operation includes at least one of the following: format conversion, data cleaning.

[0007] Optionally, multiple rounds of processing are performed on the target multimodal data, including: calling a big data processing engine to distribute the target multimodal data to a preset cluster, and batch calling a large model service in the preset cluster to process the target multimodal data, wherein the processing type includes at least one of the following: text analysis, image recognition, speech recognition, and video analysis.

[0008] Optionally, recording target metadata of the target multimodal data in each round of processing flow includes: calling a distributed data structure to record target metadata of the target multimodal data in each round of processing flow, wherein the target metadata includes at least one of the following: file name, processing time, and processing type.

[0009] Optionally, recording target metadata in two processing flows having a data lineage relationship includes: using a target data structure to record target metadata in two processing flows having a data lineage relationship, wherein the target data structure is used to record a parent metadata and at least one corresponding child metadata.

[0010] Optionally, the target metadata of two processing flows with data lineage relationship is recorded. The method also includes: updating a preset graph database with the target metadata as a graph node and the data lineage relationship between the two processing flows with data lineage relationship as an edge, wherein the graph database is used to store metadata information of multimodal data in the processing flows with data lineage relationship.

[0011] Optionally, the preset graph database is updated with each target metadata as a graph node and the data lineage relationship as an edge, including: updating the graph database with the target metadata as a graph node and the conversion relationship between the file names in the target metadata of two processing flows with data lineage relationship as an edge.

[0012] According to another aspect of an embodiment of the present application, a multimodal data lineage information management device is also provided, including: an acquisition module for acquiring target multimodal data; a processing module for performing multiple rounds of processing on the target multimodal data, and recording target metadata of the target multimodal data in each round of processing flow; a judgment module for determining whether there is a data lineage relationship between any two processing flows, based on whether the processing result of the prior processing flow is consistent with the processing object of the subsequent processing flow; and a recording module for recording the target metadata of two processing flows that have a data lineage relationship.

[0013] According to another aspect of an embodiment of the present application, a computer program product is also provided, the computer program product comprising: a computer program, wherein when the computer program is executed by a processor, the above-mentioned method for managing bloodline information of multimodal data is implemented.

[0014] According to another aspect of an embodiment of the present application, an electronic device is also provided, which includes: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the above-mentioned bloodline information management method of multimodal data through the computer program.

[0015] In an embodiment of the present application, target multimodal data is acquired; multiple rounds of processing are performed on the target multimodal data, and the target metadata of the target multimodal data in each round of processing flow is recorded; for any two processing flows, whether there is a data lineage relationship between the two processing flows is determined based on whether the processing result of the previous processing flow is consistent with the processing object of the subsequent processing flow; and the target metadata of the two processing flows with data lineage relationship is recorded. The technical effect of automatic recording and visual analysis of lineage relationships in large-scale multimodal data processing flows is achieved, and the purpose of enhancing data processing transparency, traceability and optimizing data processing flows is achieved. Thereby solving the technical problem that the related technology lacks management of data lineage relationships when processing large-scale multimodal data, resulting in low processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 It is a flowchart of an optional multi-modal data bloodline information management method according to an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of an optional data kinship relationship according to an embodiment of the present application;

[0019] Figure 3 is a schematic structural diagram of an optional multi-modal data bloodline information management device according to an embodiment of the present application;

[0020] Figure 4 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] In order to better understand the embodiments of the present application, some nouns or terms that appear in the description of the embodiments of the present application are first translated and explained as follows:

[0024] Data Lineage: also known as data origin or data lineage, refers to the natural relationship between data during its entire life cycle, from generation, processing, fusion, circulation to final extinction. It records the link relationship of data generation. These relationships are similar to human blood relationships, so they are called data lineage relationships.

[0025] Apache Spark: It is a fast and general computing engine designed for large-scale data processing. Spark has the advantages of Hadoop MapReduce, but it is different from MapReduce in that the intermediate output of the job can be stored in memory, so there is no need to read and write HDFS (Hadoop Distributed File System). Therefore, Spark is better suited for MapReduce algorithms that require iteration, such as data mining and machine learning. In addition, Spark is implemented in the Scala language and uses Scala as its application framework. Unlike Hadoop, Spark and Scala can be tightly integrated, where Scala can operate distributed data sets as easily as operating local collection objects.

[0026] Large models: refer to neural network models with a large number of parameters and complex structures. These models usually require a lot of computing resources and data to train, and perform well in various artificial intelligence tasks.

[0027] Struct List: is a data type used to define linked list structures in C language. A linked list is a data structure in which each element (node) consists of two parts: a data field (used to store actual data) and a pointer field (used to point to the next or previous element). The following is an example of using Struct List to implement a single linked list:

[0028] typedef struct List{

[0029] int data;

[0030] Struct list*next;

[0031] }List;

[0032] In the above example, data is a variable that stores data, and next is a pointer to the same structure type, which is used to connect to the next node. It should be noted that if you want to implement a doubly linked list, you only need to add a pointer to the previous node.

[0033] Example 1

[0034] According to an embodiment of the present application, a method for managing bloodline information of multimodal data is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] Figure 1 is a flow chart of a method for managing blood relationship information of multimodal data provided in an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0036] Step S102, acquiring target multimodal data.

[0037] In the technical solution provided in the above step S102, multimodal data refers to data containing multiple different modes or types, each mode can be a different type of data, such as text, picture, audio, video, etc. These different modal data may be related to jointly express a complex real-world scene or scenario, wherein each modality provides a different perspective on the data and together constitutes a more comprehensive description. For example, in the field of autonomous driving, the above target multimodal data can be image data from a camera, point cloud data from a lidar, motion data from a sensor, etc.

[0038] Step S104, performing multiple rounds of processing on the target multimodal data, and recording the target metadata of the target multimodal data in each round of processing.

[0039] In the technical solution provided in the above step S104, the system performs multiple rounds of processing on the acquired target multimodal data, wherein the type of each round of processing is determined by the type of the processing object. For example, if the modality type of the processing object is video, then the type of the current round of processing is video analysis; if the modality type of the processing object is image, then the type of the current round of processing is image recognition. At the same time, the system can also record the target metadata in each round of processing flow, wherein the target metadata refers to the description information of the data in each round of processing flow.

[0040] Step S106, for any two processing flows, determine whether there is a data lineage relationship between the two processing flows based on whether the processing result of the previous processing flow is consistent with the processing object of the subsequent processing flow.

[0041] In the technical solution provided in the above step S106, for any two processing flows, the system will check whether the processing result of the previous processing flow is consistent with the processing object of the subsequent processing flow; if they are consistent, it means that the subsequent processing flow inherits the result of the previous processing flow, so it can be determined that the two processing flows have a data lineage relationship.

[0042] Step S108, recording target metadata of two processing flows having a data lineage relationship.

[0043] In the technical solution provided in the above step S108, in order to facilitate the subsequent tracing of the processing process of the multimodal data and make the source and evolution process of the data clearly visible, the system can record the target metadata of the two processing flows with data lineage relationship.

[0044] The following describes the various steps of the method for managing bloodline information of multimodal data in conjunction with a specific implementation process.

[0045] As an optional implementation, in the technical solution provided in the above step S102, the system may obtain the target multimodal data by the following method, including:

[0046] Step S1021, obtaining initial multimodal data to be processed from a data source, wherein the data type of the multimodal data includes at least one of the following: picture, audio, video, document;

[0047] Step S1022, performing a preprocessing operation on the initial multimodal data to obtain target multimodal data, wherein the preprocessing operation includes at least one of the following: format conversion, data cleaning.

[0048] In the above embodiment, the system can identify and obtain the initial multimodal data to be processed from a variety of supported data sources (such as distributed file systems HDFS, databases, etc.), wherein the data source may contain data of types such as pictures, audio, video or documents. Then, the system automatically detects the type of the initial multimodal data obtained to ensure that the data such as pictures, audio, video or documents can be correctly processed later; then, according to the data type and subsequent processing requirements, perform format conversion operations, such as converting pictures from JPEG format to PNG format, or converting audio from MP3 to WAV format, to meet the input requirements of subsequent data analysis; finally, clean the initial multimodal data after the format conversion, including removing invalid data, processing missing values, standardizing formats, etc., to ensure data quality and smooth processing process, so as to obtain the target multimodal data.

[0049] As an optional implementation, in the technical solution provided in the above step S104, the system can process the target multimodal data by the following method, including: calling the big data processing engine to distribute the target multimodal data to a preset cluster, and batch calling the big model service in the preset cluster to process the target multimodal data, wherein the processing type includes at least one of the following: text analysis, image recognition, speech recognition, and video analysis.

[0050] In the above embodiment, the above big data processing engine is preferably Apache Spark. The big model service is a way to use the big model as a service, providing the computing resources and functions of the big model through the network, so that other systems or users can use these big models through simple API calls, such as image recognition, text generation, speech recognition and other services.

[0051] Therefore, by utilizing Spark's distributed computing capabilities, the system can distribute the target multimodal data to the Spark cluster and call large model services in the Spark cluster in batches to process the data. For example: text analysis on documents, image recognition on images, speech recognition on audio, and video analysis on videos.

[0052] Furthermore, in the technical solution provided in the above step S104, the system can also use the following method to record the target metadata of the target multimodal data in each round of processing flow, including: calling a distributed data structure to record the target metadata of the target multimodal data in each round of processing flow.

[0053] The above-mentioned distributed data structure may be a DataFrame, wherein DataFrame is a tabular data structure including a set of ordered columns, each column may be of a different value type (such as a numerical value, a string, a Boolean type, etc.), similar to a table in a relational database, and supports the processing and operation of structured data. The target metadata stored in each column includes at least one of the following: file name (the name of the data object, used to uniquely identify the data file), file path (which refers to the location of the file in the storage system), file size, processing time (specific events of data processing, including start and end time), data type (format type of data), processing type (specific data processing operation performed), etc.

[0054] As an optional implementation, in the technical solution provided in the above step S108, the system can record the target metadata in two processing flows with a data lineage relationship in the following method, including: using a target data structure to record the target metadata in two processing flows with a data lineage relationship. The target data structure is used to record a parent metadata and at least one corresponding child metadata.

[0055] For example, when processing video data, you can first extract frames from the video to decompose the video into multiple images; then, when processing each image, you can identify the content of the image, label the image, and segment the labeled content to switch the large image into multiple small images. By analyzing the above two rounds of processing, it can be seen that in each round of processing, the post-processing process inherits the processing results of the previous processing process, and the previous processing process and the post-processing process are in a one-to-many conversion relationship (video frame extraction obtains multiple images, and image segmentation obtains multiple small images). Therefore, it is determined that there is a data lineage relationship between the two processing processes, and their data lineage relationship is as follows: Figure 2 In view of this situation, the embodiment of the present application adopts a target data structure (ie, a Struct List structure) to record the target metadata information of the target multimodal data before and after each round of conversion.

[0056] In addition, the system can also record the target metadata of two processing flows with data lineage relationship according to the following method, including: taking the target metadata as the graph node and the data lineage relationship between the two processing flows with data lineage relationship as the edge, and updating the preset graph database. Among them, the above-mentioned graph database is used to store the metadata information of multimodal data in the processing flow with data lineage relationship, and storing the data lineage relationship of multimodal data through the graph database can not only achieve efficient storage and query, but also use the graphical interface display technology of the graph database to realize the visualization of data lineage relationship, which not only helps data processing engineers and data analysts to quickly understand the entire life cycle of data, but also improves the transparency and operability of data processing.

[0057] Specifically, when the system uses a graph database to store data lineage relationships, it can use the target metadata (file names in it) as graph nodes and the conversion relationship between the file names in the target metadata of two processing flows with data lineage relationships as edges to update the graph database. As data processing proceeds, the system can update the data lineage information in the graph database in real time and dynamically render the lineage relationship on the visual interface, allowing users to track the data processing status in real time.

[0058] Based on the solutions defined in the above steps S102 to S108, it can be known that the method provided by the implementation of this application has the following technical advantages compared with the existing multimodal data management method:

[0059] (1) Using the distributed computing framework of the big data processing engine to process multimodal data can significantly improve the data processing speed and meet the requirements of real-time and high throughput.

[0060] (2) The target metadata of the target multimodal data in each round of processing is recorded through a distributed data structure, eliminating the need to manually write additional recording logic and simplifying the maintenance of blood relationships.

[0061] (3) The target data structure is used to record the target metadata of two processing flows with data lineage relationships, ensuring the transmission of lineage information during batch processing. Even if the data objects undergo multiple conversions, the lineage relationship remains intact, providing a reliable basis for data traceability.

[0062] (4) Using a graph database to store blood relationships can not only efficiently store complex relationships, but also support fast queries, making blood relationship analysis and data processing audits more efficient.

[0063] (5) Transparent data processing processes and traceable data lineage enhance users’ trust in the data processing system, while also promoting the improvement of data quality and ensuring the accuracy and consistency of processed data.

[0064] Example 2

[0065] According to an embodiment of the present application, a multimodal data lineage information management device for implementing the multimodal data lineage information management method in embodiment 1 is also provided. Figure 3 As shown, the multimodal data blood relationship information management device at least includes: an acquisition module 32, a processing module 34, a judgment module 36 and a recording module 38, wherein:

[0066] An acquisition module 32, used to acquire target multimodal data;

[0067] The processing module 34 is used to perform multiple rounds of processing on the target multimodal data and record the target metadata of the target multimodal data in each round of processing;

[0068] The judgment module 36 is used to determine whether there is a data lineage relationship between any two processing flows according to whether the processing result of the previous processing flow is consistent with the processing object of the subsequent processing flow;

[0069] The recording module 38 is used to record target metadata of two processing flows that have a data lineage relationship.

[0070] It should be noted that each module in the bloodline information management device for multimodal data in the embodiment of the present application corresponds one by one to each implementation step of the bloodline information management method for multimodal data in Example 1. Since a detailed description has been given in Example 1, some details not reflected in this embodiment can be referred to Example 1 and will not be elaborated here.

[0071] Example 3

[0072] According to an embodiment of the present application, a computer program product is also provided, which includes a computer program, wherein when the computer program is executed by a processor, the bloodline information management method of multimodal data in Example 1 is implemented.

[0073] According to an embodiment of the present application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the bloodline information management method of multimodal data in Example 1 by running the computer program.

[0074] According to an embodiment of the present application, a processor is also provided, which is used to run a computer program, wherein the bloodline information management method of multimodal data in Example 1 is executed when the computer program is running.

[0075] According to an embodiment of the present application, an electronic device is also provided, which includes: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the bloodline information management method of multimodal data in Example 1 through the computer program.

[0076] Specifically, the following steps are executed when the computer program is running: acquiring target multimodal data; performing multiple rounds of processing on the target multimodal data, and recording target metadata of the target multimodal data in each round of processing flow; for any two processing flows, determining whether there is a data lineage relationship between the two processing flows based on whether the processing result of the previous processing flow is consistent with the processing object of the subsequent processing flow; and recording the target metadata of the two processing flows that have a data lineage relationship.

[0077] As an optional implementation, the electronic device may be in the form of a mobile terminal, a computer terminal or a similar computing device. Figure 4 FIG. 1 shows a hardware structure block diagram of an electronic device for implementing a method for managing bloodline information of multimodal data. Figure 4 As shown, the electronic device 40 may include one or more (402a, 402b, ..., 402n are used to illustrate) processors 402 (the processor 402 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 404 for storing data, and a transmission device 406 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4 More or fewer components as shown, or with Figure 4 Different configurations are shown.

[0078] It should be noted that the one or more processors 402 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the electronic device 40. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0079] The memory 404 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the bloodline information management method of multimodal data in the embodiment of the present application. The processor 402 executes various functional applications and data processing by running the software programs and modules stored in the memory 404, that is, to implement the vulnerability detection method of the above-mentioned application. The memory 404 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 404 may further include a memory remotely arranged relative to the processor 402, and these remote memories may be connected to the electronic device 40 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0080] The transmission device 406 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the electronic device 40. In one example, the transmission device 406 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0081] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the electronic device 40 .

[0082] The serial numbers of the above embodiments are only for description and do not represent the advantages or disadvantages of the embodiments.

[0083] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0086] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0087] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0088] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for managing bloodline information of multimodal data, characterized in that: include: Acquire target multimodal data; Performing multiple rounds of processing on the target multimodal data, and recording target metadata of the target multimodal data in each round of processing; For any two of the processing flows, determining whether there is a data lineage relationship between the two processing flows based on whether the processing result of the previous processing flow is consistent with the processing object of the subsequent processing flow; The target metadata of the two processing flows having a data lineage relationship is recorded.

2. The method according to claim 1, characterized in that: Obtain target multimodal data, including: Acquire initial multimodal data to be processed from a data source, wherein the data type of the multimodal data includes at least one of the following: picture, audio, video, document; A preprocessing operation is performed on the initial multimodal data to obtain the target multimodal data, wherein the preprocessing operation includes at least one of the following: format conversion and data cleaning.

3. The method according to claim 1, characterized in that The target multimodal data is subjected to multiple rounds of processing, including: Call a big data processing engine to distribute the target multimodal data to a preset cluster, and call a large model service in batches in the preset cluster to process the target multimodal data, wherein the type of processing includes at least one of the following: text analysis, image recognition, speech recognition, and video analysis.

4. The method according to claim 1, characterized in that: Record the target metadata of the target multimodal data in each round of processing, including: The distributed data structure is called to record target metadata of the target multimodal data in each round of processing flow, wherein the target metadata includes at least one of the following: file name, processing time, and processing type.

5. The method according to claim 1, characterized in that The target metadata in the two processing flows with data lineage relationship is recorded, including: A target data structure is used to record target metadata in the two processing flows having a data lineage relationship, wherein the target data structure is used to record a parent metadata and at least one corresponding child metadata.

6. The method according to claim 1, characterized in that Recording target metadata of the two processing flows having a data lineage relationship, the method further includes: The preset graph database is updated with the target metadata as a graph node and the data lineage relationship between two processing flows with data lineage relationship as an edge, wherein the graph database is used to store metadata information of multimodal data in processing flows with data lineage relationship.

7. The method according to claim 6, characterized in that Taking each of the target metadata as a graph node and the data blood relationship as an edge, updating the preset graph database includes: The graph database is updated by taking the target metadata as graph nodes and the conversion relationship between the file names in the target metadata of two processing flows with data lineage relationship as edges.

8. A blood relationship information management device for multimodal data, characterized in that: include: An acquisition module, used to acquire target multimodal data; A processing module, used to perform multiple rounds of processing on the target multimodal data and record target metadata of the target multimodal data in each round of processing; A judgment module, for determining, for any two of the processing flows, whether there is a data lineage relationship between the two processing flows according to whether the processing result of the previous processing flow is consistent with the processing object of the subsequent processing flow; The recording module is used to record the target metadata of the two processing flows having a data lineage relationship.

9. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the method for managing bloodline information of multimodal data as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the bloodline information management method of multimodal data as described in any one of claims 1 to 7 through the computer program.

Images