Data fusion method and device, storage medium and electronic equipment

Abstract

The invention discloses a data fusion method and device, a storage medium and electronic equipment. The method comprises the following steps: in response to access of a current media asset platform, acquiring historical media asset metadata of target media assets in a historical period from a plurality of heterogeneous platforms; based on a current metadata vector corresponding to the current media asset metadata, screening out a group of historical metadata vectors of which the similarity meets a first threshold value from a vector database; executing accurate matching on the group of historical metadata vectors to obtain a target historical metadata vector; and performing conflict detection based on the metadata attribute to obtain a target detection result, and performing fusion processing on the target historical media asset metadata corresponding to the target historical metadata vector and the current media asset metadata based on the target detection result to obtain fused media asset metadata. The technical problem that the consistency of the cross-platform media asset metadata is difficult to guarantee in the related technology is solved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a data fusion method and apparatus, storage medium and electronic device. Background Technology

[0002] In the management of media asset metadata on existing video interactive platforms, especially in the management of media asset metadata on large video platforms or digital media libraries, in order to ensure data consistency between multiple platforms, web crawler tools are usually used to collect media asset metadata from a single or a few cooperating platforms, and the collected media asset metadata is then merged manually using a simple matching logic of media asset name + basic attributes.

[0003] However, for the same media resource (hereinafter referred to as media asset), the media asset information may differ on different video interactive platforms, such as broadcast time and actor list. Using the above-mentioned method of manually reviewing and correcting each item for data fusion makes it difficult to ensure the accuracy of the fused data, thus causing technical problems in related technologies that make it difficult to guarantee the consistency of cross-platform media asset metadata.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a data fusion method and apparatus, storage medium and electronic device to at least solve the technical problem of difficulty in ensuring the consistency of cross-platform media asset metadata in related technologies.

[0006] According to one aspect of the embodiments of this application, a data fusion method is provided, comprising: collecting historical media asset metadata of a target media asset within a historical period from multiple heterogeneous platforms in response to access to a current media asset platform; selecting a set of historical metadata vectors whose similarity satisfies a first threshold from a vector database based on the current metadata vector corresponding to the current media asset metadata, wherein the current media asset metadata is data on the current media asset platform, and the vector database contains historical metadata vectors corresponding to the historical media asset metadata; performing precise matching on a set of historical metadata vectors to obtain a target historical metadata vector; performing conflict detection based on metadata attributes to obtain a target detection result, and performing fusion processing on the target historical media asset metadata corresponding to the target historical metadata vector and the current media asset metadata based on the target detection result to obtain fused media asset metadata.

[0007] Optionally, the above-mentioned method of selecting a set of historical metadata vectors with similarity satisfying a first threshold from the vector database based on the current metadata vector corresponding to the current media asset metadata includes: preprocessing the historical media asset metadata to obtain standard historical media asset metadata with a unified format; obtaining the historical metadata vector corresponding to the standard historical media asset metadata; and selecting a set of historical metadata vectors that satisfy the first threshold based on the cosine similarity between the current metadata vector and the historical metadata vector.

[0008] Optionally, the above-mentioned precise matching of a set of historical metadata vectors to obtain the target historical metadata vector includes: sequentially obtaining each historical metadata vector from the set of historical metadata vectors as the current historical metadata vector; extracting the first set of attribute information of the current historical media asset metadata corresponding to the current historical metadata vector; extracting the second set of attribute information of the current media asset metadata, wherein the attribute types of the second set of attribute information are the same as those of the first set of attribute information; sequentially determining the cosine similarity between the attribute information of the same type in the first set of attribute information and the second set of attribute information to obtain a set of similarities; based on the target weight corresponding to each type of attribute information, performing a weighted summation of the set of similarities to determine the comprehensive similarity between the current historical metadata vector and the current metadata vector; and determining the media asset represented by the current historical media asset metadata and the media asset represented by the current media asset metadata as the same media asset when the comprehensive similarity is greater than or equal to a second threshold.

[0009] Optionally, before performing a weighted summation of a set of similarities based on the target weights corresponding to each type of attribute information, the above method further includes: obtaining multiple initial weights for multiple attribute information based on the media asset type of the target media asset, wherein both the first set of attribute information and the second set of attribute information include multiple attribute information; and dynamically adjusting the multiple initial weights based on historical matching accuracy to obtain the target weights.

[0010] Optionally, the above method further includes: in the process of dynamically adjusting multiple initial weights based on historical matching accuracy, in response to ambiguity in one of the multiple attribute information, increasing the weight of one of the attribute information.

[0011] Optionally, the above-mentioned conflict detection based on metadata attributes to obtain target detection results includes: obtaining a first conflict detection result when the metadata attribute is time information and the current publication time of the current media asset metadata is inconsistent with the target historical publication time of the target historical media asset metadata corresponding to the target historical metadata vector; obtaining a second conflict detection result when the metadata attribute is subject information and the current subject list of the current media asset metadata is inconsistent with the target historical subject list of the target historical media asset metadata; and obtaining a third conflict detection result when the metadata attribute is text information and the current text description information of the current media asset metadata is inconsistent with the target historical text description information of the target historical media asset metadata, wherein the target detection result includes at least one of the first conflict detection result, the second conflict detection result, and the third conflict detection result.

[0012] Optionally, the above method further includes at least one of the following: when the target detection result only includes the first conflict detection result, the current subject list is consistent with the target historical subject list, and the current text description information is consistent with the target historical text description information, comparing the current publication time and the target historical publication time to obtain a target comparison result, wherein the target comparison result indicates that the current publication time is later than the target historical publication time; based on the target comparison result, replacing the target historical publication time in the target historical media asset metadata with the current publication time to obtain the fused media asset metadata; when the target detection result only includes the second conflict detection result, the current publication time is consistent with the target historical publication time ... publication time, comparing the current publication time and the target historical publication time to obtain a target comparison result, comparing the current publication time and the target historical publication time to obtain a target comparison result, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time, comparing the current publication time and the target historical publication time If the target historical text description information is consistent, the union of the current subject list and the target historical subject list is taken to obtain the merged subject list; the target historical subject list in the target historical media asset metadata is replaced with the merged subject list to obtain the fused media asset metadata; if the target detection result only includes the third conflict detection result, the current release time and the target historical release time are consistent, and the current subject list and the target historical subject list are consistent, the current media asset metadata and the target historical media asset metadata are fused based on the text vector similarity between the current text vector corresponding to the current text description information and the target historical text vector corresponding to the target historical text description information to obtain the fused media asset metadata.

[0013] Optionally, the above-mentioned fusion processing of current media asset metadata and target historical media asset metadata based on the text vector similarity between the current text vector corresponding to the current text description information and the target historical text vector corresponding to the target historical text description information includes: when the text vector similarity is greater than or equal to a third threshold and the current text description information contains historical text description information, updating the historical text description information in the target historical media asset metadata to the current text description information to obtain fused media asset metadata; or when the text vector similarity is greater than or equal to a third threshold and the current text description information contains historical text description information with partial overlap, merging the historical text description information and the current text description information to obtain fused media asset metadata.

[0014] Optionally, the above method further includes: when the target detection result includes the first conflict detection result and the second conflict detection result, and the current text description information is consistent with the target historical text description information, comparing the current publication time and the target historical publication time to obtain a target comparison result; based on the target comparison result, replacing the target historical publication time in the target historical media asset metadata with the current publication time, and replacing the target historical subject list in the target historical media asset metadata with a merged subject list to obtain fused media asset metadata, wherein the merged subject list is obtained by taking the union of the current subject list and the target historical subject list.

[0015] Optionally, after fusing the target's historical metadata vector with the current media metadata based on the target detection results, the method further includes: synchronizing the fused media metadata to the vector database and updating the data index relationship in the vector database.

[0016] According to another aspect of the embodiments of this application, a data fusion apparatus is also provided, comprising: a collection unit, configured to collect historical media asset metadata of a target media asset within a historical period from multiple heterogeneous platforms in response to access of a current media asset platform; a filtering unit, configured to filter a set of historical metadata vectors whose similarity satisfies a first threshold from a vector database based on a current metadata vector corresponding to the current media asset metadata, wherein the current media asset metadata is data on the current media asset platform, and the vector database contains historical metadata vectors corresponding to the historical media asset metadata; a matching unit, configured to perform precise matching on a set of historical metadata vectors to obtain a target historical metadata vector; and a first processing unit, configured to perform conflict detection based on metadata attributes to obtain a target detection result, and perform fusion processing on the target historical media asset metadata corresponding to the target historical metadata vector and the current media asset metadata based on the target detection result to obtain fused media asset metadata.

[0017] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program for executing the above-described data fusion method when the electronic device is run.

[0018] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0019] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the data fusion method described above through the computer program.

[0020] By employing the embodiments provided in this application, after access to the current media asset platform, a set of historical metadata vectors closely related to the current metadata vector of the current media asset is selected from massive historical media asset metadata through vector similarity calculation. Combined with precise matching of multi-dimensional metadata attributes and intelligent conflict decision-making, data inconsistency issues are automatically handled, significantly reducing the intervention of manual review. Simultaneously, it effectively solves the limitations of traditional matching methods caused by similar media asset names, abbreviations, or translation differences. This achieves an automated process for data collection and fusion, enhances the adaptability to heterogeneous data, and ensures the consistency of media asset data across platforms. Attached Figure Description

[0021] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0022] Figure 1 This is a schematic diagram illustrating an application scenario of an optional data fusion method according to an embodiment of this application;

[0023] Figure 2 This is a flowchart of an optional data fusion method according to an embodiment of this application;

[0024] Figure 3 This is an overall architecture diagram of an optional data fusion method according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of an optional multi-dimensional matching process according to an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of an optional data conflict decision according to an embodiment of this application;

[0027] Figure 6This is an optional automatic update closed-loop timing diagram according to an embodiment of this application;

[0028] Figure 7 This is a schematic diagram of an optional data fusion apparatus according to an embodiment of this application;

[0029] Figure 8 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] The technical solutions in this application will comply with legal regulations during implementation. When performing operations according to the technical solutions in the embodiments, the data used will not involve user privacy, ensuring that the operation process is compliant and legal while guaranteeing data security. In addition, when the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of related data must comply with the relevant regulations and standards of the relevant countries or regions.

[0033] According to one aspect of the embodiments of this application, a data fusion method is provided. As an optional implementation, the above-described data fusion method can be applied, but is not limited to, to applications such as... Figure 1 The application scenarios shown are as follows. In, for example... Figure 1In the application scenario shown, the target terminal 102 can communicate with the server 106 via network 104, but is not limited to this. The server 106 can perform operations on the database 108, such as write or read data operations. The target terminal 102 may include, but is not limited to, a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen may be used to display video images corresponding to the target media assets using the technical solution of this application on the target terminal 102. The processor may be used to respond to the human-computer interaction operations, execute corresponding operations, or generate corresponding instructions and send the generated instructions to the server 106. The memory is used to store relevant processing data, such as historical media asset metadata, current metadata vector, and target historical metadata vector.

[0034] Optionally, in this embodiment, the target terminal can be a terminal configured with a target client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The target client may be a video client, instant messaging client, browser client, educational client, etc. The network may include, but is not limited to, wired network and wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that enable wireless communication. The server may be a single server, a server cluster composed of multiple servers, or a cloud server.

[0035] The technical solution of this application belongs to the intersection of data processing and intelligent media asset management. Specifically, it involves the technology of collecting, integrating and maintaining Internet media asset metadata. It is applicable to scenarios such as audio and video platforms and digital media libraries that require integrated management of media asset information from multiple sources. It is especially suitable for the intelligent data production and operation system of various comprehensive video platforms, and can provide accurate data support for core businesses such as media asset recommendation, copyright management, and content operation.

[0036] To address the aforementioned issue of consistency in cross-platform media asset metadata, this application proposes a data fusion method. Figure 2 This is a flowchart of a data fusion method according to an embodiment of this application, which includes the following steps S202 to S208.

[0037] It should be noted that the data fusion method shown in steps S202 to S208 can be, but is not limited to, executed by an electronic device. The electronic device can be, but is not limited to, [the following]. Figure 1 The target terminal or server shown.

[0038] Step S202: In response to the access of the current media asset platform, collect historical media asset metadata of the target media asset within the historical period from multiple heterogeneous platforms;

[0039] Step S204: Based on the current metadata vector corresponding to the current media asset metadata, select a set of historical metadata vectors whose similarity meets the first threshold from the vector database. Here, the current media asset metadata is the data on the current media asset platform, and the vector database contains historical metadata vectors corresponding to the historical media asset metadata.

[0040] Step S206: Perform precise matching on a set of historical metadata vectors to obtain the target historical metadata vector;

[0041] Step S208: Perform conflict detection based on metadata attributes to obtain target detection results, and perform fusion processing on the target historical metadata vector corresponding to the target historical metadata and the current media metadata to obtain fused media metadata.

[0042] Before explaining the technical solutions in this embodiment, the technical terms involved in this application will be briefly introduced.

[0043] Media asset metadata refers to the attribute information describing media resources (such as film, television, variety shows, music, etc.), including the title of the work, director, actors, production time, duration, type, plot summary, etc. It is key data for media resource management and recommendation systems.

[0044] Heterogeneous platforms: These refer to data platforms on the Internet that come from different sources and have different formats, such as video websites, music streaming media, and news websites. Each platform may have different data structures and representation methods.

[0045] Vector database: A database for storing and retrieving vector data, such as FAISS, is used to quickly find the record most similar to the target vector. It is commonly used in recommendation systems, search engines and other scenarios.

[0046] Multi-dimensional matching: This refers to the process of media asset metadata integration, which not only relies on the work name for matching, but also considers multiple attributes such as director, actors, duration, and genre, in order to improve the accuracy and comprehensiveness of the matching.

[0047] Intelligent conflict decision-making: refers to the system automatically detecting and processing inconsistencies or conflicts in media asset metadata, such as differences in broadcast time and actor list, and automatically selecting the optimal data or marking items that require manual review through logical algorithms and rule bases.

[0048] To facilitate understanding, let's first combine... Figure 3 The overall architecture diagram is shown, and the above data fusion method is briefly described using video files as the target media asset.

[0049] like Figure 3 As shown in the diagram, the overall architecture for implementing the data fusion method in this application may include, but is not limited to, a data acquisition and adaptation layer, a vector generation and storage layer, a multi-dimensional matching layer, a conflict decision layer, a data storage layer, and an update and iteration layer. Each layer is briefly described below.

[0050] The data acquisition and adaptation layer is primarily responsible for collecting media asset metadata from multiple data sources. This layer includes multiple data collectors that automatically crawl data from different platforms via API interfaces or web crawlers. The collectors can not only recognize and adapt to different data formats (such as JSON, XML, and HTML), but also automatically convert them into a unified metadata standard format for easier processing in subsequent layers. Furthermore, this layer incorporates anti-crawler strategies to ensure the stability and efficiency of data acquisition.

[0051] Vector Generation and Storage Layer: After the collected data enters this layer, the system uses a pre-trained natural language processing model (such as BERT) to vectorize the unstructured text data, transforming texts such as plot summaries and character introductions into dense vectors, achieving a unified representation of heterogeneous data. The vector data is then stored in a vector database (such as a FAISS-based vector database), supporting fast vector similarity retrieval and providing a foundation for subsequent media asset matching.

[0052] Multi-dimensional matching layer: Responsible for intelligent matching of media asset metadata. Specifically, it first performs a rapid coarse screening using a vector retrieval engine to find candidate media assets with high similarity to the newly collected data. Then, the weight calculation module dynamically adjusts the weights of each attribute based on information such as media asset type and historical matching accuracy. The similarity calculation module then performs a comprehensive similarity calculation on the candidate media assets based on these weights. When the similarity exceeds a certain threshold, the system determines that the two are the same media asset and enters the data fusion stage.

[0053] Conflict Decision Layer: When conflicts are encountered during data fusion, the conflict detector first identifies the conflict type and assesses its credibility. Based on the assessment results and a pre-set decision rule base, it automatically selects the most accurate data or triggers a manual review process. The conflict decision process for textual description information will be described in detail below with specific examples.

[0054] Data storage layer: Stores the final converged media asset metadata, including all structured and unstructured information, as well as processed vector data. The data is organized in a unified format for easy access and analysis by business systems.

[0055] Update and Iteration Layer: Located at the top of the architecture diagram, this layer is responsible for automating the data update and fusion process without manual intervention. It also includes a feedback learning loop that continuously optimizes matching rules, model parameters, and decision logic based on historical fusion results, enabling the system to self-iterate and upgrade.

[0056] These layers are closely interconnected, forming a closed-loop data fusion and update process. From data acquisition to vector generation, then to multi-dimensional matching and conflict decision-making, and finally to data storage and iterative updates, each layer has a clearly defined function and works collaboratively to form the core architecture of the media asset metadata automatic fusion system. Through this overall architecture, the system can efficiently and intelligently process multi-source heterogeneous data, improve the fusion quality and automation level of media asset metadata, and provide solid technical support for the intelligent data production platform of video platforms.

[0057] In this embodiment, it is assumed that the platform used for data fusion processing of multiple media asset platforms is an intelligent data generation platform. When this intelligent platform connects to a new video platform (such as platform C), the system will automatically start the data collection process. For example, through a pre-configured general crawler adapter, it will crawl metadata related to the target media asset from multiple platforms (such as platform A and platform B) that have been connected in the historical period over the past 5 years. For example, for a newly released movie "Campus Life 2024", the system will crawl information such as its name, director, main actors, and plot summary from various platforms to form an original metadata set.

[0058] The collected historical media metadata is preprocessed to extract its core attributes (such as work title, director, and lead actors), and its unstructured text information (plot summary) is transformed into dense vectors using a pre-trained BERT model, resulting in a set of historical metadata vectors. Then, by calculating the cosine similarity between the current metadata vector and each historical metadata vector in the set, a group of historical metadata vectors with a similarity higher than a preset first threshold is selected to form a "candidate media asset list".

[0059] Based on the candidate media asset list, the system will perform more detailed multi-dimensional attribute matching (i.e., precise matching). By comparing attributes such as director, actors, production time, genre, and duration, combined with a dynamic weight allocation algorithm, the system calculates the comprehensive similarity between each historical media asset and the target media asset. For example, for "Campus Life" and "Campus Life 2024," the system will determine whether they are the same media asset through weighted calculation based on dimensions such as director, lead actors, and plot synopsis vector. If the comprehensive similarity exceeds a second threshold, it is confirmed as the same media asset, and the target historical metadata vector is obtained.

[0060] Further comparison of media asset metadata between Platform C and historical platforms (Platforms A and B) automatically detects inconsistencies in metadata attributes, such as broadcast time and cast list, generating a "conflict detection report." For detected conflicts, the system will automatically select the most accurate data based on a preset decision rule base and data credibility assessment mechanism, or generate a manual review task when data differences are difficult to judge automatically. For example, regarding the difference in broadcast time between Platform C and Platform A for "Campus Life 2024," the system will assess the credibility of the two platforms and select the data from the platform with the more recent update or higher credibility as the final result. The merged media asset metadata will contain the most accurate and comprehensive attribute information provided by all platforms, forming merged media asset metadata.

[0061] In this embodiment, vector retrieval enhancement technology, combined with a multi-dimensional matching system, and through intelligent weight allocation and conflict data processing units, achieves efficient and accurate fusion of media asset metadata, improving the richness and accuracy of the data. For example, in film and television media assets, by dynamically adjusting the weights of dimensions such as "director + lead actor + video fingerprint vector," the system can maintain a high matching rate even when dealing with similar names, abbreviations, or translation differences.

[0062] Meanwhile, the conflict decision-making mechanism combines the credibility of data sources and update time to automatically select the optimal data, reducing the cost of manual review and making data updates an automated and controllable process. This significantly improves the efficiency of data maintenance and updates, providing strong data support for the core businesses of video platforms such as media asset recommendation, copyright management, and content operation.

[0063] Through the embodiments provided in this application, after accessing the current media asset platform, a set of historical metadata vectors closely related to the current metadata vector of the current media asset is selected from massive historical media asset metadata through vector similarity calculation. Combined with precise matching of multi-dimensional metadata attributes and intelligent conflict decision-making, data inconsistency issues are automatically handled, significantly reducing the intervention of manual review. This effectively solves the limitations of traditional matching methods caused by similar media asset names, abbreviations, or translation differences. It achieves an automated process for data collection and data fusion, enhances the adaptability to heterogeneous data, and ensures the consistency of media asset data across platforms.

[0064] As an optional example, the above method, based on the current metadata vector corresponding to the current media asset metadata, selects a set of historical metadata vectors from the vector database whose similarity meets a first threshold, including:

[0065] By preprocessing the historical media asset metadata, a standardized historical media asset metadata with a unified format is obtained;

[0066] Obtain the historical metadata vector corresponding to the standard historical media asset metadata;

[0067] Based on the cosine similarity between the current metadata vector and the historical metadata vector, a set of historical metadata vectors that meet the first threshold is selected.

[0068] In the process of automatic media asset metadata fusion, the first step is to preprocess the historical media asset metadata collected from multiple heterogeneous platforms to standardize the data format and ensure compatibility and consistency in subsequent data processing. Preprocessing includes steps such as data cleaning, attribute extraction, and format conversion.

[0069] For example, the collected media asset metadata for the film "Campus Life 2024" might include attributes such as director, actors, and plot synopsis, but the data structure might be in different formats such as XML, JSON, or HTML. The system will automatically identify and convert these data formats using a general-purpose crawler adapter, uniformly converting all metadata into the data specification format of the aforementioned intelligent data production platform, facilitating subsequent vectorization and matching operations. During preprocessing, the system will also perform basic data cleaning, such as removing redundant information and standardizing date formats, to ensure data accuracy and usability.

[0070] After standardizing the format of historical media asset metadata, the system further processes unstructured text information, transforming it into structured vector data to achieve efficient retrieval and accurate matching. For example, for the plot summary text of the film "Campus Life," the system uses a pre-trained BERT model to convert it into a 768-dimensional dense vector. This vector can capture the core semantic features of the text, facilitating subsequent vector retrieval.

[0071] In addition, the system will also convert feature identifiers such as video frame fingerprints and audio fingerprints to ensure that all types of historical media data can be encoded into vectors of a unified dimension, thereby building a comprehensive and efficient vector database.

[0072] Once data from the new platform C is integrated, a vector retrieval process will be executed. This involves calculating the cosine similarity between the current media asset metadata vector (e.g., the text vector of "Campus Life 2024") and historical metadata vectors to locate highly relevant historical media asset metadata. Cosine similarity is an indicator that measures the directional consistency between two vectors, ranging from -1 to 1, with values ​​closer to 1 indicating higher similarity. The system sets a first threshold, meaning that only historical metadata vectors with similarity reaching or exceeding this value will be filtered out to form a candidate media asset list. For example, for the current metadata vector V2 of "Campus Life 2024," the system searches the FAISS vector database to find the top 10 historical metadata vectors with the highest similarity; the historical media asset metadata associated with these vectors constitutes the candidate media asset list.

[0073] This embodiment details the core process of the vector retrieval enhancement module, namely, how to filter historical records highly relevant to the current media asset metadata from the vector database through preprocessing, vectorization, and similarity calculation. First, the system preprocesses the historical media asset metadata, unifying the data format and structure to ensure accurate parsing and use of all metadata. Then, unstructured text and feature identification information are converted into vectors using the BERT model and feature mapping algorithm, constructing a FAISS database containing millions of vectors, supporting millisecond-level similarity retrieval. Finally, when new media asset metadata is accessed, the system quickly filters a set of historical metadata vectors whose similarity meets a first threshold by calculating the cosine similarity between its vector and the vectors in the database, forming a candidate media asset list. This provides precise positioning and scope for subsequent accurate attribute matching and data fusion.

[0074] The entire process fully leverages the efficiency of vector retrieval and the accuracy of multi-dimensional matching, effectively solving core challenges in data fusion, such as the variability of names and the heterogeneity of data. It enhances the automation and intelligence of data fusion, providing data support for media asset management, copyright transactions, and content operation on video platforms.

[0075] As another alternative example, the above performs exact matching on a set of historical metadata vectors to obtain a target historical metadata vector, including:

[0076] Each historical metadata vector is sequentially retrieved from a set of historical metadata vectors and used as the current historical metadata vector;

[0077] Extract the first set of attribute information of the current historical media asset metadata corresponding to the current historical metadata vector;

[0078] Extract the second set of attribute information from the current media asset metadata, wherein the attribute type of the second set of attribute information is the same as that of the first set of attribute information;

[0079] The cosine similarity between the same type of attribute information in the first group of attribute information and the second group of attribute information is determined in turn to obtain a set of similarity;

[0080] Based on the target weight corresponding to the attribute information of each type, a set of similarities is weighted and summed to determine the comprehensive similarity between the current historical metadata vector and the current metadata vector;

[0081] If the overall similarity is greater than or equal to the second threshold, the media asset represented by the current historical media asset metadata and the media asset represented by the current media asset metadata are determined to be the same media asset.

[0082] This embodiment proposes, for example Figure 4The multi-dimensional matching process shown aims to achieve accurate matching. The specific process is as follows:

[0083] S11, retrieve historical media asset metadata;

[0084] The process begins with the input of raw metadata, which is the historical media asset metadata within a historical period obtained by the acquisition and adaptation layer from multiple platforms through web crawlers or API interfaces. This includes structured data (such as work name, year, director, actors, etc.) and unstructured text (such as plot summary, character introduction).

[0085] S12, Extract attribute information from historical media asset metadata;

[0086] First, Natural Language Processing (NLP) technology is used to parse the acquired historical media metadata and automatically extract key attributes, including basic attributes (such as name, year, duration, etc.), creative team attributes (such as director, lead actor, etc.), content feature attributes (such as plot summary keywords, theme tags, etc.), and feature identifiers (such as unique codes, video frame fingerprints, and audio fingerprints).

[0087] S13, Based on the extracted attribute information, generate a historical metadata vector;

[0088] For unstructured text data, the system calls a pre-trained language model (such as BERT) for semantic encoding, transforming the text data into vector data to achieve a quantitative representation of the text content, facilitating subsequent similarity calculations. Simultaneously, feature identifiers such as video frame fingerprints and audio fingerprints are also converted into vectors of a unified dimension to meet the storage and retrieval requirements of the vector database.

[0089] S14, similarity calculation is used to perform a rough screening of vectors;

[0090] Vector retrieval is performed using the FAISS vector database. By calculating the cosine similarity between new data vectors and existing vectors, the top N candidate media asset data are selected as the initial range for subsequent precise matching. This step significantly reduces the scale of the matching operation and improves processing efficiency.

[0091] S15, Dynamic Weight Allocation;

[0092] Based on the vector retrieval results, the weights of multiple matching dimensions are dynamically adjusted according to media asset type and historical matching accuracy. For example, the weights of director, lead actor, and video fingerprint vectors in film and television media assets may be higher to ensure matching accuracy within specific media asset types.

[0093] S16, Similarity calculation;

[0094] By combining the weight coefficients after dynamic weight allocation, a comprehensive similarity calculation is performed on the multi-dimensional attributes of candidate media asset metadata, including name, year, director, actors, vector similarity, etc., to comprehensively evaluate the matching degree.

[0095] S17, Matching is determined based on the calculated comprehensive similarity score;

[0096] The following three methods of determination are permissible, but not limited to:

[0097] (1) Similarity score ≥ 90%: The system automatically determines that the new media asset metadata and the current historical media asset metadata are metadata of the same media asset, without the need for manual review;

[0098] (2) Similarity scores between 60% and 90%: The system determines that a match may exist, but manual review and confirmation are required to avoid false matches;

[0099] (3) Similarity score <60%: The system determines that the new data does not match the existing data and may belong to different media assets. In this case, there is no need to perform a fusion operation.

[0100] S18, Data Fusion;

[0101] For current historical media asset metadata that is determined to be a match, the system will perform data fusion operations, including but not limited to updating time, supplementing subject information, etc., and combine the conflict handling strategies in the following embodiments to merge the most accurate and comprehensive metadata information to form fused media asset metadata.

[0102] S19, Updates and Iterations.

[0103] The merged media asset metadata will be synchronized to the database of the intelligent data production platform, and the index relationships of the vector database will be updated to reflect the latest media asset status. Specifically, this includes the system's ability to continuously analyze historical fusion results, optimize weight parameters and decision rules, and achieve self-iterative upgrades.

[0104] The multi-dimensional matching flowchart demonstrates how the system analyzes and compares media asset metadata from multiple perspectives, effectively solving matching challenges such as similar names, abbreviations, and translation differences, improving the accuracy and efficiency of data fusion, and providing high-quality media asset metadata support for video platforms.

[0105] As an optional implementation, before weighted summation of a set of similarities based on the target weights corresponding to the attribute information of each type, the above method also includes:

[0106] Based on the media asset type of the target media asset, multiple initial weights of multiple attribute information are obtained, wherein the first set of attribute information and the second set of attribute information both include multiple attribute information;

[0107] During the integration of media asset metadata, the system sets initial weights for different attribute information based on the type of the target media asset (e.g., Campus Life 2024) (film, variety show, music, etc.). This is because different types of media assets have varying degrees of importance for various attributes (such as director, actors, duration, song style, etc.) in their metadata.

[0108] For example, for a film, information about the director and lead actors is often more crucial than the year or genre, as they better represent the characteristics and quality of the work. Therefore, the system will preset an initial set of weights for film and television media assets, such as: director 30%, lead actors 25%, runtime 10%, genre 10%, etc., to ensure that these core attributes receive more attention and consideration during the integration process.

[0109] The system periodically analyzes the fusion records of historical media asset metadata, calculating the accuracy of each set of attribute information in its media asset matching process. The accuracy rate reflects the effectiveness of that attribute in identifying media asset similarity. Through machine learning algorithms, the system can dynamically adjust the initial weights based on changes in accuracy to optimize subsequent media asset fusion effects. For example, if historical records show that the accuracy rate of plot summary keywords reaches 95% for film and television media assets, while the accuracy rate of song style attributes in music media assets is only 75%, the system might increase the weight of plot summary keywords for film and television media assets to 35%, while decreasing the weight of song style attributes for music media assets to 5%, ensuring a more reasonable weight allocation that better reflects the actual needs of media asset identification.

[0110] This dynamic weight adjustment mechanism is based on the following example: In the process of integrating film and television media assets, the system analyzed nearly 20,000 integration records from the past year and found that the combination of "director + lead actor + video fingerprint vector" had an accuracy rate as high as 98%, far exceeding other attribute combinations. Therefore, the system automatically increased the weights of these attributes from the original 30%, 25%, and 15% to 35%, 30%, and 20%, respectively, while correspondingly decreasing the weights of attributes such as name and year, further improving the overall matching accuracy of film and television media assets to over 99%. Simultaneously, the system also performed similar weight adjustments and optimizations for the combination of "guest list + broadcasting platform + genre tag vector" for variety show media assets, ensuring the accuracy and efficiency of identifying various types of media assets.

[0111] This embodiment aims to continuously optimize the intelligent decision-making level of media asset metadata fusion through a dynamic weight adjustment mechanism. First, the system presets an initial set of weights based on the type of media asset to guide the relative importance of different attribute information during the fusion process. Then, by monitoring and analyzing historical media asset fusion results, the system evaluates the actual effect of each set of attribute information in the matching process, i.e., the historical matching accuracy. Based on this feedback, the system dynamically adjusts the weights, so that in subsequent fusions, attribute information that performs better receives higher weights in the calculation, while its weight is reduced, thus making the overall matching strategy more aligned with actual application scenarios and improving the accuracy and efficiency of media asset identification.

[0112] The aforementioned self-learning and optimization capabilities not only enhance the system's adaptability and robustness but also reduce the number of manual interventions, providing stable and efficient data support services for the video platform's intelligent data production platform.

[0113] As an optional example, the above method also includes:

[0114] In the process of dynamically adjusting multiple initial weights based on historical matching accuracy, in response to ambiguity in one of the multiple attribute information, the weight of one of the attribute information is increased.

[0115] During the matching process, the system utilizes NLP (Natural Language Processing) technology and string matching algorithms to perform semantic analysis and difference detection on the attribute information in the media asset metadata. For example, the name of the film and television work "Campus Life 2024" may be simplified to "Campus Life 1" on different acquisition platforms or displayed as different Chinese translations due to translation differences. In this case, the "media asset name" attribute information may be ambiguous.

[0116] After identifying ambiguous attributes (such as media asset names), the system further analyzes their attribute types. In this embodiment, "media asset name" is basic attribute information, which may cause matching difficulties between data from different platforms.

[0117] The system reviews historical media asset fusion records to assess the impact of weight allocation on matching performance when encountering ambiguities in the same attributes. For example, historical records show that when encountering ambiguities in "media asset name," increasing the weight of both "video fingerprint vector" and "keyword" attributes significantly improves the accuracy of media asset recognition. This is because even with differences in name, the system can still capture the core features of the work through deep comparison of video fingerprints and keywords, thus making accurate matching decisions.

[0118] Based on this analysis, the system increased the weight of the "video fingerprint vector" from 15% to 25%, while adjusting the weight of the "keyword" attribute from 10% to 15%. This weight adjustment strategy significantly improved the accuracy of media asset matching under conditions of ambiguous names and reduced the number of times manual intervention was required.

[0119] After weighting adjustments, the system will rely more on high-credibility attributes such as "video fingerprint vector" and "keywords" for similarity calculation in subsequent media asset fusion processes. Even if there are differences in "media asset names", the same media asset can be identified through in-depth comparison of other attributes, which improves the intelligence and robustness of fusion.

[0120] To address the ambiguity of attribute information in media asset metadata fusion, a strategy of increasing the weight of ambiguous attributes is proposed, effectively improving the system's matching accuracy and robustness. When common attributes such as media asset names show naming differences or inconsistent language translations, the system automatically enhances the weight of high-confidence attributes such as video fingerprint vectors and keywords, ensuring that even when the main attribute information does not completely match, the same work can be identified through deep comparison of secondary attributes. This mechanism significantly reduces matching errors caused by ambiguity, while also reducing the frequency and complexity of manual review.

[0121] As an optional example, the above-mentioned conflict detection based on metadata attributes yields target detection results, including:

[0122] When the metadata attribute is time information and the current publication time of the current media asset metadata is inconsistent with the target historical publication time of the target historical media asset metadata corresponding to the target historical metadata vector, the first conflict detection result is obtained.

[0123] When the metadata attribute is subject information and the current subject list of the current media asset metadata is inconsistent with the target historical subject list of the target historical media asset metadata, the second conflict detection result is obtained.

[0124] When the metadata attribute is text information and the current text description information of the current media asset metadata is inconsistent with the target historical text description information of the target historical media asset metadata, a third conflict detection result is obtained, wherein the target detection result includes at least one of the first conflict detection result, the second conflict detection result and the third conflict detection result.

[0125] This embodiment further describes the conflict detection mechanism in the media asset metadata fusion process. Through detailed attribute-level conflict identification, it ensures the accuracy and consistency of the fused data. The implementation process of the above conflict detection mechanism is described in detail below using three metadata attributes with different properties as examples.

[0126] (1) Time information conflict detection;

[0127] When the system is integrating metadata, if it detects a discrepancy between the release time of the current media asset metadata (data collected by the new platform) and the release time of the target historical media asset metadata (data already existing in the system), the system will perform the first conflict detection. For example, a film or television work may be displayed on the new platform C as having a release date of March 1, 2024, while the target historical metadata recorded on the intelligent data production platform shows a release date of March 2, 2024. By comparing the time information, the system identifies the mismatch between the two time points, thus generating the first conflict detection result and prompting a review and decision-making process for this time information.

[0128] (2) Detection of conflicts in main information;

[0129] The main information typically refers to the key creative personnel and production companies involved in the work. If the main information list in the current media asset metadata differs from the main information list in the target historical media asset metadata, the system will generate a second conflict detection result. For example, when the cast list of variety show D provided by the new platform C includes three guest actors not listed in the historical media asset metadata record, the system will compare the main information and find that the cast list is inconsistent, thus triggering a second conflict detection alarm, indicating that this difference may stem from new information supplementation or data errors, requiring further verification and decision-making.

[0130] (3) Text information conflict detection.

[0131] For unstructured data, such as plot summaries and artist biographies, if there are significant differences between the current media asset metadata and the target historical metadata, the system will identify and generate a third conflict detection result. For example, in a fusion operation, the system found that the plot description of a variety show in the target historical media asset metadata retrieved from the database was too brief, while the description provided by the new platform C was more detailed. Through text comparison, it was determined that there was a significant difference between the two descriptions, thus triggering the third conflict detection. This may be due to the omission of original information or the new platform providing an updated description. The target historical media asset metadata and the current media asset metadata are determined to be metadata describing the same media asset.

[0132] By implementing a refined conflict detection process, conflicts in time, subject, and text information during media asset metadata fusion were effectively identified and addressed, improving the accuracy and consistency of data integration. The system can capture differences in publication time, subject lists, and descriptive text in real time, avoiding fusion errors caused by data ambiguity. Especially for text information, vector similarity calculations enable precise detection even for minor differences. This mechanism reduces the burden of manual review and accelerates the data processing flow.

[0133] As an optional implementation, the above method also includes at least one of the following:

[0134] If the target detection result only includes the first conflict detection result, the current subject list is consistent with the target's historical subject list, and the current text description information is consistent with the target's historical text description information, then the target comparison result is obtained by comparing the current publication time and the target's historical publication time. The target comparison result indicates that the current publication time is later than the target's historical publication time. Based on the target comparison result, the target's historical publication time in the target's historical media asset metadata is replaced with the current publication time to obtain the fused media asset metadata.

[0135] If the target detection result only includes the second conflict detection result, the current release time is consistent with the target's historical release time, and the current text description information is consistent with the target's historical text description information, then the union of the current subject list and the target's historical subject list is taken to obtain the merged subject list; the target's historical subject list in the target's historical media asset metadata is replaced with the merged subject list to obtain the fused media asset metadata;

[0136] When the target detection results only include the third conflict detection results, the current release time is consistent with the target's historical release time, and the current subject list is consistent with the target's historical subject list, the current media asset metadata and the target's historical media asset metadata are fused based on the text vector similarity between the current text vector corresponding to the current text description information and the target's historical text vector corresponding to the target's historical text description information, to obtain fused media asset metadata.

[0137] In this embodiment, a media asset metadata fusion technology solution that adopts specific fusion strategies for different conflict types is further described, aiming to improve the efficiency and accuracy of data fusion through a refined processing flow.

[0138] Fusion strategy for time-related conflicts: When the system only receives the first conflict detection result, i.e., the current media asset metadata's publication time is later than the publication time in the target historical metadata, while the main information and text description information remain consistent, the system will perform a time information update operation. For example, for the TV series "Campus Life," the latest publication time recorded in the system is March 2, 2024, while the new platform C provides an updated record showing that a special edition of the TV series was released on March 3, 2024. In this case, the system determines that the time information has been updated and replaces the publication time in the historical data with the new, later publication time to reflect the latest situation.

[0139] Subject Information Fusion Strategy: If the detection results only show differences in subject information (second conflict detection result), and the publication time matches the text description information, the system will adopt a subject information merging strategy. For example, when merging metadata of a variety show, the system finds that the data of the new platform C includes several special guests not listed in the database. In this case, the system will not simply replace them, but will perform a union operation between the current subject list and the historical subject list, merging all subject information to form a more comprehensive "merged subject list." Finally, the subject list in the target historical data will be updated to this merged list, ensuring the integrity of the subject information.

[0140] Text Information Fusion Strategy: When the system detects significant differences in text description information (third conflict detection result), but the publication time and subject information remain consistent, the system will determine the text information fusion method based on the vector similarity calculation result. For example, the system finds that the synopsis of a documentary on new platform C differs significantly from existing records. The system calculates the text vector similarity between the two. If the similarity is high (e.g., >80%), it indicates that the core content described is the same, only the expression or details differ. In this case, the system can adopt more detailed descriptive information as the fused text description, improving the richness of metadata. If the similarity is low, it may involve different versions or different perspectives of description, requiring manual review to ensure the accuracy of the fused information.

[0141] When identifying conflicts in single metadata attributes during data fusion through a conflict detection mechanism, such as inconsistencies in time, subject, or text information, precise fusion strategies are adopted based on the type of conflict. These strategies include updating time information, merging subject information, and optimizing text information based on vector similarity, ensuring that each conflict is properly handled. This series of automated fusion operations not only significantly improves data processing efficiency and reduces the number of manual reviews, but also guarantees the accuracy and richness of the fused data through intelligent decision-making.

[0142] As an optional implementation, the above-mentioned method of fusing current media asset metadata and target historical media asset metadata based on the text vector similarity between the current text vector corresponding to the current text description information and the target historical text vector corresponding to the target historical text description information includes:

[0143] If the text vector similarity is greater than or equal to the third threshold and the current text description information contains historical text description information, the historical text description information in the target historical media asset metadata is updated to the current text description information to obtain the fused media asset metadata; or if the text vector similarity is greater than or equal to the third threshold and the current text description information contains some overlap with the historical text description information, the historical text description information and the current text description information are merged to obtain the fused media asset metadata.

[0144] In this embodiment, a fusion strategy based on vector similarity and information completion is proposed to address the textual information differences during the media asset metadata fusion process. This strategy aims to ensure the accuracy and completeness of textual descriptions through intelligent algorithms. The following is a detailed description and example analysis of the two core fusion strategies in the solution.

[0145] (1) Complete Information Supplement Strategy: When the system determines through vector similarity calculation that the current text description information is highly similar to the target historical text description information (similarity greater than or equal to the third threshold), and the current text description information contains all the content of the target historical text description information, the system will perform a complete information supplement operation. For example, the synopsis of movie E on the new platform C not only includes the historical synopsis in the target historical media asset metadata, but also additional information about the movie's special effects team, which is considered a supplementary description of the movie's information. The system will automatically recognize this inclusion relationship and update the historical text description information in the target historical media asset metadata to more detailed current text description information, so as to achieve seamless expansion and upgrading of information.

[0146] (2) Partial Information Merging Strategy: In another scenario, if the vector similarity between the current text description and the historical text description still meets the third threshold, but they do not completely encompass each other, but rather overlap in some descriptive content, the system adopts a partial information merging strategy. Assume that for a music album F, the new platform C provides an introduction to the artist's tour background, while the database focuses on the music creation process; both contribute to understanding the album's overall background. In this case, the system will not simply update or replace the information, but will choose to merge the intersection of the historical and current text descriptions while retaining their respective unique information, forming a richer and more comprehensive text description to reflect the fusion effect of multiple data perspectives.

[0147] In the context of media asset metadata fusion, an intelligent text information fusion mechanism solves the problem of inconsistencies in text description information, ensuring high quality and standards for data fusion. After identifying cases where the vector similarity between the current and historical text description information meets the standard, the system further determines whether they constitute a complete inclusion or partial intersection relationship. If the current description information contains all the content of the historical information, the system will directly use the current description information for updating; if the two have intersection but are not completely identical, the system will adopt a merging strategy to integrate the information from both, retaining their respective characteristics and supplements, thereby obtaining the fused media asset metadata. This fusion strategy fully considers the redundancy and complementarity of text information, avoiding information loss or errors that may be caused by simple replacement, while also taking into account the contribution of different data sources, improving the comprehensiveness and accuracy of the fused data.

[0148] As an optional implementation, the above method also includes:

[0149] If the target detection results include the first conflict detection results and the second conflict detection results, and the current text description information is consistent with the target's historical text description information, the target comparison results are obtained by comparing the current release time and the target's historical release time.

[0150] Based on the target comparison results, the target historical publication time in the target historical media asset metadata is replaced with the current publication time, and the target historical subject list in the target historical media asset metadata is replaced with the merged subject list to obtain the fused media asset metadata. The merged subject list is obtained by taking the union of the current subject list and the target historical subject list.

[0151] (1) Decision-making under conflict between time information and subject information: When the system detects that the release time of the current media asset metadata is inconsistent with the release time of the target historical media asset metadata (first conflict detection result), and at the same time finds that there is a difference between the current subject list and the target historical subject list (second conflict detection result), but the text description information remains consistent, the system will first compare the current release time and the target historical release time to obtain the target comparison result. For example, for movie F, the information provided by the new platform C shows that its release date is March 4, 2024, while the time recorded by the intelligent data production platform is March 3, 2024. At the same time, it detects that the film's subject list on the new platform includes several cameo actors who are not in the intelligent data production platform. In this case, the system will first determine which time point is closer to the present to determine the most accurate release time, and then update the release time in the target historical data according to this decision.

[0152] (2) Merging and Integration of Subject Information: The current subject list and the target historical subject list are merged to obtain a more comprehensive "merged subject list". After confirming the update release time, the subject list provided by the new platform C is combined with the original subject list of the intelligent data generation platform to ensure that all participants in the creation are included. This means that regardless of whether the list provided by the new platform C or the records of the intelligent data generation platform are valuable information, they will be integrated into the final metadata. This merging strategy avoids information loss due to the limitations of a single data source, making media asset metadata more accurate and complete.

[0153] To more clearly understand the above data conflict decision-making process, the following will combine... Figure 5 Further explanation is needed.

[0154] The data conflict decision-making process details the intelligent decision-making process of the media asset metadata automatic fusion system when handling multi-source data conflicts. Specifically, it includes:

[0155] S21, Initiate conflict decision-making;

[0156] The process begins with the system detecting conflicts that occur during data fusion. These conflicts may stem from differences in information between different data acquisition platforms, including but not limited to inconsistencies in data such as time, personnel, and text descriptions.

[0157] S22, detects conflict type;

[0158] The system first automatically identifies the specific type of conflict. Common categories include: "time-related conflicts" (such as differences in release date and broadcast time), "personnel-related conflicts" (such as incomplete or incorrect director, actor, and guest lists), and "text-related conflicts" (such as significant differences in plot synopsis and character introductions).

[0159] S23, Vector similarity calculation;

[0160] For text-based conflicts, the system invokes the vector retrieval enhancement module to calculate the vector similarity between the current text description and the target's historical text description. This calculation is based on a pre-trained language model (such as BERT), which converts the text content into vector representations, and then uses algorithms such as cosine similarity to calculate the semantic similarity between the two.

[0161] S24, Credibility Assessment;

[0162] For each type of conflict, the system evaluates the credibility of the data source. Evaluation factors include the platform's authority, the accuracy of historical data, and the data's update time. For example, in time-related conflicts, data from official video platforms is generally considered more credible, and platforms with higher historical data accuracy will receive higher credibility scores.

[0163] S25, Data Comparison and Decision Making;

[0164] Time-related conflicts: If the time data of the platform with the highest credibility score is similar to that of other platforms (score difference ≤ 5 points), the system will prioritize the data with the more recent update time as the fusion result.

[0165] Personnel conflicts: The personnel lists are merged by taking the union of sets, and vector similarity calculation is used to help determine the correlation between personnel identities, so as to ensure complete and accurate personnel information.

[0166] Text-related conflicts: When the similarity of text description information vectors is lower than a set threshold (e.g., 60%), the system generates a manual review task. Reviewers make judgments based on the provided difference comparison report (including vector similarity analysis results) to determine the final text description information.

[0167] S26, Decision complete.

[0168] After completing the conflict decision, the system synchronizes the updated media asset metadata to the fusion database and updates the data index relationships in the vector database to ensure data consistency and retrieval efficiency. Simultaneously, the results of manual review are fed back to the decision rule base for optimizing subsequent conflict decisions, achieving a closed-loop iterative decision-making process.

[0169] The above steps demonstrate how the system automatically identifies and handles conflicts in multi-source data. This not only reduces the workload of manual intervention but also improves the accuracy and completeness of media asset metadata, providing a solid technical guarantee for the intelligent data production platform of the video platform. Through the combination of intelligent decision-making and data update mechanisms, the system can effectively respond to various emergencies during data fusion, ensuring the timeliness and reliability of media asset information, thereby providing strong data support for content recommendation, copyright management, and other businesses.

[0170] As an optional example, after fusing the target's historical metadata vector with the current media metadata based on the target detection results, the above method further includes:

[0171] Synchronize the merged media asset metadata to the vector database and update the data index relationships in the vector database.

[0172] This embodiment describes the update steps after integrating media asset metadata, particularly regarding the synchronization of the vector database and the updating of index relationships, to ensure data consistency and retrieval efficiency. The following is an explanation of the update operations in the scheme.

[0173] Once the media asset metadata has been integrated, the system synchronizes this integrated data to the vector database. For example, after time information updates and subject information merging, the metadata for the film "Campus Life 2024" was automatically updated, including a more accurate release date and a complete list of participants. The system then synchronizes this updated information, including vector data generated based on the new text description information, to the FAISS vector database to reflect the latest media asset status.

[0174] Updating index relationships is crucial for efficient retrieval in vector databases. In a vector database, each media asset metadata is indexed to its corresponding vector for quick location and retrieval. When the merged data is synchronized to the vector database, the system automatically updates these index relationships to ensure that the vectors of the new data can be correctly associated and retrieved. Taking "Campus Life 2024" as an example, the updated vector data V' needs to establish an index relationship with the newly established merged metadata to ensure that subsequent queries can quickly locate the merged metadata record containing the most complete information.

[0175] To better understand the update and iteration process, the following will combine... Figure 6 The automatic update closed-loop timing diagram shown further describes it.

[0176] Step 1: Add Collection Rules for Platform C. The system administrator or the automatic configuration module adds the collection rules (such as API interface address, access token) for the new Platform C to the collection adaptation layer. These rules allow the system to identify platform X-specific data formats (such as XML) and configure corresponding parsing strategies, ensuring the automation and accuracy of data collection. No new crawler code needs to be written manually, reducing platform integration costs and time consumption.

[0177] Step 2: Initiate API Request (Automatic Collection) The system automatically accesses the API interface of Platform X based on the newly added collection rules to collect media asset metadata within a recent period, including structured data (such as name, year, actors, etc.) and unstructured text data (such as plot summary, character introduction, etc.).

[0178] Step 3: Data Cleaning and Transformation. The collected data is cleaned by the data cleaning module to remove irrelevant or erroneous information. Then, the general crawler adapter converts the heterogeneous data format (such as XML) into a unified metadata standard format for easy subsequent processing.

[0179] Step 4: Sending Text Data (Generating Vectors) The cleaned and transformed text data is sent to a vectorization model (such as BERT) to generate corresponding vector representations. These vector data are then stored in a vector database (FAISS) to support fast vector similarity retrieval.

[0180] Step 5: Add new vectors (incremental update). The vector data generated in step 4 will be updated to the vector database in real time to maintain the timeliness of the database and support immediate fusion needs.

[0181] Step 6: Query similar vectors (FAISS retrieval) Compare the newly acquired current metadata vector with the historical metadata vectors in the vector database to find the records with the highest similarity. These records constitute the initial matching candidates.

[0182] Step 7: Sending Matching Results and Conflict Resolution. Based on the vector retrieval results, the system identifies successfully matched media asset data and also detects data conflicts that need to be addressed. These conflicting data, along with the matching results, are sent to the conflict decision layer.

[0183] Step 8: Execution Decision Process Conflict Decision-Making The decision-making layer automatically or semi-automatically determines the best data source to resolve data conflicts based on factors such as conflict type, the authority and accuracy of the data source, and the data update time. For conflicts that are difficult to resolve automatically, a manual review task is generated for reviewers to make the judgment.

[0184] Step 9: After confirming that the fused data written into the decision is correct, it is written into the fusion database, becoming the official media asset metadata in the system. Simultaneously, the system updates the data version management information, recording the source, time, modified content, and vector data changes of this update, providing a basis for subsequent data analysis and system optimization.

[0185] Step 10: Monitoring Logs and Analysis Results. By monitoring system logs, collect various indicators (such as the efficiency of vector retrieval, matching accuracy, conflict handling results, etc.) during the data fusion and update process, and conduct regular analysis.

[0186] Step 11: Optimize data collection / matching / decision rules. Based on the analysis results, automatically adjust the data collection rules, matching algorithm parameters (such as vector similarity threshold), and decision logic, and continuously iterate to optimize system performance and improve the overall quality and efficiency of data fusion.

[0187] Through this series of automated, closed-loop updates, the system can automatically and efficiently collect, match, integrate, and update media asset metadata when a new platform is added or when existing platform data is updated. At the same time, through continuous optimization, it ensures the accuracy and real-time performance of data integration.

[0188] As an alternative, the aforementioned data fusion method can be applied to scenarios where information consistency needs to be addressed across different platforms. In this scenario, the fusion processing of media asset metadata is the core process. Traditional technical solutions mainly employ a combination of single-platform data collection and manual fusion. In the data collection stage, most platforms rely on customized web crawlers to collect media asset metadata from a single or a few partner platforms, resulting in a narrow scope of collection and a lack of unified adaptation capabilities for heterogeneous data across multiple platforms (such as different formats of plot summaries and differentiated actor information annotations).

[0189] In the integration process, the system mainly relies on a simple matching logic of "media asset name + basic attributes". For example, it compares "work name + year" to determine whether they are the same media asset. However, when faced with scenarios such as similar names, abbreviations, or differences in translation, the matching accuracy is generally less than 70%. For data conflicts after integration (such as the same film and television show inconsistent broadcast times on different platforms), the system mainly relies on manual review and correction of each item, which is inefficient and easily affected by subjective human judgment.

[0190] In the maintenance phase, there is a lack of automated update mechanisms. When the target platform updates its media asset information (such as adjustments to film and television release dates or supplementation of actor information), the collection and fusion process needs to be manually triggered, resulting in a delay in data timeliness.

[0191] To address the aforementioned shortcomings, this embodiment addresses the issue that information from different platforms often differs from and overlaps during the process of collecting media asset metadata from the internet using web crawlers. Obtaining complete media asset metadata typically requires collecting information across multiple platforms and fusing the multi-source data.

[0192] During the data update and maintenance process, with the inclusion of new data collection platforms, the system needs to automatically integrate the newly acquired data into the existing data to make the overall data richer and more accurate.

[0193] During the data fusion process, the current mainstream fusion judgment criteria are "media asset name + basic attributes". However, when faced with similar names, abbreviations, or differences in name translation, the matching accuracy is low, which poses a challenge to accurate data fusion.

[0194] Specifically, it provides a method for constructing a multi-dimensional matching mechanism, breaking through the traditional model of matching based solely on media asset names. It constructs a multi-dimensional matching system that includes core attributes of media assets (such as director, actors, production time, duration, type, plot summary, keywords, etc.) and feature identifiers (such as unique codes, fingerprint information), thus solving the limitations of name matching.

[0195] It also provides an intelligent weight allocation module, which is mainly designed to address the differences in the importance of different attributes in media asset matching. For example, for film and television media assets, information about core personnel such as directors and lead actors has a higher weight than secondary attributes; for music media assets, singers, albums, and release years have higher weights, and the matching accuracy is improved by dynamically adjusting the weights.

[0196] It also provides a conflict data processing unit, including a newly added conflict data detection and decision-making unit. When data from multiple platforms differs (e.g., the duration displayed for the same media asset may vary slightly across different platforms), it selects the optimal data based on factors such as the credibility of the data source (platform authority, historical data accuracy), and data update time. For conflicts that cannot be automatically decided, it generates a manual review task and provides difference comparison information. An automatic update and iteration module is also included, primarily featuring an automatic data update trigger mechanism. When a new platform is included in the collection scope or existing platform data is updated, the system automatically initiates the fusion process without manual intervention. Simultaneously, the module has self-iterative capabilities, continuously optimizing matching rules and weight parameters by analyzing the accuracy of historical fusion results.

[0197] Optionally, in this embodiment, the construction process of the multi-dimensional media asset matching system may include, but is not limited to, the following:

[0198] (1) Core Attribute Extraction Module: Through NLP (Natural Language Processing) technology, the metadata collected from multiple platforms is parsed, and the core attributes of the media assets are automatically extracted, including basic attributes (name, year, type, duration), creative team attributes (director, screenwriter, lead actor, production company), content feature attributes (plot summary keywords, theme tags, character relationships), and feature identifiers (unique media asset code, video frame fingerprint, audio fingerprint), forming 12-15 matching dimensions, breaking through the limitations of traditional single name matching.

[0199] (2) Vector Retrieval Enhancement Module: The vector generation process mainly targets unstructured text data (such as plot summaries, character introductions, and media reviews), using a pre-trained language model (such as the BERT-based Chinese model) for semantic encoding to transform the text into 768-dimensional dense vectors; for feature identifiers such as video frame fingerprints and audio fingerprints, a feature mapping algorithm is used to convert them into vectors of a unified dimension (768-dimensional), achieving vector normalization for multiple types of data; the construction of the vector database includes building a system based on FAISS (Facebook AI Similarity). The vector database of Search stores multi-dimensional vector data of existing media assets, supporting millisecond-level vector similarity retrieval. Simultaneously, it establishes an association index between vectors and original metadata, ensuring that search results can be quickly traced back to complete metadata information. Combining vector coarse screening and precise matching, after newly collected media asset data enters the system, it first performs coarse screening in the vector database through the vector retrieval module, calculating the cosine similarity between the new data vector and the vectors in the database, and selecting the top 10 candidate media assets with the highest similarity (coarse screening threshold set to ≥60%), significantly narrowing the scope of subsequent precise matching. Then, it performs precise comparison of multi-dimensional attributes (basic attributes, creative team attributes, etc.) of the candidate media assets, combining dynamic weights to calculate the comprehensive similarity, ultimately determining whether they are the same media asset, balancing matching efficiency and accuracy.

[0200] (3) The dynamic weight calculation unit is mainly based on the preset weight benchmark of media asset type (film, variety show, music, documentary), and then dynamically adjusts the weight of each dimension in combination with the historical matching accuracy. For example, in film and television media assets, the weight of "director + lead actor + video fingerprint vector" accounts for 60%, and in variety show media assets, the weight of "guest list + broadcasting platform + theme tag vector" accounts for 55%. At the same time, when a certain dimension (such as name) is ambiguous, the weight of high credibility dimensions such as vector similarity and fingerprint information is automatically increased to ensure matching accuracy.

[0201] (4) Similarity calculation model: An improved cosine similarity algorithm is adopted to quantify and calculate multi-dimensional attributes and vector data, set matching thresholds (e.g., a comprehensive similarity of ≥85% is judged as the same media asset), and optimize the algorithm for special scenarios: such as name abbreviation scenarios, string fuzzy matching (edit distance ≤2) combined with plot summary vector similarity (≥75%) is used to assist in the judgment; in name translation difference scenarios, after connecting to a multilingual dictionary database to realize the normalization of translation names, vector retrieval is used to filter semantically similar candidate media assets to improve cross-language matching ability.

[0202] Furthermore, intelligent data conflict decision-making mechanisms may include, but are not limited to, the following:

[0203] (1) Conflict detection module: During the fusion process, the system automatically compares data of the same dimension from multiple platforms and marks the differences (e.g., Platform A shows the broadcast time of a certain film and television show as January 10, 2025, while Platform B shows it as January 12, 2025, and classifies them according to conflict type (time, personnel, numerical, and text). For text-related conflicts (e.g., differences in plot summary), the degree of difference can be initially judged by vector similarity calculation (e.g., similarity < 60% is judged as a high-difference conflict, which requires key review).

[0204] (2) Data Credibility Assessment Unit: Construct a platform credibility scoring system, combining platform authority (e.g., official video platforms are more credible than third-party information platforms), historical data accuracy (the degree of consistency between the platform's data and the final confirmed data in the past 6 months), and data update time (prioritize data with more recent update time) to score the credibility of each platform's data (out of 100 points); if a platform's data is verified to be semantically consistent with official information through vector retrieval multiple times, an additional 5-10 credibility points can be added.

[0205] (3) Conflict Decision Rule Base: Preset decision rules for different types of conflicts. For example, for time-related conflicts, select the time data of the platform with the highest credibility score. If the score difference is ≤5 points, take the data with the more recent update time. For personnel-related conflicts (such as differences in the actor list), cross-validation (such as comparing the production company's announcement and the actor's studio information) is used to confirm the conflict. At the same time, the vector retrieval module can be called to compare the relevant personnel introduction text to help determine the relevance of personnel identities. For conflicts that cannot be automatically decided by rules (such as plot outlines with high differences), a manual review task is generated and a difference comparison report (including vector similarity analysis results) is pushed. The review results are updated to the rule base in sync to optimize subsequent decisions.

[0206] This embodiment also proposes a fully automated fusion and update closed-loop process, which may include, but is not limited to, the following modules:

[0207] (1) Automated data collection and adaptation module: Develop a general crawler adapter that supports API interfaces or web page structures of different platforms, automatically parse heterogeneous data formats (JSON, XML, HTML) and convert them into a unified metadata standard format (based on the data specifications of the intelligent data production platform) without the need for manual adjustment of crawler rules; at the same time, it has built-in anti-crawling strategies (such as dynamic IP pool and request frequency control) to ensure the stability of data collection; after the data collection is completed, it automatically triggers the vector generation module to convert unstructured data into vectors and update them to the vector database.

[0208] (2) Automatic fusion triggering mechanism: Two triggering modes are set: one is timed triggering (such as starting full data fusion at 2 am every day and updating the vector database index synchronously), and the other is event triggering (such as automatically starting initial fusion when a new platform is included, triggering incremental fusion when the target platform data is updated, and performing vector coarse screening after the incremental data generates vectors before entering the precise matching stage). The fusion process does not require manual intervention and the entire log is traceable.

[0209] (3) Data update and iteration module: The fused data is synchronized to the database of the intelligent data production platform in real time, and a data version management mechanism is established to record the source, time, modified content and vector data changes of each update; at the same time, the historical fusion results are analyzed regularly (e.g. monthly), the matching accuracy of each dimension, the coarse screening accuracy of vector retrieval, and the correctness of conflict decision are calculated, and the weight parameters, vector generation model (e.g., fine-tuning the pre-trained language model to adapt to the semantics of the media asset domain) and decision rules are automatically optimized to realize the system's self-iterative upgrade.

[0210] To better understand the above solution, the following section will take the integration and processing of media asset metadata of the intelligent data production platform into the new video platform (C platform) as an example to describe its overall implementation process in detail.

[0211] S31, Preparations for the implementation of the plan;

[0212] (1) Configure a general crawler adapter: Add the interface parameters of the C platform (API interface address, access token) to the system, set the data collection scope (film and television, variety show media assets), the adapter automatically recognizes the C platform data format (XML) and configures the parsing rules, without the need to manually write crawler code; at the same time, deploy the BERT-base Chinese pre-trained model (which has been fine-tuned through historical media asset text data to adapt to the semantics of film and television, variety show fields), and build the FAISS vector database (using IVF_FLAT index, supporting efficient retrieval of 1 million vectors).

[0213] (2) Initialize weights and rules: Initialize dimension weights for film and television and variety show media assets respectively (e.g., for film and television, “Director 30% + Leading actor 20% + Video fingerprint vector 15% + Name 15% + Year 5% + Other 15%”), and import the conflict decision rule library (e.g., time-based conflict decision rules, text-based conflict vector similarity judgment criteria).

[0214] S32, Data Acquisition and Preprocessing;

[0215] (1) Start event trigger mode: Due to the inclusion of the new platform, the system automatically triggers the collection task. The crawler adapter accesses the C platform API according to the configuration parameters to collect metadata of movies and variety shows launched in the past 5 years (a total of 5,000 records). The collected data includes structured attributes (name, year, actors, etc.) and unstructured text (plot summary, character introduction, a total of about 100,000 words).

[0216] (2) Heterogeneous data conversion and vector generation: The adapter converts the collected XML format data into a unified metadata format (e.g., converting the “Actor Name (Character Name)” format under the “Leading Actor” field of the C platform into the “Actor Name - Character Name” format); at the same time, it calls the BERT model to perform vector encoding on the unstructured text to generate a 768-dimensional vector. For example, a film plot summary “tells a story about a future space encountering a crisis, and a group of young people embarking on a rescue journey” is encoded to obtain vector V1. The vector is then associated with the corresponding media asset ID and stored in the FAISS vector database.

[0217] S33, multi-dimensional matching (including vector retrieval) and fusion;

[0218] (1) Vector coarse screening: After the vector V2 of a certain film and television "Campus Life 2024" is generated on platform C, the system calls the FAISS vector database to search and calculate the cosine similarity between V2 and the existing film and television vectors in the database. The top 10 candidate media assets with similarity (similarity range 65%-92%) are screened out. According to the search results, the similarity between V2 and the existing data "Campus Life" vector V3 is determined to be 92%, which means that vectors V2 and V3 are used to describe the same media asset. At this time, the precise matching stage is entered.

[0219] (2) Core attribute extraction and precise matching: The NLP module extracts the core attributes of "Campus Life 2024" (director, lead actors, year, duration, etc.) and calculates the comprehensive similarity by combining dynamic weights: name similarity 90% (the name on platform C is "Campus Life 2024", while the current name is "Campus Life"), director similarity 100%, lead actor similarity 95%, vector similarity 92%, and the comprehensive similarity is calculated to be 94.5% (≥85% matching threshold), which is determined to be the same media asset.

[0220] (3) Data fusion: The new data of the film and television on the C platform (such as information on the 3 guest actors that were not included and links to high-definition posters) are added to the existing data. At the same time, the vector data of the media asset is updated (the newly added character introduction text after fusion generates vector V4, which is merged with the original vector V3 to form V5 and updated to the vector database) to form a complete metadata record.

[0221] S34, Conflict detection and handling (executed during data fusion);

[0222] (1) Conflict detection: It was found that the plot introduction text of a variety show on the C platform was significantly different from the existing text. The similarity between the two was calculated to be 58% (<60%), which was marked as a high-difference text conflict. At the same time, the broadcast time displayed on the C platform was March 1, 2024, while the existing data was March 2, 2024, which was marked as a time conflict.

[0223] (2) Credibility assessment: Platform C has a credibility score of 88 (authoritative video platform with a historical accuracy rate of 92%. Due to the match between the text data vector retrieval and authoritative information in the past 3 times, an additional 5 points are added, making the actual score 93). The credibility score of the existing data source platform is 92 (authoritative production platform with a historical accuracy rate of 95%). The time-related conflict score difference is 1 point (≤5 points).

[0224] (3) Conflict decision: For time-related conflicts, select the data with the more recent update time (the data on platform C was updated on February 28, 2024, and the existing data was updated on February 25, 2024), and unify the broadcast time to February 28, 2024; for text-related conflicts, due to low vector similarity, generate a manual review task, push two versions of plot introduction text and vector similarity analysis report, and after review, confirm that the text on platform C is more complete, update it to standard text and regenerate vectors, and update it to the vector database simultaneously.

[0225] S35, Updates and Iterations.

[0226] (1) Data synchronization: The merged data is synchronized to the database of the intelligent data production platform in real time for media asset recommendation and copyright management business calls, with a synchronization delay of ≤5 minutes; the vector database updates the index synchronously to ensure the accuracy of subsequent retrieval.

[0227] (2) System Iteration: One month later, the system analyzed the results of this fusion and found that the accuracy of the vector retrieval screening reached 96% (the proportion of correct media assets included in the Top 10 candidates). The weight of "vector similarity" in film and television media assets was automatically increased from 15% to 18%. At the same time, the BERT model parameters were fine-tuned (100 highly dissimilar text data confirmed by manual review were added) to make the text vector encoding more in line with the semantics of the media asset domain. The accuracy of subsequent text-type conflict vector judgment was increased to 82%.

[0228] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0229] According to another aspect of the embodiments of this application, as follows is also provided Figure 7 The data fusion apparatus shown includes:

[0230] The acquisition unit 702 is used to collect historical media asset metadata of the target media asset within a historical period from multiple heterogeneous platforms in response to the access of the current media asset platform.

[0231] The filtering unit 704 is used to filter a set of historical metadata vectors whose similarity meets a first threshold from the vector database based on the current metadata vector corresponding to the current media asset metadata. The current media asset metadata is the data on the current media asset platform, and the vector database contains historical metadata vectors corresponding to the historical media asset metadata.

[0232] Matching unit 706 is used to perform precise matching on a set of historical metadata vectors to obtain the target historical metadata vector;

[0233] The first processing unit 708 is used to perform conflict detection based on metadata attributes, obtain target detection results, and perform fusion processing on the target historical metadata vector corresponding to the target historical metadata and the current media metadata to obtain fused media metadata.

[0234] Optionally, the above-mentioned filtering unit 704 includes:

[0235] The first processing module is used to preprocess historical media asset metadata to obtain standardized historical media asset metadata with a unified format.

[0236] The first acquisition module is used to acquire the historical metadata vector corresponding to the standard historical media asset metadata.

[0237] The filtering module is used to filter out a set of historical metadata vectors that meet the first threshold based on the cosine similarity between the current metadata vector and the historical metadata vector.

[0238] Optionally, the matching unit 706 includes:

[0239] The second acquisition module is used to sequentially acquire each historical metadata vector from a set of historical metadata vectors as the current historical metadata vector;

[0240] The first extraction module is used to extract the first set of attribute information of the current historical media asset metadata corresponding to the current historical metadata vector;

[0241] The second extraction module is used to extract the second set of attribute information of the current media asset metadata, wherein the attribute type of the second set of attribute information is the same as that of the first set of attribute information;

[0242] The second processing module is used to sequentially determine the cosine similarity between the same type of attribute information in the first set of attribute information and the second set of attribute information to obtain a set of similarity;

[0243] The third processing module is used to perform a weighted summation of a set of similarities based on the target weights corresponding to the attribute information of each type, and to determine the comprehensive similarity between the current historical metadata vector and the current metadata vector.

[0244] The fourth processing module is used to determine the media asset represented by the current historical media asset metadata and the media asset represented by the current media asset metadata as the same media asset when the comprehensive similarity is greater than or equal to the second threshold.

[0245] Optionally, the matching unit 706 further includes:

[0246] The third acquisition module is used to acquire multiple initial weights of multiple attribute information based on the media asset type of the target media asset before performing a weighted summation of a set of similarities based on the target weights corresponding to the attribute information of each type. The first set of attribute information and the second set of attribute information both include multiple attribute information.

[0247] The adjustment module is used to dynamically adjust multiple initial weights based on historical matching accuracy to obtain the target weight.

[0248] Optionally, the above-mentioned device further includes:

[0249] The first acquisition unit is used to acquire multiple initial weights of multiple attribute information based on the media asset type of the target media asset, wherein the first set of attribute information and the second set of attribute information both include multiple attribute information.

[0250] The adjustment unit is used to dynamically adjust multiple initial weights based on historical matching accuracy to obtain the target weight.

[0251] Optionally, the above-mentioned device further includes:

[0252] The second processing unit is used to increase the weight of one of the attribute information when there is ambiguity in one of the attribute information during the process of dynamically adjusting multiple initial weights based on historical matching accuracy.

[0253] Optionally, the first processing unit 708 includes:

[0254] The fifth processing module is used to obtain the first conflict detection result when the metadata attribute is time information and the current publication time of the current media asset metadata is inconsistent with the target historical publication time of the target historical media asset metadata corresponding to the target historical metadata vector.

[0255] The sixth processing module is used to obtain the second conflict detection result when the metadata attribute is subject information and the current subject list of the current media asset metadata is inconsistent with the target historical subject list of the target historical media asset metadata.

[0256] The seventh processing module is used to obtain a third conflict detection result when the metadata attribute is text information and the current text description information of the current media asset metadata is inconsistent with the target historical text description information of the target historical media asset metadata. The target detection result includes at least one of the first conflict detection result, the second conflict detection result and the third conflict detection result.

[0257] Optionally, the above method further includes:

[0258] The third processing unit is configured to perform at least one of the following: when the target detection result includes only the first conflict detection result, the current subject list is consistent with the target historical subject list, and the current text description information is consistent with the target historical text description information, compare the current publication time and the target historical publication time to obtain a target comparison result, wherein the target comparison result indicates that the current publication time is later than the target historical publication time; based on the target comparison result, replace the target historical publication time in the target historical media asset metadata with the current publication time to obtain the fused media asset metadata; when the target detection result includes only the second conflict detection result, the current publication time is consistent with the target historical publication time ... current publication time is later than the target historical publication time, and the current text description information is consistent with the target historical text description information, compare the current publication time and the target historical publication time to obtain a target comparison result, wherein the current publication time is later than the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the current publication time and the target historical publication time, compare the If the target historical text description information is consistent, the union of the current subject list and the target historical subject list is taken to obtain the merged subject list; the target historical subject list in the target historical media asset metadata is replaced with the merged subject list to obtain the fused media asset metadata; if the target detection result only includes the third conflict detection result, the current release time and the target historical release time are consistent, and the current subject list and the target historical subject list are consistent, the current media asset metadata and the target historical media asset metadata are fused based on the text vector similarity between the current text vector corresponding to the current text description information and the target historical text vector corresponding to the target historical text description information to obtain the fused media asset metadata.

[0259] Optionally, the third processing unit mentioned above includes:

[0260] The update module is used to update the historical text description information in the target historical media asset metadata to the current text description information when the text vector similarity is greater than or equal to the third threshold and the current text description information contains historical text description information, thereby obtaining the fused media asset metadata; or the merging module is used to merge the historical text description information and the current text description information when the text vector similarity is greater than or equal to the third threshold and the current text description information contains historical text description information with some overlap, thereby obtaining the fused media asset metadata.

[0261] Optionally, the above-mentioned device further includes:

[0262] The comparison module is used to compare the current publication time and the target's historical publication time to obtain the target comparison result when the target detection results include the first conflict detection result and the second conflict detection result, and the current text description information is consistent with the target's historical text description information.

[0263] The replacement module is used to replace the target historical publication time in the target historical media asset metadata with the current publication time, and replace the target historical subject list in the target historical media asset metadata with the merged subject list, based on the target comparison results, to obtain the merged media asset metadata. The merged subject list is obtained by taking the union of the current subject list and the target historical subject list.

[0264] Optionally, the above-mentioned device further includes:

[0265] The fourth processing unit is used to synchronize the fused media asset metadata to the vector database and update the data index relationship in the vector database after fusing the target historical metadata vector with the current media asset metadata based on the target detection results.

[0266] It should be noted that the embodiments of the data fusion device described here can refer to the embodiments of the data fusion method described above, and will not be repeated here.

[0267] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described data fusion method is also provided. This electronic device may be... Figure 1 The target terminal or server is shown. This embodiment uses the electronic device as an example to illustrate the concept. Figure 8 As shown, the electronic device includes a memory 802 and a processor 804. The memory 802 stores a computer program, and the processor 804 is configured to execute the steps in any of the above method embodiments via the computer program.

[0268] Optionally, the aforementioned electronic device may be located in at least one of a plurality of network devices of the computer.

[0269] Optionally, the processor described above can be configured to perform the following steps via a computer program:

[0270] S1, in response to the access of the current media asset platform, collects historical media asset metadata of the target media asset within the historical period from multiple heterogeneous platforms;

[0271] S2, based on the current metadata vector corresponding to the current media asset metadata, select a set of historical metadata vectors whose similarity meets the first threshold from the vector database, where the current media asset metadata is the data on the current media asset platform, and the vector database contains historical metadata vectors corresponding to the historical media asset metadata;

[0272] S3 performs precise matching on a set of historical metadata vectors to obtain the target historical metadata vector;

[0273] S4. Conflict detection is performed based on metadata attributes to obtain target detection results. Based on the target detection results, the target historical metadata vector corresponding to the target historical metadata vector is fused with the current media metadata to obtain fused media metadata.

[0274] Alternatively, as those skilled in the art will understand, Figure 8 The structure shown is for illustrative purposes only. Figure 8 This does not limit the structure of the aforementioned electronic devices or electronic equipment. For example, electronic devices or electronic equipment may also include components that are more... Figure 8 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 8 The different configurations shown.

[0275] The memory 802 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data fusion method and apparatus in this embodiment. The processor 804 executes various functional applications and data processing by running the software programs and modules stored in the memory 802, thereby realizing the aforementioned data fusion method. The memory 802 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 802 may further include memory remotely located relative to the processor 804, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 802 may be used, but is not limited to, to store historical media asset metadata, current metadata vectors, and historical metadata vectors. As an example, such as... Figure 8 As shown, the memory 802 may include, but is not limited to, the acquisition unit 702, filtering unit 704, matching unit 706, and first processing unit 708 in the data fusion device. Furthermore, it may include, but is not limited to, other module units in the data fusion device, which will not be elaborated upon in this example.

[0276] Optionally, the transmission device 806 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 806 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 806 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0277] In addition, the above-mentioned electronic device also includes: a display 808 for displaying video images of media resources; and a connection bus 8010 for connecting various module components in the above-mentioned electronic device.

[0278] In other embodiments, the target terminal or server described above can be a node in a distributed system. This distributed system can be a blockchain system, formed by connecting multiple nodes through network communication. The nodes can form a point-to-point network, and any type of computing device, such as a server or target terminal, can become a node in the blockchain system by joining this point-to-point network.

[0279] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data fusion method provided in various optional implementations of the aforementioned server verification processing, wherein the computer program is configured to execute the steps in any of the above-described method embodiments at runtime.

[0280] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store a computer program for performing the following steps:

[0281] S1, in response to the access of the current media asset platform, collects historical media asset metadata of the target media asset within the historical period from multiple heterogeneous platforms;

[0282] S2, based on the current metadata vector corresponding to the current media asset metadata, select a set of historical metadata vectors whose similarity meets the first threshold from the vector database, where the current media asset metadata is the data on the current media asset platform, and the vector database contains historical metadata vectors corresponding to the historical media asset metadata;

[0283] S3 performs precise matching on a set of historical metadata vectors to obtain the target historical metadata vector;

[0284] S4. Conflict detection is performed based on metadata attributes to obtain target detection results. Based on the target detection results, the target historical metadata vector corresponding to the target historical metadata vector is fused with the current media metadata to obtain fused media metadata.

[0285] Optionally, in embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0286] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the target terminal. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0287] The sequence numbers of the embodiments in this application are merely for description and do not represent the superiority or inferiority of the embodiments. If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods in the various embodiments of this application.

[0288] In the above embodiments of this application, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. It should be understood that the disclosed client can be implemented in other ways in the several embodiments provided in this application. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0289] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0290] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data fusion method characterized by, The method comprises the following steps: in response to access of a current media platform, collecting historical media metadata of target media in a historical period from a plurality of heterogeneous platforms; based on a current metadata vector corresponding to current media metadata, filtering a group of historical metadata vectors with a similarity satisfying a first threshold from a vector database, wherein the current media metadata is data on the current media platform, and the vector database contains historical metadata vectors corresponding to the historical media metadata; performing accurate matching on the group of historical metadata vectors to obtain a target historical metadata vector; based on metadata attributes, performing conflict detection to obtain a target detection result, and based on the target detection result, performing fusion processing on target historical media metadata corresponding to the target historical metadata vector and the current media metadata to obtain fused media metadata.

2. The method of claim 1, wherein, The method comprises the following steps: by preprocessing the historical media metadata, obtaining standard historical media metadata with unified format; obtaining the historical metadata vector corresponding to the standard historical media metadata; based on the cosine similarity between the current metadata vector and the historical metadata vector, filtering the group of historical metadata vectors satisfying the first threshold.

3. The method of claim 1, wherein, The method comprises the following steps: obtaining each historical metadata vector as a current historical metadata vector from the group of historical metadata vectors in turn; extracting a first group of attribute information of the current historical media metadata corresponding to the current historical metadata vector; extracting a second group of attribute information of the current media metadata, wherein the second group of attribute information is of the same type as the first group of attribute information; determining the cosine similarity between the same type attribute information in the first group of attribute information and the second group of attribute information in turn to obtain a group of similarities; based on the target weight corresponding to each type of attribute information, performing weighted summation on the group of similarities to determine the comprehensive similarity between the current historical metadata vector and the current metadata vector; in the case that the comprehensive similarity is greater than or equal to a second threshold, determining the media represented by the current historical media metadata and the media represented by the current media metadata as the same media.

4. The method of claim 3, wherein, Before the step of performing weighted summation on the group of similarities based on the target weight corresponding to each type of attribute information, the method further comprises the following steps: based on the media type of the target media, obtaining a plurality of initial weights of a plurality of attribute information, wherein the first group of attribute information and the second group of attribute information both include the plurality of attribute information; based on the historical matching accuracy, dynamically adjusting the plurality of initial weights to obtain the target weight.

5. The method of claim 4, wherein, The method further comprises the following steps: in the process of dynamically adjusting the plurality of initial weights based on the historical matching accuracy, in response to the existence of ambiguity in one of the plurality of attribute information, increasing the weight of the one attribute information.

6. The method of claim 1, wherein, The conflict detection is performed based on the metadata attribute, and a target detection result is obtained, including: In a case where the metadata attribute is time information and a current publishing time of the current media asset metadata is inconsistent with a target historical publishing time of target historical media asset metadata corresponding to the target historical media asset metadata vector, a first conflict detection result is obtained; In a case where the metadata attribute is subject information and a current subject list of the current media asset metadata is inconsistent with a target historical subject list of the target historical media asset metadata, a second conflict detection result is obtained; In a case where the metadata attribute is text information and a current text description information of the current media asset metadata is inconsistent with target historical text description information of the target historical media asset metadata, a third conflict detection result is obtained, wherein the target detection result includes at least one of the first conflict detection result, the second conflict detection result, and the third conflict detection result.

7. The method of claim 6, wherein, The method further includes at least one of: In a case where the target detection result only includes the first conflict detection result, the current subject list is consistent with the target historical subject list, and the current text description information is consistent with the target historical text description information, a target comparison result is obtained by comparing the current publishing time and the target historical publishing time, wherein the target comparison result indicates that the current publishing time is later than the target historical publishing time; and based on the target comparison result, the target historical publishing time in the target historical media asset metadata is replaced by the current publishing time to obtain fused media asset metadata; In a case where the target detection result only includes the second conflict detection result, the current publishing time and the target historical publishing time are consistent, and the current text description information is consistent with the target historical text description information, a merged subject list is obtained by taking a union set of the current subject list and the target historical subject list; and the target historical subject list in the target historical media asset metadata is replaced by the merged subject list to obtain the fused media asset metadata; In a case where the target detection result only includes the third conflict detection result, the current publishing time and the target historical publishing time are consistent, and the current subject list is consistent with the target historical subject list, a text vector similarity between a current text vector corresponding to the current text description information and a target historical text vector corresponding to the target historical text description information is used to perform fusion processing on the current media asset metadata and the target historical media asset metadata to obtain the fused media asset metadata.

8. The method of claim 7, wherein, The fusion processing on the current media asset metadata and the target historical media asset metadata based on the text vector similarity between the current text vector corresponding to the current text description information and the target historical text vector corresponding to the target historical text description information includes: In a case where the text vector similarity is greater than or equal to a third threshold value and the current text description information contains the historical text description information, the historical text description information in the target historical media asset metadata is updated to the current text description information to obtain the fusion media asset metadata; or In a case where the text vector similarity is greater than or equal to the third threshold value and the current text description information contains a partial intersection of the historical text description information, the historical text description information and the current text description information are merged to obtain the fusion media asset metadata.

9. The method of claim 6, wherein, The method further comprises: In a case where the target detection result includes the first conflict detection result and the second conflict detection result and the current text description information is consistent with the target historical text description information, a target comparison result is obtained by comparing the current publishing time and the target historical publishing time; Based on the target comparison result, the target historical publishing time in the target historical media asset metadata is replaced by the current publishing time, and the target historical subject list in the target historical media asset metadata is replaced by a merged subject list to obtain fusion media asset metadata, wherein the merged subject list is obtained by taking a union set of the current subject list and the target historical subject list.

10. The method according to any one of claims 1 to 9, characterized in that, After the fusion processing of the target historical media asset metadata corresponding to the target historical metadata vector and the current media asset metadata based on the target detection result, the method further comprises: The fusion media asset metadata is synchronized to the vector database, and a data index relationship in the vector database is updated.

11. A data fusion apparatus characterized by comprising: Comprise: The acquisition unit is configured to acquire historical media asset metadata of target media assets in a historical period from a plurality of heterogeneous platforms in response to access of a current media asset platform; The screening unit is configured to screen a group of historical metadata vectors with a similarity satisfying a first threshold value from a vector database based on a current metadata vector corresponding to current media asset data, wherein the current media asset data is data on the current media asset platform, and the vector database contains historical metadata vectors corresponding to the historical media asset data; The matching unit is configured to perform accurate matching on the group of historical metadata vectors to obtain a target historical metadata vector; The first processing unit is configured to perform conflict detection based on metadata attributes to obtain a target detection result, and perform fusion processing of target historical media asset metadata corresponding to the target historical metadata vector and the current media asset data based on the target detection result to obtain fusion media asset metadata.

12. A computer readable storage medium, characterized in that, The computer-readable storage medium comprises a stored program, wherein the program can be run by a terminal device or a computer to execute the method described in any one of claims 1 to 10.

13. An electronic device comprising a memory and a processor, characterized in that The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 10 by using the computer program.

Images