Metadata automatic processing method and apparatus, terminal device, and storage medium

Abstract

The application is suitable for the technical field of data processing, and provides a metadata automatic processing method and device, terminal equipment and storage medium. The method comprises the following steps: performing format identification on the obtained original data, determining a target sampling strategy according to the format identification result, and reading a data segment as sampling data according to the target sampling strategy; determining the matching degree of the sampling data and a plurality of predetermined modalities according to the byte distribution characteristics of the sampling data; performing feature extraction according to a feature extraction rule to obtain feature description data of each predetermined modality; performing synthesis processing according to the matching degree to generate multi-modal feature description data; matching in an algorithm performance portrait database based on the multi-modal feature description data to determine a target metadata processing algorithm; and performing metadata extraction processing on the original data based on the target metadata processing algorithm to obtain metadata. The application can realize fine processing of different characteristic data and improve the accuracy and processing efficiency of metadata extraction.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, terminal device and storage medium for automatic metadata processing. Background Technology

[0002] With the rapid development of big data technology, massive amounts of data from different sources and modalities are growing dramatically. Automatically extracting metadata from this massive amount of data is the foundation for effective data management, retrieval, and analysis.

[0003] Currently, common metadata extraction methods typically employ pre-configured processing rules, which define corresponding processing strategies for data of specific formats or types and then execute metadata extraction according to those strategies. However, with the increasing diversity of data types and sources, the complexity and variety of metadata extraction tasks have significantly increased. How to achieve refined processing of data with different characteristics and improve the accuracy and efficiency of metadata extraction is a problem that needs to be solved. Summary of the Invention

[0004] This application provides an automatic metadata processing method, apparatus, terminal device, and storage medium, which can achieve refined processing of different feature data and improve the accuracy and efficiency of metadata extraction.

[0005] In a first aspect, embodiments of this application provide an automatic metadata processing method, including: Obtain the raw data to be processed; The original data is format-identified, and a target sampling strategy is determined from multiple predetermined sampling strategies based on the format identification result. Data segments are then read from the original data as sampled data according to the target sampling strategy. Based on the byte distribution characteristics of the sampled data, the degree of matching between the sampled data and multiple predetermined modalities is determined; Based on the feature extraction rules corresponding to each predetermined modality, feature extraction is performed on the sampled data to obtain feature description data for each predetermined modality; Based on the matching degree, the feature description data of the predetermined modality is synthesized to generate multimodal feature description data of the original data; Based on the multimodal feature description data, a matching algorithm performance profile database is performed to determine the target metadata processing algorithm that is suitable for the original data; Metadata is extracted from the original data based on the target metadata processing algorithm.

[0006] Secondly, embodiments of this application provide an automatic metadata processing apparatus, comprising: The raw data acquisition unit is used to acquire the raw data to be processed. The data sampling unit is used to perform format recognition on the raw data, determine the target sampling strategy from a variety of predetermined sampling strategies based on the format recognition result, and read data segments from the raw data as sampled data according to the target sampling strategy. A modality matching degree determination unit is used to determine the matching degree between the sampled data and multiple predetermined modalities based on the byte distribution characteristics of the sampled data; The feature data extraction unit is used to extract features from the sampled data according to the feature extraction rules corresponding to each predetermined mode, so as to obtain feature description data of each predetermined mode; A multimodal data synthesis unit is used to synthesize the feature description data of the predetermined modality according to the matching degree, and generate multimodal feature description data of the original data. The target algorithm determination unit is used to match the multimodal feature description data in the algorithm performance profile database to determine the target metadata processing algorithm that is suitable for the original data. The metadata processing unit is used to perform metadata extraction processing on the original data based on the target metadata processing algorithm to obtain metadata.

[0007] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the automatic metadata processing method as described in the first aspect above.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the automatic metadata processing method as described in the first aspect above.

[0009] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the automatic metadata processing method described in the first aspect above.

[0010] In this embodiment, the original data to be processed is first acquired and its format is identified. Based on the identification results, a target sampling strategy matching the original data format is determined from multiple predetermined sampling strategies. Representative data segments are read from the original data as sampled data according to the target sampling strategy, which significantly reduces the amount of data to be processed and reduces computational overhead. Then, based on the byte distribution characteristics of the sampled data, the matching degree between the sampled data and multiple predetermined modalities is determined. Based on the feature extraction rules corresponding to each predetermined modality, feature extraction is performed on the sampled data to obtain feature description data for each modality, so that the content features under different modalities can be fully extracted. Then, based on the matching degree, the feature description data of each modality is synthesized to generate multimodal feature description data that uniformly represents the content and structural attributes of the original data. Then, based on the multimodal feature description data, matching is performed in the algorithm performance profile database to determine the target metadata processing algorithm that is suitable for the original data. This realizes matching the corresponding processing algorithm according to the content features of the data itself, avoiding the accuracy loss caused by different feature data being processed by the same algorithm. Finally, based on the target metadata processing algorithm, metadata extraction processing is performed on the original data to obtain metadata. The proposed solution achieves collaborative processing of data format recognition, adaptive sampling, multimodal feature synthesis, and dynamic algorithm matching, enabling refined processing of data with different features and effectively improving the accuracy and efficiency of metadata extraction. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the implementation of the automatic metadata processing method provided in this application embodiment; Figure 2 This is a flowchart illustrating a specific implementation of the metadata automatic processing method provided in this application for identifying the original data format; Figure 3 This is a flowchart illustrating a specific implementation of the automatic metadata processing method provided in this application to determine the degree of matching. Figure 4 This is a flowchart illustrating a specific implementation of the automatic metadata processing method for constructing an algorithm performance profile database provided in this application embodiment; Figure 5 This is a flowchart illustrating a specific implementation of step S106 in the automatic metadata processing method provided in this application embodiment; Figure 6This is a flowchart illustrating a specific implementation of the automatic metadata processing method for updating the algorithm performance profile database provided in this application embodiment; Figure 7 This is a structural block diagram of the metadata automatic processing device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0013] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0014] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0015] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0016] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0017] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0018] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0019] By way of example and not limitation, the automatic metadata processing method provided in this application is applicable to various types of terminal devices that require automatic metadata processing. Specific terminal devices may include mobile phones, tablets, wearable devices, laptops, ultra-mobile personal computers (UMPCs), desktop computers, and servers, etc. This application does not impose any restrictions on the specific type of terminal device.

[0020] Figure 1 The implementation flow of the automatic metadata processing method provided in this application embodiment is illustrated. The method flow includes steps S101 to S107. The specific implementation principle of each step is as follows: Step S101: Obtain the raw data to be processed.

[0021] Raw data refers to digital content of any format that is to be processed for metadata extraction. Raw data can originate from various sources, including but not limited to: files in a system, byte streams transmitted over a network, objects stored in a data lake, and data objects in a storage system. The format of raw data is unrestricted; it can be in multiple modalities or even streaming data not persisted to a file. For example, raw data could be a JPEG digital photograph, a CSV file containing sales records, an MP3 recording of a meeting, or a ZIP archive containing multiple documents and images. The acquisition of raw data does not require prior knowledge of its exact format or content type.

[0022] Step S102: Perform format recognition on the raw data, determine the target sampling strategy from multiple predetermined sampling strategies based on the format recognition result, and read data segments from the raw data as sampled data according to the target sampling strategy. The data segments shall not exceed a preset size.

[0023] Format identification of raw data refers to determining the encoding format or file format of the raw data. For example, image data may be in JPEG or PNG format, audio data may be in MP3 or WAV format, and compressed data may be in ZIP format. The results of format identification are used to subsequently determine the target sampling strategy that matches the format of the raw data from a variety of predetermined sampling strategies.

[0024] In one possible implementation, the original data is format-identified using a hierarchical progressive strategy. The format identification does not rely on a single information source but is performed step by step through a multi-level judgment mechanism to ensure that identification results can still be obtained even when some information is missing or abnormal.

[0025] As one possible implementation of this application, such as Figure 2 As shown, the format of the raw data is identified, and execution is carried out according to a hierarchical progressive strategy, including: A1: Perform first-level identification, which includes identifying the format of the original data based on the characteristic bytes of the byte stream header of the original data.

[0026] The feature byte (i.e., magic number) is a specific byte sequence located at the beginning of a byte stream that serves as a format identifier.

[0027] In one possible implementation, first-level identification is the most confident identification method. Since the feature byte is an identifier explicitly defined in the format definition, the identification result based on the feature byte is deterministic, and most common data formats can be identified in this way. By performing first-level identification, the format of the original data can be identified quickly and accurately.

[0028] A2: If the first level of identification fails to identify the format of the original data, then the second level of identification is performed, which includes identifying the format of the original data based on the file extension of the original data.

[0029] A file extension is the string appended to a filename after the last period. In operating systems, file extensions are typically used to indicate the file's format. For example, ".txt" usually indicates text format, ".log" usually indicates log format, and ".csv" usually indicates comma-separated value format. When the raw data has a usable file extension, its underlying type can be inferred from the extension. For formats lacking standard magic number signature bytes (such as plain text or log files), first-level identification will fail due to the missing signature bytes; in such cases, identification can be performed based on the file extension.

[0030] In one possible implementation, the confidence level of the second-level identification is lower than that of the first-level identification; that is, the confidence level for identifying file extensions is lower than that for identifying feature bytes. In this embodiment, the first-level identification is executed first as a priority, and the second-level identification is only executed if the first-level identification cannot handle the problem (e.g., the confidence score is lower than a preset threshold, such as 80%, which is considered as not identified). The second-level identification supplements the first-level identification and can cover scenarios where there is no standard magic number but the file extension is standardized. By executing the second-level identification, the file extension is used as an auxiliary identification clue to identify data formats that lack feature bytes.

[0031] A3: If the second level of identification fails to identify the format of the original data, then the third level of identification is performed, which includes identifying the format of the original data based on the statistical characteristics of the byte stream of the original data.

[0032] Statistical characteristics refer to quantitative features obtained through statistical analysis of byte streams, including but not limited to byte entropy and the percentage of printable characters. Byte entropy measures the randomness of a byte sequence. The percentage of printable characters refers to the proportion of bytes in a byte stream that are printable ASCII characters (encoding range 32 to 126) out of the total number of bytes. For example, plaintext data has low byte entropy (because the character distribution is regular) and a printable character percentage higher than 90%; encrypted or highly compressed data has high byte entropy (because the byte distribution is nearly uniform and random) and a low printable character percentage. By analyzing statistical characteristics such as byte entropy and the percentage of printable characters, the type of data can be identified, for example, distinguishing plaintext or logs from encrypted or compressed data.

[0033] In one possible implementation, the byte entropy is calculated by: for all possible values ​​(0 to 255) in the byte stream, calculating the frequency of each value, and then calculating the information entropy based on the frequency, thereby obtaining the byte entropy.

[0034] In one possible implementation, the third-level identification has the lowest confidence. If the first two levels fail to yield identification results, format classification is performed by analyzing the statistical characteristics of the sampled data's byte stream, without relying on any external metadata. The third-level identification serves as a last resort mechanism, only reverting to it when the second-level identification fails. This ensures that even if a file lacks a magic number and its extension is missing or incorrect, a reasonable format judgment can still be obtained. By performing third-level identification, utilizing the statistical characteristics of the byte stream as the final identification method, data formats lacking characteristic bytes and having no or missing file extensions are identified, ensuring a reasonable judgment of the original data's format can be made under various boundary conditions.

[0035] Through the above three-layer progressive format recognition mechanism, a complete recognition link is formed from high confidence to low confidence and from low overhead to high overhead. In most cases, only the low-overhead first layer is needed to complete the recognition, and in boundary cases, subsequent layers ensure full-scene coverage.

[0036] In this embodiment, the data category to which the original data belongs is determined based on the format recognition result, and the predetermined sampling strategy corresponding to the data category is determined as the target sampling strategy. The predetermined sampling strategy refers to a pre-set sampling rule corresponding to the data category, used to determine the sampling read position and read size. The target sampling strategy refers to a sampling rule selected from multiple predetermined sampling strategies based on the format recognition result of the original data, applicable to the original data. Data categories are types divided according to the information carrying format of the data, including text, image, audio, video, structured data, and compressed / container categories. Specifically, the text category covers plain text, code files, and log files; the image category covers still images in formats such as JPEG and PNG; the audio category covers audio in formats such as MP3 and WAV; the video category covers video in formats such as MP4 and AVI; the structured data category covers data in formats such as CSV, JSON, Parquet, and XML; and the compressed / container category covers formats such as ZIP, RAR, PDF, and PPTX.

[0037] Based on the identified format of the raw data, it is determined which of several data categories the format belongs to, and then the predetermined sampling strategy corresponding to that data category is determined as the target sampling strategy. For example, when the raw data format is identified as JPEG, this format belongs to the image category, and the target sampling strategy is the sampling strategy corresponding to the image category; when the raw data format is identified as ZIP, this format belongs to the compressed file category, and the target sampling strategy is the sampling strategy corresponding to the compressed file category. By determining the corresponding sampling strategy according to the identified data format, the sampling method is matched with the content distribution characteristics of the data.

[0038] According to the reading position and reading size specified by the target sampling strategy, data segments are read from the original data, and the total amount of data segments read does not exceed the preset size. In one possible implementation, the preset size is a pre-set upper limit for the amount of sampled data, which can be 12 kilobytes (KB).

[0039] One possible implementation includes sampling strategies for each data category: For text data, bytes of a preset size are read from the beginning, middle, and end of the file and then merged to capture key information distributed at the beginning, middle, and end. For example, 4KB is read from the beginning, 4KB from the middle, and 4KB from the end of the original data, and the three segments are merged as sampled data.

[0040] For image data, the complete header of the raw data (containing EXIF ​​information) and the 2KB compressed data block immediately following the header are read and combined as sampled data. The file header carries all the metadata information of the image, and the compressed data block contains frequency domain coefficients that reflect the visual content, allowing feature extraction without decoding to a complete pixel image.

[0041] For audio data, the first preset size of the tag information is read, and the starting frame of the audio data is located. Several seconds of audio frame data are then read. The tag information provides basic audio attributes, and the first few seconds of audio frames are sufficient to extract stable acoustic features. For example, for audio data, the first 4KB of the ID3 tag is read, and the first audio frame is located and at least 2 seconds of frame data are read. These two are then combined as sampled data. If the sampled audio frame data is less than 2 seconds, all available frame data is used.

[0042] For video data, a preset size of bytes is read from the file header and footer to extract keyframe data and a portion of the audio frame data. The header and footer carry the video's metadata, keyframes provide visual scene features, and audio frames support acoustic feature extraction. For example, the header and footer of the original data (4KB and 4KB respectively) are read, and approximately 8KB of the first keyframe is extracted as sampled data. If an audio track is detected, an additional 2 seconds of data from the first audio frame is extracted, bringing the total sampled data to no more than 16KB. If the sampled keyframes and audio frames are not aligned in timestamp, features can be extracted independently and then synthesized using a weighted summation method, or the timestamp offset between the keyframes and audio frames can be recorded as auxiliary features and incorporated into the structured modality.

[0043] For structured data types, a preset size of bytes is read from the beginning and end of the file. The beginning contains information such as header field names, key names, and data types, while the end carries statistical information such as row group metadata. For example, the first 2KB and the last 4KB of the original data are read and combined to form sampled data.

[0044] For compressed files and container-type data, a preset size of bytes is read from the end of the file. This area typically contains a central directory or structural index information, allowing direct access to structured information such as the internal subfile list, compression ratio, and offset. For example, for compressed file-type data, the last 12KB of the original data's central directory area is read as sampled data.

[0045] In one possible implementation, for streaming data that cannot be accessed randomly, the data stream is read sequentially in bytes of a preset size, and the total amount of data segments read does not exceed the preset size, such as 12KB.

[0046] In one possible implementation, when the original data is encrypted, the encrypted data is not sampled; instead, an exception is thrown directly, and no sampled data is generated.

[0047] In this embodiment, the sampling strategy is matched according to the format of the original data, and the most representative data segments are extracted from the original data of different formats as sampling data. This ensures that the read data segments can represent the content and structural features of the data of that type, while keeping the data volume within a preset range. This can significantly reduce the amount of data processing and improve processing efficiency.

[0048] Step S103: Determine the degree of matching between the sampled data and multiple predetermined modes based on the byte distribution characteristics of the sampled data.

[0049] Multiple predefined modalities refer to the types of data content representation, including text modality, image modality, audio modality, and structured modality. Video data, after sampling, is covered by both image and audio modalities. Compressed and containerized data, after sampling, are parsed into structured records and categorized as structured data modality. Structured modality refers to the modality type corresponding to structured data. Structured data refers to data with a clearly defined structure, whose content can be parsed into fields / keys / rows / columns / nested records, including but not limited to data in formats such as CSV, JSON, Parquet, and XML.

[0050] The byte distribution characteristics of sampled data refer to the distribution pattern of byte values ​​within the range of values. The degree of matching refers to the strength of the association between the sampled data and each predetermined modality, used to indicate the probability that the sampled data belongs to each predetermined modality.

[0051] In one possible implementation, anomaly detection is performed on the sampled data (e.g., detecting whether all sampled data are zero values ​​or all are the same byte values; detecting whether the byte entropy falls within the normal range; detecting whether there are known file corruption markers, etc.). If the sampled data passes the anomaly detection, the degree of matching between the sampled data and multiple predetermined modalities is determined based on the byte distribution characteristics of the sampled data.

[0052] As one possible implementation of this application Figure 3 This application provides a specific implementation flow for determining the degree of matching between sampled data and multiple predetermined modalities in an automatic metadata processing method, which is detailed below: B1: Truncate or pad the byte sequence of the sampled data to a preset length, and normalize each byte value to a preset numerical range to obtain a standardized sequence.

[0053] The distribution of byte values ​​in a byte sequence reflects the content characteristics of the original data. For example, the byte values ​​of text data are concentrated within the encoding range corresponding to printable characters, the byte value distribution of image compressed data has a specific pattern, the byte value distribution of audio data is related to the statistical characteristics of sound wave sampling values, and the byte value distribution of structured data is related to the specific encoding format.

[0054] The preset length refers to the length of the standardized sequence that is set in advance. Since the byte sequence lengths obtained after sampling from raw data of different formats and sizes may be different, it is necessary to unify the byte sequences of different lengths to the same length in order to facilitate subsequent matching degree calculations based on data of a unified format.

[0055] When the actual byte length of the sampled data exceeds the preset length, only the first preset length of bytes is retained, and the excess portion is discarded, i.e., the byte sequence is truncated. When the actual byte length of the sampled data is less than the preset length, zero-value bytes are added to the end of the byte sequence to make the sequence length reach the preset length.

[0056] By normalizing to a preset numerical range, the values ​​of each byte in the byte sequence are mapped from their original range to the preset numerical range. The original value range of each byte is 0 to 255, and the preset numerical range can be 0 to 1. The normalization method can be to divide each byte value by 255, mapping the value to the range of 0 to 1. After truncation or zero-padding and normalization, a standardized sequence with a fixed length and a uniform numerical range is obtained.

[0057] B2: Based on the byte distribution characteristics of the standardized sequence, calculate the degree of matching between the sampled data and each predetermined modality.

[0058] In one possible implementation, the byte distribution characteristics of the normalized sequence are obtained by statistically analyzing the frequency of each byte value. Specifically, the proportion of each possible value (e.g., 0 to 255) appearing in the normalized sequence relative to the total length is calculated, resulting in a frequency distribution vector of length 256. This frequency distribution vector reflects the statistical characteristics of the sampled data and serves as the basis for determining the modality to which the data belongs.

[0059] When calculating the degree of matching between the sampled data and each predetermined modality, the byte distribution characteristics of the standardized sequence are compared with the typical byte distribution characteristics of each predetermined modality to obtain the degree of matching between the sampled data and each predetermined modality. The degree of matching can be expressed in probabilistic form, and the sum of the degree of matching values ​​for each predetermined modality is 1.

[0060] In one possible implementation, the typical byte distribution characteristics of each predetermined modality can be obtained in advance through statistical methods. Specifically, sample data of multiple known modalities are acquired, with no less than 1000 samples for each modality. The modality label of each sample data is determined according to its file format (e.g., JPEG format sample data is labeled as image modality, and MP3 format sample data is labeled as audio modality). The byte sequence is truncated or padded with zeros to a preset length and normalized according to the aforementioned B1 method to obtain the standardized sequence of each sample. The frequency distribution vector of each sample is obtained by statistically analyzing the proportion of the occurrence of each possible value (0 to 255) in the standardized sequence of each sample to the total length. For multiple samples under the same modality, the arithmetic mean of the frequency distribution vectors of each sample is calculated by dimension to obtain the typical byte distribution characteristics of each modality.

[0061] For example, the byte sequence of the sampled data is truncated or padded with zeros to a fixed length of N bytes, where N can be 8192. Then, each byte value is divided by 255 to obtain a normalized sequence. Based on this byte distribution characteristic, the matching degree of the sampled data belonging to the text modality, image modality, audio modality, and structured modality is calculated respectively. This matching degree can be expressed in probabilistic form. When the sampled data comes from a text file, its byte distribution characteristic shows a high proportion of printable characters, which is highly consistent with the typical distribution characteristics of the text modality. The calculated matching degree is 0.85 for the text modality, 0.05 for the image modality, 0.05 for the audio modality, and 0.05 for the structured modality. When the sampled data comes from a video file, since the video contains both image and audio components, its byte distribution characteristic comprehensively reflects the statistical characteristics of the image and audio. The calculated matching degree is 0.45 for the image modality, 0.40 for the audio modality, 0.10 for the text modality, and 0.05 for the structured modality.

[0062] The matching degree obtained through the above method is a soft classification result, meaning that a data point can simultaneously possess the possibility of belonging to multiple modalities, rather than being forcibly classified into a single modality. The soft classification method can more accurately reflect the multimodal characteristics of the data and improve the robustness of subsequent synthesis.

[0063] In one possible implementation, the degree of matching is calculated based on byte distribution characteristics and preset matching rules to determine the degree of matching between the sampled data and each predetermined modality.

[0064] In this embodiment, by determining the modality attribution of the sampled data at the content level, a basis is provided for determining the fusion weight of each modality feature description data in the subsequent synthesis process, thereby improving the accuracy of multimodal feature extraction.

[0065] Step S104: Based on the feature extraction rules corresponding to each predetermined modality, perform feature extraction on the sampled data to obtain feature description data for each predetermined modality.

[0066] Feature extraction rules refer to pre-defined processing methods for converting sampled data into feature description data specific to each predefined modality. Each predefined modality corresponds to a unique feature extraction rule: text modality corresponds to text feature extraction rules, image modality to image feature extraction rules, audio modality to audio feature extraction rules, and structured modality to structured data feature extraction rules. Each feature extraction rule independently extracts features from the sampled data, resulting in fixed-dimensional feature description data. For example, the feature description data obtained by each feature extraction rule has a 64-dimensional dimension.

[0067] In one possible implementation, the text feature extraction rules include: decoding and segmenting the sampled data, extracting semantic features of the text through word frequency statistics and keyword matching, and generating text feature description data. For example, the text feature extraction rules process the sampled data as follows: attempt to decode the sampled data using UTF-8 encoding; if decoding fails, revert to Latin-1 encoding; segment the decoded text into a word sequence using a preset vocabulary, and truncate or pad the sequence to a fixed length of 512; statistically analyze the frequency of each word in the sequence and the co-occurrence relationship between words, and generate 64-dimensional text feature description data based on the statistical results and preset feature extraction mapping rules.

[0068] In one possible implementation, the image feature extraction rules include: skipping the file header marker in the sampled data and directly extracting the byte stream of the compressed data block to form a one-dimensional sequence; performing multi-scale segmented statistics on the one-dimensional sequence, calculating the mean, variance, and gradient values ​​between adjacent bytes in each segment, extracting edge intensity features based on the gradient value distribution, and extracting texture complexity features based on the combination of mean and variance to obtain image feature description data without decoding the compressed data into pixel images. Specifically, multi-scale segmented statistics refers to segmenting the one-dimensional sequence at different granularities (e.g., 32 bytes, 64 bytes, and 128 bytes respectively), and calculating the mean, variance, and gradient statistics for each segment at each granularity; the mean reflects the average brightness level of each segment, the variance reflects the fluctuation of pixel values ​​within each segment, and the gradient value reflects the variation amplitude between adjacent bytes; edges correspond to regions with higher gradient values, and textures are manifested as periodic changes in the mean and variance. For example, the image feature extraction rule processes the sampled data as follows: skipping the header marker of the original data, directly extracting the byte stream of the compressed data block (for JPEG format, extracting the data block after the start scan marker; for PNG format, extracting the image data block; for image formats not explicitly listed, using the primary chunk / frame defined in the format specification as the extraction object; if the primary chunk cannot be located, directly using the complete byte stream of the sampled data for multi-scale segmented statistics), normalizing each byte to the interval between 0 and 1, forming a one-dimensional sequence of length 2048; performing segmented statistics on this one-dimensional sequence at segment granularities of 32 bytes, 64 bytes, and 128 bytes, respectively calculating the mean, variance, and gradient values ​​between adjacent bytes in each segment; combining and mapping the statistics obtained at each segment granularity into 64-dimensional image feature description data according to a preset feature extraction mapping rule.

[0069] In one possible implementation, the audio feature extraction rules include: parsing audio tag information to obtain the sampling rate and bit depth, decoding the audio frame into pulse code modulation data and resampling it, extracting acoustic features through statistical spectral distribution features, and generating audio feature description data. For example, the audio feature extraction rules process the sampled data as follows: parsing the ID3 tag to obtain the sampling rate and bit depth, decoding the first 2 seconds of audio frames into pulse code modulation format, resampling to 16kHz, and performing a short-time Fourier transform on the resampled data to obtain a spectrogram; if the sampled audio frame data is less than 2 seconds, all available frame data are used, and the duration ratio is recorded. This duration ratio is used as an adaptive adjustment factor for subsequent features (for example, this duration ratio can be used as a correction coefficient for the degree of audio modality matching. When the actual sampling duration is less than 2 seconds, the confidence of the degree of audio modality matching is reduced proportionally; or in the weighted summation, the duration ratio is directly multiplied by the degree of audio modality matching as an attenuation factor); statistically analyzing the energy distribution, peak frequency, and other statistics of each frequency band in the spectrogram, and generating 64-dimensional audio feature description data according to the preset feature extraction mapping rules.

[0070] In one possible implementation, the structured data feature extraction rules include: using a parser to automatically identify the format of the sampled data, extracting structural skeleton information from the sampled data, including column names or key names, data type distribution, nesting depth, and row number estimation, without relying on an external data schema file; and generating structured feature description data through rule encoding. For example, the structured data feature extraction rules generate 64-dimensional feature description data through the following rule encoding method: Column names or key names are hashed and mapped to 16 buckets, with each bucket using 1 bit to indicate the existence of a key name with a corresponding hash value, generating 16-dimensional features; the proportion of columns of basic types such as numeric, character, date, and boolean is statistically analyzed, with each type occupying 2 dimensions (one dimension being the proportion value, and the other an indicator of the type's existence), generating 16-dimensional features; structural complexity features are extracted (including normalized nesting depth, average number of columns or keys per row, estimated number of rows after logarithmic scaling, and proportion of null values), generating 16-dimensional features; one-hot encoding is performed on various common structured formats (including JSON, CSV or TSV, Parquet, Avro, XML, HTML tables, Excel, and Protobuf), generating 8-dimensional features; statistical features such as the average length and character entropy of column names or key names are extracted, generating 8-dimensional features; and the above features are concatenated to obtain 64-dimensional structured feature description data. When parsing fails, a zero vector is output and a warning is recorded.

[0071] In one possible implementation, for video data, the sampled data includes image keyframes and audio frames, which are processed by image feature extraction rules and audio feature extraction rules, respectively, to obtain feature description data of the image modality and feature description data of the audio modality.

[0072] In one possible implementation, for compressed and container-type data, the sampled data is the central directory area at the end, which contains structured record information such as internal file paths and sizes before and after compression. This information is processed by structured data feature extraction rules to obtain structured modality feature description data.

[0073] By using the feature extraction rules corresponding to each modality, feature description data for each modality is extracted from the sampled data, providing feature representations for each modality in subsequent multimodal feature synthesis.

[0074] Step S105: Based on the matching degree, the feature description data of the predetermined modality is synthesized to generate multimodal feature description data of the original data.

[0075] The matching degree includes the matching degree value between the sampled data and the predetermined modality. The feature description data of each predetermined modality is the feature description data determined according to the feature extraction rules corresponding to each predetermined modality. The dimension of each feature description data is fixed and uniform, for example, 64 dimensions.

[0076] In one possible implementation, the matching degree between the sampled data and each predetermined modality is used as a weighting coefficient to perform a weighted summation of the feature description data of the predetermined modality to obtain intermediate synthetic data; the intermediate synthetic data is then subjected to feature mapping processing to generate the multimodal feature description data.

[0077] In one possible implementation, the intermediate synthesis data is calculated according to the following formula (1): (1) in, Indicates the degree of text modality matching. Indicates the degree of matching of image modalities. Indicates the degree of audio modality matching. This indicates the degree of matching of structured modes; the sum of these four factors is 1. Feature description data representing text modalities Feature description data representing image modalities Feature description data representing audio modalities Feature description data representing structured modes.

[0078] In this embodiment, the feature description data of each modality are weighted and summed using the matching degree value as the weight, so that the features of each modality contribute to the final feature description data according to their matching degree, thus avoiding the problem of overall feature distortion caused by a hard classification error of a single modality.

[0079] In one possible implementation, the intermediate synthesized data is processed by feature mapping to map it into feature description data of a preset dimension. This feature description data of the preset dimension is multimodal feature description data, which simultaneously contains the content attributes and structural attributes of the original data, and is a unified quantitative representation of the content and structural features of the original data.

[0080] In this embodiment, the feature mapping processing of the intermediate synthetic data includes: mapping the values ​​of each dimension in the intermediate synthetic data to multimodal feature description data of a preset dimension according to a preset expansion rule. For example, the intermediate synthetic data is 64-dimensional, the preset dimension is 256-dimensional, and the expansion rule includes: sequentially mapping the 64 dimensions of the intermediate synthetic data to the first 64 dimensions of the multimodal feature description data, and filling the remaining 192 dimensions of the multimodal feature description data with preset values; or, copying the values ​​of each dimension of the intermediate synthetic data to multiple corresponding dimensions of the multimodal feature description data. The weighted summed intermediate synthetic data is mapped to a preset unified feature space, so that the data after fusion of different modalities is still output with fixed dimensions.

[0081] In this embodiment, the feature description data of each modality is integrated into a unified multimodal feature description data, so that the feature description data can comprehensively reflect the feature information of the original data under different modalities. At the same time, the feature dimension is fixed to a unified dimension through feature mapping processing, which facilitates subsequent algorithm matching processing.

[0082] Step S106: Based on the multimodal feature description data, match it in the algorithm performance profile database to determine the target metadata processing algorithm that is suitable for the original data.

[0083] The algorithm performance profile database is a pre-built database used to record the historical processing performance of various candidate metadata processing algorithms on different feature categories. Specifically, the database records historical performance metrics for multiple candidate metadata processing algorithms on different feature categories. These historical performance metrics are quantitative data used to measure the processing effectiveness of candidate metadata processing algorithms on specific feature categories. Performance metrics include, but are not limited to, values ​​across multiple dimensions such as accuracy, recall, processing time, and memory usage.

[0084] The embodiments of this application match the corresponding processing algorithm according to the content characteristics of the original data, so as to realize the dynamic adaptation of the processing strategy and the data content.

[0085] For example, for simple plain text document-type raw data, if the algorithm performance profile database records that lightweight rule-based algorithms have shorter processing time and achieve the required extraction accuracy under the corresponding feature category, then the lightweight rule-based algorithm can be identified as the target metadata processing algorithm. For complex text-type raw data containing a large number of named entities, if the algorithm performance profile database records that learning-based processing algorithms have higher extraction accuracy under the corresponding feature category, then the learning-based processing algorithm can be identified as the target metadata processing algorithm.

[0086] As one possible implementation of this application Figure 4 This application illustrates a specific implementation flow of constructing an algorithm performance profile database in the metadata automatic processing method provided in the embodiments of this application, which is described in detail below: C1: Obtain the sample dataset. This sample dataset is a reference data set used to build the algorithm performance profile database.

[0087] The sample dataset contains multiple samples, and the source and modality of each sample are unrestricted. It can include text data, image data, audio data, structured data, compressed and containerized data, and video data. The size of the sample dataset can be determined based on the number of candidate metadata processing algorithms and the complexity of the feature space. For example, the sample dataset can be between 50,000 and 100,000 samples, and it should cover multiple data categories to ensure that the constructed performance profile database has generalization ability across different feature types of data.

[0088] C2: Extract multimodal feature description data for each sample in the sample dataset.

[0089] For each sample in the sample dataset, the process of format recognition, sampling, matching degree determination, feature extraction, and synthesis as described above is used to generate multimodal feature description data corresponding to that sample. This multimodal feature description data is fixed-dimensional data (e.g., 256 dimensions), which maps the original content and structural attributes of each sample to a unified feature space, enabling samples of different modalities to be quantized and represented in the same feature space. Each sample in the sample dataset corresponds to a feature point located in the feature space.

[0090] For example, the sample dataset contains a plain text document, a JPEG image, and a WAV audio file. These three samples are processed through format recognition, sampling, matching degree determination, feature extraction, and synthesis to generate three unified-dimensional multimodal feature description data, representing the content and structural attributes of each sample. By converting each sample in the sample dataset into a unified-dimensional feature description data representation, each sample can be measured and compared within the same feature space.

[0091] C3: Cluster the multimodal feature description data, and divide the feature space into multiple feature categories based on the clustering results.

[0092] Clustering can employ the K-means clustering algorithm, which aims to divide feature points in the feature space into multiple groups based on their spatial proximity. Each group corresponds to a feature category. Samples within the same feature category are close to each other in the feature space, indicating that these samples are similar in content and structural attributes. After clustering, each group constitutes a feature category, and all feature categories collectively cover the entire feature space.

[0093] In one possible implementation, the number of clusters can be determined using a data-driven approach. For example, different numbers of clusters can be scanned within a preset range, which can be from 5 to 30. A reasonable number of clusters can be automatically determined through clustering effect evaluation indicators. Specifically, the sum of squared errors and the average profile coefficient corresponding to each value are calculated. The number of clusters is determined by comprehensively considering the inflection point of the sum of squared errors curve and the maximum value of the average profile coefficient, while ensuring that the number of samples in each feature category is not less than a preset threshold (e.g., 20).

[0094] After determining the number of feature categories, the feature space is divided into multiple feature categories. Each feature category has a feature category center, which is the arithmetic mean vector of the multimodal feature description data of all samples in that feature category. Each feature category contains several samples. Samples within each feature category have high similarity in content features, while samples between different feature categories have differences in content features.

[0095] For example, clustering can divide the feature space into multiple feature categories, which may include categories such as "high-resolution natural images," "plain text documents," "voice recordings," and "spreadsheets." Each feature category corresponds to a region in the feature space, and this region has a corresponding feature category center.

[0096] C4: For each feature category and each candidate metadata processing algorithm, calculate the average performance index of the candidate metadata processing algorithm on the feature category, and store the average performance index as the historical performance index of the candidate metadata processing algorithm on the feature category.

[0097] Candidate metadata processing algorithms refer to various algorithms that can be scheduled to perform metadata extraction processing, including rule-based processing algorithms, statistical method-based processing algorithms, and learning-based processing algorithms.

[0098] In one possible implementation, rule-based processing algorithms may include regular expression-based field extraction algorithms and file header parsing-based metadata extraction algorithms; statistical method-based processing algorithms may include word frequency statistics-based keyword extraction algorithms and field distribution statistics-based structured data summary generation algorithms; and learning-based processing algorithms may include classifier-based name and place name recognition algorithms and cluster analysis-based scene classification algorithms.

[0099] For example, the regular expression-based field extraction algorithm performs the following processing on the text data: the text data is segmented into multiple sentences according to punctuation marks and whitespace characters; for each sentence, a preset list of place names is used for matching, the list of place names includes the names of administrative divisions at all levels such as provinces, cities, districts, counties, and townships, as well as common natural place names; the string in the sentence that successfully matches the list of place names is output as the place name extraction result.

[0100] For example, a metadata extraction algorithm based on file header parsing performs the following processing on image data: reads the header bytes of the image file; locates the metadata fields in the header according to the image format, for JPEG format, locates the application segment after the start marker, and for PNG format, locates the text block before the image data block; parses the shooting device information (such as the “Make” field and the “Model” field) and resolution information (such as the “ImageWidth” field and the “ImageLength” field) from the located metadata fields; and outputs the parsing results as shooting device and resolution metadata.

[0101] For example, the keyword extraction algorithm based on word frequency statistics performs the following processing on text data: the text data is segmented to obtain a word sequence; stop words are removed from the word sequence, and the stop words include words in a preset stop word list; the number of times the remaining word elements appear in the word sequence is counted to obtain the word frequency of each word element; the word elements are sorted in descending order of word frequency, and the word elements that are in the first preset number (e.g., the first 10) after sorting are taken as keywords.

[0102] For example, the structured data summary generation algorithm based on field distribution statistics performs the following processing on structured data: reads the field names and corresponding data types of the structured data; for numeric fields, calculates the mean and maximum values ​​of all values ​​in the field; for character fields, calculates the frequency of each character value in the field, and takes the character value with the highest frequency as the representative value of the field; and combines the field names, data types, and corresponding statistical results into a structured summary output.

[0103] When constructing the performance profile database, for each feature category and each candidate metadata processing algorithm, the candidate metadata processing algorithm is applied to each sample belonging to that feature category to obtain the metadata extraction performance metrics for each sample. Performance metrics include accuracy, recall, processing time, and memory usage. For accuracy and recall, a field-level calculation principle is adopted. For discrete output, accuracy is the number of correctly predicted fields divided by the total number of predicted output fields, and recall is the number of correctly predicted fields divided by the total number of labeled ground truth fields. For continuous output, the text similarity between the output result and the labeled ground truth is calculated. When the similarity reaches a preset threshold, it is considered correct, and accuracy and recall are calculated in the same way. When processing multiple types of metadata at once, a weighted average accuracy and weighted average recall are calculated using the proportion of fields in each category as weights, serving as the overall accuracy and overall recall for that processing.

[0104] For multiple samples belonging to the same feature category, the arithmetic mean of the performance metrics obtained by the candidate algorithm for each sample is calculated to obtain the mean performance metric of the candidate algorithm for that feature category. For example, if a feature category contains 100 samples, and the accuracy of a candidate algorithm for processing these 100 samples is 0.75, 0.80, 0.70, ..., then the sum of these 100 accuracy values ​​and the result divided by 100 gives the mean accuracy of the candidate algorithm for that feature category. Similarly, the mean recall, mean processing time, and mean memory usage are calculated separately.

[0105] The calculated average performance metric is stored as the historical performance metric of the candidate metadata processing algorithm for that feature category in the algorithm performance profile database. After traversing all feature categories and all candidate algorithms, the algorithm performance profile database records the historical performance at the intersection of each feature category and each candidate algorithm, forming a performance profile matrix with "feature category" and "candidate algorithm" as two-dimensional indexes.

[0106] In one possible implementation, candidate metadata processing algorithms are incorporated in a plug-in manner, allowing new algorithms to be added at any time. When adding a new candidate metadata processing algorithm, it is not necessary to re-evaluate all samples. Instead, a small number of representative samples are extracted from each feature category to run the new algorithm, calculate the average performance metric for each feature category, and write it to the database. This completes the initialization of the performance profile for the new algorithm.

[0107] In one possible implementation, during the initial construction phase, for each feature category, a preset number of representative samples are extracted from that feature category; for each candidate metadata processing algorithm, the candidate metadata processing algorithm is run on the representative samples of all feature categories to obtain the performance index of the candidate metadata processing algorithm on each feature category; if a candidate metadata processing algorithm does not obtain a performance index on a certain feature category, the average performance index of the candidate metadata processing algorithm on all feature categories is stored as the initial performance index on that feature category.

[0108] The average performance metrics are stored as historical performance indicators of candidate metadata processing algorithms on feature categories, forming an algorithm performance profile database. This database provides a quantitative reference for subsequent algorithm determination based on multimodal feature description data, enabling the algorithm determination process to refer to the real historical performance of each algorithm on similar data, rather than relying on preset static rules.

[0109] In this embodiment, the construction of the algorithm performance profile database first involves acquiring a sample dataset covering multiple modal types. Each sample is then transformed into multimodal feature description data of a unified dimension. Clustering is used to divide the feature space into multiple feature categories, ensuring that samples with similar content and structural attributes are grouped into the same category. Historical performance metrics are then calculated and stored for each feature category and each candidate metadata processing algorithm. By establishing a mapping relationship between feature space regions and historical algorithm performance, the expected performance of each candidate algorithm on data from different feature categories is recorded. During actual metadata extraction processing, the system can query similar feature categories based on the multimodal feature description data of the data to be processed, and then query the historical performance of each candidate algorithm within that feature category. This allows for prediction of the performance of each candidate algorithm without actual trial and error, providing a data foundation and quantitative basis for dynamically selecting appropriate metadata processing algorithms based on data content characteristics. This helps improve the accuracy and processing efficiency of algorithm selection during metadata extraction.

[0110] As one possible implementation of this application Figure 5 A specific implementation flow of step S106 in the automatic metadata processing method provided in this application embodiment is shown below: D1: Calculate the similarity between the multimodal feature description data and the center of each feature category in the algorithm performance profile database.

[0111] The algorithm performance profiling database contains multiple feature categories, each with a center. The center of each feature category is a representative vector obtained by clustering the multimodal feature description data of each sample in the sample dataset during the database construction. The center of a feature category is the mean vector of the multimodal feature description data of all samples within that category.

[0112] Similarity measures how close multimodal feature description data is to the centers of each feature category in the feature space. Similarity can be calculated using methods such as cosine similarity or Euclidean distance. A higher similarity between multimodal feature description data and the center of a particular feature category indicates that the content features of the original data are closer to the data type represented by that feature category.

[0113] In one possible implementation, the cosine similarity method is used to calculate the similarity: the multimodal feature description data and the center of a certain feature category are respectively normalized using the L2 norm, and then the dot product of the normalized vectors is calculated. The dot product value is the cosine similarity. The cosine similarity value ranges from -1 to 1. The closer the value is to 1, the more consistent the directions of the two in the feature space, that is, the higher the similarity between the original data and the center of the feature category.

[0114] D2: Based on the similarity, determine at least one target feature category from the feature categories.

[0115] The target feature category is one or more feature categories selected from all feature categories included in the algorithm performance profiling database based on their similarity to the multimodal feature description data, and which are similar to the original data content features. When determining the target feature category, a preset number of feature categories with the highest similarity ranking can be selected as the target feature categories, ordered from highest to lowest similarity. The preset number can be set according to actual needs, for example, selecting the top 3 or top 5 feature categories.

[0116] D3: Obtain the historical performance metrics of multiple candidate metadata processing algorithms recorded in the algorithm performance profile database on the at least one target feature category.

[0117] For each candidate metadata processing algorithm, its historical performance metrics on the target feature category are read from the algorithm performance profile database. If a candidate metadata processing algorithm has multiple historical performance records on a certain target feature category, its comprehensive value (such as the mean) is obtained.

[0118] D4: Determine the target metadata processing algorithm from the multiple candidate metadata processing algorithms based on the historical performance indicators.

[0119] Based on the historical performance metrics of each candidate metadata processing algorithm on the target feature category, a comprehensive evaluation is conducted to determine the most suitable candidate metadata processing algorithm for processing the current raw data as the target metadata processing algorithm.

[0120] In one possible implementation, the comprehensive evaluation includes: weighting the historical performance metrics of candidate metadata processing algorithms on each target feature category using the similarity between each target feature category and the multimodal feature description data as weights, to obtain the prediction score of the candidate metadata processing algorithm; and determining the candidate metadata processing algorithm with the highest prediction score as the target metadata processing algorithm. The historical performance metrics may include multiple dimensions, and the weight of each dimension can be configured according to the application scenario. For example, in scenarios emphasizing accuracy, the weight of the accuracy dimension is increased; in scenarios emphasizing real-time performance, the weight of the processing time dimension is increased.

[0121] The target metadata processing algorithm is determined based on historical performance indicators, so that the algorithm determination process is driven by the real content characteristics of the original data and takes into account the historical performance of each candidate algorithm on similar data. This avoids the problems of accuracy loss and resource mismatch caused by the same algorithm processing data with different content in the same format.

[0122] In this embodiment, the similarity between the centers of the multimodal feature description data and the algorithm performance profile database is calculated to quantify the matching degree between the original data and each feature category. Then, based on the similarity, at least one target feature category that best matches the original data is determined from multiple feature categories to narrow down the subsequent query range. Historical performance indicators of each candidate metadata processing algorithm on these target feature categories are obtained, and historical performance data of each candidate algorithm on data types that match the content features of the original data are obtained. Based on a comprehensive evaluation of historical performance indicators, the target metadata processing algorithm with the best overall performance is determined from multiple candidate algorithms. This ensures that the determined target metadata processing algorithm is the algorithm with the best historical performance for the content features of the current original data. It realizes dynamic matching of the corresponding processing algorithm according to the content features of the data itself, avoiding the accuracy loss caused by different feature data being processed by the same algorithm under the static configuration method, and helps to improve the accuracy and processing efficiency of metadata extraction.

[0123] Step S107: Perform metadata extraction processing on the original data based on the target metadata processing algorithm to obtain metadata.

[0124] The target metadata processing algorithm is a defined metadata processing algorithm adapted to the characteristics of the original data content. This target metadata processing algorithm can be a rule-based extraction algorithm or an extraction algorithm based on statistical methods.

[0125] Metadata extraction processing refers to inputting complete raw data into a target metadata processing algorithm, which then parses and extracts information from the raw data, outputting structured metadata. Metadata is structured descriptive information automatically extracted from raw data. For text data, metadata may include named entities (such as names of people, places, and organizations), keywords, summaries, topic classifications, sentiment indicators, etc.; for image data, metadata may include the shooting device, resolution, color features, object detection results (object category and location), scene classification, etc.; for audio data, metadata may include sampling rate, duration, speech-to-text transcription, speaker identification, audio event classification (such as applause, alarms), etc.; for structured data, metadata may include field names, data type distribution, null value rate, number of rows or columns, statistical summaries (such as mean, maximum and minimum values), etc.

[0126] In one possible implementation, the metadata is output in a pre-defined unified data structure, including source information fields, content metadata fields, and extracted metadata fields, which facilitates subsequent data management, retrieval, and analysis.

[0127] In one possible implementation, the metadata output by the target metadata processing algorithm is encapsulated into a unified JSON structure. The JSON structure includes: a first top-level key for recording the source information of the original data; a second top-level key for recording the content metadata of the original data, wherein the content metadata contains at least one metadata object in the form of a list, and each metadata object contains a type field, a numeric field, and a confidence field; and a third top-level key for recording the metadata information of this metadata extraction and processing.

[0128] The embodiments of this application perform metadata extraction processing on the original data based on the target metadata processing algorithm to obtain accurate and complete metadata, and complete the end-to-end automatic processing from the original data to the structured metadata information.

[0129] As one possible implementation of this application, the algorithm performance profile database records historical performance metrics of multiple candidate metadata processing algorithms on different feature categories. After each metadata extraction and processing is completed, the database is updated using the actual performance information generated in this processing, so that the historical performance metrics can continuously reflect the latest performance of each algorithm, forming a closed-loop self-optimization mechanism. Figure 6 This application illustrates a specific implementation flow of updating the algorithm performance profile database in the metadata automatic processing method provided in the embodiments of this application, which is described in detail below: E1: Obtain the algorithm identifier of the target metadata processing algorithm, the multimodal feature description data, and the actual performance information of the target metadata processing algorithm in performing metadata extraction processing.

[0130] Algorithm identifiers are used to distinguish different candidate metadata processing algorithms. In the algorithm performance profile database, each candidate metadata processing algorithm corresponds to a unique algorithm identifier, which can be the algorithm name, version number, or other unique code. This identifier allows the database to locate the historical performance metrics of each algorithm across different feature categories. Actual performance information refers to data generated during the metadata extraction process of the target metadata processing algorithm on the current raw data, reflecting the algorithm's performance in this processing iteration. This actual performance information includes processing time, success or failure status, and subsequent feedback on the quality of the results. Processing time refers to the time elapsed from the start of metadata extraction to obtaining the metadata result in a single processing step, which can be recorded in milliseconds. The purpose of obtaining actual performance information is to use the actual performance data of this processing as a feedback signal, providing a data source for subsequently updating the algorithm's historical performance metrics across the corresponding feature categories.

[0131] In one possible implementation, result quality feedback information can be obtained in various ways. For example, users can correct or rate the automatically extracted metadata, and the difference between the corrected fields and the original results can be used as a quality signal; or the quality of metadata can be inferred from changes in metrics of downstream tasks; or automated verification methods can be used to periodically sample and evaluate test data with standard metadata, using the consistency verification results as a quality proxy indicator. The result quality feedback information obtained in the above ways can be expressed as metrics such as accuracy and recall for each field. Accuracy refers to the correctness of the extracted results, which can be determined by the ratio of the number of correctly predicted fields to the total number of predicted output fields, and recall is determined by the ratio of the number of correctly predicted fields to the total number of fields labeled with true values.

[0132] For example, when the target metadata processing algorithm is an image metadata extraction algorithm, obtain the algorithm identifier "IMG-EXT-V3", obtain the multimodal feature description data generated in this processing, and record the processing time, as well as the actual performance information such as the accuracy and recall of each field in the extraction result.

[0133] E2: Determine the feature category to which the multimodal feature description data belongs in the feature space.

[0134] The feature space is the space in which multimodal feature description data resides, and this space is pre-divided into multiple feature categories through clustering. Each feature category has a center, which is the arithmetic mean of the multimodal feature description data of all samples in that feature category.

[0135] The feature category to which the multimodal feature description data belongs in the feature space refers to the feature category that best matches the multimodal feature description data of the current original data among the multiple feature categories divided when constructing the algorithm performance profile database. By calculating the similarity between the multimodal feature description data and the centers of each feature category in the algorithm performance profile database, the feature category with the highest similarity is determined as the feature category to which the multimodal feature description data belongs.

[0136] For example, if the multimodal feature description data generated after feature extraction of a landscape photo has the highest center similarity with the feature category "high-resolution natural image" in the database, then "high-resolution natural image" is determined as the feature category to which the multimodal feature description data belongs.

[0137] By identifying the feature category, the performance feedback of this processing is accurately assigned to the corresponding feature region, so that the update of historical performance indicators can reflect the changes in the algorithm's performance on specific types of data, rather than updating a global indicator in a general way.

[0138] E3: Based on the actual performance information, update the historical performance indicators of the target metadata processing algorithm on the feature category in the algorithm performance profile database.

[0139] Using the actual performance information obtained in this process, the historical performance indicators of the target metadata processing algorithm stored in the algorithm performance profile database for the corresponding feature categories are refreshed and corrected, so that the refreshed and corrected historical performance indicators can comprehensively reflect the historical cumulative performance and recent performance.

[0140] In one possible implementation, the algorithm performance profile database can be updated using an exponential moving average method. Specifically, when new actual performance information is obtained, the historical performance indicators stored for that feature category by the target metadata processing algorithm are read. The historical performance indicators and the actual performance information are then weighted and summed according to a preset smoothing coefficient. The result of this weighted sum is stored in the algorithm performance profile database as the updated historical performance indicator. The preset smoothing coefficient can be set according to actual needs; for example, a smoothing coefficient of 0.1 indicates that the new data accounts for 0.1% of the updated indicator, and the original historical data accounts for 0.9%. This method can maintain historical statistical stability while allowing the performance indicators to respond quickly to recent changes.

[0141] For example, the image metadata extraction algorithm had an average historical accuracy of 85% in the "high-resolution natural image" feature category. The actual accuracy of this processing was 90%, and the smoothing coefficient was 0.1. Therefore, the updated average historical accuracy was 85% multiplied by 0.9 plus 90% multiplied by 0.1, which is 85.5%.

[0142] In this embodiment, by continuously updating the algorithm performance profile database with the actual performance information from each processing step, historical performance metrics can continuously reflect the latest performance of each candidate metadata processing algorithm on different types of data, ensuring that the algorithm determination strategy remains optimal as data and algorithms evolve. For example, when an algorithm is upgraded, its performance improvement in the new version will be gradually reflected in the historical performance metrics through successive feedback; when the data distribution changes, the actual performance changes of the algorithm in each category will also be captured and updated in a timely manner. This closed-loop update mechanism ensures that the historical performance metrics used in subsequent algorithm determination processes always remain consistent with actual performance, thereby continuously guaranteeing the accuracy of algorithm matching and further improving the accuracy of metadata extraction.

[0143] As can be seen from the above, in this embodiment, the original data to be processed is first acquired and its format is identified. Based on the identification result, a target sampling strategy matching the original data format is determined from multiple predetermined sampling strategies. Representative data segments are read from the original data as sampled data according to the target sampling strategy, which significantly reduces the amount of data to be processed and reduces computational overhead. Then, based on the byte distribution characteristics of the sampled data, the matching degree between the sampled data and multiple predetermined modalities is determined. Based on the feature extraction rules corresponding to each predetermined modality, feature extraction is performed on the sampled data to obtain feature description data for each modality, so that the content features under different modalities can be fully extracted. Then, based on the matching degree, the feature description data of each modality is synthesized to generate multimodal feature description data that uniformly represents the content and structural attributes of the original data. Then, based on the multimodal feature description data, matching is performed in the algorithm performance profile database to determine the target metadata processing algorithm that is suitable for the original data. This realizes matching the corresponding processing algorithm according to the content features of the data itself, avoiding the accuracy loss caused by different feature data being processed by the same algorithm. Finally, based on the target metadata processing algorithm, metadata extraction processing is performed on the original data to obtain metadata. The proposed solution achieves collaborative processing of data format recognition, adaptive sampling, multimodal feature synthesis, and dynamic algorithm matching, enabling refined processing of data with different features and effectively improving the accuracy and efficiency of metadata extraction.

[0144] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0145] Corresponding to the automatic metadata processing method described in the above embodiments, Figure 7 A structural block diagram of the metadata automatic processing device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0146] Reference Figure 7 The automatic metadata processing device includes: a raw data acquisition unit 71, a data sampling unit 72, a modality matching degree determination unit 73, a feature data extraction unit 74, a multimodal data synthesis unit 75, a target algorithm determination unit 76, and a metadata processing unit 77, wherein: Raw data acquisition unit 71 is used to acquire raw data to be processed; The data sampling unit 72 is used to perform format recognition on the raw data, determine the target sampling strategy from a variety of predetermined sampling strategies based on the format recognition result, and read data segments from the raw data as sampled data according to the target sampling strategy. The modality matching degree determination unit 73 is used to determine the matching degree between the sampled data and multiple predetermined modalities based on the byte distribution characteristics of the sampled data; The feature data extraction unit 74 is used to extract features from the sampled data according to the feature extraction rules corresponding to each predetermined mode, so as to obtain feature description data of each predetermined mode. The multimodal data synthesis unit 75 is used to synthesize the feature description data of the predetermined modality according to the matching degree to generate the multimodal feature description data of the original data. The target algorithm determination unit 76 is used to match the multimodal feature description data in the algorithm performance profile database to determine the target metadata processing algorithm that is suitable for the original data. Metadata processing unit 77 is used to perform metadata extraction processing on the original data based on the target metadata processing algorithm to obtain metadata.

[0147] As one possible implementation of this application, the step of format recognition of the original data and execution according to a hierarchical progressive strategy includes: Perform first-level identification, which includes identifying the format of the original data based on the characteristic bytes of the byte stream header of the original data; If the first level of identification fails to identify the format of the original data, then the second level of identification is performed, which includes identifying the format of the original data based on the file extension of the original data; If the second level of identification fails to identify the format of the original data, then the third level of identification is performed, which includes identifying the format of the original data based on the statistical characteristics of the byte stream of the original data.

[0148] As one possible implementation of this application, the modal matching degree determination unit 73 is specifically used for: The byte sequence of the sampled data is truncated or padded with zeros to a preset length, and each byte value is normalized to a preset numerical range to obtain a standardized sequence; Based on the byte distribution characteristics of the standardized sequence, the degree of matching between the sampled data and each predetermined modality is calculated.

[0149] As one possible implementation of this application, the multimodal data synthesis unit 75 is specifically used for: Using the degree of matching between the sampled data and each predetermined modality as a weighting coefficient, the feature description data of the predetermined modality are weighted and summed to obtain intermediate synthetic data; The intermediate synthesized data is subjected to feature mapping processing to generate the multimodal feature description data.

[0150] As one possible implementation of this application, the automatic metadata processing device includes a profile database construction unit, used for: Obtain the sample dataset; Extract multimodal feature description data for each sample in the sample dataset; The multimodal feature description data is clustered, and the feature space is divided into multiple feature categories based on the clustering results; For each feature category and each candidate metadata processing algorithm, the average performance index of the candidate metadata processing algorithm on the feature category is calculated, and the average performance index is stored as the historical performance index of the candidate metadata processing algorithm on the feature category.

[0151] As one possible implementation of this application, the target algorithm determination unit 76 includes: The similarity calculation module is used to calculate the similarity between the multimodal feature description data and the centers of each feature category in the algorithm performance profile database; A target category determination module is used to determine at least one target feature category from the feature categories based on the similarity. The historical indicator acquisition module is used to acquire the historical performance indicators of multiple candidate metadata processing algorithms recorded in the algorithm performance profile database on the at least one target feature category; The target algorithm determination module is used to determine the target metadata processing algorithm from the plurality of candidate metadata processing algorithms based on the historical performance indicators.

[0152] As one possible implementation of this application, the automatic metadata processing device further includes a database update unit, used for: Obtain the algorithm identifier of the target metadata processing algorithm, the multimodal feature description data, and the actual performance information of the target metadata processing algorithm in performing metadata extraction processing; Determine the feature category to which the multimodal feature description data belongs in the feature space; Based on the actual performance information, update the historical performance metrics of the target metadata processing algorithm on the feature category in the algorithm performance profile database.

[0153] As can be seen from the above, in this embodiment, the original data to be processed is first acquired and its format is identified. Based on the identification result, a target sampling strategy matching the original data format is determined from multiple predetermined sampling strategies. Representative data segments are read from the original data as sampled data according to the target sampling strategy, which significantly reduces the amount of data to be processed and reduces computational overhead. Then, based on the byte distribution characteristics of the sampled data, the matching degree between the sampled data and multiple predetermined modalities is determined. Based on the feature extraction rules corresponding to each predetermined modality, feature extraction is performed on the sampled data to obtain feature description data for each modality, so that the content features under different modalities can be fully extracted. Then, based on the matching degree, the feature description data of each modality is synthesized to generate multimodal feature description data that uniformly represents the content and structural attributes of the original data. Then, based on the multimodal feature description data, matching is performed in the algorithm performance profile database to determine the target metadata processing algorithm that is suitable for the original data. This realizes matching the corresponding processing algorithm according to the content features of the data itself, avoiding the accuracy loss caused by different feature data being processed by the same algorithm. Finally, based on the target metadata processing algorithm, metadata extraction processing is performed on the original data to obtain metadata. The proposed solution achieves collaborative processing of data format recognition, adaptive sampling, multimodal feature synthesis, and dynamic algorithm matching, enabling refined processing of data with different features and effectively improving the accuracy and efficiency of metadata extraction.

[0154] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0155] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements... Figures 1 to 6 The steps of any automatic metadata processing method.

[0156] This application embodiment also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements... Figures 1 to 6 The steps of any automatic metadata processing method.

[0157] This application also provides a computer program product that, when run on a terminal device, causes the terminal device to execute the implementation of... Figures 1 to 6 The steps of any automatic metadata processing method.

[0158] Figure 8 This is a schematic diagram of a terminal device provided in an embodiment of this application. For example... Figure 8 As shown, the terminal device 8 in this embodiment includes: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. When the processor 80 executes the computer program 82, it implements the steps in the various metadata automatic processing method embodiments described above, for example... Figure 1 Steps S101 to S107 are shown. Alternatively, when the processor 80 executes the computer program 82, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 7 The functions of units 71 to 77 are shown.

[0159] For example, the computer program 82 may be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete this application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program 82 in the terminal device 8.

[0160] The terminal device 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8 This is merely an example of terminal device 8 and does not constitute a limitation on terminal device 8. It may include more or fewer components than shown, or combine certain components, or different components. For example, terminal device 8 may also include input / output devices, network access devices, buses, etc.

[0161] The processor 80 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0162] The memory 81 can be an internal storage unit of the terminal device 8, such as a hard disk or memory of the terminal device 8. The memory 81 can also be an external storage device of the terminal device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 8. Furthermore, the memory 81 can include both internal and external storage units of the terminal device 8. The memory 81 is used to store the computer program and other programs and data required by the terminal device. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0163] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0164] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0165] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0166] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for automatic metadata processing, characterized in that, include: Obtain the raw data to be processed; The original data is format-identified, and a target sampling strategy is determined from multiple predetermined sampling strategies based on the format identification result. Data segments are then read from the original data as sampled data according to the target sampling strategy. Based on the byte distribution characteristics of the sampled data, the degree of matching between the sampled data and multiple predetermined modalities is determined; Based on the feature extraction rules corresponding to each predetermined modality, feature extraction is performed on the sampled data to obtain feature description data for each predetermined modality; Based on the matching degree, the feature description data of the predetermined modality is synthesized to generate multimodal feature description data of the original data; Based on the multimodal feature description data, a matching algorithm performance profile database is performed to determine the target metadata processing algorithm that is suitable for the original data; Metadata is extracted from the original data based on the target metadata processing algorithm.

2. The method according to claim 1, characterized in that, The process of recognizing the format of the raw data and executing it according to a hierarchical progressive strategy includes: Perform first-level identification, which includes identifying the format of the original data based on the characteristic bytes of the byte stream header of the original data; If the first level of identification fails to identify the format of the original data, then the second level of identification is performed, which includes identifying the format of the original data based on the file extension of the original data; If the second level of identification fails to identify the format of the original data, then the third level of identification is performed, which includes identifying the format of the original data based on the statistical characteristics of the byte stream of the original data.

3. The method according to claim 1, characterized in that, The step of determining the degree of matching between the sampled data and multiple predetermined modalities based on the byte distribution characteristics of the sampled data includes: The byte sequence of the sampled data is truncated or padded with zeros to a preset length, and each byte value is normalized to a preset numerical range to obtain a standardized sequence; Based on the byte distribution characteristics of the standardized sequence, the degree of matching between the sampled data and each predetermined modality is calculated.

4. The method according to claim 1, characterized in that, The step of synthesizing the feature description data of the predetermined modality based on the matching degree to generate multimodal feature description data of the original data includes: Using the degree of matching between the sampled data and each predetermined modality as a weighting coefficient, the feature description data of the predetermined modality are weighted and summed to obtain intermediate synthetic data; The intermediate synthesized data is subjected to feature mapping processing to generate the multimodal feature description data.

5. The method according to claim 1, characterized in that, The algorithm performance profile database is constructed in the following way: Obtain the sample dataset; Extract multimodal feature description data for each sample in the sample dataset; The multimodal feature description data is clustered, and the feature space is divided into multiple feature categories based on the clustering results; For each feature category and each candidate metadata processing algorithm, the average performance index of the candidate metadata processing algorithm on the feature category is calculated, and the average performance index is stored as the historical performance index of the candidate metadata processing algorithm on the feature category.

6. The method according to claim 1, characterized in that, The process of matching the multimodal feature description data with an algorithm performance profile database to determine a target metadata processing algorithm suitable for the original data includes: Calculate the similarity between the multimodal feature description data and the center of each feature category in the algorithm performance profile database; Based on the similarity, at least one target feature category is determined from the feature categories; Obtain the historical performance metrics of multiple candidate metadata processing algorithms recorded in the algorithm performance profile database on the at least one target feature category; Based on the historical performance metrics, the target metadata processing algorithm is determined from the plurality of candidate metadata processing algorithms.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obtain the algorithm identifier of the target metadata processing algorithm, the multimodal feature description data, and the actual performance information of the target metadata processing algorithm in performing metadata extraction processing; Determine the feature category to which the multimodal feature description data belongs in the feature space; Based on the actual performance information, update the historical performance metrics of the target metadata processing algorithm on the feature category in the algorithm performance profile database.

8. An automatic metadata processing device, characterized in that, include: The raw data acquisition unit is used to acquire the raw data to be processed. The data sampling unit is used to perform format recognition on the raw data, determine the target sampling strategy from a variety of predetermined sampling strategies based on the format recognition result, and read data segments from the raw data as sampled data according to the target sampling strategy. A modality matching degree determination unit is used to determine the matching degree between the sampled data and multiple predetermined modalities based on the byte distribution characteristics of the sampled data; The feature data extraction unit is used to extract features from the sampled data according to the feature extraction rules corresponding to each predetermined mode, so as to obtain feature description data of each predetermined mode; A multimodal data synthesis unit is used to synthesize the feature description data of the predetermined modality according to the matching degree, and generate multimodal feature description data of the original data. The target algorithm determination unit is used to match the multimodal feature description data in the algorithm performance profile database to determine the target metadata processing algorithm that is suitable for the original data. The metadata processing unit is used to perform metadata extraction processing on the original data based on the target metadata processing algorithm to obtain metadata.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the automatic metadata processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the automatic metadata processing method as described in any one of claims 1 to 7.

Images