Document analysis methods, apparatus, equipment, and storage media

Abstract

This disclosure provides a document analysis method, apparatus, device, and storage medium. This disclosure relates to the field of big data, particularly to the technical fields of large models and data filtering. The specific implementation scheme is as follows: A large model is used to analyze decision reference information to determine a document filtering method and the filtering parameters required for that method; based on the document filtering method and the filtering parameters, a coarse screening of a mixed document set is performed to obtain a first document set; finely screened documents are selected from the first document set; related documents of the finely screened documents are selected from the mixed document set to obtain a document set to be described, with the finely screened documents as the core; and descriptive information for the document set to be described is generated based on the large model. In this embodiment, the large model can flexibly determine the document filtering method, filter out the required document set to be described, and provide rich descriptions.

Description

Technical Field

[0001] This disclosure relates to the field of big data, particularly to technical fields such as large models and data filtering. Background Technology

[0002] With the increasing volume of information, extracting the desired content from massive amounts of documents becomes an extremely complex task. Related technologies may require defining a series of data filtering rules, and then matching the required data from the massive dataset according to these predefined rules. The resulting data is then simply merged together, providing only a general, simplified description. Summary of the Invention

[0003] This disclosure provides a document analysis method, apparatus, device, and storage medium.

[0004] According to one aspect of this disclosure, a document analysis method is provided, comprising:

[0005] Large-scale model analysis is used to determine decision-making reference information, and to determine the document selection method and the selection parameters required for the document selection method.

[0006] Based on the file filtering method and filtering parameters, the mixed file set is coarsely filtered to obtain the first file set;

[0007] Select the finely filtered files from the first set of files;

[0008] The associated files of the finely screened files are filtered out from the mixed file set to obtain a set of files to be described with the finely screened files as the core;

[0009] Descriptive information for a set of files to be described is generated based on a large model.

[0010] According to another aspect of this disclosure, a document analysis apparatus is provided, comprising:

[0011] The decision module is used to analyze decision reference information using a large model and determine the document filtering method and the filtering parameters required for the document filtering method.

[0012] The coarse screening module is used to perform coarse screening on the mixed file set based on the file screening method and screening parameters to obtain the first file set.

[0013] The fine screening module is used to select finely screened files from the first file set;

[0014] The determination module is used to filter out the associated files of the refined files from the mixed file set, and obtain the set of files to be described with the refined files as the core;

[0015] The generation module is used to generate descriptive information for a set of files to be described based on a large model.

[0016] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0017] At least one processor; and

[0018] The memory is communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0020] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0021] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0022] In this embodiment of the disclosure, a flexible decision-making method for file filtering can be made through a large model, which can filter out the required set of files to be described and provide rich descriptions.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0024] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0025] Figure 1 This is a schematic flowchart of a document analysis method according to an embodiment of the present disclosure;

[0026] Figure 2 This is a schematic diagram of the screening process for a fine-screen document according to an embodiment of the present disclosure;

[0027] Figure 3 This is a flowchart illustrating the process of obtaining a set of files to be described, with the selected files as the core, according to an embodiment of this disclosure.

[0028] Figure 4 This is a schematic flowchart for generating description information of a set of files to be described according to an embodiment of the present disclosure;

[0029] Figure 5 This is the intended process for fine-tuning a large model according to an embodiment of this disclosure;

[0030] Figure 6 This is the intended flow of generating text by taking an image analysis example according to an embodiment of the present disclosure;

[0031] Figure 7 This is a schematic diagram of an application scenario according to an embodiment of the present disclosure;

[0032] Figure 8 This is a schematic diagram of a possible application interface according to an embodiment of the present disclosure;

[0033] Figure 9 This is a schematic diagram of the structure of a document analysis apparatus according to an embodiment of the present disclosure;

[0034] Figure 10 This is a block diagram of an electronic device used to implement the document analysis method of the embodiments of this disclosure. Detailed Implementation

[0035] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0036] The terms “first,” “second,” etc., used in this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0037] The data processing methods in related technologies are not very flexible and cannot meet the requirements. In view of this, this disclosure proposes a document analysis method that uses the powerful reasoning and understanding capabilities of large models to help filter documents and provide appropriate descriptions.

[0038] Large models, in computer science, generally refer to models with a large number of parameters or complex structures. These models can be used for various tasks, such as image recognition and natural language processing. For example, large language models are a common type of large model.

[0039] The large model used in this embodiment can be a large language model, which refers to a deep learning model trained on a large amount of text data that can generate natural language text or understand the meaning of language text. Large language models can handle various natural language tasks, such as text classification, question answering, and dialogue.

[0040] like Figure 1 The diagram shown is a flowchart illustrating the document analysis method provided in this embodiment of the disclosure, including the following:

[0041] S101 uses a large model to analyze decision reference information and determines the document filtering method and the filtering parameters required for the document filtering method.

[0042] This system allows for the pre-setting of multiple file filtering methods to construct a set of filtering methods. For example, different file filtering methods can be set for different metadata of a file. The same file filtering method can filter files based on one or more pieces of metadata. Metadata may include, for example, file size, file time information, file publication location, file type, and key elements contained in the file (such as scenic spots, people, events, knowledge points), etc. Filtering parameters are the parameters required by the filtering method. Filtering parameters can be defined based on metadata; for example, filtering based on time, filtering based on location, or even filtering based on multiple pieces of metadata.

[0043] During implementation, the large model can determine the document filtering method and parameters from a pre-set set of filtering methods based on decision reference information, enabling it to filter out the documents that the target audience needs or is interested in. Decision reference information helps the large model understand which content is critical, desirable, or potentially of interest, which helps it make reasonable decisions and filter out appropriate documents.

[0044] The files in this disclosure can be of any form, such as images, multimedia, web pages, publicly available blog posts, text files, etc. Any information carrier format, including images and / or text content, can be processed as a file in this disclosure.

[0045] Taking images as an example, a large model may consider various possible subjects such as the time the image was taken, the location of the image, and key subjects within the image (such as plants, animals, people, buildings, and scenic spots).

[0046] Understandably, different target groups, due to their varying focuses or interests, can be addressed using a large model to produce different decisions. This allows for personalized decision outcomes, improving the alignment between the decisions and the target group's needs.

[0047] Furthermore, embodiments of this disclosure can proactively generate filtering methods by leveraging the reasoning capabilities and understanding of target objects within a large model. This further enhances the flexibility of file filtering and increases the diversity of the final generated content.

[0048] It should be noted that all information involving user privacy is collected in strict compliance with relevant laws and policies, and with the user's informed consent. While protecting the security of users' personal data, it will not be used for any unauthorized purposes.

[0049] S102, based on the file filtering method and filtering parameters, perform coarse filtering on the mixed file set to obtain the first file set.

[0050] That is, based on the filtering parameters, files that meet the requirements of the file filtering method are filtered from the mixed file set to obtain the first file set.

[0051] S103, Select the finely filtered files from the first set of files.

[0052] This can be understood as obtaining a first set of files through coarse screening, and then obtaining finely screened files based on this step. This allows for progressive screening of the required files, so that when multiple files are ultimately retained as finely screened files, each finely screened file has strong representativeness and is unlikely to be duplicated.

[0053] S104: Filter out the associated files of the finely screened files from the mixed file set to obtain a set of files to be described with the finely screened files as the core.

[0054] In this process, based on the similarity of various features, feature combinations, key elements, etc. of the finely screened documents, similar documents or documents with strong correlations can be selected from the mixed document set as related documents of the finely screened documents.

[0055] Taking plant and animal images as an example, other images that also contain at least some of the plants and animals in the finely screened files are selected as associated files.

[0056] Taking a knowledge point as an example, files containing extended content and detailed descriptions of that knowledge point are selected as associated files.

[0057] The specific association rules can be predefined, and different association rules can be set for different types of files. This disclosure does not limit this.

[0058] S105, Generate description information for the set of files to be described based on the large model.

[0059] Large language models have document comprehension capabilities, and can comprehensively consider various information from different documents to provide rich descriptions.

[0060] In this embodiment, the large model, constrained by decision reference information, automatically provides appropriate file filtering methods and parameters based on its strong knowledge background and reasoning capabilities. This facilitates coarse screening of mixed file sets, improving the flexibility and accuracy of file filtering. Furthermore, fine screening further filters high-quality files from the coarsely screened files, making it easier to select a set of related files to be described from the mixed file set based on these finely screened files. Then, the large model organizes the set of files to be described, providing descriptive information. In this embodiment, constrained by decision reference information, a set of files to be described with certain relationships is automatically obtained through coarse screening, fine screening, and related file analysis. The entire process improves the flexibility and efficiency of file analysis. In addition, the large model can generate rich descriptions based on relationships, generating diverse content that fits the theme of the files to be described, thereby improving the quality of the generated content and avoiding limitations on simple, general descriptions.

[0061] In this embodiment of the disclosure, the decision reference information may include at least one of the following:

[0062] (1) The target object’s annotation information on the file, wherein the annotation information is used to indicate whether to pay attention to the corresponding file.

[0063] For example, annotation information is used to describe the target object's filtering intent for files. If the target object can specify that a certain type or type of file should be kept, then the files that need to be kept can be filtered out. As another example, the target object can specify that a certain type or type of file should be filtered out.

[0064] For example, when working with files, the target audience can perform actions such as saving or categorizing them. This action indicates the target audience's annotation of the file, thus obtaining annotation information. Files in categories that the target audience is not interested in can be removed. The target audience can also indicate whether they are interested in a file, requiring a focus on files that the target audience is interested in and the removal of files that are not. Regardless of the specific action, it is sufficient if it reflects the target audience's preferences or needs.

[0065] (2) Contextual information of the environment in which the target object is located.

[0066] For example, the context of the target object's environment can be the target object's current location information, the special meaning of the current time, such as January 1st being New Year's Day, or a certain anniversary of the target object.

[0067] For example, with text-based documents, one can determine the target's focus based on at least one round of conversation. For instance, when the target is organizing knowledge points related to technology A, the explanations of those points, application examples, and underlying principles can be used as contextual information.

[0068] (3) Requirements for the file describing the target object.

[0069] In some embodiments, the target object can directly input its specific requirements into the large model. The large model understands the requirements and finds files that meet them. For example, filtering New Year photos from an album, filtering instructional instructions on photography techniques from a knowledge base, or filtering events from a specific time period among many events.

[0070] In this embodiment of the disclosure, by analyzing the target object's annotation information on the file, the potential interests and needs of the target object can be understood, thereby providing a more personalized file decision-making reference. By considering the context of the target object's environment, decision-making services tailored to the current situation can be provided. Based on the file requirements described by the target object, their specific file needs can be better met, improving the target object's satisfaction.

[0071] In implementation, the process of fine-tuning from the first file set can be tailored to the specific needs of the application; this disclosure does not limit this approach. To facilitate understanding and accurate selection of the finely screened files, this disclosure provides a scheme for selecting files based on data filtering and / or data sorting. In implementation, data filtering employs a metadata filtering strategy to filter out files whose metadata does not meet the requirements. Data sorting can be based on the file scoring results. Specifically, a scoring-based filtering strategy can be used to filter out files whose scoring results do not meet the expected results.

[0072] During the fine screening process, data filtering can be performed on the first set of files (i.e., the coarse-screened files) first, followed by data sorting to obtain the fine-screened files; alternatively, data sorting can be performed first, followed by data filtering. It's even possible to perform both data filtering and data sorting operations simultaneously on the first set of files, and then filter out unsuitable files based on the results of both operations. Regardless of the method used, the fine-screened files can be obtained from the remaining files.

[0073] This disclosure embodiment uses metadata filtering and data scoring results to measure file quality from multiple dimensions, so as to accurately select high-quality files.

[0074] After obtaining the refined files, data filtering operations are performed to obtain files associated with the refined files, thus constructing a set of files to be described. Then, based on the data generation method of the large model, descriptive information for the set of files to be described is obtained. Furthermore, regarding the large model, to adapt it to the application scenarios of this disclosure embodiment, the large model has been fine-tuned; the implementation of the large model fine-tuning will be briefly described later.

[0075] The following sections provide a detailed description of the data filtering, data sorting, data selection, data generation, and large model fine-tuning mentioned above.

[0076] 1. Data Filtering

[0077] When processing a large number of files, preliminary screening based on the file filtering methods and parameters provided by the large model can avoid wasting resources on unnecessary files, making the entire workflow smoother and more efficient. The purpose of coarse screening is to recall as many relevant files as possible to improve the recall rate, but the quality and specific content of the files are not of great concern. To further filter out the required files, the files can be further filtered at the metadata granularity, which can be implemented as follows:

[0078] Step A1: Obtain the metadata of the files in the first file set.

[0079] During implementation, if some files are filtered out based on the scoring results first, then only the metadata of the remaining files needs to be obtained for data filtering. If data filtering is performed first, and then scoring is performed, then the metadata of all files in the first file set needs to be obtained.

[0080] Step A2: Based on the metadata filtering strategy, filter out files whose metadata does not meet the requirements from the first file set.

[0081] For example, when the first set of documents consists of knowledge documents, the metadata can include the document's author, publication year, journal name, citation count, keywords, and technical field. For instance, if only articles published within the last five years are of interest, the metadata can be defined as publication date; based on this metadata filtering strategy, articles published more than five years ago can be filtered out. As another example, if only articles by a specific author and journal are of interest, the metadata can be defined as specifying the author and journal, thereby filtering out articles from authors and journals not specified.

[0082] For example, the first document set contains 100 articles, but some of them were published in earlier years or in journals we don't pay much attention to. By setting a meta-information filtering strategy, requiring articles to be published after 2019 and from a specified set of top-tier journals, articles that do not meet the requirements can be automatically filtered out, leaving only about 70 articles. This filters out some ineligible documents and improves the efficiency of subsequent processing.

[0083] In this embodiment, by first obtaining the file's metadata and applying a filtering strategy, irrelevant or unsuitable files can be accurately and quickly excluded at the metadata granularity, reducing resource consumption in subsequent analysis and processing. Metadata provides key file attributes, and filtering strategies based on these attributes ensure that the remaining files more accurately meet the requirements, thereby improving the accuracy of the results. Through the meta-filtering strategy, the focus can be concentrated on the most valuable files, avoiding wasting computational resources on irrelevant or low-value files.

[0084] In some embodiments, the meta-information used by the meta-information filtering strategy can be determined based on the analysis of decision reference information from a large model.

[0085] Furthermore, contextual information about the target object's environment can be used as baseline information. A reasonable metadata filtering strategy can then be generated based on this. This can be implemented as follows: determining the baseline information and target information of a preset type associated with the baseline information; generating the required core metadata based on the target information of the preset type to obtain the metadata filtering strategy; wherein, this metadata filtering strategy is used to retain files containing the core metadata.

[0086] During implementation, the context information can be determined based on the actual application scenario.

[0087] For example, in some scenarios, when using a large model to help a target audience organize information, the core points of interest to the target audience can be determined based on the dialogue history of multi-turn conversations, word frequency analysis, or intent recognition, serving as baseline information. For instance, if the target audience is interested in technology A, then the terminology related to technology A can be used as baseline information to identify relevant individuals, journals, etc., and these can be designated as target information of a preset type. During implementation, a knowledge graph can be used to identify nodes associated with the baseline information, and these associated nodes can then be used as target information of a preset type. In this application scenario, generating core metadata constrained by target information can be understood as treating the baseline and target information as key information that needs to be protected in a document, and using the generated filtering strategy to filter out unnecessary documents as much as possible.

[0088] For example, in other scenarios, when organizing images, the current date, the date the image was taken, and the geographical location can be used as baseline information. For historical events, the anniversary can be used as baseline information to facilitate the organization of appropriate images. This is particularly useful when journalists compile images from interviews and create accompanying text.

[0089] Furthermore, in other scenarios, if it is necessary to organize photos into an album, the target's current location and time can be used as the base information. When the base information is a commemorative holiday or anniversary, it can help the target to filter out old photos from the same location and / or time period, and organize the album around a specific theme. Of course, whether to use the method provided in this disclosure to organize the album is up to the target's needs. If the target agrees to access the corresponding album and enables the function, and provided that relevant laws and regulations are met, a memory album can be organized for the target at a suitable location and / or on a suitable date.

[0090] In this embodiment, by determining baseline information and preset event types, a reasonable metadata filtering strategy can be decided based on the current situation, thereby quickly identifying and filtering key information, reducing interference from irrelevant information, and improving the quality and efficiency of data filtering. By generating the required core metadata, a metadata filtering strategy for retaining files containing the core metadata can be obtained, ensuring the accuracy and completeness of the dataset, thereby improving the overall quality of data filtering.

[0091] 2. Data sorting

[0092] As described above, this embodiment supports fine screening using a scoring method. Specifically, files can be scored based on a scoring model. To improve the quality of fine screening, files within the first file set can be scored based on multiple scoring models to obtain final scores for multiple files. Based on the final scores of each file, files whose scoring results do not meet the expected results are filtered out.

[0093] In this embodiment of the disclosure, after filtering out some files based on the data filtering strategy, multiple scoring models are sampled to score each file in the first set of filtered files, and then the core files that meet the current situation are selected based on the scoring results.

[0094] Among the various scoring models, at least one scoring model is based on the first key information that matches the target object. This first key information is used to characterize the target object's target needs, which can be used to describe the target object's preferences and concerns.

[0095] In this embodiment of the disclosure, by employing multiple scoring models, documents can be evaluated from different angles and dimensions, thereby improving the accuracy and comprehensiveness of the scoring and more fully reflecting the quality and value of the documents. Screening based on the final score allows for more accurate identification of documents that match the key information of the target object, improving the efficiency and accuracy of the screening process.

[0096] Specifically, in order to filter out high-quality files that meet current needs, the implementation of scoring each file in the first file set based on multiple scoring models is as follows: Figure 2 As shown:

[0097] S201, based on the first scoring model of file quality, scores the files in the first file set and obtains the first score of multiple files in the first file set.

[0098] File quality refers to a specific feature, attribute, or a combination of features and attributes. For example, for images, file quality could be image integrity, image clarity, etc. For text files, file quality can be determined based on key elements, such as keyword coverage in a keyword set, coverage of preset metadata, file authority, publication time, etc.

[0099] S202, based on the first score, select the second set of files.

[0100] During implementation, based on the first score, the files in the first file set are sorted, and the top-ranked parts are selected as the second file set, thereby reducing the size of the file set and improving the processing efficiency of subsequent operations.

[0101] S203, using a large model to score the second set of documents based on the first key information that matches the target object, and obtain the second score of each document in the second set of documents; where the higher the degree of matching with the first key information, the higher the second score of the document.

[0102] In one possible implementation, the first key information of the target object can be obtained based on the target object's preference information and the context information of the target object's environment. For example, if the target object is more interested in a certain type of file, then when scoring the second set of files, the files that are closer to that type of file will receive higher scores.

[0103] S204, a second scoring model trained based on the second key information of the target object scores the second set of files to obtain the third score of each file in the second set of files.

[0104] This second key information is used to characterize the operational features of the target object. For example, the second scoring model is trained using the interactive history behavior features of the target object. This interactive history can be the interactive behavior with the set of files to be described generated by the method provided in this embodiment, such as whether it is browsed, the browsing duration, likes, favorites, downloads, etc. Alternatively, it can be the operational history behavior features of the target object on other existing file resources.

[0105] S205, perform a weighted sum of the first score, second score, and third score of each file in the second file set to obtain the final score of each file in the second file set.

[0106] The first, second, and third scores of each file in the second file set are assigned corresponding weights, and then weighted and summed to obtain the final score. Over time, each model can be optimized based on the target object. Therefore, by analyzing the model's performance and the interactive behavior feedback of the target object, the weight ratios corresponding to each score can be continuously adjusted, thereby improving the efficiency of the screening process.

[0107] In this disclosed implementation, by combining multiple scoring models, the quality of documents can be more accurately assessed, and the degree of matching with the needs of the target audience can be evaluated. Through an automated scoring and filtering process, the most suitable documents can be found more quickly when processing a large number of documents, improving efficiency. By comprehensively considering scores from multiple dimensions, the value or suitability of each document can be more accurately evaluated, leading to more optimized decisions.

[0108] During implementation, regardless of the filtering strategy employed, the remaining files can be individually designated as refined screening files. For example, if data filtering is performed before data sorting, multiple intermediate files are selected from the second file set based on the final scores of each file, and these intermediate files are then designated as refined screening files. That is, if multiple files remain, each file can serve as a separate refined screening file, forming a set of files to be described centered around that refined screening file, and generating the final descriptive information. This allows for the generation of multiple sets of files to be described for the target audience to view, improving their experience.

[0109] In addition, in some other embodiments, the associated files selected based on each fine-screen file can be aggregated and then reclassified by the large model to obtain a set of files to be described corresponding to at least one category.

[0110] 3. Data Filtering

[0111] Based on the preceding description, the selected files are key files that meet the current requirements. Therefore, further data filtering can be used to obtain the associated files of the selected files, and this can be used to construct the set of files to be described. To filter out the associated files of the selected files from the mixed file set, resulting in a set of files to be described centered around the selected files, the following operations are performed for each candidate file in the mixed file set. The specific implementation steps are as follows: Figure 3 As shown:

[0112] S301, obtain the metadata of candidate files and the metadata of the refined files.

[0113] S302, determine the correlation between the metadata of candidate documents and the metadata of the refined documents.

[0114] S303: If the relevance is greater than a preset threshold, the candidate files are assigned to the set of files to be described.

[0115] S304: Filter out candidate files if the relevance is less than or equal to a preset threshold.

[0116] During implementation, for each candidate file, its metadata needs to be extracted. The metadata can be determined based on actual needs, and this embodiment does not limit its scope. For example, metadata refers to information about the file, describing its basic characteristics, status, attributes, etc. Specifically, it may include filename, creation date, modification date, file size, file type, location, and may even include a summary and keywords. These keywords can be extracted from the file; for example, image recognition models can identify keywords from images, and for text files, keywords can be extracted. Simultaneously, it is also necessary to obtain metadata of the same type from the refined files. The metadata of the refined files represents the characteristics of the specific type or topic of the files required.

[0117] By using some file comparison or text matching algorithms to compare the similarity between the metadata of the two, the correlation between the candidate file and the screened file can be calculated.

[0118] If the correlation between a candidate file and a screened file exceeds a preset threshold, the candidate file can be considered to belong to the specific type or topic being searched for, and it will be included in the set of files to be described. Conversely, if the correlation between a candidate file and a screened file is lower than or equal to the preset threshold, then the candidate file may not belong to the set of files to be described centered on the screened files, meaning its correlation is low, and it can be filtered out without further processing.

[0119] In this embodiment of the disclosure, by comparing the correlation between the metadata of candidate files and the carefully selected files, files related to a specific topic or content represented by the carefully selected files can be efficiently filtered from a large number of files. Related files can be easily and quickly filtered out by setting a preset threshold.

[0120] 4. Data Generation

[0121] For each carefully selected file, descriptive information for a set of files to be described is generated based on the large model, with that model at its core. The specific implementation steps are as follows: Figure 4 As shown:

[0122] S401 uses a large model to analyze the relationships between files in the file set to be described, and outputs the final file set in the file set to be described that has the relationships.

[0123] During implementation, feature analysis, feature transformation, and feature generation are performed on each file in the set of documents to be described to obtain more feature information. Then, a large model is used to analyze the feature relationships between each file. These relationships could include, for example, the travel history of the same tourist attraction, a summary of the development of the same event, or a brief introduction and application of the same knowledge point.

[0124] Feature analysis can be understood as extracting features from the content of a file. For example, for image / video files, image features are extracted. For text files, text features are extracted. Through feature analysis, key information can be extracted for analysis by large models.

[0125] Feature transformation can be understood as converting features that large language models cannot directly process into information formats that large language models can process. For example, location information, whose original data format might be latitude and longitude, can be converted into a textual description of a point of interest or city. Similarly, time information can be converted from year-removed descriptions into textual representations. Furthermore, extracted textual and image features can be further transformed into features that large language models can understand through a representation transformation neural network.

[0126] Feature generation refers to deriving more related features from known features using a large model. In practice, related features can be derived based on feature types (such as time, location, and event), or corresponding related features can be obtained from a pre-built feature map, thus generating the derived related features. For example, features related to time information can be derived based on that time information; specifically, for Qixi Festival (Chinese Valentine's Day), vocabulary related to Valentine's Day can be added.

[0127] The large model understands and mines files with certain relationships based on the features of the input. During implementation, the number of files in the final file set can be limited to display the most important and core content.

[0128] S402, with relationships as its core theme, uses a large model to generate descriptive information for the final file set.

[0129] Relationships can be one-dimensional or multi-dimensional. Taking photo filtering in an album as an example, with time as the single-dimensional relationship, such as a photo set of New Year's photos from each year, the generated description might be, "Last year's New Year was filled with festive spirit at home; this year's New Year was still wonderful in an unfamiliar city." With time and location as multi-dimensional relationships, such as a photo set of a city today, the generated description might be, "This morning, I enjoyed a delicious breakfast in a hutong in a certain city; this afternoon, the sunlight in the courtyard was still warm; as night fell, the commercial street was illuminated by colorful neon lights, becoming vibrant."

[0130] In this embodiment of the disclosure, by using a large model to analyze the relationships between files in the file set to be described, and taking the relationships as the core theme, the large model generates descriptive information for the file set to be described. This can generate clear and fluent copy around a core theme, combined with user-preferred language styles.

[0131] During implementation, for each screened file, a separate set of files to be described is generated, centered around that screened file. This results in a separate set of files to be described for each screened file. Then, descriptive information for each set of files to be described is generated based on the large model. For example, if images 1 and 2 are a set of images M selected based on the temporal features of screened file A, then there is a temporal correlation between these images. If images 2, 3, 4, and 5 are a set of images N selected based on the geographical location features of screened file A, then there is the same geographical location correlation between these images. The large model generates different descriptions for each set of images based on the different correlations between them.

[0132] It is understandable that for the same set of files to be described in the fine screening process, different relationships can be mined to obtain at least one final set of files, thereby generating different descriptions.

[0133] During implementation, it can be required that for the same set of files to be described, the main key relationships that fit the current situation be mined out, and descriptive information be generated accordingly.

[0134] Furthermore, when multiple carefully selected files exist, each file is used as a baseline to identify its associated files, resulting in a separate set of files to be described for each selected file. This leads to multiple sets of files to be described. These sets are then aggregated to form a single set of files to be classified. This single set is then reclassified using a large model based on the discovered associations, resulting in a final set of files for each category. For each final set, a description is generated based on its associations. Alternatively, the final set of files with the most relevant associations to the target object's context can be selected from multiple final sets to generate descriptive information.

[0135] For example, M carefully selected files yield M sets of files to be described. A large model is used to re-examine the relationships within these M sets and classify them into N categories. The relationships within each category are then matched with the contextual information of the target object's environment. The category with the highest matching degree is selected from the N categories to generate the final file set, with the relationship relationship as the core theme of the description.

[0136] Alternatively, you can select the top p categories with the highest matching degree to generate descriptions for the target object to view.

[0137] 5. Fine-tuning of the large model

[0138] To make reasonable use of the large model to complete the corresponding tasks, the model's output can be optimized through fine-tuning before using the large model to perform document analysis. Furthermore, after fine-tuning, during the process of using the large model to complete the document analysis tasks provided in the embodiments of this disclosure, the large model can be further iteratively optimized based on the real-time feedback from the target object.

[0139] In the initial stage, to adapt the large language model to the current task requirements and improve its performance in the current scenario, such as... Figure 5 As shown, positive and negative feedback data from the target object can be introduced. Furthermore, to protect the privacy of the target object, data anonymization is required first, removing data fields involving privacy. Then, a labeled dataset is constructed using the positive and negative feedback data and corresponding file information. The model is then fine-tuned based on the labeled dataset. The training task during model fine-tuning may include at least one of the following (…). Figure 5 (Not shown in the image):

[0140] 1) Determine appropriate document filtering methods and parameters based on decision reference information;

[0141] 2) Scoring of each file is based on the first key information of the target object and the information of each file in the second file set;

[0142] 3) Analyze the relationships between the files in the file set to be described;

[0143] 4) The ability to generate descriptive information based on relationships.

[0144] The model parameters are optimized by minimizing the loss function of the model for each task on the training set, thereby improving its performance on the current task. Based on this, such as... Figure 5 As shown, the fine-tuned model is evaluated and tested. This yields a final model that better suits the current task requirements.

[0145] For tasks lacking training sample sets, the large model can be optimized based on the feedback from the target object to the output results during its use, so that the large model can be continuously iterated and updated, generating a more accurate set of description files and their descriptive information.

[0146] In summary, taking image analysis as an example, the specific implementation flow of the document analysis method provided in this disclosure embodiment is as follows: Figure 6 As shown, it includes:

[0147] 1) Selection of data source: In this embodiment of the disclosure, the data source may include: image metadata, description information of the target object, and context information of the current environment of the target object.

[0148] 2) In the basic recall method stage, as described in the relevant part of step S101, multiple document screening methods can be pre-set to construct a set of screening methods so that the large model can infer and decide which document screening method is applicable. Figure 6 In the basic recall method stage, a set of filtering methods is constructed by acquiring preset file filtering methods such as recall rule 1, recall rule 2, and recall rule 3. Among them, recall rule 1 can be constructed based on the image's time information, recall rule 2 can be constructed based on the image's geographical location information, and recall rule 3 can be constructed based on the theme expressed by the image.

[0149] 3) In Figure 6 In the data retrieval phase, the large model, based on the set of filtering methods provided in the previous process and combined with decision reference information (such as information about the target object, including its description and the context of its current environment), determines the appropriate file filtering method and the filtering parameters required for that method. A coarse screening is then performed using the file filtering method and parameters to obtain the first set of files.

[0150] In order to adapt to the application of large models, when some information in the decision reference information is not text, it will be converted into natural language text and then used by the large model for analysis and decision-making, so as to give the specific screening method and screening parameters used for the current image recall.

[0151] 4) Data Filtering Stage: As described above, filtering can be performed on the recalled image data based on a metadata filtering strategy. That is, the first file set is filtered based on the metadata filtering strategy. In this stage, predefined metadata dimensions are used as the judgment criteria. For example... Figure 6 The diagram shows how filtering rules 1, 2, and 3 are used to traverse the first file set and filter out files that do not meet the requirements for metadata.

[0152] 5) In the data sorting stage, images can first be scored based on the first scoring model to obtain a first score (i.e., coarse ranking score), and then sorted based on the coarse ranking score to select the second set of files. Then, the second set of files is scored using the second scoring model to obtain a fine ranking score (i.e., the third score), and a large model is used to obtain a large model score (i.e., the second score) as a coarse ranking result. Finally, based on the coarse ranking score, fine ranking score, and large model score, a final score is obtained, and the second set of files is sorted based on the final score to select the finely filtered files.

[0153] 6) During the data filtering stage, for each image used as a fine-screening file, based on the image metadata of the fine-screening file, association data retrieval rules are used to recall images associated with the fine-screening file, thus completing the selection of associated data. That is, this achieves the process described above. Figure 3 The content.

[0154] 7) Finally, using a large model, feature analysis, feature transformation, and feature generation are performed on a set of images centered around the refined files to obtain more feature information. The large language model is then required to analyze this set of images, identify the relationships among them, and provide a small number of images with these relationships to obtain the final image set. Finally, the large model generates descriptive text for this set of images based on the core theme expressed by these relationships, and can further incorporate the preferences of the target audience.

[0155] In some embodiments, the solutions provided in this disclosure can be applied to recommendation systems. For example... Figure 7 As shown, after launching the application, its recommendation system can be triggered. The recommendation system can generate personalized materials. Specifically, based on the file analysis method provided in this embodiment, personalized materials (i.e., a set of related files and descriptions) are generated according to the target object's file metadata and information, and then fed back to the target object through the recommendation system. Taking a photo album system as an example, the final result fed back to the target object is as follows: Figure 8As shown, the method provided in this embodiment ultimately selects several pictures from the album, uses a large model to generate relevant titles, such as "New Year's Notes," with the corresponding description: "On October 14, 2022, I wandered through the autumn of **. The morning was rushed at the airport, the lunch break was quick, I met a handsome person, and in the afternoon I enjoyed the beautiful afterglow of the sky, and a lazy cat. Life is a complete picture composed of these fragments!"

[0156] Based on the same technical concept, this disclosure also provides a document analysis device 900, such as... Figure 9 As shown, it includes:

[0157] Decision module 901 is used to analyze decision reference information using a large model and determine the document filtering method and the filtering parameters required for that document filtering method.

[0158] The coarse screening module 902 is used to perform coarse screening on the mixed file set based on the file screening method and the screening parameters to obtain the first file set;

[0159] The fine screening module 903 is used to select finely screened files from the first file set;

[0160] The determination module 904 is used to filter out the associated files of the refined file from the mixed file set, so as to obtain the file set to be described with the refined file as the core;

[0161] The generation module 905 is used to generate description information for the set of files to be described based on the large model.

[0162] In some embodiments, the fine screening module is specifically used for:

[0163] Perform at least one of the following data filtering strategies to filter the first set of files in order to obtain the refined file from the remaining files:

[0164] Metadata filtering strategies are used to filter out files whose metadata does not meet the requirements;

[0165] The scoring-based filtering strategy is used to filter out files whose scoring results do not meet the expected results.

[0166] In some embodiments, the fine screening module is specifically used for:

[0167] Retrieve the metadata of the files in this first file set;

[0168] Based on this metadata filtering strategy, files whose metadata does not meet the requirements are filtered out from the first file set.

[0169] In some embodiments, the fine screening module includes:

[0170] The scoring unit is used to score the files in the first file set based on multiple scoring models to obtain the final scores of multiple files;

[0171] The filter is used to filter out files whose scores do not meet the expected results based on the final scores of each file.

[0172] In some embodiments, the scoring unit is specifically used for:

[0173] Based on the first scoring model of file quality, the files in the first file set are scored to obtain the first score of multiple files in the first file set;

[0174] Based on this first score, a second set of files is selected;

[0175] Using this large model, based on the first key information that matches the target object, a score is assigned to the second set of documents, resulting in a second score for each document within the second set; wherein, the higher the degree of matching with the first key information, the higher the second score of the document; and...

[0176] The second scoring model, trained based on the second key information of the target object, scores the second set of files to obtain the third score of each file in the second set of files.

[0177] The first, second, and third scores of each file in the second file set are weighted and summed to obtain the final score of each file in the second file set.

[0178] The first key information is used to characterize the target requirements of the target object; the second key information is used to characterize the operational characteristics of the target object.

[0179] In some embodiments, the fine screening module is specifically used to determine the remaining files as the fine screening files.

[0180] In some embodiments, the determining module is specifically configured to perform the following for each candidate file in the mixed file set:

[0181] Obtain the metadata of the candidate file and the metadata of the refined file;

[0182] Determine the correlation between the metadata of the candidate file and the metadata of the refined file;

[0183] If the correlation is greater than a preset threshold, the candidate file is assigned to the set of files to be described.

[0184] If the relevance is less than or equal to the preset threshold, the candidate file is filtered out.

[0185] In some embodiments, the decision reference information includes at least one of the following:

[0186] The target object's annotation information for the file, which is used to indicate whether the corresponding file is of interest;

[0187] The context information of the environment in which the target object exists;

[0188] The target object describes the file requirements.

[0189] In some embodiments, a policy generation module is also included, for:

[0190] Determine the baseline information, and the target information of a preset type associated with that baseline information;

[0191] Based on the target information of the preset type, the required core meta-information is generated, and the meta-information filtering strategy is obtained. The meta-information filtering strategy is used to retain files containing the core meta-information.

[0192] In some embodiments, the generation module is specifically used for:

[0193] This large model is used to analyze the relationships between files in the file set to be described, and outputs the final set of files in the file set to be described that have the relationships.

[0194] Using this relationship as the core theme, the descriptive information for the final file set is generated using this large model.

[0195] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0196] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0197] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0198] Figure 10A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0199] like Figure 10 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1002 or a computer program loaded from storage unit 1008 into random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0200] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0201] The computing unit 1001 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the file analysis method. For example, in some embodiments, the file analysis method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the file analysis method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the file analysis method by any other suitable means (e.g., by means of firmware).

[0202] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0203] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0204] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0205] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0206] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0207] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0208] Based on the aforementioned electronic devices, this disclosure also provides a vehicle that may include electronic devices, and may also include communication components, a display screen for realizing a human-machine interface, and an information collection device for collecting information about the surrounding environment, etc., wherein the communication components, the display screen, the information collection device and the electronic devices are communicatively connected.

[0209] According to embodiments of this disclosure, the electronic device can be integrated with the communication component, display screen, and information acquisition device, or it can be separately configured with the communication component, display screen, and information acquisition device.

[0210] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0211] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A document analysis method, comprising: A large model is used to analyze decision reference information and determine the document screening method and the screening parameters required for the document screening method. Based on the file filtering method and the filtering parameters, the mixed file set is coarsely filtered to obtain the first file set; Select the finely filtered files from the first file set; From the mixed file set, the associated files of the finely screened files are filtered out to obtain a set of files to be described with the finely screened files as the core; Based on the large model, generate description information for the set of files to be described; The decision reference information includes at least one of the following: The annotation information of the target object on the file, wherein the annotation information is used to indicate whether the corresponding file is of interest; The context information of the environment in which the target object is located; The file requirements described by the target object; The step of generating description information for the set of files to be described based on the large model includes: For each file in the set of files to be described, feature analysis, feature transformation, and feature generation are performed to extract the related feature information that the large model can understand. Based on the aforementioned association feature information, the large model is used to analyze the potential association relationships between files in the file set to be described, and the final file set with the aforementioned association relationships is output from the file set to be described. Using the aforementioned relationships as the core theme, the large model is used to generate personalized description information for the final file set.

2. The method according to claim 1, wherein, The step of selecting refined files from the first file set includes: Perform at least one of the following data filtering strategies to filter the first set of files to obtain the refined files from the remaining files: Metadata filtering strategies are used to filter out files whose metadata does not meet the requirements; The scoring-based filtering strategy is used to filter out files whose scoring results do not meet the expected results.

3. The method according to claim 2, wherein, Based on the metadata filtering strategy, data filtering is performed on the first file set, including: Obtain the metadata of the files in the first file set; Based on the metadata filtering strategy, files whose metadata does not meet the requirements are filtered out from the first file set.

4. The method according to claim 2, wherein, The filtering strategy based on the scoring results filters the first file set, including: The files in the first file set are scored based on multiple scoring models to obtain the final scores of multiple files; Based on the final score of each file, files whose scores do not meet the expected results are filtered out.

5. The method according to claim 4, wherein, The process of scoring files within the first file set based on multiple scoring models yields final scores for multiple files, including: Based on the first scoring model of file quality, the files in the first file set are scored to obtain the first score of multiple files in the first file set; Based on the first score, a second set of files is selected; Using the aforementioned large model and based on the first key information matching the target object, the second file set is scored to obtain a second score for each file within the second file set; wherein, the higher the degree of matching with the first key information, the higher the second score of the file; and... The second scoring model, trained based on the second key information of the target object, scores the second file set to obtain the third score of each file in the second file set. The first, second, and third scores of each file in the second file set are weighted and summed to obtain the final score of each file in the second file set. The first key information is used to characterize the target requirements of the target object; the second key information is used to characterize the operational characteristics of the target object.

6. The method according to any one of claims 2-5, wherein, The step of obtaining the refined screening file from the remaining files includes: The remaining files are then designated as the finely screened files.

7. The method according to claim 1, wherein, The step of filtering the associated files of the refined files from the mixed file set to obtain a set of files to be described with the refined files as the core includes: For each candidate file in the aforementioned mixed file set, execute the following respectively: Obtain the metadata of the candidate files and the metadata of the refined files; Determine the correlation between the metadata of the candidate files and the metadata of the refined screening files; If the correlation is greater than a preset threshold, the candidate files are assigned to the set of files to be described. If the relevance is less than or equal to the preset threshold, the candidate files are filtered out.

8. The method according to claim 2, further comprising: Determine the baseline information and the target information of a preset type associated with the baseline information; Based on the target information of the preset type, the required core meta-information is generated to obtain the meta-information filtering strategy, wherein the meta-information filtering strategy is used to retain files containing the core meta-information.

9. A document analysis device, comprising: The decision module is used to analyze decision reference information using a large model and determine the document filtering method and the filtering parameters required for the document filtering method. The coarse screening module is used to coarsely screen the mixed file set based on the file screening method and the screening parameters to obtain a first file set; The fine screening module is used to select finely screened files from the first file set; The determination module is used to filter out the associated files of the finely screened files from the mixed file set to obtain a set of files to be described with the finely screened files as the core. A generation module is used to generate description information for the set of files to be described based on the large model; The decision reference information includes at least one of the following: The target object's annotation information for the file, the annotation information being used to indicate whether to pay attention to the corresponding file; The context information of the environment in which the target object is located; The file requirements described by the target object; Specifically, the generation module is used for: For each file in the set of files to be described, feature analysis, feature transformation, and feature generation are performed to extract the related feature information that the large model can understand. Based on the aforementioned association feature information, the large model is used to analyze the potential association relationships between files in the file set to be described, and the final file set with the aforementioned association relationships is output from the file set to be described. With the aforementioned relationships as the core theme, the large model is used to generate personalized description information for the final file set, supporting semantic expression of single-dimensional or multi-dimensional relationships.

10. The apparatus according to claim 9, wherein the fine screening module is specifically used for: Perform at least one of the following data filtering strategies to filter the first set of files to obtain the refined files from the remaining files: Metadata filtering strategies are used to filter out files whose metadata does not meet the requirements; The scoring-based filtering strategy is used to filter out files whose scoring results do not meet the expected results.

11. The apparatus according to claim 10, wherein, The fine screening module is specifically used for: Obtain the metadata of the files in the first file set; Based on the metadata filtering strategy, files whose metadata does not meet the requirements are filtered out from the first file set.

12. The apparatus according to claim 10, wherein, The fine screening module includes: The scoring unit is used to score the files in the first file set based on multiple scoring models to obtain the final scores of multiple files; The filter is used to filter out files whose scores do not meet the expected results based on the final scores of each file.

13. The apparatus according to claim 12, wherein, The scoring unit is specifically used for: Based on the first scoring model of file quality, the files in the first file set are scored to obtain the first score of multiple files in the first file set; Based on the first score, a second set of files is selected; Using the aforementioned large model and based on the first key information matching the target object, the second file set is scored to obtain a second score for each file within the second file set; wherein, the higher the degree of matching with the first key information, the higher the second score of the file; and... The second scoring model, trained based on the second key information of the target object, scores the second file set to obtain the third score of each file in the second file set. The first, second, and third scores of each file in the second file set are weighted and summed to obtain the final score of each file in the second file set. The first key information is used to characterize the target requirements of the target object; the second key information is used to characterize the operational characteristics of the target object.

14. The apparatus according to any one of claims 10-13, wherein, The fine screening module is specifically used to identify the remaining files as the fine screening files.

15. The apparatus according to claim 9, wherein, The determining module is specifically used to perform the following for each candidate file in the mixed file set: Obtain the metadata of the candidate files and the metadata of the refined files; Determine the correlation between the metadata of the candidate files and the metadata of the refined screening files; If the correlation is greater than a preset threshold, the candidate files are assigned to the set of files to be described. If the relevance is less than or equal to the preset threshold, the candidate files are filtered out.

16. The apparatus of claim 10, further comprising a strategy generation module, configured to: Determine the baseline information and the target information of a preset type associated with the baseline information; Based on the target information of the preset type, the required core metadata is generated, and the metadata filtering strategy is obtained, wherein... The metadata filtering strategy is used to retain files containing the core metadata.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images