A high information density set generation method, device, equipment and storage medium

Abstract

The application discloses a high information density set generation method, device and equipment and a storage medium. The method comprises the following steps: obtaining original demand information for indicating generation of a high information density set. Based on a pre-trained text generation model and the original demand information, a target retrieval condition for data indexing is generated, wherein the text generation model comprises a text generation model corresponding to each retrieval stage, and the target retrieval condition at least comprises a primary retrieval condition, a secondary retrieval condition and a fine screening retrieval condition. Based on a pre-constructed index database, multi-level information retrieval is performed according to the target retrieval condition, and target retrieval data is obtained. Based on pre-set event information extraction rules, information fusion extraction is performed according to the target retrieval data, and a high information density set corresponding to the original demand information is generated. Through layer-by-layer filtering retrieval, the application improves the retrieval precision, performs information compression on the retrieval result, improves the information density, and meets the high-precision application demand.

Description

Technical Field

[0001] This invention relates to the field of information extraction and fusion technology, and in particular to a method, apparatus, device and storage medium for generating high information density sets. Background Technology

[0002] Currently, with the rapid development of the Internet and the Internet of Things, the scale of information data is growing exponentially, and its form has broken through the single text format, increasingly exhibiting diversified and integrated characteristics, covering multiple modalities such as images, audio, and video. In order to efficiently extract high-value information from massive and heterogeneous data, information retrieval and information set construction technologies have gradually become core research directions.

[0003] However, in the crucial steps of constructing high-information-density datasets, solutions for multimodal data processing and information set construction generally suffer from inconsistent processing mechanisms, low storage and indexing efficiency, and a lack of coarse-to-fine, hierarchical data management and filtering mechanisms. It is difficult to dynamically adjust the retrieval granularity according to different task requirements, resulting in low query efficiency, high result redundancy, and a significant waste of computing and storage resources.

[0004] These shortcomings result in high redundancy and insufficient information accuracy in the constructed high information density dataset, leading to poor performance in scenarios requiring rapid and accurate responses, such as public opinion analysis and intelligence assessment, thus limiting its practical application effectiveness. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and storage medium for generating high information density sets, which can filter and retrieve data layer by layer to improve retrieval accuracy, compress the retrieval results to increase information density, and meet the needs of high-precision applications.

[0006] According to one aspect of the present invention, a method for generating high information density sets is provided. The method includes: Obtain the original requirement information used to indicate the generation of a high information density set; Based on the pre-trained text generation model and the original demand information, target retrieval conditions for data indexing are generated. The text generation model includes text generation models corresponding to each level of retrieval stage, and the target retrieval conditions include at least primary retrieval conditions, secondary retrieval conditions, and refined retrieval conditions. Based on a pre-built index database, multi-level information retrieval is performed according to the target retrieval conditions to obtain target retrieval data, wherein the index database is constructed based on multimodal data; Based on pre-defined event information extraction rules, information is fused and extracted according to the target retrieval data to generate a high information density set corresponding to the original demand information.

[0007] According to another aspect of the present invention, an apparatus for generating high information density sets is provided. The apparatus includes: The demand information acquisition module is used to acquire the original demand information used to indicate the generation of a high information density set; The retrieval condition generation module is used to generate target retrieval conditions for data indexing based on a pre-trained text generation model and the original requirement information. The text generation model includes text generation models corresponding to each level of retrieval stage, and the target retrieval conditions include at least primary retrieval conditions, secondary retrieval conditions, and fine screening retrieval conditions. The data retrieval determination module is used to perform multi-level information retrieval based on a pre-built index database and according to the target retrieval conditions to obtain target retrieval data, wherein the index database is constructed based on multimodal data; The information extraction and fusion module is used to perform information fusion and extraction based on the target retrieval data according to the pre-set event information extraction rules, and generate a high information density set corresponding to the original demand information.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the high information density set generation method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the method for generating a high information density set as described in any embodiment of the present invention.

[0010] The technical solution of this invention involves acquiring original demand information to indicate the generation of a high information density set. Based on a pre-trained text generation model and the original demand information, target retrieval conditions for data indexing are generated. Based on a pre-built index database, multi-level information retrieval is performed according to the target retrieval conditions, achieving layer-by-layer filtering from massive data to high-value candidate data, improving retrieval performance and result accuracy, and obtaining high-value target retrieval data. Based on pre-defined event information extraction rules, information fusion and extraction are performed on the target retrieval data, achieving information compression and redundancy removal of the filtered high-value target retrieval data, increasing information density while retaining key information, and generating a high information density set that meets specific needs and satisfies high-precision application requirements.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a method for generating a high information density set according to an embodiment of the present invention; Figure 2 This is a structural diagram of a high information density set generation device according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device that implements the method for generating high information density sets according to embodiments of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Figure 1This is a flowchart illustrating a method for generating a high information density set according to an embodiment of the present invention. This embodiment is applicable to situations where high-density information is extracted based on information requirements. The method can be executed by a high information density set generation device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes: S101. Obtain the original requirement information used to indicate the generation of a high information density set.

[0017] The original requirement information refers to the construction requirements input from users or upstream business modules. This information may include elements such as keywords, topic scope, time interval, geographical scope, entity relationships, information granularity, and output scale. The original requirement information will serve as control parameters throughout the entire process, guiding the formulation and execution of strategies at each stage, including retrieval, filtering, and compression.

[0018] Specifically, the system obtains the original requirement information from the requirement input interface, which instructs the generation of a high information density set. For example, the original requirement information could be: "Please describe the national climate change situation in 2024 using approximately 100 words." In this original requirement information, the keywords are: 2024 weather, national, climate change; the time interval is: the 12 months of 2024; the geographical scope is: a preset region; the information granularity is: national overview level; and the output scale is: a 100-word summary.

[0019] S102. Based on the pre-trained text generation model and the original demand information, generate target retrieval conditions for data indexing.

[0020] It should be explained that the technical solution of the present invention adopts a multi-level retrieval method, from large-scale raw data to high information density datasets, to simulate the cognitive process of experts in the field in handling complex information tasks: first, extensive collection, then focusing on relevant fields, and finally precise identification.

[0021] The invention employs corresponding generative models and search criteria for each level of the retrieval process. For example, the invention may include three retrieval stages, and correspondingly, the text generation model includes text generation models for each level of the retrieval process. The target search criteria include at least primary search criteria, secondary search criteria, and refined search criteria. The text generation models for each level of the retrieval process are trained based on the search requirements at each level.

[0022] Specifically, for each retrieval stage, the corresponding text generation model and original requirement information are used to generate target retrieval conditions for data indexing at each stage.

[0023] For example, in the case of the initial retrieval stage, the original demand information is input into the text generation model corresponding to the initial retrieval stage to generate retrieval conditions, and the initial retrieval conditions are obtained based on the output of the text generation model corresponding to the initial retrieval stage.

[0024] In the secondary retrieval stage, the original demand information is input into the text generation model corresponding to the secondary retrieval stage to generate retrieval conditions, and the secondary retrieval conditions are obtained based on the output of the text generation model corresponding to the secondary retrieval stage.

[0025] In the fine screening and retrieval stage, the original demand information is input into the text generation model corresponding to the fine screening and retrieval stage to generate retrieval conditions, and the fine screening and retrieval conditions are obtained based on the output of the text generation model corresponding to the fine screening and retrieval stage.

[0026] S103. Based on a pre-built index database, perform multi-level information retrieval according to the target retrieval conditions to obtain target retrieval data. The index database is constructed based on multimodal data.

[0027] Specifically, by using the search conditions at each stage of the search process, multi-level information retrieval is performed sequentially in the index database, thereby enabling the retrieval of target data from massive amounts of multimodal data.

[0028] For example, the step of performing multi-level information retrieval based on a pre-built index database and the target retrieval conditions to obtain target retrieval data includes: performing a primary retrieval in the index database based on the primary retrieval conditions to obtain primary retrieval data; performing a secondary retrieval based on the primary retrieval data and the secondary retrieval conditions to obtain secondary retrieval data; and performing a refined retrieval based on the secondary retrieval data and the refined retrieval conditions to obtain target retrieval data. The screening precision of the refined search criteria is higher than that of the secondary search criteria, and the screening precision of the secondary search criteria is higher than that of the primary search criteria; the number of events in the primary search data is higher than that in the secondary search data, and the number of events in the secondary search data is higher than that in the target search data.

[0029] Specifically, primary search criteria are used to perform a primary search in the index database, returning a broad range of primary search data, potentially reaching millions in scale, to ensure comprehensive coverage of the relevant information domain. Secondary search criteria are then used to perform a secondary search from these millions of primary search data, yielding tens of thousands of secondary search data. Finally, refined search criteria are used to refine the search from these tens of thousands of secondary search data, resulting in hundreds of target search data.

[0030] For example, in this invention, the primary search criteria include keyword Boolean logic combinations, semantic vector features, and time or geographic filtering parameters; the secondary search criteria include semantic features of topic tags, key summaries, and named entities; and the refined search criteria include multidimensional features of similarity scores, weight thresholds, and time proximity.

[0031] Accordingly, the step of performing a secondary search based on the primary search data and the secondary search conditions to obtain secondary search data includes: extracting semantic features from the primary search data to obtain event semantic features of each primary search data, and constructing a material database based on the event semantic features of each primary search data; and performing a secondary search in the material database based on the secondary search conditions to obtain secondary search data.

[0032] The initial search data, after preliminary screening, is processed to retain richer semantic features, facilitating semantic matching and topic clustering. A hybrid architecture of inverted and forward indexes supports precise semantic retrieval, thereby obtaining a material database. Then, based on secondary search criteria, a secondary search is performed within this material database to obtain secondary search data.

[0033] Furthermore, during the fine-screening stage, a filtering database can be constructed by vectorizing all secondary search data. This facilitates approximate searches based on similarity and reduces irrelevant data from entering the final information set. By combining fine-screening criteria with multi-dimensional constraints such as similarity calculation, weighted ranking, and temporal proximity, highly relevant fine-screening results are obtained from the filtering database, yielding target search data within the top 100.

[0034] This invention, through its layered design, not only improves data retrieval speed but also significantly reduces computation and storage costs, and provides a high-quality data source for the subsequent construction of high-information-density sets.

[0035] For example, retrieval examples at each retrieval stage can be shown below: In the initial search stage, the initial search criteria can be: keyword combination: ("2024 weather" OR "climate change") AND "China"; time filter: January 1, 2024 – December 31, 2024; geographical filter: target region. The initial search process includes performing an initial search on an index database with hundreds of millions of entries, obtaining approximately 1,200,000 initial search data entries from across the country, including weather news, meteorological bureau announcements, research reports, and climate topics on social media.

[0036] In the secondary search stage, secondary search criteria can include: semantic features: seasonal changes, extreme weather types (high temperature, cold wave, rainstorm, typhoon, etc.), and degree of impact; and deep feature alignment: highly relevant to the theme of "national climate trends". The secondary search process includes: performing a secondary search on a database of millions of materials, and filtering out approximately 18,000 secondary search results with abstracts and subject tags (such as "heavy rainfall in the south", "cold wave in the north", and "typhoon impact").

[0037] During the fine screening and retrieval phase, the fine screening and retrieval criteria can be: priority based on time proximity (events occurring within the past year); similarity matching weight > 0.85; and sorting priority: scope of influence > media authority > event representativeness. The fine screening process includes: importing 18,000 candidate data into a screening database of tens of thousands (vectorized storage), and then, through fine screening search criteria, narrowing the final screening results down to 130 highly relevant materials, covering target search data such as major weather events and trends throughout the year.

[0038] In this invention, the index database construction process includes: Multimodal data is collected using preset collection methods. The modal types of the multimodal data include text, image, audio, and video data. For each modal type, modal data features are extracted according to the corresponding feature extraction rules. These features are then processed using feature encoding to generate data index information, and an index database is constructed based on this information. Multimodal data, such as text (news, papers, social media content, etc.), images (photos, screenshots, charts, etc.), audio (interview recordings, meeting minutes, etc.), and video (surveillance videos, video lectures, etc.), is collected through preset collection methods such as API interfaces, web crawlers, batch file import, and real-time streaming media access. In video / audio data, frame images and audio transcribed into text are automatically extracted to achieve a unified feature space mapping and obtain modal data features.

[0039] For example, for text data, a language model is used to extract semantic features such as topic, keywords, named entities, and sentiment. For image data, a multimodal large model is invoked to extract features such as object category, scene description, and image embedding vectors, and then a natural language description is generated. For audio / video data, speech recognition is used to convert audio into text, and semantic fusion is performed by combining the results of image frame analysis.

[0040] The extracted modal data features are encoded into a unified format (such as JSON) and index records are generated, supporting keyword retrieval and vector similarity retrieval to obtain data index information. This data index information is stored in an index database for initial retrieval across the entire dataset.

[0041] S104. Based on the pre-set event information extraction rules, information fusion and extraction are performed on the target retrieval data to generate a high information density set corresponding to the original demand information.

[0042] Among them, event information extraction rules can be data compression instructions. Event information extraction rules need to retain key information types (such as technical principles, key parameters, performance indicators, etc.), eliminate redundant information (such as duplicate backgrounds, non-core details), and provide precise instructions for compression processing, such as information fusion strategies (such as the merging method of multiple related records).

[0043] For example, event information extraction rules may include: retaining information such as time, location, major weather phenomena, and impacts (e.g., disaster type); removing information such as repetitive background descriptions, irrelevant minor events, and secondary details; and merging strategies such as combining similar events into a single sentence (e.g., merging multiple typhoon events into "coastal areas were hit by typhoons").

[0044] Specifically, based on the event information extraction rules, the target retrieval data is fused and refined to obtain a high information density set corresponding to the original demand information.

[0045] For example, the step of generating a high information density set corresponding to the original demand information by performing information fusion and extraction based on the target retrieval data according to the pre-set event information extraction rules includes: determining sample data and label data for model training according to the event information extraction rules; training a preset fusion and extraction model based on the sample data and the label data to obtain a target fusion and extraction model; and inputting the target retrieval data into the target fusion and extraction model to perform information fusion and extraction to generate a high information density set corresponding to the original demand information.

[0046] In this invention, sample data and label data for model training can be constructed using event information extraction rules. A preset fusion extraction model is then trained using the sample data and label data to obtain a target fusion extraction model (e.g., a large text generation model). This target fusion extraction model is then used to perform information compression and fusion on the refined target retrieval data to obtain a high information density set.

[0047] The compression and fusion process includes operations such as redundant information removal, similar information aggregation, and key information enhancement. While ensuring information integrity, it improves information density and structural compactness, so that the output results can meet the requirements of high-precision applications.

[0048] The resulting high-information-density dataset is output in structured data format, which can be directly used for high-precision tasks such as large-scale model prompt word examples, intelligence analysis, public opinion monitoring, and scientific research data screening. Output methods may include storing it in a high-density information database, providing access via an API interface, or generating downloadable data files to support real-time application and secondary utilization in multiple scenarios.

[0049] For example, the event information extraction rules were input into the target fusion and extraction model along with 130 target retrieval data entries. Redundancy removal, similar information aggregation, and key information enhancement were performed to compress the content to within 100 characters. The final high information density set generated was: "In the past year, many parts of my country experienced extreme weather, with frequent record-breaking high temperatures in summer, multiple heavy rainfalls in the south causing floods, typhoons hitting the coast, and frequent cold waves in the north during winter. Overall, the climate showed a trend of increasing volatility."

[0050] The technical solution of this invention involves acquiring original demand information to indicate the generation of a high information density set. Based on a pre-trained text generation model and the original demand information, target retrieval conditions for data indexing are generated. Based on a pre-built index database, multi-level information retrieval is performed according to the target retrieval conditions, achieving layer-by-layer filtering from massive data to high-value candidate data, improving retrieval performance and result accuracy, and obtaining high-value target retrieval data. Based on pre-defined event information extraction rules, information fusion and extraction are performed on the target retrieval data, achieving information compression and redundancy removal of the filtered high-value target retrieval data, increasing information density while retaining key information, and generating a high information density set that meets specific needs and satisfies high-precision application requirements.

[0051] Figure 2 This is a schematic diagram of a high information density set generation device provided in an embodiment of the present invention. Figure 2 As shown, the device includes: The demand information acquisition module 301 is used to acquire the original demand information used to indicate the generation of a high information density set; The retrieval condition generation module 302 is used to generate target retrieval conditions for data indexing based on a pre-trained text generation model and the original demand information. The text generation model includes text generation models corresponding to each level of retrieval stage, and the target retrieval conditions include at least primary retrieval conditions, secondary retrieval conditions, and fine screening retrieval conditions. The data retrieval determination module 303 is used to perform multi-level information retrieval based on a pre-built index database and according to the target retrieval conditions to obtain target retrieval data, wherein the index database is constructed based on multimodal data; The information extraction and fusion module 304 is used to perform information fusion and extraction based on the target retrieval data according to the pre-set event information extraction rules, and generate a high information density set corresponding to the original demand information.

[0052] Optionally, the screening precision of the refined search criteria is higher than that of the secondary search criteria, and the screening precision of the secondary search criteria is higher than that of the primary search criteria; The primary search criteria include keyword Boolean logic combinations, semantic vector features, and time or location filtering parameters. The secondary search criteria include topic tags, key summaries, and semantic features of named entities; The refined screening retrieval conditions include multidimensional features such as similarity score, weight threshold, and time proximity.

[0053] Optionally, the search condition generation module 302 is specifically used for: In the initial retrieval stage, the original demand information is input into the text generation model corresponding to the initial retrieval stage to generate retrieval conditions, and the initial retrieval conditions are obtained based on the output of the text generation model corresponding to the initial retrieval stage.

[0054] Optionally, the data retrieval determination module 303 includes: The primary data determination unit is used to perform a primary search in the index database according to the primary search conditions to obtain primary search data. The secondary data determination unit is used to perform a secondary search based on the primary search data and the secondary search conditions to obtain secondary search data. The target data determination unit is used to perform fine screening based on the secondary search data and the fine screening search conditions to obtain target search data. The number of events in the primary search data is higher than that in the secondary search data, and the number of events in the secondary search data is higher than that in the target search data.

[0055] Optional, secondary data determination unit, specifically used for: Semantic features are extracted from the primary search data to obtain the event semantic features of each primary search data, and a material database is constructed based on the event semantic features of each primary search data. Based on the secondary search criteria, a secondary search is performed in the material database to obtain secondary search data.

[0056] Optionally, the apparatus further includes an index database construction module. The index database construction module is used for: Multimodal data is collected through a preset collection method, wherein the modal types of the multimodal data include text data, image data, audio data, and video data; For each modality type, modal data features are extracted from the multimodal data according to the feature extraction rules corresponding to the modality type; The modal data features are processed by feature encoding to generate data index information, and an index database is constructed based on the data index information.

[0057] Optional, the information extraction and fusion module 304 is specifically used for: Based on the event information extraction rules, determine the sample data and label data used for model training; The preset fusion and extraction model is trained based on the sample data and the label data to obtain the target fusion and extraction model. The target retrieval data is input into the target fusion and refinement model for information fusion and refinement, generating a high information density set corresponding to the original demand information.

[0058] The technical solution of this invention involves acquiring original demand information to indicate the generation of a high information density set. Based on a pre-trained text generation model and the original demand information, target retrieval conditions for data indexing are generated. Based on a pre-built index database, multi-level information retrieval is performed according to the target retrieval conditions, achieving layer-by-layer filtering from massive data to high-value candidate data, improving retrieval performance and result accuracy, and obtaining high-value target retrieval data. Based on pre-defined event information extraction rules, information fusion and extraction are performed on the target retrieval data, achieving information compression and redundancy removal of the filtered high-value target retrieval data, increasing information density while retaining key information, and generating a high information density set that meets specific needs and satisfies high-precision application requirements.

[0059] The high information density set generation apparatus provided in the embodiments of the present invention can execute the high information density set generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0060] Figure 3A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0061] like Figure 3 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0062] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0063] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as methods for generating high information density sets.

[0064] In some embodiments, the method for generating a high information density set can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the high information density set generation method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the high information density set generation method by any other suitable means (e.g., by means of firmware).

[0065] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0066] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0067] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0068] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0069] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0070] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0071] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0072] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating high information density sets, characterized in that, include: Obtain the original requirement information used to indicate the generation of a high information density set; Based on the pre-trained text generation model and the original demand information, target retrieval conditions for data indexing are generated. The text generation model includes text generation models corresponding to each level of retrieval stage, and the target retrieval conditions include at least primary retrieval conditions, secondary retrieval conditions, and refined retrieval conditions. Based on a pre-built index database, multi-level information retrieval is performed according to the target retrieval conditions to obtain target retrieval data, wherein the index database is constructed based on multimodal data; Based on pre-defined event information extraction rules, information is fused and extracted according to the target retrieval data to generate a high information density set corresponding to the original demand information.

2. The method according to claim 1, characterized in that, The screening accuracy of the refined search criteria is higher than that of the secondary search criteria, and the screening accuracy of the secondary search criteria is higher than that of the primary search criteria. The primary search criteria include keyword Boolean logic combinations, semantic vector features, and time or location filtering parameters. The secondary search criteria include topic tags, key summaries, and semantic features of named entities; The refined screening retrieval conditions include multidimensional features such as similarity score, weight threshold, and time proximity.

3. The method according to claim 1, characterized in that, The pre-trained text generation model and the original demand information are used to generate target retrieval conditions for data indexing, including: In the initial retrieval stage, the original demand information is input into the text generation model corresponding to the initial retrieval stage to generate retrieval conditions, and the initial retrieval conditions are obtained based on the output of the text generation model corresponding to the initial retrieval stage.

4. The method according to claim 1, characterized in that, The method, based on a pre-built index database, performs multi-level information retrieval according to the target retrieval conditions to obtain target retrieval data, including: Based on the primary search criteria, a primary search is performed in the index database to obtain primary search data. A secondary search is performed based on the primary search data and the secondary search conditions to obtain secondary search data. Based on the secondary search data and the refined search conditions, a refined search is performed to obtain the target search data; The number of events in the primary search data is higher than that in the secondary search data, and the number of events in the secondary search data is higher than that in the target search data.

5. The method according to claim 4, characterized in that, The step of performing a secondary search based on the primary search data and the secondary search conditions to obtain secondary search data includes: Semantic features are extracted from the primary search data to obtain the event semantic features of each primary search data, and a material database is constructed based on the event semantic features of each primary search data. Based on the secondary search criteria, a secondary search is performed in the material database to obtain secondary search data.

6. The method according to claim 1, characterized in that, The index database construction process includes: Multimodal data is collected through a preset collection method, wherein the modal types of the multimodal data include text data, image data, audio data, and video data; For each modality type, modal data features are extracted from the multimodal data according to the feature extraction rules corresponding to the modality type; The modal data features are processed by feature encoding to generate data index information, and an index database is constructed based on the data index information.

7. The method according to claim 1, characterized in that, The process involves refining and integrating information based on pre-defined event information extraction rules and the target retrieval data to generate a high-information-density set corresponding to the original demand information, including: Based on the event information extraction rules, determine the sample data and label data used for model training; The preset fusion and extraction model is trained based on the sample data and the label data to obtain the target fusion and extraction model. The target retrieval data is input into the target fusion and refinement model for information fusion and refinement, generating a high information density set corresponding to the original demand information.

8. A device for generating high information density sets, characterized in that, include: The demand information acquisition module is used to acquire the original demand information used to indicate the generation of a high information density set; The retrieval condition generation module is used to generate target retrieval conditions for data indexing based on a pre-trained text generation model and the original requirement information. The text generation model includes text generation models corresponding to each level of retrieval stage, and the target retrieval conditions include at least primary retrieval conditions, secondary retrieval conditions, and fine screening retrieval conditions. The data retrieval determination module is used to perform multi-level information retrieval based on a pre-built index database and according to the target retrieval conditions to obtain target retrieval data, wherein the index database is constructed based on multimodal data; The information extraction and fusion module is used to perform information fusion and extraction based on the target retrieval data according to the pre-set event information extraction rules, and generate a high information density set corresponding to the original demand information.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the method for generating a high information density set according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for generating a high information density set as described in any one of claims 1-7.

Images