Large-capacity text clue retrieval method, system, product and medium based on enhanced retrieval

Abstract

The present application discloses a method, system, product and medium for large-capacity text clue retrieval based on enhanced retrieval, and the method specifically includes: S100, dividing large-capacity text information into blocks; S200, converting the divided text information blocks into text high-dimensional feature vector matrices; S300, introducing a dynamic feature library that stores features of different business types; S400, based on the prompt words and business types input by the user, obtaining the target business feature vector matrix in the dynamic feature library, and obtaining the associated prompt words associated with the target business feature vector matrix; S500, generating a retrieval unit, scanning the text information blocks, marking the scanning results that meet the retrieval unit hit conditions as hit text information blocks, and outputting them; S600, replying with the associated prompt words within the range of the hit text information blocks. The present application can perform accurate and efficient intelligent clue retrieval for large-capacity texts, and provide case handlers with accurate replies based on prompt words.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a method, system, product and medium for large-capacity text clue retrieval based on enhanced retrieval. Background Art

[0002] In the information age, criminal activity is highly digitized and transregional, making electronic data a core source of evidence in criminal investigations. For example, chat logs from instant messaging tools are often used as a key data source for analysis during electronic evidence collection in public security investigations. Electronic data is characterized by its vast volume and complex structure. For example, a single case may involve tens or even hundreds of thousands of chat logs and transaction flow data from various instant messaging tools or information platform apps. Traditional manual screening methods suffer from low efficiency, high costs, and the tendency to miss important information.

[0003] Currently, general-purpose large language models have made significant progress in semantic understanding and natural language generation in complex scenarios. Large language models are gradually being applied to electronic data management, case clue induction, and evidence collection. However, the current technical architecture still has many problems in practical application.

[0004] First, due to the massive volume of electronic data, natural language understanding based on large language models takes a long time, has high feature complexity, and is limited by system resources, resulting in low efficiency and difficulty in performing feature matching. Some technologies use a "stop on first hit" retrieval strategy to improve retrieval efficiency, but this in turn prevents the system from completing a full content scan, which in turn creates the risk of missing retrieved fragments. Secondly, retrieval based on large language models cannot guarantee the accuracy of retrieval results. The accuracy of retrieval results depends on the degree of match between the hit fragment and the retrieved content, which has a certain degree of randomness, and the accuracy of the first hit is difficult to guarantee. Furthermore, due to the limited depth of semantic understanding, scanning electronic data makes it difficult to capture hidden information or coded information, resulting in insufficient recall rates. Furthermore, it is also impossible to avoid hallucinations caused by large language models, resulting in inaccurate information provided. Summary of the Invention

[0005] The purpose of this application is to overcome the shortcomings of the existing technology and provide a large-capacity text clue retrieval method, system, product and medium based on enhanced retrieval, which can perform accurate and efficient intelligent clue retrieval of large-capacity text and provide case handlers with accurate answers based on prompt words.

[0006] In the first aspect, the present application provides a large-capacity text clue retrieval method based on enhanced retrieval, which adopts the following technical solutions:

[0007] S100, dividing the large amount of text information into blocks based on a certain number of characters, and dividing the large amount of text information into a number of continuous text information blocks;

[0008] S200, through a large language processing model, converts each text information block into a high-dimensional feature vector matrix of the text;

[0009] S300: Introducing a dynamic feature library, which stores service type feature vector matrices that differentiate between different service types. Each service type feature vector matrix is ​​composed of several service vector matrices related to the service type, and the service vector matrix represents specific features of a specific service type.

[0010] S400: Based on the prompt word and service type input by the user, features of the prompt word are extracted using a large language processing model. Then, under the corresponding service type in the dynamic feature library, a target service feature vector matrix is ​​obtained from the feature vector matrix of the corresponding service type. The prompt word and the target service feature vector matrix are associated to obtain an associated prompt word. Based on the prompt word input by the user, several target service feature vector matrices are obtained.

[0011] S500, for each target business feature vector matrix, a retrieval unit is generated, the window range of the retrieval unit and the hit condition of the text information block are set, each retrieval unit independently performs a scan on the text high-dimensional feature vector matrix of the text information block based on the target business feature vector matrix, when the spatial similarity between the two meets the hit condition preset by the retrieval unit, the corresponding text information block is marked as a hit text information block, and the corresponding hit text information block is output, and the hit text information block finally output is the sum of the hit text information blocks marked by each retrieval unit; for any retrieval unit, a corresponding scanning thread is created, each scanning thread is responsible for the text information block scanning task of the corresponding retrieval unit, and multiple scanning threads execute the scanning task in parallel;

[0012] S600, within the scope of the hit text information block, respond to the associated prompt word based on the large language processing model.

[0013] Through the above technical solution, the large-capacity text information is divided into blocks and then a high-dimensional feature vector matrix is ​​extracted. Then, a dynamic feature library classified according to business type is introduced. The prompt words entered by the user are classified according to the business type and matched with the dynamic feature library to obtain the target business feature vector matrix. The high-dimensional feature vector matrix of the large-capacity text information is then scanned. After hitting, the hit text information block is output and the prompt word is replied. This application improves the accuracy of the search content based on the enhanced search of user prompt words and dynamic feature library; this application performs a full search of large-capacity text information, and outputs and replies to the hit text information blocks, preventing the large language model from generating hallucinations while ensuring the search scope and recall rate; this application replies based on the prompt words after association, which improves the relevance of the reply content to the user's actual reply target.

[0014] Through the above technical solution, multiple retrieval units are used to construct multi-parallel retrieval channels, which can achieve semantic coverage, feature fusion, enhance the complementarity between features, explore potential related clues, and improve the recall rate and completeness rate of retrieval; the multi-threaded parallel retrieval strategy can improve retrieval efficiency.

[0015] Preferably, for any retrieval unit, the retrieval range consists of a main text information block and several adjacent text information blocks of the main text information block.

[0016] Preferably, the adjacent text information blocks include adjacent contexts of text information published in the same instant messaging tool or information publishing platform software, and text information published in different instant messaging tools and / or information publishing platform software within the same time period.

[0017] Through the above technical solution, a dynamically sliding retrieval window formed by the main text information block and the adjacent text information block is realized, which can realize continuous semantic capture of long texts in context, semantic association of text information across information sources, ensure the restoration of fragmented chat information, and improve retrieval coverage and accuracy.

[0018] Preferably, in S500, after marking the corresponding text information block as a hit text information block for the first time, the scanning task is continued until the scanning of the text high-dimensional feature vector matrix of all text information blocks is completed, and all marked hit text information blocks are sorted according to spatial similarity, and a set number of hit text information blocks are output from high to low.

[0019] Through the above technical solution, after performing a full search of large-capacity text information, this application prioritizes it based on spatial similarity and then outputs the hit text information blocks. The output quantity can be adjusted according to user needs to ensure that the hit text information blocks that are strongly related to the user's intention are preferentially displayed. Users can directly focus on highly relevant hit text information blocks without manually traversing all the results, thereby improving the ability to identify key evidence.

[0020] Preferably, the large-capacity text clue retrieval method based on enhanced retrieval further includes S700, updating the business type feature vector matrix in the dynamic feature library by manually supplementing or modifying business type features based on the hit text information block.

[0021] Through the above technical solution, by manually identifying the hit text information blocks, the dynamic feature library can be updated with new clue features that change over time (such as new aliases and code words), so that the large language processing model retrieval strategy based on the dynamic feature library can adapt to new types of secret and complex clue scanning and collection services.

[0022] In a second aspect, the present application provides a large-capacity text clue retrieval system based on enhanced retrieval, comprising a text segmentation module, a text information processing module, a dynamic feature library module, a retrieval unit generation module, a text information scanning module, and an output module;

[0023] The text segmentation module divides large amounts of text information into several continuous text information blocks with a certain number of characters as a limit;

[0024] The text information processing module converts text information blocks into high-dimensional feature vector matrices through a large language processing model;

[0025] The dynamic feature library module stores the service type feature vector matrices of different service types;

[0026] The retrieval unit generation module matches the target service feature vector matrix of the corresponding service type in the dynamic feature library module based on the prompt word and service type input by the user, then generates a retrieval unit and associates the prompt word with the target service feature vector matrix to obtain the associated prompt word;

[0027] The text information scanning module scans the high-dimensional feature vector matrix of the text information block based on the target business feature vector matrix based on the retrieval unit. If the scanning result meets the hit condition preset by the retrieval unit, the corresponding text information block is marked as a hit text information block;

[0028] The output module outputs the corresponding hit text information block, and within the range of the hit text information block, it performs the response of the prompt word after association based on the large language processing model.

[0029] In a third aspect, the present application provides a computer program product, which includes a computer program or instructions, so that the computer program or instructions can implement the steps in the above-mentioned large-capacity text clue retrieval method based on enhanced retrieval.

[0030] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned large-capacity text clue retrieval method based on enhanced retrieval.

[0031] In summary, this application includes at least one of the following beneficial technical effects:

[0032] 1. This application can achieve full scanning of large amounts of text information, achieve full coverage of the content, and avoid missing potential clues.

[0033] 2. This application introduces a dynamic feature library. After the prompt words entered by the user are extracted through the feature extraction of the large language model, the target business feature vector matrix is ​​matched according to the business type, and the high-dimensional feature vector matrix of the text information block is scanned based on the target business feature vector matrix, thereby improving the relevance of clue retrieval with user intention and business modality, and improving retrieval accuracy.

[0034] 3. This application performs retrieval based on multiple retrieval units, and each retrieval unit performs retrieval based on a main text information block and several adjacent text information blocks, thereby improving the depth of semantic understanding, improving the clue coverage of cross-platform group chats, and successfully capturing hidden conversation patterns, effectively improving the recall rate of retrieval.

[0035] 4. This application reduces the difficulty for users to identify key clues and improves judgment efficiency by sorting and outputting the spatial similarity of the hit text information blocks.

[0036] 5. This application fully utilizes system resources, significantly reducing the processing time for large amounts of text information. It also reduces hardware resource consumption and the average computing cost per case. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the flow of a large-capacity text clue retrieval method based on enhanced retrieval in an embodiment of the present application;

[0038] Figure 2 Schematic diagram of a process for obtaining a hit text information block through a plurality of search units in an embodiment of the present application;

[0039] Figure 3 Schematic diagram of the architecture of a large-capacity text clue retrieval system based on enhanced retrieval in an embodiment of the application;

[0040] Figure 4 This is a schematic diagram of the structure of a computer device in an embodiment of the application. DETAILED DESCRIPTION

[0041] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed, but as long as they are within the scope of the present application, they are protected by patent law.

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. It should be noted that in the optional embodiments of the present application, when the embodiments in the present application are applied to specific products or technologies, the object information and other related data involved need to obtain the object's permission or consent, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained through the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0043] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document, unless otherwise specified, generally indicates an "or" relationship between the related objects.

[0044] The embodiments of the present application are described in further detail below with reference to the accompanying drawings.

[0045] In one embodiment of the present application, see Figure 1 , provides a large-capacity text clue retrieval method based on enhanced retrieval, comprising the following steps:

[0046] S100 , dividing the large-capacity text information into blocks based on a certain number of characters, and dividing the large-capacity text information into several continuous text information blocks.

[0047] It should be noted that the character limit for chunking large text messages is pre-set based on the business type and application scenario. The character limit can be set by selecting a specific business type or adjusted by the user. For example, business scenarios are categorized into Category A basic information verification scenarios, Category B multi-source evidence chain analysis scenarios, and Category C cross-domain comprehensive assessment scenarios. Category A basic information verification scenarios have a token limit of approximately 1,500 characters. These scenarios, such as ID number verification and case code matching, use fixed input fields (e.g., 18-digit ID number + name) and output Boolean values ​​or preset labels. Since these scenarios don't require long text generation, the token limit is relatively low. Category B multi-source evidence chain analysis scenarios, due to the need to process unstructured data and simultaneously load PDF reports, email text, and call log transcripts, each file consumes an average of 800-1,200 tokens. Therefore, considering multi-file scenarios, the token limit is set to 5,000 characters. For Class C cross-domain comprehensive analysis scenarios, the scenarios involve the need to integrate multimodal data with structured data. For example, cases involving organized crime require simultaneous analysis of fund maps (structured tables), surveillance video descriptions (unstructured text), and communication network topologies (graph data). After encoding, the tokens occupy an upper limit exceeding the normal limit, so the token limit needs to be customized based on the specific scenario and hardware computing power.

[0048] S200, through the large language processing model, converts each text information block into a high-dimensional feature vector matrix of the text. The optional types of the large language processing model used are Qwen2.5 or DeepSeek R1. As a preferred option, in this embodiment, the Qwen model is used. The reasons are based on the following two aspects: 1. Enterprise-level application maturity advantage: As the preferred solution of this embodiment, the core advantage of the Qwen model lies in its deep scene adaptation capability that has been verified by long-term business. In this application, technical personnel have fully deployed the Qwen series of models in the electronic data intelligent analysis application model since 2023. Through continuous scenario-based iteration, a customized capability covering the entire process of case analysis has been built: In a pilot work on the analysis of a certain Internet-related case, the intelligent text mining function developed based on Qwen achieved an accuracy rate of up to 82.3% for the description scene hit rate (an increase of 11.5% compared to the general model). This empirical data fully verifies its technical robustness in the analysis scenario, providing a reliable technical foundation for the technical solution in this application. 2. Technical Advantages of Model Architecture: Considering the technical characteristics of large language models, the Qwen model demonstrates significantly superior engineering performance compared to similar products. Its ultra-long context processing capability of 128k tokens (compared to DeepSeek R1's 64k limit) increased the recall rate of key information in a 100,000-word case file by 27% in testing, perfectly meeting the patent's ultra-long token restrictions for Class C business types (cross-domain comprehensive analysis scenarios). Furthermore, during the fine-tuning phase, the model requires only 500 annotated data points to achieve a task accuracy of over 90% (reducing annotation costs by 37.5% compared to competing products). This feature is crucial in handling confidential cases where annotation resources are scarce.

[0049] S300: A dynamic feature library is introduced. The dynamic feature library stores service type feature vector matrices that distinguish different service types. Each service type feature vector matrix is ​​actually composed of several service vector matrices related to the service type. The service vector matrix represents specific features of a specific service type.

[0050] Specifically, electronic data, differentiated by business type, involves different case description scenarios, broadly categorized into economic and financial scenarios, violence scenarios, and cyber scenarios. Within the case description scenario, specific business divisions are made for the crime and investigation types involved, and corresponding dynamic feature libraries are introduced.

[0051] It should be noted that the dynamic feature library can be derived from an existing keyword library based on business type classification and matching through feature vector extraction, or trained using an artificial neural network using a training set of clues labeled with features and business types, or manually input into the feature library based on expert experience and extracting feature vectors. The dynamic feature library can be adjusted and expanded at any time based on actual needs and can be connected to other corresponding case feature libraries.

[0052] At step S400, based on the prompt word and service type input by the user, the large language processing model extracts features of the prompt word. Then, under the corresponding service type in the dynamic feature library, a target service feature vector matrix is ​​obtained from the feature vector matrix of the corresponding service type. Furthermore, the prompt word is associated with the target service feature vector matrix to obtain an associated prompt word. This enhances the relevance of the prompt word to the service type, thereby improving the relevance of the response content to the user's intent and service type when the large language processing model subsequently extracts and responds to the prompt word.

[0053] It should be noted that, for user input, the business type is selected from a drop-down menu using preset existing business types, and the prompt word is input using natural language.

[0054] S500: Generate a retrieval unit to scan the high-dimensional feature vector matrix of the text information block based on the target business feature vector matrix. Then, when the spatial similarity between the two meets the hit condition preset by the retrieval unit, mark the corresponding text information block as a hit text information block and output the corresponding hit text information block.

[0055] It should be noted that the parameters of the search unit are specifically set by the user, including the search unit window range, the hit condition for the text information block, etc. Specifically, in this embodiment, the hit condition can be a spatial similarity threshold of the text high-dimensional feature vector matrix with respect to the target business feature vector matrix, and only the hit text information blocks whose spatial similarity exceeds the threshold are output.

[0056] S600: Responses are made to the associated prompt word within the scope of the hit text information block based on the large language processing model. Strictly limiting the scope of the large language processing model's responses prevents the model from referencing or creating clues outside the scope of the provided text information, preventing model slack and strictly ensuring the authenticity of the clues. If the large language processing model cannot provide a response based on the user's prompt word, it only provides the hit text information block and outputs an empty set for the response content. The user is then asked to update the prompt word to adjust the search and response strategy.

[0057] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0058] Further, in another embodiment, see Figure 2 Based on the prompt word input by the user, several target business feature vector matrices are obtained. For each target business feature vector matrix, a retrieval unit is generated. Each retrieval unit independently scans the high-dimensional feature vector matrix of the text information block. The final output hit text information block is the sum of the hit text information blocks marked by each retrieval unit.

[0059] It should be understood that since the prompt word is input by the user through natural language, after semantic understanding and feature extraction of the prompt word through a large language processing model, multiple different features will be generated. Generating target business feature vector matrices for different features, generating different retrieval units, and scanning subsequent clues can achieve semantic coverage, feature fusion, and improve the recall rate of retrieval. Users can set the parameters of different retrieval units uniformly, or set the parameters of each retrieval unit separately based on the specific features represented by the retrieval unit.

[0060] More specifically, for each retrieval unit, a corresponding scanning thread is created. Each scanning thread is responsible for scanning the text information block of the corresponding retrieval unit, and multiple scanning threads execute scanning tasks in parallel. Through the solution of this embodiment, dense vector calculations can be accelerated by GPU processing, and different scanning tasks can be assigned to different computing nodes. This improves data throughput, avoids a single channel becoming a performance bottleneck, improves hardware resource utilization, and significantly reduces the processing time of large amounts of text information.

[0061] Furthermore, in another embodiment, for any retrieval unit, the retrieval range consists of the main text information block and several adjacent text information blocks of the main text information block.

[0062] More specifically, the adjacent text information blocks include adjacent contexts of text information published in the same instant messaging tool or information publishing platform software, and text information published in different instant messaging tools and / or information publishing platform software within the same time period.

[0063] It should be noted that when setting the parameters of the search unit, the user can selectively set whether to include text information published in different instant messaging tools and / or information publishing platform software as adjacent text information blocks. If not, the adjacent text information blocks only include text information published in the same instant messaging tool or information publishing platform software. The search unit can set the parameter for adjacent text blocks to the number of adjacent text blocks. For example, if it is set to 3, then text information blocks within 3 adjacent to the main text information block will be included as adjacent text information blocks. The search unit can also set the parameter for adjacent text blocks to the same time interval. For example, if the same time interval is set to 2 days, then all text information blocks that occurred within two days of the main text information block will be included as adjacent text information blocks. If it is required, the adjacent text information blocks will include text information published in different instant messaging tools and / or information publishing platform software. When setting the search unit, it is necessary to additionally set the different time intervals. For example, if the different time interval is set to 1 day, then all text information published in different instant messaging tools and / or information publishing platform software within one day of the main text information block will be included as adjacent text information blocks. The weights of the feature vector matrix of adjacent text information blocks relative to the main text information block can be specifically set and adjusted. Through the above technical solution, an adjustable dynamic sliding window is established for clue retrieval, which can achieve contextual semantic understanding and clue capture of long, continuous texts of the same source. It can also break down barriers between heterogeneous text information, establish connections between heterogeneous texts based on time intervals, and conduct semantic comprehensive understanding. This achieves semantic associations across a single text information block, significantly improving the hit accuracy and recall rate of clues, and can also identify hidden code words and other conversation patterns.

[0064] More specifically, after marking the corresponding text information block as a hit text information block for the first time, continue to execute the scanning task until the scanning of the text high-dimensional feature vector matrix of all text information blocks is completed. Sort all marked hit text information blocks according to spatial similarity, and output a set number of hit text information blocks from high to low. For multiple retrieval units, it is necessary to wait until the scanning tasks of all scanning threads are completed before summarizing all hit text information blocks. The user can set a comprehensive sorting of all hit text information blocks, or sort the hit text information blocks corresponding to different retrieval units separately. The specific output quantity is set by the user. For example, the user can choose to output the TOP 5 or TOP 10 hit text information blocks.

[0065] In another embodiment, the large-capacity text clue retrieval method based on enhanced retrieval further includes S700, which updates the business type feature vector matrix in the dynamic feature library by manually supplementing or modifying business type features based on the hit text information block. Taking anti-money laundering as an example, criminals will adopt increasingly complex and covert money laundering methods and communication codes to evade detection. Therefore, it is necessary to dynamically adapt to new money laundering methods and adjust, update, and maintain the dynamic feature library. Because the hit text information block is text retrieved and output through a large language model clue, it is easy for experts to analyze and summarize new features, thereby improving the real-time and adaptability of the system, simplifying the analysis difficulty for experts and improving analysis efficiency.

[0066] In another embodiment of the present application, please refer to Figure 3 , provides a large-capacity text clue retrieval system based on enhanced retrieval, including a text segmentation module 1, a text information processing module 2, a dynamic feature library module 3, a retrieval unit generation module 4, a text information scanning module 5 and an output module 6.

[0067] The text segmentation module 1 divides a large amount of text information into a number of continuous text information blocks according to the user's settings and with a certain number of characters as a limit.

[0068] The text information processing module 2 converts the text information block into a high-dimensional feature vector matrix of the text through a large language processing model.

[0069] The dynamic feature library module 3 stores the service type feature vector matrices of different service types.

[0070] The retrieval unit generation module 4 matches the target business feature vector matrix under the corresponding business type in the dynamic feature library module 3 based on the prompt word and business type input by the user, and then generates several retrieval units and associates the prompt word with the target business feature vector matrix to obtain the associated prompt word.

[0071] The text information scanning module 5 scans the high-dimensional feature vector matrix of the text information block based on the target business feature vector matrix based on the retrieval unit. If the scanning result meets the hit condition preset by the retrieval unit, the corresponding text information block is marked as a hit text information block.

[0072] The output module 6 outputs the corresponding hit text information block, and within the range of the hit text information block, performs an associated prompt word response based on the large language processing model.

[0073] In other embodiments, the large-capacity text clue retrieval system based on enhanced retrieval further includes an updating module 7 for updating the dynamic feature library module based on the hit text information block output by the output module.

[0074] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the large-capacity text clue retrieval system based on enhanced retrieval described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0075] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0076] In another embodiment of the present application, a computer program product is provided. The computer program product includes a computer program or instructions, so that the computer program or instructions can perform the steps in the above-mentioned large-capacity text clue retrieval method based on enhanced retrieval.

[0077] In another embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the large-capacity text clue retrieval method based on enhanced retrieval are implemented.

[0078] In another embodiment of the present application, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a large-capacity text clue retrieval method based on enhanced retrieval, large-capacity text information and related data of a dynamic feature library. The network interface of the computer device is used to communicate with an external terminal through a network connection to realize the input of external user prompt words, business types and parameter adjustments, and output clue retrieval results to the user. When the computer program is executed by the processor, it realizes a large-capacity text clue retrieval method based on enhanced retrieval.

[0079] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive).

[0080] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0081] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A large-capacity text clue retrieval method based on enhanced retrieval, characterized in that: The steps include: S100, dividing the large amount of text information into blocks based on a certain number of characters, and dividing the large amount of text information into a number of continuous text information blocks; S200, through a large language processing model, converts each text information block into a high-dimensional feature vector matrix of the text; S300: Introducing a dynamic feature library, which stores service type feature vector matrices that differentiate between different service types. Each service type feature vector matrix is ​​composed of several service vector matrices related to the service type, and the service vector matrix represents specific features of a specific service type. The dynamic feature library is obtained by extracting feature vectors from a keyword library based on business type classification matching, or by inputting a clue training set with business type labels into an artificial neural network for training; S400: Based on the prompt word and service type input by the user, features of the prompt word are extracted using a large language processing model. Then, under the corresponding service type in the dynamic feature library, several target service feature vector matrices are obtained from the feature vector matrix of the corresponding service type. The prompt word is then associated with the target service feature vector matrix to obtain an associated prompt word. S500, for each target business feature vector matrix, a retrieval unit is generated, the window range of the retrieval unit and the hit condition of the text information block are set, each retrieval unit independently performs a scan on the text high-dimensional feature vector matrix of the text information block based on the target business feature vector matrix, when the spatial similarity between the two meets the hit condition preset by the retrieval unit, the corresponding text information block is marked as a hit text information block, and the corresponding hit text information block is output, and the hit text information block finally output is the sum of the hit text information blocks marked by each retrieval unit; for any retrieval unit, a corresponding scanning thread is created, each scanning thread is responsible for the text information block scanning task of the corresponding retrieval unit, and multiple scanning threads execute the scanning task in parallel; S600, within the scope of the hit text information block, respond to the associated prompt word based on the large language processing model.

2. The large-capacity text clue retrieval method based on enhanced retrieval according to claim 1 is characterized in that: For any retrieval unit, the retrieval range consists of the main text information block and several adjacent text information blocks of the main text information block.

3. The large-capacity text clue retrieval method based on enhanced retrieval according to claim 2 is characterized in that: The adjacent text information blocks include adjacent contexts of text information published in the same instant messaging tool or information publishing platform software, and text information published in different instant messaging tools and / or different information publishing platform software within the same time interval.

4. The large-capacity text clue retrieval method based on enhanced retrieval according to claim 1 is characterized in that: In S500, after the corresponding text information block is marked as a hit text information block for the first time, the scanning task is continued until the scanning of the text high-dimensional feature vector matrix of all text information blocks is completed, and all marked hit text information blocks are sorted according to spatial similarity, and a set number of hit text information blocks are output from high to low.

5. The large-capacity text clue retrieval method based on enhanced retrieval according to claim 1 is characterized in that: The method further includes S700, updating the business type feature vector matrix in the dynamic feature library by manually supplementing or modifying the business type features based on the hit text information block.

6. A large-capacity text clue retrieval system based on enhanced retrieval, characterized by: It includes text segmentation module, text information processing module, dynamic feature library module, retrieval unit generation module, text information scanning module and output module; The text segmentation module divides large amounts of text information into several continuous text information blocks with a certain number of characters as a limit; The text information processing module converts text information blocks into high-dimensional feature vector matrices through a large language processing model; The dynamic feature library module stores business type feature vector matrices for different business types. Each business type feature vector matrix is ​​composed of several business vector matrices related to the business type, and the business vector matrix represents the specific characteristics of a particular business type. The dynamic feature library is obtained by extracting feature vectors from an existing keyword vocabulary based on business type classification matching, or by inputting a clue training set with business type labels into an artificial neural network for training. The retrieval unit generation module matches the prompt word and service type input by the user in the dynamic feature library module to obtain several target service feature vector matrices under the corresponding service type. Then, for each target service feature vector matrix, a retrieval unit is generated and the prompt word is associated with the target service feature vector matrix to obtain the associated prompt word. The text information scanning module scans the high-dimensional feature vector matrix of the text information block based on the target business feature vector matrix based on each retrieval unit. If the scanning result meets the hit condition preset by the retrieval unit, the corresponding text information block is marked as a hit text information block; The output module outputs the sum of the hit text information blocks marked by each retrieval unit, and responds to the prompt words after association based on the large language processing model within the range of the hit text information blocks.

7. A computer program product, characterized in that The computer program product includes a computer program or instructions, so that the computer program or instructions can implement the steps of the large-capacity text clue retrieval method based on enhanced retrieval according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large-capacity text clue retrieval method based on enhanced retrieval are implemented as described in any one of claims 1 to 5.

Images