Information retrieval method and device and medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides an information retrieval method and device and a medium, which can segment to-be-processed data according to a configured segmentation strategy to obtain the to-be-processed data, and improve the organization and retrieval efficiency of the data. Performing vectorization processing on the segmented data to obtain vector data, and storing the vector data into an embedded database so as to facilitate subsequent similarity retrieval; the problem optimization is performed on the information retrieval request, so that the retrieval content can be better matched, and the retrieval accuracy is improved; a target cue word is generated by utilizing similar knowledge matched in an embedded database and a pre-configured cue word template, so that the retrieval efficiency and accuracy are further improved; according to the method, the retrieval model is retrieved according to the information retrieval request, the retrieval model is utilized to respond based on the target prompt word, and flexible retrieval of the model not only optimizes the retrieval effect, but also reduces the retrieval cost.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an information retrieval method, device and medium. Background Art

[0002] With the continuous development of artificial intelligence technology, using models for information retrieval has become an important technical tool in various fields.

[0003] However, due to the diversity of information, the personalized needs of users, and the high requirements for retrieval efficiency, the information retrieval methods used in the existing technology are no longer applicable to various technical scenarios for efficient retrieval. Summary of the invention

[0004] In view of the above, it is necessary to provide an information retrieval method, device and medium to solve the problems of low retrieval efficiency and low accuracy.

[0005] An information retrieval method, the information retrieval method comprising: Collecting an initial text file and preprocessing the initial text file to obtain data to be processed; Splitting the data to be processed according to the configured segmentation strategy to obtain segmented data; Performing vectorization processing on the segmented data to obtain vector data, and storing the vector data in an embedded database; When an information retrieval request triggered by a target user is received, the information retrieval request is optimized to obtain a target question, including: analyzing the hierarchical structure of the topics contained in the information retrieval request, and establishing a target topic tree according to the hierarchical structure; obtaining user preference data of the target user as target preference data, and obtaining a user portrait of the target user as a target portrait; obtaining a topic priority corresponding to the target portrait, and adjusting the topic priority according to the target preference data to obtain a target priority; optimizing the information retrieval request according to the target priority to obtain the target question; Vectorizing the target question to obtain a question vector, and using the question vector to perform similarity retrieval in the embedded database to obtain similar knowledge; Generate a target prompt word using the similar knowledge and a pre-configured prompt word template; Retrieving a retrieval model according to the information retrieval request, and using the retrieval model to respond based on the target prompt word to obtain an information retrieval result; The information retrieval result is fed back to the target user.

[0006] According to a preferred embodiment of the present invention, the collecting of the initial text file and preprocessing the initial text file to obtain the data to be processed includes: Collecting text files in various formats as the initial text files; Extract the text content in the text file of each format using the loader corresponding to each format; The extracted text content is segmented to obtain the data to be processed.

[0007] According to a preferred embodiment of the present invention, before segmenting the data to be processed according to the configured segmentation strategy, the method further includes: Acquire a data source of the data to be processed, and split the data to be processed according to the data source of the data to be processed to obtain at least one sub-data set; Get each file in each sub-dataset and split each file into multiple data fields; The context correlation between every two data fields is calculated, and the relationship between different data fields is marked according to the context correlation between every two data fields.

[0008] According to a preferred embodiment of the present invention, segmenting the data to be processed according to the configured segmentation strategy to obtain segmented data includes: Obtaining user feedback data and usage habit data of the target user; Obtaining a target strategy from a pre-configured segmentation strategy pool according to the user feedback data and usage habit data of the target user; Using the target strategy to segment the data to be processed to obtain the segmented data; The target strategy includes one or more of the following strategies: intelligent word segmentation strategy, intelligent tagging strategy, and semantic understanding strategy; Wherein, for the intelligent word segmentation strategy, each data field is segmented using a configured word segmentation algorithm; Wherein, for the intelligent labeling strategy, entity labeling is performed on each segmentation obtained after the segmentation process; Among them, for the semantic understanding strategy, intent recognition is performed on each data field to obtain a recognition result, and each data field is segmented according to the recognition result.

[0009] According to a preferred embodiment of the present invention, before obtaining the subject priority corresponding to the target portrait, the method further includes: Obtaining a historical information retrieval request, and analyzing a hierarchical structure of topics included in the historical information retrieval request as a historical hierarchical structure; Establishing a historical theme tree according to the historical hierarchical structure; Searching each historical topic in the historical topic tree respectively to obtain historical search results corresponding to each historical topic; Get pre-built user profiles; For each user in the user portrait, obtaining each user's response to the historical search results corresponding to each historical topic; The priority of each topic in the historical topic tree corresponding to each user is determined according to each user's response to the historical search result corresponding to each historical topic.

[0010] According to a preferred embodiment of the present invention, the step of optimizing the information retrieval request to obtain a target question further comprises: Obtaining each metadata in the information retrieval request; Calculate the correlation between every two metadata, and mark each metadata according to the correlation between every two metadata; Expand the retrieval dimension of each marked metadata to obtain multiple retrieval conditions; The information search request is adjusted according to the multiple search conditions to obtain the target question.

[0011] According to a preferred embodiment of the present invention, the step of using the retrieval model to respond based on the target prompt word to obtain the information retrieval result includes: Based on the keyword search technology, the search model is used to respond based on the target prompt word to obtain a first search result; Based on the semantic retrieval technology, the retrieval model is used to respond based on the target prompt word to obtain a second retrieval result; Based on the graph retrieval technology, the retrieval model is used to respond based on the target prompt word to obtain a third retrieval result; Obtaining a first weight corresponding to the keyword search technology, obtaining a second weight corresponding to the semantic search technology, and obtaining a third weight corresponding to the graph search technology; Calculate a weighted sum of the first search result, the second search result, and the third search result according to the first weight, the second weight, and the third weight to obtain a fused search result; Using a prediction model to predict the value of each information unit in the fusion search result; Rearrange each information unit in descending order of its value to obtain an information unit sequence; Acquire the pre-set information units from the information unit sequence to form the information retrieval result; The first weight, the second weight, and the third weight are configured according to the contribution of each search technology to the search results.

[0012] An information retrieval device, comprising: A preprocessing unit, used for collecting an initial text file and preprocessing the initial text file to obtain data to be processed; A segmentation unit, used to segment the data to be processed according to the configured segmentation strategy to obtain segmented data; A vectorization unit, used for performing vectorization processing on the segmented data to obtain vector data, and storing the vector data in an embedded database; The optimization unit is used to optimize the information retrieval request when receiving the information retrieval request triggered by the target user to obtain the target question, including: analyzing the hierarchical structure of the topics contained in the information retrieval request, and establishing a target topic tree according to the hierarchical structure; obtaining the user preference data of the target user as the target preference data, and obtaining the user portrait of the target user as the target portrait; obtaining the topic priority corresponding to the target portrait, and adjusting the topic priority according to the target preference data to obtain the target priority; optimizing the information retrieval request according to the target priority to obtain the target question; A retrieval unit, configured to perform vectorization processing on the target question to obtain a question vector, and perform similarity retrieval in the embedded database using the question vector to obtain similar knowledge; A generating unit, configured to generate a target prompt word by using the similar knowledge and a pre-configured prompt word template; A response unit, configured to retrieve a retrieval model according to the information retrieval request, and respond based on the target prompt word using the retrieval model to obtain an information retrieval result; A feedback unit is used to feed back the information retrieval result to the target user.

[0013] A computer device, comprising: a memory storing at least one instruction; and A processor executes the instructions stored in the memory to implement the information retrieval method.

[0014] A computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the information retrieval method.

[0015] It can be seen from the above technical solutions that the present invention can segment the data to be processed according to the configured segmentation strategy to obtain the data to be processed, thereby improving the data organization and retrieval efficiency; vectorize the segmented data to obtain vector data, and store the vector data in an embedded database for subsequent similarity retrieval; optimize the information retrieval request to better match the retrieved content, thereby improving the retrieval accuracy; generate target prompt words using similar knowledge matched in the embedded database and a pre-configured prompt word template, thereby further improving the retrieval efficiency and accuracy; retrieve the retrieval model according to the information retrieval request, and use the retrieval model to respond based on the target prompt word. The flexible retrieval of the model not only optimizes the retrieval effect, but also reduces the retrieval cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a preferred embodiment of the information retrieval method of the present invention; Figure 2 It is a functional module diagram of a preferred embodiment of the information retrieval device of the present invention; Figure 3 It is a schematic diagram of the structure of a computer device of a preferred embodiment of the information retrieval method of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] like Figure 1 FIG. 1 is a flow chart of a preferred embodiment of the information retrieval method of the present invention. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0019] The information retrieval method is applied to one or more computer devices, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and whose hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0020] The computer device may be any electronic product that can perform human-computer interaction with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0021] The computer device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0022] The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0023] Among them, Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0024] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0025] The network where the computer device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0026] S10, collecting an initial text file, and preprocessing the initial text file to obtain data to be processed.

[0027] In this embodiment, the initial text file may include, but is not limited to, text files in formats such as HTML (HyperText Markup Language), JSON (JavaScript Object Notation), CSV (Comma-Separated Values), PDF (Portable Document Format), and DOCX (Microsoft Word Open XML Document).

[0028] In this embodiment, the collecting of the initial text file and preprocessing of the initial text file to obtain the data to be processed includes: Collecting text files in various formats as the initial text files; Extract the text content in the text file of each format using the loader corresponding to each format; The extracted text content is segmented to obtain the data to be processed.

[0029] The initial text file may be collected through investigation, experiment, observation or from other existing data sets.

[0030] The loaders may include, but are not limited to: pdfloader, htmlloader and jsonloader, etc.

[0031] Among them, the extracted text content can be segmented by using a word segmentation method such as SplitWord, and the present invention does not limit the word segmentation algorithm used.

[0032] Through the above embodiments, text files in various formats can be processed specifically to facilitate subsequent use, thereby improving generalization capabilities.

[0033] Of course, in other embodiments, noise (such as unnecessary parts such as empty fields) in the initial text file may also be removed to improve data quality.

[0034] The initial text files may also be uniformly converted into one format, such as converting the text into lowercase, deleting punctuation marks, replacing abbreviations, etc., to facilitate subsequent processing.

[0035] Duplicate data in the initial text file may also be deleted to reduce the amount of calculation and ensure the accuracy of the analysis results.

[0036] The processed data can also be verified to ensure data quality and integrity, such as checking the consistency, completeness and accuracy of the data.

[0037] S11, segmenting the data to be processed according to the configured segmentation strategy to obtain segmented data.

[0038] In this embodiment, before segmenting the data to be processed according to the configured segmentation strategy, the method further includes: Acquire a data source of the data to be processed, and split the data to be processed according to the data source of the data to be processed to obtain at least one sub-data set; Get each file in each sub-dataset and split each file into multiple data fields; The context correlation between every two data fields is calculated, and the relationship between different data fields is marked according to the context correlation between every two data fields.

[0039] The data source may include, but is not limited to, various data channels such as databases, documents, online resources, etc. By refining the data source, a multi-level data structure is introduced to enhance the data strength and ensure the diversity and comprehensiveness of the data.

[0040] Each file can be marked with a unique identifier, and each file can include multiple data fields. By structuring the file, the file can be converted into a structured format consisting of multiple data fields (including fields or documents such as data tables and charts), so as to facilitate subsequent analysis and retrieval.

[0041] Among them, the cosine similarity algorithm or artificial intelligence model can be used to calculate the contextual correlation between every two data fields, so as to improve the relevance of the retrieval results through the intersection of the context.

[0042] In this embodiment, the data to be processed is segmented according to the configured segmentation strategy to obtain segmented data including: Obtaining user feedback data and usage habit data of the target user; Obtaining a target strategy from a pre-configured segmentation strategy pool according to the user feedback data and usage habit data of the target user; Using the target strategy to segment the data to be processed to obtain the segmented data; The target strategy includes one or more of the following strategies: intelligent word segmentation strategy, intelligent tagging strategy, and semantic understanding strategy; Wherein, for the intelligent word segmentation strategy, each data field is segmented using a configured word segmentation algorithm; Among them, for the intelligent labeling strategy, each segmentation obtained after the segmentation process is labeled with an entity; for example, entities such as names, places, and times can be labeled; Among them, for the semantic understanding strategy, intent recognition is performed on each data field to obtain a recognition result, and each data field is segmented according to the recognition result. For example, a model such as BERT (Bidirectional Encoder Representations from Transformers) can be used for intent recognition.

[0043] In the above embodiments, by segmenting the data and labeling the entities, the accuracy of information parsing can be improved; by identifying the user's search intention, the effective matching of information in the search process can be ensured; and by dynamically adjusting the segmentation strategy based on user feedback and usage habits, the adaptability of subsequent searches can be improved.

[0044] S12, performing vectorization processing on the segmented data to obtain vector data, and storing the vector data in an embedded database.

[0045] In this embodiment, the segmented data may be converted into vectors using an Embedding model to obtain the vector data, so as to facilitate effective identification during similarity retrieval.

[0046] In this embodiment, by storing the vector data in an embedded database, support can be provided for subsequent retrieval.

[0047] The embedded database may include, but is not limited to, faiss, milvus or pinecone databases, etc. The embedded database supports the storage of vector data and similarity retrieval.

[0048] S13, when receiving an information retrieval request triggered by a target user, optimizing the information retrieval request to obtain a target question, including: analyzing the hierarchical structure of the topics contained in the information retrieval request, and establishing a target topic tree according to the hierarchical structure; obtaining the user preference data of the target user as target preference data, and obtaining the user portrait of the target user as a target portrait; obtaining the topic priority corresponding to the target portrait, and adjusting the topic priority according to the target preference data to obtain a target priority; optimizing the information retrieval request according to the target priority to obtain the target question.

[0049] In this embodiment, the target user may be any user having a search requirement.

[0050] In this embodiment, the information retrieval request may be triggered through a designated user interaction interface, and the present invention does not limit the triggering method of the information retrieval request.

[0051] Among them, the hierarchical structure of the topics contained in the information retrieval request can be analyzed according to the type and attributes of the information. The information retrieval request may include topics, sub-topics and specific content levels. For example, the topic may be technology, the sub-topic may be artificial intelligence, quantum computing, etc., the content levels included in artificial intelligence may be machine learning, etc., and the content levels included in quantum computing may be cloud computing, etc.

[0052] Among them, searching according to the topic priority corresponding to different user portraits can be carried out in depth from macro to micro level, thereby improving the flexibility and pertinence of information acquisition.

[0053] Among them, dynamically adjusting the search priority according to user preferences can specifically display the most concerned information to users, thereby improving the accuracy of the search.

[0054] In this embodiment, before obtaining the subject priority corresponding to the target portrait, the method further includes: Obtaining a historical information retrieval request, and analyzing a hierarchical structure of topics included in the historical information retrieval request as a historical hierarchical structure; Establishing a historical theme tree according to the historical hierarchical structure; Searching each historical topic in the historical topic tree respectively to obtain historical search results corresponding to each historical topic; Get pre-built user profiles; For each user in the user portrait, obtaining each user's response to the historical search results corresponding to each historical topic; The priority of each topic in the historical topic tree corresponding to each user is determined according to each user's response to the historical search result corresponding to each historical topic.

[0055] Among them, each user's response to the historical search results corresponding to each historical topic can be feedback to each historical search result, such as likes or dislikes, etc., to identify the user's recognition of these historical search results. For example: an initial weight can be configured for each historical search result. When it is detected that the user likes any historical search result, the weight is +1; when it is detected that the user dislikes any historical search result, the weight is -1. Finally, the weight of each historical search result is counted, and different historical search results are sorted according to the weight of each historical search result, so as to determine the priority of the topic corresponding to each historical search result.

[0056] The user portrait can be configured according to the gender, age, occupation, preferences, etc. of each user, so as to display search results for similar users according to the topic priority preferred by different user types, so as to improve the user's recognition of the search results.

[0057] In this embodiment, optimizing the information retrieval request to obtain a target question further includes: Obtaining each metadata in the information retrieval request; Calculate the correlation between every two metadata, and mark each metadata according to the correlation between every two metadata; Expand the retrieval dimension of each marked metadata to obtain multiple retrieval conditions; The information search request is adjusted according to the multiple search conditions to obtain the target question.

[0058] The metadata may include, but is not limited to, creation date, author, subject tag, etc. Introducing rich metadata information in the search process can support more accurate search results and enhance the traceability of information.

[0059] Among them, by analyzing the relationship between metadata, the potential connection between information can be identified, thereby helping users better understand the search results.

[0060] Among them, after calculating the correlation between each two metadata, for the information unit corresponding to each metadata, the correlation matrix can be used to represent the potential connection between the information units. Specifically, when there is shared metadata between two information units, the correlation matrix takes a value of 1; when there is no shared metadata between two information units, the correlation matrix takes a value of. For example: an information unit can be represented as follows: M={creation date=2024-01-01, author=Zhang San, topic tag={technology, AI}}.

[0061] Among them, after the search dimension is expanded, the relevance, accuracy and efficiency of the search can be further improved. For example, the search condition can be expressed as {creation date [d1, d2], author = a, topic tag {t1, t2}}, the search will be filtered within the range limited by the search condition, thereby improving the pertinence of the search results and improving the search efficiency.

[0062] S14, vectorizing the target question to obtain a question vector, and using the question vector to perform similarity search in the embedded database to obtain similar knowledge.

[0063] In this embodiment, an Embedding model may be used to vectorize the target question to obtain the question vector.

[0064] In this embodiment, the similar knowledge can be obtained by calculating the cosine similarity and performing similarity search in the embedded database.

[0065] S15, generating a target prompt word by using the similarity knowledge and a pre-configured prompt word template.

[0066] In this embodiment, the similar knowledge may be integrated into the prompt word template, thereby obtaining the target prompt word that is more convenient for model recognition.

[0067] Since the prompt word template is a pre-configured standard template and the similar knowledge is knowledge pre-maintained in the database, the target prompt word finally obtained can simultaneously have a structure and knowledge that are easy to understand by a large model, further improving the retrieval efficiency and accuracy of the model.

[0068] S16, calling a retrieval model according to the information retrieval request, and using the retrieval model to respond based on the target prompt word to obtain an information retrieval result.

[0069] In this embodiment, a large language model of corresponding size and model can be retrieved as the retrieval model according to different retrieval requirements, which can reduce the model deployment cost while meeting the retrieval performance.

[0070] In this embodiment, the using of the retrieval model to respond based on the target prompt word to obtain the information retrieval result includes: Based on the keyword search technology, the search model is used to respond based on the target prompt word to obtain a first search result; Based on the semantic retrieval technology, the retrieval model is used to respond based on the target prompt word to obtain a second retrieval result; Based on the graph retrieval technology, the retrieval model is used to respond based on the target prompt word to obtain a third retrieval result; Obtaining a first weight corresponding to the keyword search technology, obtaining a second weight corresponding to the semantic search technology, and obtaining a third weight corresponding to the graph search technology; Calculate a weighted sum of the first search result, the second search result, and the third search result according to the first weight, the second weight, and the third weight to obtain a fused search result; Using a prediction model to predict the value of each information unit in the fusion search result; Rearrange each information unit in descending order of its value to obtain an information unit sequence; Acquire the pre-set information units from the information unit sequence to form the information retrieval result; The first weight, the second weight, and the third weight are configured according to the contribution of each search technology to the search results.

[0071] The prediction model may be a machine learning model, such as a model based on ranking learning or a recommendation system algorithm, etc. The prediction model may be used to rearrange the search results based on multi-dimensional features such as metadata, search scores, and user preferences.

[0072] The pre-set positions may be the first 10 positions, the first 5 positions, etc., and may be specifically configured according to different retrieval requirements.

[0073] Among them, by integrating the search results obtained by different search technologies and eliminating redundancy, the uniqueness and relevance of information can be improved.

[0074] Among them, by further sorting and optimizing the fusion search results, users can be ensured to obtain the most valuable information.

[0075] Among them, multiple search methods are integrated for mixed search, and the search results are optimized and aligned, which improves the accuracy of the search results and the comprehensive coverage of information.

[0076] S17, feeding back the information retrieval result to the target user.

[0077] In this embodiment, the information retrieval result may be fed back to the target user by means of email or the like, or may be directly displayed on a designated user interaction interface.

[0078] It can be seen from the above technical solutions that the present invention can segment the data to be processed according to the configured segmentation strategy to obtain the data to be processed, thereby improving the data organization and retrieval efficiency; vectorize the segmented data to obtain vector data, and store the vector data in an embedded database for subsequent similarity retrieval; optimize the information retrieval request to better match the retrieved content, thereby improving the retrieval accuracy; generate target prompt words using similar knowledge matched in the embedded database and a pre-configured prompt word template, thereby further improving the retrieval efficiency and accuracy; retrieve the retrieval model according to the information retrieval request, and use the retrieval model to respond based on the target prompt word. The flexible retrieval of the model not only optimizes the retrieval effect, but also reduces the retrieval cost.

[0079] like Figure 2As shown, it is a functional module diagram of a preferred embodiment of the information retrieval device of the present invention. The information retrieval device 11 includes a preprocessing unit 110, a segmentation unit 111, a vectorization unit 112, an optimization unit 113, a retrieval unit 114, a generation unit 115, a response unit 116, and a feedback unit 117. The module / unit referred to in the present invention refers to a series of computer program segments that can be executed by a processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0080] The preprocessing unit 110 is used to collect an initial text file and preprocess the initial text file to obtain data to be processed; The segmentation unit 111 is used to segment the data to be processed according to the configured segmentation strategy to obtain segmented data; The vectorization unit 112 is used to perform vectorization processing on the segmented data to obtain vector data, and store the vector data in an embedded database; The optimization unit 113 is used to optimize the information retrieval request when receiving the information retrieval request triggered by the target user to obtain the target question, including: analyzing the hierarchical structure of the topics contained in the information retrieval request, and establishing a target topic tree according to the hierarchical structure; obtaining the user preference data of the target user as the target preference data, and obtaining the user portrait of the target user as the target portrait; obtaining the topic priority corresponding to the target portrait, and adjusting the topic priority according to the target preference data to obtain the target priority; optimizing the information retrieval request according to the target priority to obtain the target question; The retrieval unit 114 is used to perform vectorization processing on the target question to obtain a question vector, and use the question vector to perform similarity retrieval in the embedded database to obtain similar knowledge; The generating unit 115 is used to generate a target prompt word by using the similar knowledge and a pre-configured prompt word template; The response unit 116 is used to retrieve a retrieval model according to the information retrieval request, and use the retrieval model to respond based on the target prompt word to obtain an information retrieval result; The feedback unit 117 is used to feed back the information retrieval result to the target user.

[0081] It can be seen from the above technical solutions that the present invention can segment the data to be processed according to the configured segmentation strategy to obtain the data to be processed, thereby improving the data organization and retrieval efficiency; vectorize the segmented data to obtain vector data, and store the vector data in an embedded database for subsequent similarity retrieval; optimize the information retrieval request to better match the retrieved content, thereby improving the retrieval accuracy; generate target prompt words using similar knowledge matched in the embedded database and a pre-configured prompt word template, thereby further improving the retrieval efficiency and accuracy; retrieve the retrieval model according to the information retrieval request, and use the retrieval model to respond based on the target prompt word. The flexible retrieval of the model not only optimizes the retrieval effect, but also reduces the retrieval cost.

[0082] like Figure 3 FIG. 1 is a schematic diagram of the structure of a computer device for implementing a preferred embodiment of the information retrieval method of the present invention.

[0083] The computer device 1 may include a memory 12 , a processor 13 , and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13 , such as an information retrieval program.

[0084] Those skilled in the art will appreciate that the schematic diagram is merely an example of the computer device 1 and does not constitute a limitation on the computer device 1. The computer device 1 may be a bus-type structure or a star-type structure. The computer device 1 may also include more or less other hardware or software than shown in the diagram, or a different arrangement of components. For example, the computer device 1 may also include input and output devices, network access devices, etc.

[0085] It should be noted that the computer device 1 is only an example, and other existing or future electronic products that are suitable for the present invention should also be included in the protection scope of the present invention and included here by reference.

[0086] The memory 12 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 12 may be an internal storage unit of the computer device 1, such as a mobile hard disk of the computer device 1. In other embodiments, the memory 12 may also be an external storage device of the computer device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 1. Further, the memory 12 may also include both an internal storage unit of the computer device 1 and an external storage device. The memory 12 may not only be used to store application software and various types of data installed in the computer device 1, such as the code of the information retrieval program, etc., but may also be used to temporarily store data that has been output or is to be output.

[0087] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the computer device 1, and uses various interfaces and lines to connect various components of the entire computer device 1, and executes various functions and processes data of the computer device 1 by running or executing programs or modules (such as executing information retrieval programs, etc.) stored in the memory 12, and calling data stored in the memory 12.

[0088] The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned various information retrieval method embodiments, for example Figure 1 Steps shown.

[0089] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into a preprocessing unit 110, a segmentation unit 111, a vectorization unit 112, an optimization unit 113, a retrieval unit 114, a generation unit 115, a response unit 116, and a feedback unit 117.

[0090] The above-mentioned integrated unit implemented in the form of software function modules can be stored in a computer-readable storage medium. The above-mentioned software function modules are stored in a storage medium and include several instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute the part of the information retrieval method described in each embodiment of the present invention.

[0091] If the module / unit integrated in the computer device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware devices through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0092] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0093] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0094] The blockchain referred to in this invention is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the blockchain underlying platform, platform product service layer, and application service layer.

[0095] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The bus is represented by only one straight line, but it does not mean that there is only one bus or one type of bus. The bus is configured to realize the connection and communication between the memory 12 and at least one processor 13, etc.

[0096] Although not shown, the computer device 1 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 13 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, and power status indicators. The computer device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0097] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the computer device 1 and other computer devices.

[0098] Optionally, the computer device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the computer device 1 and to display a visual user interface.

[0099] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0100] It can be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0101] Combination Figure 1 , the memory 12 in the computer device 1 stores a plurality of instructions to implement an information retrieval method, and the processor 13 can execute the plurality of instructions to implement: Collecting an initial text file and preprocessing the initial text file to obtain data to be processed; Splitting the data to be processed according to the configured segmentation strategy to obtain segmented data; Performing vectorization processing on the segmented data to obtain vector data, and storing the vector data in an embedded database; When an information retrieval request triggered by a target user is received, the information retrieval request is optimized to obtain a target question, including: analyzing the hierarchical structure of the topics contained in the information retrieval request, and establishing a target topic tree according to the hierarchical structure; obtaining user preference data of the target user as target preference data, and obtaining a user portrait of the target user as a target portrait; obtaining a topic priority corresponding to the target portrait, and adjusting the topic priority according to the target preference data to obtain a target priority; optimizing the information retrieval request according to the target priority to obtain the target question; Vectorizing the target question to obtain a question vector, and using the question vector to perform similarity retrieval in the embedded database to obtain similar knowledge; Generate a target prompt word using the similar knowledge and a pre-configured prompt word template; Retrieving a retrieval model according to the information retrieval request, and using the retrieval model to respond based on the target prompt word to obtain an information retrieval result; The information retrieval result is fed back to the target user.

[0102] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0103] It should be noted that the data involved in this case were all obtained legally.

[0104] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0105] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0106] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0108] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0109] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0110] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the present invention can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An information retrieval method, characterized in that: The information retrieval method comprises: Collecting an initial text file and preprocessing the initial text file to obtain data to be processed; Splitting the data to be processed according to the configured segmentation strategy to obtain segmented data; Performing vectorization processing on the segmented data to obtain vector data, and storing the vector data in an embedded database; When an information retrieval request triggered by a target user is received, the information retrieval request is optimized to obtain a target question, including: analyzing the hierarchical structure of the topics contained in the information retrieval request, and establishing a target topic tree according to the hierarchical structure; obtaining user preference data of the target user as target preference data, and obtaining a user portrait of the target user as a target portrait; obtaining a topic priority corresponding to the target portrait, and adjusting the topic priority according to the target preference data to obtain a target priority; optimizing the information retrieval request according to the target priority to obtain the target question; Vectorizing the target question to obtain a question vector, and using the question vector to perform similarity retrieval in the embedded database to obtain similar knowledge; Generate a target prompt word using the similar knowledge and a pre-configured prompt word template; Retrieving a retrieval model according to the information retrieval request, and using the retrieval model to respond based on the target prompt word to obtain an information retrieval result; The information retrieval result is fed back to the target user.

2. The information retrieval method according to claim 1, characterized in that: The collecting of the initial text file and preprocessing the initial text file to obtain the data to be processed includes: Collecting text files in various formats as the initial text files; Extract the text content in the text file of each format using the loader corresponding to each format; The extracted text content is segmented to obtain the data to be processed.

3. The information retrieval method according to claim 1, characterized in that: Before segmenting the data to be processed according to the configured segmentation strategy, the method further includes: Acquire a data source of the data to be processed, and split the data to be processed according to the data source of the data to be processed to obtain at least one sub-data set; Get each file in each sub-dataset and split each file into multiple data fields; The context correlation between every two data fields is calculated, and the relationship between different data fields is marked according to the context correlation between every two data fields.

4. The information retrieval method according to claim 3, characterized in that: The data to be processed is segmented according to the configured segmentation strategy to obtain segmented data including: Obtaining user feedback data and usage habit data of the target user; Obtaining a target strategy from a pre-configured segmentation strategy pool according to the user feedback data and usage habit data of the target user; Using the target strategy to segment the data to be processed to obtain the segmented data; The target strategy includes one or more of the following strategies: intelligent word segmentation strategy, intelligent tagging strategy, and semantic understanding strategy; Wherein, for the intelligent word segmentation strategy, each data field is segmented using a configured word segmentation algorithm; Wherein, for the intelligent labeling strategy, entity labeling is performed on each segmentation obtained after the segmentation process; Among them, for the semantic understanding strategy, intent recognition is performed on each data field to obtain a recognition result, and each data field is segmented according to the recognition result.

5. The information retrieval method according to claim 1, characterized in that: Before obtaining the subject priority corresponding to the target portrait, the method further includes: Obtaining a historical information retrieval request, and analyzing a hierarchical structure of topics included in the historical information retrieval request as a historical hierarchical structure; Establishing a historical theme tree according to the historical hierarchical structure; Searching each historical topic in the historical topic tree respectively to obtain historical search results corresponding to each historical topic; Get pre-built user profiles; For each user in the user portrait, obtaining each user's response to the historical search results corresponding to each historical topic; The priority of each topic in the historical topic tree corresponding to each user is determined according to each user's response to the historical search result corresponding to each historical topic.

6. The information retrieval method according to claim 1, characterized in that: Optimizing the information retrieval request to obtain a target question further includes: Obtaining each metadata in the information retrieval request; Calculate the correlation between every two metadata, and mark each metadata according to the correlation between every two metadata; Expand the retrieval dimension of each marked metadata to obtain multiple retrieval conditions; The information search request is adjusted according to the multiple search conditions to obtain the target question.

7. The information retrieval method according to claim 1, characterized in that: The using the retrieval model to respond based on the target prompt word to obtain the information retrieval result includes: Based on the keyword search technology, the search model is used to respond based on the target prompt word to obtain a first search result; Based on the semantic retrieval technology, the retrieval model is used to respond based on the target prompt word to obtain a second retrieval result; Based on the graph retrieval technology, the retrieval model is used to respond based on the target prompt word to obtain a third retrieval result; Obtaining a first weight corresponding to the keyword search technology, obtaining a second weight corresponding to the semantic search technology, and obtaining a third weight corresponding to the graph search technology; Calculate a weighted sum of the first search result, the second search result, and the third search result according to the first weight, the second weight, and the third weight to obtain a fused search result; Using a prediction model to predict the value of each information unit in the fusion search result; Rearrange each information unit in descending order of its value to obtain an information unit sequence; Acquire the pre-set information units from the information unit sequence to form the information retrieval result; The first weight, the second weight, and the third weight are configured according to the contribution of each search technology to the search results.

8. A computer device, characterized in that: The computer device comprises: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the information retrieval method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the information retrieval method according to any one of claims 1 to 7.

Images