Information retrieval method and device based on reinforcement learning, electronic equipment and medium

Abstract

The embodiment of the invention provides an information retrieval method and device based on reinforcement learning, electronic equipment and a medium, and belongs to the technical field of financial science and technology and artificial intelligence. The method comprises the following steps: performing knowledge retrieval on business retrieval indication information, and performing retrieval behavior value identification on coded retrieval information through a retrieval behavior and a reward factor to obtain a retrieval behavior value; identifying a retrieval stop condition of the coded retrieval information according to the retrieval behavior value; and generating target retrieval data based on the retrieval stop condition and the encoded retrieval information. According to the embodiment of the invention, knowledge retrieval is carried out on the business retrieval indication information, so that the knowledge information most relevant to the retrieval indication can be preliminarily matched, the retrieval stop condition is determined according to the retrieval behavior value, the target retrieval data is generated based on the retrieval stop condition, the optimal stop point during retrieval can be determined, and the retrieval efficiency is improved. According to the method, the retrieval strategy is dynamically adjusted, the output of high-quality data is generated, and the accuracy of information retrieval is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, is applicable to the field of financial technology, and in particular, to an information retrieval method and device, electronic device and medium based on reinforcement learning. Background Art

[0002] At present, the traditional retrieval-augmented generation (RAG) model is usually used to preliminarily collect documents related to the retrieval, merge these documents with the user query, input them into the encoder for encoding, and finally generate answers through the decoder to realize information retrieval based on reinforcement learning. For example, in the financial technology scenario, assuming that the user's retrieval information is: "If the vehicle is damaged by floods, what repair costs will the purchased car insurance cover?" RAG retrieves preliminary documents related to "floods" and "car insurance" from the insurance terms database based on the retrieval information, and generates target retrieval data by encoding and decoding the preliminary documents and retrieval information. According to the full insurance terms of the insurance, if your car is damaged by floods, the insurance will cover the cost of repairing or replacing damaged parts, including the engine, body and interior. However, the retrieval process of this method is usually static, that is, after one retrieval, it enters the retrieval data generation stage, and cannot realize the ability to dynamically adjust the retrieval strategy, resulting in low accuracy of information retrieval. Therefore, how to improve the accuracy of information retrieval has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose an information retrieval method and device, electronic device and medium based on reinforcement learning, aiming to improve the accuracy of information retrieval.

[0004] To achieve the above purpose, a first aspect of an embodiment of the present application proposes an information retrieval method based on reinforcement learning, the method comprising:

[0005] Obtaining business search indication information;

[0006] Performing knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain retrieval knowledge information;

[0007] Encoding the service search indication information and the search knowledge information to obtain encoded search information;

[0008] Acquire the retrieval action instruction of the encoded retrieval information as the retrieval behavior of the intelligent agent, and acquire the reward parameter of the retrieval behavior as the reward factor of the intelligent agent;

[0009] Identify the retrieval behavior value of the encoded retrieval information through the retrieval behavior and the reward factor to obtain the retrieval behavior value;

[0010] Identifying a retrieval stop condition for the coded retrieval information according to the retrieval behavior value;

[0011] The target search data of the service search instruction information is generated based on the search stop condition and the coded search information.

[0012] In some embodiments, the identifying the retrieval behavior value of the encoded retrieval information through the retrieval behavior and the reward factor to obtain the retrieval behavior value includes:

[0013] Obtaining an expected value of a retrieval behavior of the encoded retrieval information in a future state;

[0014] Obtaining a discount factor and a learning rate of the encoded retrieval information of the current state;

[0015] The retrieval behavior value of the encoded retrieval information in the current state is identified according to the discount factor, the learning rate, the retrieval behavior, the reward factor and the expected value of the retrieval behavior to obtain the retrieval behavior value.

[0016] In some embodiments, the retrieval behavior value includes a continue retrieval behavior value and a stop retrieval behavior value;

[0017] The step of identifying the retrieval stop condition of the coded retrieval information according to the retrieval behavior value includes:

[0018] Compare the continue retrieval behavior value and the stop retrieval behavior value to obtain behavior value comparison data; the behavior value comparison data is used to indicate that the continue retrieval behavior value is greater than the stop retrieval behavior value or the continue retrieval behavior value is less than or equal to the stop retrieval behavior value;

[0019] When the behavior value comparison data is used to indicate that the continuing search behavior value is greater than the stopping search behavior value, continuing to search for the coded search information;

[0020] When the behavior value comparison data is used to indicate that the continue retrieval behavior value is less than or equal to the stop retrieval behavior value, the retrieval of the coded retrieval information is stopped, and the continue retrieval behavior value less than or equal to the stop retrieval behavior value is used as the stop retrieval behavior value.

[0021] In some embodiments, performing knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain the retrieved knowledge information includes:

[0022] Acquire the number of search terms in the business search indication information, and acquire business knowledge documents and search knowledge documents containing the number of search terms from the knowledge base;

[0023] Obtain the search term frequency of the search term number appearing in the business knowledge document;

[0024] Determining the inverse document frequency of the number of search terms according to the proportion of the search knowledge documents in the business knowledge documents;

[0025] The correlation scores between the business knowledge documents and the business search indication information are obtained according to the search term frequency and the inverse document frequency, and the business knowledge document with the highest score is selected from the correlation scores as the search knowledge information.

[0026] In some embodiments, obtaining the reward parameter of the retrieval behavior as the reward factor of the agent includes:

[0027] Obtaining the penalty coefficient, penalty growth rate, number of searches, search text quality score and quality score weight of the search behavior as the reward parameters;

[0028] A reward factor for the agent is generated according to the reward parameter.

[0029] In some embodiments, encoding the service search indication information and the search knowledge information to obtain the encoded search information includes:

[0030] splicing the business search indication information and the search knowledge information to obtain spliced ​​search information;

[0031] Performing word segmentation processing on the concatenated search information to obtain word segmentation search information;

[0032] Performing word embedding processing on the word segmentation retrieval information to obtain a retrieval information vector;

[0033] Position encoding is performed on the retrieval information vector to obtain the encoded retrieval information.

[0034] In some embodiments, the generating the target retrieval data of the service retrieval indication information based on the retrieval stop condition and the coded retrieval information includes:

[0035] Acquire initial search information of the service search indication information based on the search stop condition;

[0036] Merging the initial search information and the coded search information to obtain fused search indication information;

[0037] Performing attention processing on the fused retrieval indication information to obtain a fused retrieval indication feature;

[0038] The fusion search indication feature is subjected to search word prediction to obtain a target search word sequence, and the target search word sequence is converted into the target search data.

[0039] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes an information retrieval device based on reinforcement learning, the device comprising:

[0040] A retrieval indication information acquisition module, used to acquire business retrieval indication information;

[0041] A knowledge retrieval module, used to perform knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain retrieved knowledge information;

[0042] An encoding module, used for encoding the service search indication information and the search knowledge information to obtain encoded search information;

[0043] A retrieval behavior and reward factor acquisition module, used to obtain the retrieval action instruction of the encoded retrieval information as the retrieval behavior of the intelligent agent, and obtain the reward parameter of the retrieval behavior as the reward factor of the intelligent agent;

[0044] A retrieval behavior value identification module is used to identify the retrieval behavior value of the coded retrieval information through the continued retrieval behavior and the reward factor to obtain the continued retrieval behavior value; and to identify the retrieval behavior value of the coded retrieval through the stopped retrieval behavior and the reward factor to obtain the stopped retrieval behavior value;

[0045] A stop search condition identification module, used for identifying a stop search condition of the coded search information according to the continue search behavior value and the stop search behavior value;

[0046] A search information generating module is used to generate target search data of the service search indication information based on the search stop condition and the coded search information.

[0047] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0048] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0049] The information retrieval method and device, electronic device and medium based on reinforcement learning proposed in the present application obtain business retrieval indication information; perform knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain retrieval knowledge information; encode the business retrieval indication information and the retrieval knowledge information to obtain encoded retrieval information; obtain the retrieval action instructions of the encoded retrieval information as the retrieval behavior of the intelligent agent, and obtain the reward parameters of the retrieval behavior as the reward factor of the intelligent agent; identify the retrieval behavior value of the encoded retrieval information through the retrieval behavior and the reward factor to obtain the retrieval behavior value; identify the retrieval stop condition of the encoded retrieval information according to the retrieval behavior value; and generate target retrieval data of the business retrieval indication information based on the retrieval stop condition and the encoded retrieval information. The present application firstly performs knowledge retrieval on the business retrieval indication information from a preset knowledge base, and can preliminarily match the knowledge information most relevant to the retrieval indication; secondly, by identifying the value of the retrieval behavior through the retrieval behavior and the reward factor, it can be determined whether it is more valuable to continue the retrieval or stop the retrieval, which is helpful for the subsequent optimization of the retrieval strategy and facilitates the improvement of the accuracy of information retrieval based on reinforcement learning; finally, the retrieval stop condition is determined according to the retrieval behavior value, and the target retrieval data is generated based on the retrieval stop condition, which can determine the optimal stopping point during the retrieval, realize the dynamic adjustment of the retrieval strategy, and generate high-quality data output, which significantly improves the accuracy of information retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flowchart of an information retrieval method based on reinforcement learning provided in an embodiment of the present application;

[0051] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0052] Figure 3 yes Figure 1 Flow chart of step S103 in FIG.

[0053] Figure 4 yes Figure 1 Flow chart of step S104 in FIG.

[0054] Figure 5 yes Figure 1 Flow chart of step S105 in FIG.

[0055] Figure 6 yes Figure 1 Flow chart of step S106 in FIG.

[0056] Figure 7 yes Figure 1 Flow chart of step S107 in FIG.

[0057] Figure 8is a schematic diagram of the structure of an information retrieval device based on reinforcement learning provided in an embodiment of the present application;

[0058] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0062] First, some nouns involved in this application are analyzed:

[0063] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0064] Based on this, the embodiments of the present application provide an information retrieval method and device, an electronic device and a medium based on reinforcement learning, aiming to improve the accuracy of information retrieval.

[0065] The information retrieval method and device based on reinforcement learning, electronic device and medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the information retrieval method based on reinforcement learning in the embodiments of the present application is described.

[0066] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0067] The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, processing / interaction systems, mechatronics, etc. Artificial intelligence software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0068] The information retrieval method based on reinforcement learning provided in the embodiment of the present application relates to the field of artificial intelligence technology. The information retrieval method based on reinforcement learning provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements an information retrieval method based on reinforcement learning, etc., but is not limited to the above forms.

[0069] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0070] Figure 1 is an optional flowchart of the information retrieval method based on reinforcement learning provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S107.

[0071] Step S101: obtaining service search indication information.

[0072] Step S102: perform knowledge search on the business search indication information from a preset knowledge base to obtain search knowledge information.

[0073] Step S103, encoding the service search indication information and the search knowledge information to obtain encoded search information.

[0074] Step S104, obtaining the retrieval action instruction of the encoded retrieval information as the retrieval behavior of the intelligent agent, and obtaining the reward parameter of the retrieval behavior as the reward factor of the intelligent agent.

[0075] Step S105, identifying the retrieval behavior value of the encoded retrieval information through the retrieval behavior and the reward factor to obtain the retrieval behavior value.

[0076] Step S106, identifying the retrieval stop condition of the coded retrieval information according to the retrieval behavior value.

[0077] Step S107: generating target retrieval data of the service retrieval indication information based on the retrieval stop condition and the coded retrieval information.

[0078] Steps S101 to S107 shown in the embodiment of the present application are as follows: obtaining business retrieval indication information; performing knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain retrieval knowledge information; encoding the business retrieval indication information and the retrieval knowledge information to obtain encoded retrieval information; obtaining retrieval action instructions of the encoded retrieval information as the retrieval behavior of the intelligent agent, and obtaining reward parameters of the retrieval behavior as the reward factor of the intelligent agent; identifying the retrieval behavior value of the encoded retrieval information through the retrieval behavior and the reward factor to obtain the retrieval behavior value; identifying the retrieval stop condition of the encoded retrieval information according to the retrieval behavior value; and generating target retrieval data for the business retrieval indication information based on the retrieval stop condition and the encoded retrieval information. The present application firstly performs knowledge retrieval on the business retrieval indication information from a preset knowledge base, and can preliminarily match the knowledge information most relevant to the retrieval indication; secondly, by identifying the value of the retrieval behavior through the retrieval behavior and the reward factor, it can be determined whether it is more valuable to continue the retrieval or stop the retrieval, which is helpful for the subsequent optimization of the retrieval strategy and facilitates the improvement of the accuracy of information retrieval based on reinforcement learning; finally, the retrieval stop condition is determined according to the retrieval behavior value, and the target retrieval data is generated based on the retrieval stop condition, which can determine the optimal stopping point during the retrieval, realize the dynamic adjustment of the retrieval strategy, and generate high-quality data output, which significantly improves the accuracy of information retrieval.

[0079] In step S101 of some embodiments, specifically, the business retrieval indication information is used to instruct the agent to generate corresponding information retrieval data.

[0080] For example, in the financial field, the business search indication information may be: I want to purchase a travel insurance policy that can cover my medical expenses during the travel period.

[0081] See also Figure 2 In some embodiments, step S102 includes but is not limited to steps S201 to S204:

[0082] Step S201, obtaining the number of search terms in the business search indication information, and obtaining the business knowledge document and the search knowledge document containing the number of search terms from the knowledge base.

[0083] Step S202, obtaining the search term frequency of the search term number appearing in the business knowledge document.

[0084] Step S203: determining the inverse document frequency of the number of search terms according to the proportion of the search knowledge documents in the business knowledge documents.

[0085] Step S204: Obtain the correlation scores between the business knowledge documents and the business search indication information according to the search term frequency and the inverse document frequency, and select the business knowledge document with the highest score from the correlation scores as the search knowledge information.

[0086] In step S201 of some embodiments, specifically, the number of search terms refers to the number of search terms in the business search indication information.

[0087] For example, in the field of finance and insurance, the search terms in the business search indication information include health insurance, annuity, and investment return rate, etc.

[0088] Specifically, the knowledge base may be a database for an insurance company to store and manage insurance product information.

[0089] Specifically, business knowledge documents refer to all documents stored in the knowledge base.

[0090] Specifically, the retrieval knowledge document refers to a single document or record information containing retrieval term data in the knowledge base.

[0091] For example, in the field of financial insurance, if the search term is auto insurance, the searched knowledge documents are detailed information documents of auto insurance products (including auto insurance terms, coverage, price and deductible, etc.), auto insurance products evaluated and fed back by users, and auto insurance-related regulations, etc.

[0092] In step S202 of some embodiments, specifically, the search term frequency refers to the number of times a search term appears in a search knowledge document.

[0093] Furthermore, in order to eliminate the influence of document length on word frequency, the word frequency can be normalized by dividing the word frequency by the total number of words in the business knowledge document.

[0094] In step S203 of some embodiments, specifically, the inverse document frequency is used to measure the prevalence of a search term in all documents. If a term appears in many search knowledge documents, its inverse document frequency value will be lower, otherwise it will be higher.

[0095] Specifically, the inverse document frequency can be determined by the following formula:

[0096]

[0097] Among them, IDF(q i ) represents the inverse document frequency of the i-th search term, q i represents the i-th search term, N represents the total number of business knowledge documents, m(q i ) represents the number of retrieved knowledge documents containing the i-th search term.

[0098] In this embodiment, the inverse document frequency of the number of search terms is determined according to the proportion of the searched knowledge documents in the business knowledge documents, which helps to identify terms that are uncommon in documents but important in specific documents, and can improve the search ranking of documents containing rare but important information, thereby improving the quality of knowledge retrieval data.

[0099] In step S204 of some embodiments, specifically, the relevance score is used to describe the degree of association between all business knowledge documents and the search terms in the business search indication information.

[0100] Specifically, the relevance score between each search term and each business knowledge document can be determined by calculating the relevance score, and these scores are added together to obtain the relevance score between the entire business search indication information and the business knowledge document. The higher the score, the more relevant the business knowledge document is to the business search indication information.

[0101] Specifically, the relevance score can be determined by the following formula:

[0102]

[0103] Among them, Score(D,Q) represents the relevance score between the business knowledge document D and the business search indication information Q, n represents the total number of search terms in the business search indication information Q, and IDF(q i ) represents the inverse document frequency of the i-th search term, q i represents the i-th search term, f(q i ,D) represents the frequency of the i-th search term in the business knowledge document D, k1 represents the term frequency adjustment parameter, b represents the business knowledge document length adjustment parameter, |D| represents the length of the business knowledge document, and avgdl represents the average length of all business knowledge documents in the knowledge base.

[0104] Specifically, the business knowledge document with the highest score may be selected as the retrieval knowledge information.

[0105] In this embodiment, the correlation score between the business knowledge document and the business retrieval indication information is obtained based on the search term frequency and the inverse document frequency, which can accurately measure the degree of correlation between the business knowledge document and the business retrieval indication information, and facilitate effectively distinguishing which business knowledge documents are more in line with the user's query intention, thereby providing more accurate knowledge retrieval results.

[0106] See also Figure 3 In some embodiments, step S102 includes but is not limited to steps S301 to S304:

[0107] Step S301, splicing the business search indication information and the search knowledge information to obtain spliced ​​search information.

[0108] Step S302, performing word segmentation processing on the concatenated search information to obtain word segmentation search information.

[0109] Step S303: perform word embedding processing on the word segmentation retrieval information to obtain a retrieval information vector.

[0110] Step S304: Position-encode the search information vector to obtain encoded search information.

[0111] In step S301 of some embodiments, specifically, the spliced ​​retrieval information refers to information obtained by splicing the business retrieval instruction information and the retrieval knowledge information.

[0112] For example, in the financial field, if the business search indication information is what health insurance products are recommended, then relevant health insurance documents are found from the knowledge base, and the health insurance documents are spliced ​​with what health insurance products are recommended to form health insurance spliced ​​search information.

[0113] In step S302 of some embodiments, specifically, segmenting the search information refers to segmenting the concatenated search information into a plurality of tokens having semantic meanings.

[0114] Specifically, the word segmentation retrieval information can be implemented by segmenting the spliced ​​retrieval information through the Tokenize (word segmentation operation) of the coding layer, that is, the word segmentation operation is implemented through Tokenize (Q⊕D), where Q represents business retrieval indication information, D represents business knowledge documents, and ⊕ represents splicing processing.

[0115] In step S303 of some embodiments, specifically, the retrieval information vector refers to the word segmentation retrieval information represented by a high-dimensional vector space, and the word segmentation retrieval information represented by the retrieval information vector can capture the semantic information between words.

[0116] Specifically, word embedding processing can be achieved by embedding the word segmentation retrieval information through a reinforcement learning agent, that is, word embedding processing can be achieved through Embedding (Tokenize (Q⊕D)).

[0117] In step S304 of some embodiments, specifically, position encoding is a process of adding position information to the retrieval information vector, so that the information retrieval model can understand the order of words in the sentence.

[0118] Specifically, for each position of the retrieval information vector, two position vectors are generated using different frequencies of the sine function and the cosine function, and these position vectors are added to the word embedding vector, so that the information retrieval model can distinguish the semantic differences of the same word embedding at different positions.

[0119] In this embodiment, by position encoding the retrieval information vector, the semantic order information of the retrieval information vector can be captured, and the contextual semantics of the retrieval information vector can be learned to more accurately understand the sentence structure of the spliced ​​retrieval information, thereby accurately identifying the semantic meaning of the spliced ​​retrieval information.

[0120] In step S104 of some embodiments, specifically, since the search action instruction includes a continue search instruction and a stop search instruction, the search behavior includes a continue search behavior and a stop search behavior.

[0121] Specifically, in reinforcement learning (RL), the agent is the learning subject in reinforcement learning, responsible for selecting and executing actions in the environment, and adjusting the behavior strategy according to the feedback of the action to maximize the cumulative reward.

[0122] Specifically, in reinforcement learning, behavior is the action taken by an agent in a specific state. The action space of the agent usually includes two behaviors: continue retrieval and stop retrieval. The agent needs to make a choice between these two retrieval behaviors.

[0123] See also Figure 4 In some embodiments, step S104 includes but is not limited to steps S401 to S402:

[0124] Step S401, obtaining the penalty coefficient, penalty growth rate, number of searches, search text quality score and quality score weight of the search behavior as reward parameters.

[0125] Step S402, generating a reward factor for the agent according to the reward parameter.

[0126] In step S401 of some embodiments, specifically, the penalty coefficient represents a penalty coefficient for continuing the retrieval behavior.

[0127] Specifically, the penalty growth rate increases as the number of retrievals increases.

[0128] Specifically, the retrieval text quality score refers to the retrieved text quality score associated with the encoded retrieval information.

[0129] Specifically, in reinforcement learning, these reward parameters can provide immediate reward feedback to the agent's behavior.

[0130] In step S402 of some embodiments, the reward factor may be determined by the following formula:

[0131] r=(1-a)*(β*QualityScore(output)-C)-a*(δ+λ*L)

[0132] Among them, r is the reward factor after taking the retrieval behavior a, a represents the retrieval behavior of the agent, including continuing the retrieval behavior and stopping the retrieval to generate the retrieval text behavior, δ represents the penalty coefficient for continuing the retrieval, λ represents the penalty growth rate, and the penalty increases with the increase of the number of retrievals, L represents the number of retrievals so far, QualityScore (output) represents the frequency function of the retrieval text quality score, β represents the quality score weight, and C represents the fixed cost of the retrieval.

[0133] For example, in the interaction between retrieval behavior and reward factor, when the retrieval behavior a=1 (continue retrieval), the reward factor is a negative value, that is, the penalty for continuing retrieval, and the size of the reward factor increases with the increase in the number of retrievals. When a=0 (stop retrieval and generate retrieval text), the reward factor is determined based on the retrieval text quality score generated after the quality score weight is adjusted minus the fixed cost C.

[0134] Specifically, the reward factor represents the degree of goodness of the agent's retrieval behavior in the encoding state of encoding retrieval information, and the reward factor can be positive, negative or zero. By adjusting the size and direction of the reward factor, the agent can be guided to learn the correct behavioral strategy, which helps to improve the accuracy of subsequent information retrieval.

[0135] See also Figure 5 In some embodiments, step S105 includes but is not limited to steps S501 to S503:

[0136] Step S501, obtaining the expected value of the retrieval behavior of the encoded retrieval information of the future state.

[0137] Step S502, obtaining the discount factor and learning rate of the coding retrieval information of the current state.

[0138] Step S503, identifying the retrieval behavior value of the encoded retrieval information in the current state according to the discount factor, learning rate, retrieval behavior, reward factor and retrieval behavior expected value to obtain the retrieval behavior value.

[0139] In step S501 of some embodiments, specifically, the agent uses the encoded retrieval information and retrieval behavior of the current state to preset the encoded retrieval information of the future state. For each possible future state, the agent determines the expected value of the retrieval behavior of the encoded retrieval information of the future state by estimating the sum of future reward factors.

[0140] Furthermore, the expected value of retrieval behavior represents the expectation of future rewards and is used to guide the agent's retrieval decision process, helping the agent to select the retrieval action that is most likely to bring the highest cumulative reward.

[0141] In step S502 of some embodiments, specifically, the intelligent agent first needs to encode the business search indication information and the search knowledge information through T5-Encoder to form the encoded search information of the current state.

[0142] Specifically, the discount factor is used to indicate the importance of future rewards. The closer the discount factor is to 1, the more foresighted the value of future states will be, and the closer the discount factor is to 0, the more the value of the current state will be considered.

[0143] Specifically, the learning rate is used to control the degree to which the encoded retrieval information of the future state affects the expected value of the retrieval behavior.

[0144] In step S503 of some embodiments, specifically, the retrieval behavior value refers to the expected long-term benefit value of the retrieval behavior of the encoded retrieval information in the current state.

[0145] Specifically, DQN (Deep Q-Network) decides whether to continue retrieving information or stop retrieving and generate retrieval text. Furthermore, DQN evaluates possible retrieval behaviors based on the encoded retrieval information of the current state, and selects corresponding retrieval behaviors to maximize the expected reward of future retrieval.

[0146] Specifically, in reinforcement learning, DQN is used to help the agent make retrieval decisions in the environment. DQN selects the optimal retrieval behavior by evaluating the potential value of different retrieval behaviors to maximize the expected reward of future retrieval.

[0147] Specifically, the retrieval behavior value can be expressed by the following formula:

[0148] Q(s,a)←Q(s,a)+α[r+γmax a′ Q(s ′ ,a ′ )-Q(s,a)]

[0149] Among them, Q(s,a) represents the retrieval behavior value of taking retrieval behavior a under the encoded retrieval information s of the current state, s represents the encoded retrieval information of the current state, a represents the retrieval behavior of the agent, including continuing retrieval behavior and stopping retrieval to generate retrieval text behavior, r represents the reward factor after taking retrieval behavior a, α represents the learning rate, γ represents the discount factor, max a′ Q(s ′ ,a ′ ) represents the encoded retrieval information of the future state s ′ The future retrieval behavior a ′ The expected value of the retrieval behavior, s ′ Represents the encoded retrieval information of the future state, a′ Indicates future retrieval behavior.

[0150] For example, in the financial field, the encoded retrieval information is the coverage of health insurance. The agent evaluates the retrieval behavior value of continuing the retrieval behavior or stopping the retrieval behavior. If the agent chooses to continue the retrieval and retrieves documents related to health insurance from the knowledge base, the agent will receive a corresponding reward based on the relevance of the retrieved health insurance-related documents. The agent updates the retrieval behavior value based on the reward and the expected value of future retrieval behaviors to maximize the retrieval reward of the cumulative retrieval behavior.

[0151] In this embodiment, the retrieval behavior value of the encoded retrieval information in the current state is identified based on the discount factor, learning rate, retrieval behavior, reward factor and expected value of the retrieval behavior to obtain the retrieval behavior value. By setting gradually increasing penalties and rewards for continued retrieval, the intelligent agent is motivated to find the optimal stopping retrieval point, which can balance the retrieval depth and retrieval quality, help to subsequently optimize the retrieval strategy, and facilitate improving the accuracy of information retrieval based on reinforcement learning.

[0152] See also Figure 6 In some embodiments, step S106 includes but is not limited to steps S601 to S603:

[0153] In step S601, the continue retrieval behavior value and the stop retrieval behavior value are compared to obtain behavior value comparison data; the behavior value comparison data is used to indicate that the continue retrieval behavior value is greater than the stop retrieval behavior value or the continue retrieval behavior value is less than or equal to the stop retrieval behavior value.

[0154] Step S602, when the behavior value comparison data is used to indicate that the continuing search behavior value is greater than the stopping search behavior value, the coded search information is continued to be searched.

[0155] Step S603, when the behavior value comparison data is used to indicate that the continue retrieval behavior value is less than or equal to the stop retrieval behavior value, the coded retrieval information is stopped, and the continue retrieval behavior value is less than or equal to the stop retrieval behavior value as the stop retrieval behavior value.

[0156] In step S601 of some embodiments, specifically, retrieving the behavior value includes continuing to retrieve the behavior value and stopping retrieving the behavior value.

[0157] Specifically, the behavior value comparison data is used to indicate that the continuing retrieval behavior value is greater than the stopping retrieval behavior value or the continuing retrieval behavior value is less than or equal to the stopping retrieval behavior value.

[0158] Specifically, the agent will compare the value of the continued retrieval behavior with the value of the stopped retrieval behavior. If the value of the continued retrieval behavior is greater than the value of the stopped retrieval behavior, the agent will continue the retrieval. If the value of the continued retrieval behavior is less than or equal to the value of the stopped retrieval behavior, the agent will stop the retrieval.

[0159] In this embodiment, by comparing the value of continuing the search behavior and the value of stopping the search behavior, the behavior value comparison data is obtained, which can quantify the potential benefits of each search behavior, realize dynamic adjustment of the search strategy, and facilitate improving the accuracy of information retrieval.

[0160] In step S602 of some embodiments, specifically, when the behavior value comparison data is used to indicate that the behavior value of continuing to retrieve is greater than the behavior value of stopping to retrieve, and the penalty term in the reward factor is not sufficient to offset the potential positive reward in the future, the agent will choose to continue to retrieve, that is, the agent predicts that further retrieval can bring higher cumulative rewards.

[0161] For example, in the financial field, if the current state of the coded retrieval information is what the health insurance coverage is, then the agent evaluates that continuing to search can retrieve more information about "health insurance", and the potential value that may be obtained includes finding more detailed health insurance terms, health insurance customer evaluations, and health insurance market analysis reports.

[0162] In this embodiment, when the behavior value comparison data is used to indicate that the value of continuing the search behavior is greater than the value of stopping the search behavior, the coded search information can be further searched, which helps to deeply explore more comprehensive and detailed search information, thereby providing more accurate information retrieval data.

[0163] In step S603 of some embodiments, specifically, when the behavior value comparison data is used to indicate that the value of continuing the retrieval behavior is less than or equal to the value of stopping the retrieval behavior, that is, the penalty term in the reward factor is greater than the potential positive reward in the future, the agent will choose to stop the retrieval, that is, the agent predicts that the currently retrieved information has met the retrieval requirements, or that continuing the retrieval will not significantly improve the relevance and completeness of the retrieved information.

[0164] For example, in the financial field, if the current state of the coded retrieval information is what the health insurance coverage is, after stopping the search, the intelligent agent outputs detailed health insurance information such as basic health insurance information (including health insurance product name, product coverage), insurance liability, liability exemption, insurance period, main terms of the waiting period, claims process, claims conditions, etc.

[0165] In this embodiment, when the behavior value comparison data is used to indicate that the value of continuing the search behavior is less than or equal to the value of stopping the search behavior, the search for the coded search information is stopped, which can quickly provide the best information retrieval data and effectively avoid excessive retrieval, thereby ensuring the accuracy of information retrieval and improving the efficiency of information retrieval.

[0166] See also Figure 7 In some embodiments, step S107 includes but is not limited to steps S701 to S704:

[0167] Step S701: acquiring initial search information of service search indication information based on a search stop condition.

[0168] Step S702: merge the initial search information and the coded search information to obtain merged search indication information.

[0169] Step S703, performing attention processing on the fused retrieval indication information to obtain fused retrieval indication features.

[0170] Step S704, performing search word prediction on the fused search indication feature to obtain a target search word sequence, and converting the target search word sequence into target search data.

[0171] In step S701 of some embodiments, specifically, the initial retrieval information refers to retrieval information generated by further retrieval of the DQN network.

[0172] For example, regarding the coverage of health insurance, the initial search information further provides detailed information such as health insurance terms, insured persons, effective time, and claim conditions.

[0173] In step S702 of some embodiments, specifically, the fused search indication information includes initial search information and encoded search information.

[0174] For example, regarding what the health insurance covers, the initial search information provides detailed information on health insurance terms, insured persons, insured persons, effective time, claims conditions, etc., and the coded search information provides health insurance documents related to health insurance (such as health insurance contract documents).

[0175] In step S703 of some embodiments, specifically, since the fusion retrieval indication information is represented by an encoded high-dimensional vector space, it needs to be converted into natural language text.

[0176] Specifically, the fused retrieval indication feature refers to a low-dimensional space vector representation of the fused retrieval indication information. By performing Attention processing on the fused retrieval indication information, it is ensured that the subsequently generated retrieval data accurately reflects the retrieved information.

[0177] In step S704 of some embodiments, specifically, the retrieval word (i.e., retrieval tokens) can be predicted starting from the start tag [START], and the next word can be predicted repeatedly, and so on, until the end tag is generated or the maximum text output length is reached, a target token sequence is constructed, and the target token sequence is converted back to text form through an activation function (such as a Relu activation function) to form the final target retrieval data.

[0178] For example, in the financial field, the business search indication information is whether the travel insurance I purchased can cover the medical expenses of my illness when traveling abroad. The target search data is based on the travel insurance you purchased. If you are infected with a disease during your trip abroad, the travel insurance will cover the relevant treatment expenses, including hospitalization and outpatient treatment. For specific coverage and limits, please refer to your travel insurance policy details.

[0179] In this embodiment, by performing search word prediction on the fused search indication features to obtain a target search word sequence, and converting the target search word sequence into target search data, the encoded high-dimensional vector representation can be converted into accurate, relevant and high-quality search data, thereby significantly improving the accuracy of information retrieval.

[0180] The embodiment of the present application obtains business retrieval indication information; performs knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain retrieval knowledge information; encodes the business retrieval indication information and the retrieval knowledge information to obtain the encoded retrieval information; obtains the retrieval action instruction of the encoded retrieval information as the retrieval behavior of the intelligent body, obtains the reward parameter of the retrieval behavior as the reward factor of the intelligent body; identifies the retrieval behavior value of the encoded retrieval information through the retrieval behavior and the reward factor to obtain the retrieval behavior value; identifies the retrieval stop condition of the encoded retrieval information according to the retrieval behavior value; generates the target retrieval data of the business retrieval indication information based on the retrieval stop condition and the encoded retrieval information. The present application first performs knowledge retrieval on the business retrieval indication information from a preset knowledge base, and can preliminarily match the most relevant knowledge information of the retrieval indication; secondly, identifies the retrieval behavior value through the retrieval behavior and the reward factor, and can determine whether it is more valuable to continue the retrieval or stop the retrieval, which is helpful for the subsequent optimization of the retrieval strategy and is convenient for improving the accuracy of information retrieval based on reinforcement learning; finally, determines the retrieval stop condition according to the retrieval behavior value, and generates the target retrieval data based on the retrieval stop condition, which can determine the optimal stop point during the retrieval, realizes the dynamic adjustment of the retrieval strategy, and generates the output of high-quality data, which significantly improves the accuracy of information retrieval.

[0181] See also Figure 8 The embodiment of the present application further provides an information retrieval device based on reinforcement learning, which can implement the above-mentioned information retrieval method based on reinforcement learning, and the device includes:

[0182] A retrieval indication information acquisition module, used to acquire business retrieval indication information;

[0183] The knowledge retrieval module is used to perform knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain the retrieved knowledge information;

[0184] The encoding module is used to encode the business search indication information and the search knowledge information to obtain the encoded search information;

[0185] A retrieval behavior and reward factor acquisition module is used to obtain the retrieval action instructions encoding the retrieval information as the retrieval behavior of the intelligent agent, and obtain the reward parameters of the retrieval behavior as the reward factor of the intelligent agent;

[0186] The retrieval behavior value identification module is used to identify the retrieval behavior value of the coded retrieval information through the continued retrieval behavior and the reward factor to obtain the continued retrieval behavior value; and to identify the retrieval behavior value of the coded retrieval through the stopped retrieval behavior and the reward factor to obtain the stopped retrieval behavior value;

[0187] A stop retrieval condition identification module, used for identifying a retrieval stop condition of the coded retrieval information according to a continuing retrieval behavior value and a stop retrieval behavior value;

[0188] The retrieval information generating module is used to generate target retrieval data of the service retrieval indication information based on the retrieval stop condition and the coded retrieval information.

[0189] The specific implementation of the information retrieval device based on reinforcement learning is basically the same as the specific implementation of the information retrieval method based on reinforcement learning, and will not be repeated here.

[0190] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned information retrieval method based on reinforcement learning when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0191] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0192] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0193] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store processing systems and other applications. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the information retrieval method based on reinforcement learning in the embodiments of this application;

[0194] Input / output interface 903, used to implement information input and output;

[0195] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0196] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0197] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0198] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned information retrieval method based on reinforcement learning is implemented.

[0199] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0200] The embodiments of the present application provide an information retrieval method based on reinforcement learning, an information retrieval device based on reinforcement learning, an electronic device and a storage medium, which obtain business retrieval indication information; perform knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain retrieval knowledge information; encode the business retrieval indication information and the retrieval knowledge information to obtain encoded retrieval information; obtain retrieval action instructions of the encoded retrieval information as the retrieval behavior of the intelligent agent, and obtain reward parameters of the retrieval behavior as the reward factor of the intelligent agent; identify the retrieval behavior value of the encoded retrieval information through the retrieval behavior and the reward factor to obtain the retrieval behavior value; identify the retrieval stop condition of the encoded retrieval information according to the retrieval behavior value; and generate target retrieval data of the business retrieval indication information based on the retrieval stop condition and the encoded retrieval information. The present application firstly performs knowledge retrieval on the business retrieval indication information from a preset knowledge base, and can preliminarily match the knowledge information most relevant to the retrieval indication; secondly, by identifying the value of the retrieval behavior through the retrieval behavior and the reward factor, it can be determined whether it is more valuable to continue the retrieval or stop the retrieval, which is helpful for the subsequent optimization of the retrieval strategy and facilitates the improvement of the accuracy of information retrieval based on reinforcement learning; finally, the retrieval stop condition is determined according to the retrieval behavior value, and the target retrieval data is generated based on the retrieval stop condition, which can determine the optimal stopping point during the retrieval, realize the dynamic adjustment of the retrieval strategy, and generate high-quality data output, which significantly improves the accuracy of information retrieval.

[0201] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0202] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0203] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0204] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0205] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0206] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0207] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0208] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0210] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0211] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. An information retrieval method based on reinforcement learning, characterized in that: The method comprises: Obtaining business search indication information; Performing knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain retrieval knowledge information; Encoding the service search indication information and the search knowledge information to obtain encoded search information; Acquire the retrieval action instruction of the encoded retrieval information as the retrieval behavior of the intelligent agent, and acquire the reward parameter of the retrieval behavior as the reward factor of the intelligent agent; Identify the retrieval behavior value of the encoded retrieval information through the retrieval behavior and the reward factor to obtain the retrieval behavior value; Identifying a retrieval stop condition for the coded retrieval information according to the retrieval behavior value; The target search data of the service search instruction information is generated based on the search stop condition and the coded search information.

2. The method according to claim 1, characterized in that The step of identifying the retrieval behavior value of the encoded retrieval information through the retrieval behavior and the reward factor to obtain the retrieval behavior value includes: Obtaining an expected value of a retrieval behavior of the encoded retrieval information in a future state; Obtaining a discount factor and a learning rate of the encoded retrieval information of the current state; The retrieval behavior value of the encoded retrieval information in the current state is identified according to the discount factor, the learning rate, the retrieval behavior, the reward factor and the expected value of the retrieval behavior to obtain the retrieval behavior value.

3. The method according to claim 1, characterized in that The retrieval behavior value includes a continuing retrieval behavior value and a stopping retrieval behavior value; The step of identifying the retrieval stop condition of the coded retrieval information according to the retrieval behavior value includes: Compare the continue retrieval behavior value and the stop retrieval behavior value to obtain behavior value comparison data; the behavior value comparison data is used to indicate that the continue retrieval behavior value is greater than the stop retrieval behavior value or the continue retrieval behavior value is less than or equal to the stop retrieval behavior value; When the behavior value comparison data is used to indicate that the continuing search behavior value is greater than the stopping search behavior value, continuing to search for the coded search information; When the behavior value comparison data is used to indicate that the continue retrieval behavior value is less than or equal to the stop retrieval behavior value, the retrieval of the coded retrieval information is stopped, and the continue retrieval behavior value less than or equal to the stop retrieval behavior value is used as the stop retrieval behavior value.

4. The method according to claim 1, characterized in that The performing knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain the retrieved knowledge information includes: Acquire the number of search terms in the business search indication information, and acquire business knowledge documents and search knowledge documents containing the number of search terms from the knowledge base; Obtain the search term frequency of the search term number appearing in the business knowledge document; Determining the inverse document frequency of the number of search terms according to the proportion of the search knowledge documents in the business knowledge documents; The correlation scores between the business knowledge documents and the business search indication information are obtained according to the search term frequency and the inverse document frequency, and the business knowledge document with the highest score is selected from the correlation scores as the search knowledge information.

5. The method according to claim 1, characterized in that The step of obtaining the reward parameter of the retrieval behavior as the reward factor of the agent includes: Obtaining the penalty coefficient, penalty growth rate, number of searches, search text quality score and quality score weight of the search behavior as the reward parameters; A reward factor for the agent is generated according to the reward parameter.

6. The method according to any one of claims 1 to 5, characterized in that: The encoding of the service search indication information and the search knowledge information to obtain the encoded search information includes: splicing the business search indication information and the search knowledge information to obtain spliced ​​search information; Performing word segmentation processing on the concatenated search information to obtain word segmentation search information; Performing word embedding processing on the word segmentation retrieval information to obtain a retrieval information vector; Position encoding is performed on the retrieval information vector to obtain the encoded retrieval information.

7. The method according to any one of claims 1 to 5, characterized in that: The generating of the target retrieval data of the service retrieval indication information based on the retrieval stop condition and the coded retrieval information comprises: Acquire initial search information of the service search indication information based on the search stop condition; Merging the initial search information and the coded search information to obtain fused search indication information; Performing attention processing on the fused retrieval indication information to obtain a fused retrieval indication feature; The fusion search indication feature is subjected to search word prediction to obtain a target search word sequence, and the target search word sequence is converted into the target search data.

8. An information retrieval device based on reinforcement learning, characterized in that: The device comprises: A retrieval indication information acquisition module, used to acquire business retrieval indication information; A knowledge retrieval module, used to perform knowledge retrieval on the business retrieval indication information from a preset knowledge base to obtain retrieved knowledge information; An encoding module, used for encoding the service search indication information and the search knowledge information to obtain encoded search information; A retrieval behavior and reward factor acquisition module, used to obtain the retrieval action instruction of the encoded retrieval information as the retrieval behavior of the intelligent agent, and obtain the reward parameter of the retrieval behavior as the reward factor of the intelligent agent; A retrieval behavior value identification module is used to identify the retrieval behavior value of the coded retrieval information through the continued retrieval behavior and the reward factor to obtain the continued retrieval behavior value; and to identify the retrieval behavior value of the coded retrieval through the stopped retrieval behavior and the reward factor to obtain the stopped retrieval behavior value; A stop search condition identification module, used for identifying a stop search condition of the coded search information according to the continue search behavior value and the stop search behavior value; A search information generating module is used to generate target search data of the service search indication information based on the search stop condition and the coded search information.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the information retrieval method based on reinforcement learning as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the information retrieval method based on reinforcement learning described in any one of claims 1 to 7 is implemented.

Images