Knowledge retrieval candidate library generation method and system based on incremental pre-training optimization

Abstract

The application discloses a knowledge retrieval candidate library generation method and system based on incremental pre-training optimization, relates to the technical field of information retrieval, and comprises the following steps: collecting a first retrieval sentence and analyzing and obtaining a retrieval behavior feature vector; calculating and obtaining a normalized co-occurrence frequency; constructing a mapping function relationship to obtain a matching incremental pre-training order; using a sentence incremental optimization model to execute multi-stage expansion on the first retrieval sentence to output an expanded retrieval sentence, and then generating a knowledge retrieval candidate library. The application solves the technical problem that, in electronic digital data processing, the traditional knowledge retrieval candidate library generation method cannot dynamically adjust the incremental pre-training order based on user retrieval behavior characteristics, resulting in insufficient semantic coverage, waste of computing resources, and inaccurate candidate library generation, and achieves the technical effects of expanding the semantic coverage range, optimizing the configuration of computing resources, and accurately generating the candidate library.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information retrieval, in particular to a knowledge retrieval candidate library generation method and system based on incremental pre-training optimization. BACKGROUND

[0002] In the process of electric digital data processing, knowledge retrieval candidate library generation is crucial for accurate information acquisition. The existing technology mainly generates candidate library through fixed pre-training model expansion of retrieval sentences, which plays a role in stable retrieval scenarios. However, with the diversification of retrieval needs and the improvement of accuracy, traditional methods have limitations in electric digital data processing applications. Because the traditional method cannot dynamically adjust the incremental pre-training level according to the user's retrieval behavior characteristics, it leads to insufficient semantic coverage, waste of computing resources, and inaccurate candidate library generation in the process of electric digital data processing, which makes it difficult to meet the needs of accurate evaluation and efficient application of knowledge retrieval. SUMMARY

[0003] The present application provides a knowledge retrieval candidate library generation method and system based on incremental pre-training optimization, which is used to solve the technical problem that the traditional knowledge retrieval candidate library generation method cannot dynamically adjust the incremental pre-training level based on user retrieval behavior characteristics, resulting in insufficient semantic coverage, waste of computing resources, and inaccurate candidate library generation in the process of electric digital data processing.

[0004] In a first aspect, the present application provides a knowledge retrieval candidate library generation method based on incremental pre-training optimization, which includes: collecting a first retrieval sentence of a current user, calling a historical retrieval sentence behavior library to analyze the first retrieval sentence, and obtaining a retrieval behavior feature vector; calculating the retrieval behavior feature vector to obtain a normalized co-occurrence frequency; constructing a mapping function relationship between the normalized co-occurrence frequency sample and the incremental pre-training level sample, and using the mapping function relationship to obtain the matching incremental pre-training level corresponding to the normalized co-occurrence frequency; pre-training a sentence incremental optimization model, the sentence incremental optimization model uses the matching incremental pre-training level to perform multi-level expansion on the first retrieval sentence, outputs an expanded retrieval sentence, and generates a knowledge retrieval candidate library according to the expanded retrieval sentence.

[0005] In a second aspect of the present application, a knowledge retrieval candidate library generation system based on incremental pre-training optimization is provided, comprising: a retrieval behavior feature vector acquisition module, configured to collect a first retrieval sentence of a current user, call a historical retrieval sentence behavior library to analyze the first retrieval sentence, and acquire a retrieval behavior feature vector; a normalized co-occurrence frequency acquisition module, configured to calculate the retrieval behavior feature vector and acquire a normalized co-occurrence frequency; a mapping function relationship construction module, configured to construct a mapping function relationship between a normalized co-occurrence frequency sample and an incremental pre-training level sample, and acquire a matching incremental pre-training level corresponding to the normalized co-occurrence frequency by using the mapping function relationship; and a knowledge retrieval candidate library construction module, configured to pre-train a sentence incremental optimization model, use the matching incremental pre-training level to perform multi-level expansion on the first retrieval sentence, output an expanded retrieval sentence, and generate a knowledge retrieval candidate library according to the expanded retrieval sentence.

[0006] The one or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0007] In the present application, the first retrieval sentence of the current user is collected, the retrieval behavior feature vector is acquired by calling the historical retrieval sentence behavior library, the normalized co-occurrence frequency is calculated, the mapping function relationship between the normalized co-occurrence frequency and the incremental pre-training level is constructed, the sentence incremental optimization model is expanded by using the matching incremental pre-training level, the expanded retrieval sentence is output, and the knowledge retrieval candidate library is generated, so as to realize accurate expansion of retrieval semantics and efficient generation of the candidate library, improve the accuracy and efficiency of knowledge retrieval, and achieve the technical effects of semantic coverage expansion, optimized configuration of computing resources, and accurate generation of the candidate library. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0009] Figure 1 FIG. 1 is a flow diagram of a knowledge retrieval candidate library generation method based on incremental pre-training optimization provided by an embodiment of the present application.

[0010] Figure 2 FIG. 2 is a structural diagram of a knowledge retrieval candidate library generation system based on incremental pre-training optimization provided by an embodiment of the present application.

[0011] Legend of the drawings: retrieval behavior feature vector acquisition module 1, normalized co-occurrence frequency acquisition module 2, mapping function relationship construction module 3, and knowledge retrieval candidate library construction module 4. DETAILED DESCRIPTION

[0012] The application provides a knowledge retrieval candidate library generation method and system based on incremental pre-training optimization, which is used to solve the technical problems of insufficient semantic coverage, waste of computing resources and inaccurate candidate library generation caused by the inability of traditional knowledge retrieval candidate library generation methods to dynamically adjust the incremental pre-training level based on user retrieval behavior characteristics in electronic digital data processing.

[0013] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the application.

[0014] It should be noted that the terms "first", "second" and the like in the specification and the above drawings of the application are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or modules not clearly listed or inherent to these processes, methods, products or devices.

[0015] Embodiment one, as shown in the figure, a knowledge retrieval candidate library generation method based on incremental pre-training optimization, wherein the method comprises: Figure 1

[0016] Step A100: Collecting a first retrieval statement of a current user, calling a historical retrieval statement behavior library to analyze the first retrieval statement and obtaining a retrieval behavior feature vector.

[0017] In the embodiments of the application, the first retrieval statement is a to-be-processed retrieval statement input by the current user, which is used to trigger the knowledge retrieval candidate library generation process. The historical retrieval statement behavior library is a database storing user historical retrieval behavior data, containing effective historical retrieval statements and related features in a preset period, which is used to analyze the retrieval behavior features of the first retrieval statement.

[0018] ​Specifically, first, an original sentence input by a user for triggering knowledge retrieval is acquired. The sentence is input by the user through an interactive interface (such as a search box) and serves as an input starting point of the entire knowledge retrieval candidate library generation process. The content can include keywords, phrases, or complete sentences, such as the application of artificial intelligence in the field of science and technology. The sentence is captured in real time through the interface to provide an original data basis for subsequent calling of a historical retrieval behavior library for feature analysis, generation of an extended retrieval sentence, and construction of a candidate library.

[0019] Next, a preset period is acquired, and valid historical retrieval sentence behaviors in the period in the historical retrieval sentence behavior library are extracted. The first retrieval sentence is analyzed to acquire a retrieval behavior feature vector, and specific steps are described in detail in A110-A120.

[0020] By collecting the first retrieval sentence of the user and deeply analyzing the historical retrieval behavior features, the user's retrieval requirements are accurately described, which lays a data foundation for subsequent dynamic matching of incremental pre-training levels and efficient generation of a knowledge retrieval candidate library.

[0021] Step A200: The retrieval behavior feature vector is calculated to acquire a normalized co-occurrence frequency.

[0022] In the embodiment of the application, the normalized co-occurrence frequency is a quantitative value obtained by mapping the feature fusion index to a target interval.

[0023] Optionally, the retrieval behavior feature vector including a retrieval frequency feature vector representing historical retrieval frequency, a generation time length feature vector representing a required time length for generating a retrieval candidate library, and a retrieval feedback index vector representing a feedback quantity of a retrieval candidate library is subjected to weight calculation to acquire a feature fusion index and map it to a target interval, thereby acquiring a normalized co-occurrence frequency. Specific steps are described in detail in A210-A220.

[0024] Step A300: A mapping function relationship between the normalized co-occurrence frequency sample and the incremental pre-training level sample is constructed, and the matching incremental pre-training level corresponding to the normalized co-occurrence frequency is acquired by using the mapping function relationship.

[0025] In the embodiment of the application, the incremental pre-training level sample is a set of different levels that are preset and used to construct a mapping relationship with the normalized co-occurrence frequency sample. The optimal level corresponding to each co-occurrence frequency is acquired by calculating a score value, thereby realizing dynamic matching of the optimal incremental pre-training level according to the user retrieval behavior features.

[0026] In an embodiment of the present application, first, the score value of the normalized co-occurrence frequency sample and the incremental pre-training level sample (the mean of the information entropy of the candidate library generation quality and computing resource score) is calculated, the optimal solution of the incremental pre-training level of each co-occurrence frequency sample is obtained, and a function relationship is fitted, thereby constructing the mapping relationship between the two, which is described in detail in A310-A330.

[0027] The steps of obtaining the matching incremental pre-training level using the mapping function relationship are as follows: first, based on the constructed normalized co-occurrence frequency sample, such as {0.1, 0.3, 0.5, 0.7, 0.9}, and the incremental pre-training level sample, such as {1-5 levels}, the score value of each frequency and level combination, i.e., the mean of the information entropy of the candidate library generation quality and computing resource score, is calculated to determine the optimal solution of the level corresponding to each frequency.

[0028] Second, the mapping relationship between the frequency and the level is fitted using the constructed exponential decreasing function model (such as ), and the parameters are optimized by the least squares method to make the mean square error between the model output and the optimal solution converge to below 0.1. When the normalized co-occurrence frequency of a certain user is input as 0.6, the model calculates y=5·e^(-2.3×0.6)+0.5≈5×0.257+0.5≈1.785, which is rounded to level 2. Compared with the traditional fixed 5 levels, the operation time can be reduced, and the semantic query hit rate can be maintained above 85%.

[0029] By constructing a dynamic mapping function based on user search behavior characteristics, the normalized co-occurrence frequency and the incremental pre-training level are accurately matched, the problem of insufficient semantic coverage and resource waste caused by fixed levels in the prior art is solved, and a quantitative level matching scheme is provided for efficient generation of knowledge retrieval candidate library.

[0030] Step A400: pre-training sentence incremental optimization model, the sentence incremental optimization model uses the matching incremental pre-training level to perform multi-level expansion on the first search sentence, outputs an expanded search sentence, and generates a knowledge retrieval candidate library according to the expanded search sentence.

[0031] In an embodiment of the present application, the expanded search sentence refers to the relevant search sentence output after the first search sentence is input into the sentence incremental optimization model, the search semantic vector is extracted, L rounds of expansion iteration are performed according to the matching incremental pre-training level, and the semantic similarity is calculated and merged.

[0032] Specifically, first, the input of the pre-training sentence incremental optimization model is the initial retrieval sentence and the matched incremental pre-training level L, and the output is the expanded retrieval sentence set generated after L rounds of semantic expansion iteration, which is used to construct a knowledge retrieval candidate library. Its construction process is based on a pre-trained language model (such as BERT, T5). First, the semantic vector of the initial retrieval sentence is extracted by the encoder. Then, the semantic expansion module and the multi-task learning mechanism are integrated: the semantic expansion module includes L rounds of iterative calculation. Each round generates semantic neighbors and synonymous semantic neighbors through cosine similarity or contrastive learning, selects relevant entities to generate retrieval sentences and merges redundancies; the multi-task learning module integrates quality score and resource score, dynamically adjusts the weight through MMoE and other gating networks, and balances retrieval quality and resource consumption. The training process adopts an incremental pre-training strategy, adjusts the learning rate (such as cosine annealing) combined with new and old data, and adds original data to prevent forgetting, while introducing contrastive learning (such as SimCSE, DiffCSE) to optimize semantic vector representation, and narrowing similar semantics and pushing away dissimilar semantics through the InfoNCE loss function. In addition, the model integrates the average information entropy through dynamic weight optimization (such as based on gradient or training progress), improves the coverage of the retrieval while accurately controlling the quality of the candidate library, and finally outputs an expanded retrieval sentence set containing rich semantic dimensions.

[0033] Next, after inputting the first retrieval sentence into the sentence incremental optimization model to extract the retrieval semantic vector, L rounds of expansion iteration are performed according to the matched incremental pre-training level to obtain relevant retrieval sentences, and their semantic vectors are extracted for semantic similarity calculation. The sentences greater than a preset value are merged and output as expanded retrieval sentences. The specific steps are described in detail in A410-A430.

[0034] Finally, based on these expanded retrieval sentences, a knowledge retrieval candidate library containing rich semantic dimensions is constructed by integrating multi-dimensional retrieval expressions related to semantics. This process realizes the systematic expansion of the retrieval coverage and the accurate improvement of the quality of the candidate library through dynamic expansion and semantic optimization.

[0035] By dynamically matching the incremental pre-training level based on user retrieval behavior, combining the multi-level expansion mechanism of semantic neighbors and synonymous neighbors, and optimizing semantic similarity merging, the retrieval sentence is accurately expanded, solving the problem of insufficient semantic coverage and resource waste caused by fixed levels in the prior art, and providing a dynamic optimization model scheme for efficiently generating a knowledge retrieval candidate library.

[0036] Further, the method provided in the embodiments of the present application includes the following steps A200:

[0037] A210: Perform weight calculation on the retrieval behavior feature vector to obtain a feature fusion indicator, map the feature fusion indicator to a target interval, and obtain a normalized co-occurrence frequency.

[0038] A220: wherein the search behavior feature vector comprises a search frequency feature vector representing historical search frequency, a generation time length feature vector representing search candidate library generation time length, and a search feedback index vector representing the number of generated search candidate library feedbacks.

[0039] Specifically, first, a three-dimensional feature vector including a search frequency feature vector, a generation time length feature vector, and a search feedback index vector is extracted, for example, 12 searches in the last 30 days, an average generation time of 3.5 seconds, and 9 effective feedbacks. Second, a preset weight system (such as search frequency 0.4, generation time length 0.3, and search feedback 0.3, the specific numerical value is determined by a person skilled in the art according to the actual situation) is used for linear weighting calculation, and the calculation formula of the feature fusion index is: fusion index = 0.4 x frequency standardized value + 0.3 x time length standardized value + 0.3 x feedback standardized value.

[0040] For the determination of the standardized values of search frequency, generation time length, and search feedback index, the specific conditions are as follows: First, for the search frequency feature vector, based on the historical search frequency in the preset period (such as the last 30 days), the minimum-maximum standardization method is used to map the actual search frequency to the [0, 1] interval, for example, if the minimum value of the search frequency in the last 30 days is 0 and the maximum value is 15, the standardized value of 12 searches is (12-0) / (15-0)=0.8. Second, for the generation time length feature vector, considering that the shorter the generation time is, the more efficient it is, the time data is first converted into an efficiency index (such as 1 / generation time), and then standardized with the minimum time and the maximum time in the preset period as the boundary. If the minimum time is 2 seconds and the maximum time is 5 seconds, the efficiency value corresponding to the average time of 3.5 seconds is 1 / 3.5≈0.286, and the standardized value is (0.286-1 / 5) / (1 / 2-1 / 5)=(0.286-0.2) / 0.3≈0.086 / 0.3≈0.29. The example data can be adjusted according to the actual situation to ensure that the standardized value is in the [0, 1] interval. Finally, for the search feedback index vector, based on the number of effective feedbacks in the preset period, the minimum-maximum standardization is directly processed, if the minimum value of the feedback number is 0 and the maximum value is 10, the standardized value of 9 effective feedbacks is 9 / 10=0.9. The above standardization process takes the extreme value in the preset period as the basis to ensure that the numerical range of each feature vector is unified, providing standardized input for subsequent weight calculation.

[0041] Taking the above data as an example, the frequency standardization is 0.8, the time length standardization is 0.29, and the feedback standardization is 0.9. The calculated feature fusion index is 0.677. Finally, the fusion index is mapped to the [0, 1] target interval through a linear mapping function, and the normalized co-occurrence frequency 0.677 is obtained. This value represents the comprehensive frequency characteristics of user search behavior. Since the weight sum is 1 and the standardized values are all ≤1, the fusion index is naturally limited in the range of [0, 1], so it is directly used as the normalized co-occurrence frequency through the linear mapping function. This value directly represents the comprehensive frequency characteristics of user search behavior.

[0042] By performing multi-dimensional weight calculation and target interval mapping on the search behavior feature vector, accurate quantitative representation of user search behavior is realized, providing a scientific numerical basis for subsequent construction of a dynamic matching relationship between the behavior index and the incremental pre-training level.

[0043] Further, the method provided in the embodiments of the present application comprises the following steps A300:

[0044] A310: Calculate the score value of each co-occurrence frequency sample in the normalized co-occurrence frequency sample and each incremental pre-training level in the incremental pre-training level sample, and output a score value sample.

[0045] A320: wherein the score value is the average information entropy of the candidate library generation quality score value and the required computing resource score value of the candidate library.

[0046] A330: According to the score value sample, obtain the incremental pre-training level optimal solution of each co-occurrence frequency sample in the normalized co-occurrence frequency sample, fit the function relationship between each co-occurrence frequency sample in the normalized co-occurrence frequency sample and the corresponding incremental pre-training level optimal solution, and output the mapping function relationship.

[0047] In the embodiments of the present application, the co-occurrence frequency sample is a set of normalized co-occurrence frequency values, which is used to construct a mapping function relationship with the incremental pre-training level sample. The candidate library is a knowledge retrieval candidate library generated according to the expanded search statement, which is output by the sentence incremental optimization model after multi-level expansion of the first search statement using the matching incremental pre-training level.

[0048] Optionally, in knowledge retrieval, the existing technology usually uses a fixed incremental pre-training level when constructing a search candidate library, such as uniformly setting it to 3 levels or 5 levels, without dynamically adjusting it in combination with user search behavior characteristics, resulting in an increase in semantic coverage missing rate due to insufficient levels in the low frequency scenario of normalized co-occurrence frequency (such as <0.3), and causing waste of computing resources due to too high levels in the high frequency scenario (such as >0.7).

[0049] To solve the above problems, first, a normalized co-occurrence frequency sample set is established, such as {0.1, 0.3, 0.5, 0.7, 0.9}, and an incremental pre-training level sample set is established, such as {1, 2, 3, 4, 5}. Taking normalized co-occurrence frequency 0.1 and level 5 as an example, when calculating the score value, first, the candidate library generation quality score value is obtained (the specific steps are described in detail in A321-A322), such as semantic query hit rate 85%, consistency score annotation information 4.2 / 5, semantic relevance weight 0.7, and the comprehensive quality score is 0.85x0.4+0.84x0.3+0.7x0.3=0.812, and the computing resource score value is calculated, such as GPU occupancy rate 60%, time consumption 12 seconds, and the standardized resource score is 0.4, the information entropy of the two is respectively H(quality)=-0.812xln0.812-0.188xln0.188≈0.36, H(resource)=-0.4xln0.4-0.6xln0.6≈0.67, and the average information entropy is (0.36+0.67) / 2≈0.515, that is, the score value of this combination is 0.515, and the calculation formula of information entropy can be briefly described as: for a group of probability distribution , wherein , the information entropy is . In the binary distribution scenario, such as the probability distribution of the quality score value and the resource score value described above, the formula is simplified as , wherein , , and correspond to the proportion of the candidate library generation quality score value and the candidate library required computing resource score value in the probability distribution, respectively.

[0050] Then, all combinations of normalized co-occurrence frequency and incremental pre-training level are traversed to obtain a score value sample matrix. For example, when the co-occurrence frequency is 0.7, the score value of level 2 is calculated as 0.78, wherein the quality score is 0.92, the resource score is 0.85, and the average information entropy is 0.81, which is the highest score under this frequency, so level 2 is determined as the optimal solution.

[0051] Based on all optimal solution data, an exponential decreasing function model is used for fitting to obtain a mapping relationship such as 0.1 corresponding to level 5 and 0.9 corresponding to level 1, as shown in Table 1, wherein the construction process of the exponential decreasing function model is described in detail in A331.

[0052] By quantitatively analyzing the combination score of normalized co-occurrence frequency and incremental pre-training level, combining the information entropy average evaluation mechanism and the exponential fitting model, the dynamic matching of retrieval behavior characteristics and pre-training resources is realized, the problem of insufficient semantic coverage and resource waste caused by fixed level in the prior art is solved, and a scientific mapping basis is provided for accurately generating a knowledge retrieval candidate library.

[0053] Table 1: Normalized co-occurrence frequency sample and incremental pre-training level mapping table

[0054]

[0055] Further, the step A330 in the method provided by the embodiments of the present application comprises:

[0056] A331: fitting the function relationship between each co-occurrence frequency sample in the normalized co-occurrence frequency sample and the corresponding incremental pre-training level optimal solution by fitting a model, wherein the fitting model is an exponential decreasing function model.

[0057] Specifically, the exponential decreasing function model is used to fit the mapping function relationship between the co-occurrence frequency sample and the corresponding incremental pre-training level optimal solution, and the form is wherein a, b, g are fitting parameters, x is the normalized co-occurrence frequency, and y is the incremental pre-training level. The specific steps are as follows: first, based on the aforementioned constructed score value sample, the level optimal solution corresponding to different normalized co-occurrence frequencies is obtained, and exemplary, the sample points (0.1, 5), (0.3, 4), (0.5, 3), (0.7, 2), (0.9, 1) are collected. Secondly, the least square method is used to optimize the parameters of the exponential decreasing function, and the value of a, b, g is iteratively adjusted with the minimum mean square error of the sample points as the target. Assuming that the initial parameters are a = 4, b = 2, g = 1, the error between the predicted level and the actual optimal solution is calculated, for example, when x = 0.1, y = 4e^(-2x0.1) + 1 ≈ 4x0.8187 + 1 ≈ 4.275, and the error with the actual optimal solution 5 is 0.725, and the parameters are updated to a = 5, b = 2.3, g = 0.5 by gradient descent method, at this time, when x = 0.1, y = 5e^(-2.3x0.1) + 0.5 ≈ 5x0.7945 + 0.5 ≈ 4.472, and the error is reduced to 0.528, and finally the overall mean square error is converged to below 0.1.

[0058] The input of the model is the normalized co-occurrence frequency, the range is [0, 1], and the output is the matched incremental pre-training level, which is a positive integer, and the dynamic mapping of higher frequency and lower level is realized through the exponential decreasing characteristic. For example, when the input x = 0.2, the model output y ≈ 5e^(-2.3x0.2) + 0.5 ≈ 5x0.6387 + 0.5 ≈ 3.693, and the integer is 4 levels, which can improve the semantic query hit rate compared with the prior art fixed 3 levels; when x = 0.8, the output y ≈ 5e^(-2.3x0.8) + 0.5 ≈ 5x0.1609 + 0.5 ≈ 1.304, and the integer is 1 level, which can reduce the consumption of computing resources compared with the fixed 5 levels.

[0059] By constructing an exponential decreasing function model and fitting parameters based on the optimal solution sample, the normalized co-occurrence frequency and the incremental pre-training level are nonlinearly dynamically matched, the problem of insufficient semantic expansion or resource waste caused by fixed level in the prior art is solved, and a quantitative mapping basis is provided for efficient generation of knowledge retrieval candidate library.

[0060] Further, the step A400 in the method provided by the embodiment of the application comprises:

[0061] A410: inputting the first retrieval statement into the statement incremental optimization model to extract a retrieval semantic vector.

[0062] A420: performing L rounds of expansion iteration on the retrieval semantic vector according to the matched incremental pre-training level to obtain L rounds of relevant retrieval statements, wherein L is the number of the matched incremental pre-training level.

[0063] A430: performing semantic similarity calculation on the semantic vectors of the L rounds of relevant retrieval statements, merging relevant retrieval statements greater than a preset semantic similarity, and outputting the processed relevant retrieval statements as expanded retrieval statements.

[0064] Specifically, first, the first retrieval statement input by the user, such as “application of artificial intelligence in science and technology image”, is input into the model, and a retrieval semantic vector containing keywords is extracted by a semantic analysis module. This process can be implemented based on an encoder of a BERT or other Transformer architecture, for example, by using a multi-layer self-attention mechanism to capture semantic associations in the statement to generate a semantic vector with a dimension of 768.

[0065] Second, the retrieval semantic vector is iterated for L rounds of expansion according to the matched incremental pre-training level, including calculating a round of semantic neighborhood, selecting a round of semantically related entities from the round of semantic neighborhood to generate a round of relevant retrieval statements, and iteratively outputting L rounds of relevant retrieval statements. The specific steps are described in detail in A421-A422.

[0066] Further, the expansion iteration method further comprises obtaining a synonymous semantic neighborhood of the retrieval semantic vector, selecting synonymous semantically related entities from the synonymous semantic neighborhood to generate synonymous relevant retrieval statements, and updating the synonymous relevant retrieval statements to the L rounds of relevant retrieval statements. The specific steps are described in detail in A423-A425.

[0067] Each round of iteration generates relevant retrieval statements. For example, after 4 rounds of expansion, multiple relevant statements such as “application of deep learning in AI judgment of science and technology image” and “combination of scientific image processing and AI technology” can be generated, covering synonymous replacement and domain subdivision of the original statement.

[0068] Subsequently, the semantic vectors of all L-round expansion statements are extracted, and the semantic similarity between each two statements is calculated using the cosine similarity algorithm. The similarity threshold is preset to 0.7, and the redundant statements with a similarity greater than the threshold are merged. For example, among 20 expansion statements, 5 statements may be merged due to high semantic similarity (such as “AI technology” and “artificial intelligence technology”), and finally 15 simplified expansion retrieval statements are output. The merging mechanism can improve the semantic query hit rate in the low-frequency retrieval scenario, and reduce the computational resource consumption in the high-frequency retrieval scenario (such as normalized co-occurrence frequency 0.8). By dynamically adjusting the expansion level and the semantic similarity merging strategy, the generalization ability of the model is maintained while the retrieval efficiency in a specific field is significantly improved.

[0069] By constructing a dynamic expansion mechanism based on user retrieval behavior characteristics, combining semantic vector extraction, multi-level iterative expansion, and similarity merging optimization, accurate expansion of retrieval statements is achieved. Compared with the fixed-level static expansion mode in the prior art, the current scheme improves the semantic coverage comprehensiveness and resource utilization efficiency, and provides a dynamic optimization model scheme for efficient generation of knowledge retrieval candidate library.

[0070] Further, the step A420 in the method provided by the embodiment of the present application comprises:

[0071] A421: calculating a round of semantic neighborhood of the retrieval semantic vector.

[0072] A422: selecting a round of semantic related entities from the round of semantic neighborhood of the retrieval semantic vector, generating a round of related retrieval statements, and iteratively outputting L rounds of related retrieval statements of the retrieval semantic vector for L rounds of expansion.

[0073] In the embodiment of the present application, the semantic neighborhood is the adjacent area of the retrieval semantic vector in the semantic space, which contains a set of entities related to the semantic vector. The semantic related entity is an entity selected from the semantic neighborhood of the retrieval semantic vector, which has semantic correlation with the original retrieval statement, and is used to generate a related retrieval statement.

[0074] Specifically, first, the user input first retrieval statement, such as “application of artificial intelligence in science and technology image”, is input into the statement incremental optimization model, and a retrieval semantic vector with a dimension of 768 is extracted by the BERT-based encoder. This process uses a multi-layer self-attention mechanism to capture semantic associations in the statement, such as identifying the domain correlation between “artificial intelligence” and “science and technology image”. Subsequently, the retrieval semantic vector is iteratively expanded for L rounds according to the number L of matching incremental pre-training levels determined in the foregoing, such as 5 levels determined by the mapping function.

[0075] In each round of iteration, first, a round of semantic neighborhood of the current semantic vector is calculated. This process is based on the cosine similarity algorithm, searching for adjacent vectors with similarity greater than 0.6 in the pre-trained semantic space. For example, the first round of expansion may retrieve adjacent entities such as "deep learning" and "scientific image processing" from the knowledge graph, forming a semantic neighborhood containing 20 nodes.

[0076] Next, a round of semantically related entities is selected from the neighborhood, and a round of related search sentences is generated through template filling or sequence generation model. For example, combining "deep learning" and "scientific image AI diagnosis" generates multiple candidate sentences such as "application of deep learning in scientific image AI diagnosis".

[0077] By constructing a dynamic neighborhood expansion mechanism based on semantic vector space, combining cosine similarity calculation, knowledge graph entity retrieval and iterative sentence generation strategy, multi-dimensional expansion of search semantics is realized, providing an intelligent dynamic optimization model for efficient generation of knowledge search candidate library.

[0078] Further, the method provided in the embodiment of the application comprises the following steps:

[0079] A423: Obtain the synonymous semantic neighborhood of the search semantic vector.

[0080] A424: Select synonymous semantic related entities from the synonymous semantic neighborhood of the search semantic vector, and generate synonymous related search sentences of the search semantic vector.

[0081] A425: Update the synonymous related search sentences to the L rounds of related search sentences.

[0082] In one embodiment, first, the synonymous semantic neighborhood of the search semantic vector is obtained using a pre-trained sentence incremental optimization model. Taking the input sentence "application of artificial intelligence in scientific image" as an example, the model calculates the semantic vector of "scientific image" through the BERT word vector model, retrieves synonymous entities with a cosine similarity greater than 0.7 in the semantic space, such as "scientific image" and "image judgment", and forms a synonymous semantic neighborhood set containing multiple synonymous entities.

[0083] Secondly, 2-3 entities with the strongest semantic relevance to the original sentence (such as "scientific image" and "image analysis") are selected from the neighborhood, and synonymous related search sentences are constructed through template generation method, such as "application of artificial intelligence in scientific image" and "AI application in image analysis".

[0084] Finally, the newly generated synonymous sentences are updated to the related search sentence set obtained through L rounds of expansion iteration, realizing dynamic expansion of the semantic coverage.

[0085] By acquiring the synonymous semantic neighborhood, generating context-related sentences, and updating the extended set, the problem of low coverage and poor adaptability of synonyms in the prior art is solved, and an intelligent expansion scheme is provided to improve the comprehensiveness and accuracy of the knowledge retrieval candidate library.

[0086] Further, the step A320 in the method provided by the embodiment of the application comprises:

[0087] A321: acquiring semantic query hit rate, consistency score labeling information, and semantic relevance weight of the generated candidate library.

[0088] A322: outputting a candidate library generation quality score value according to the calculation results of the semantic query hit rate, the consistency score labeling information, and the semantic relevance weight.

[0089] Optionally, first, the semantic query hit rate is acquired, that is, the candidate library is retrieved by a query sentence of a preset test set, and the ratio of the number of correct hits to the total number of queries is counted, for example, 85 hits out of 100 test queries, with a hit rate of 85%. Second, consistency score labeling information is collected, which is scored by a person skilled in the art on the semantic consistency of the retrieval results in the candidate library and the query sentence (for example, using a 5-point system), and it is assumed that the scores of 10 samples are 4, 5, 4, 3, 5, 4, 4, 5, 3, and 4, and the average consistency score is 4.2. Finally, the semantic relevance weight is determined, which is obtained based on the experience of a domain expert or historical data training, and is used to represent the importance of semantic relevance in quality evaluation, for example, set to 0.3.

[0090] In the calculation link, the above indicators are quantitatively integrated. It is assumed that the semantic query hit rate is standardized to 0.85 (full score 1), the consistency score is standardized to 0.84 (4.2 / 5), and the semantic relevance weight is 0.3. The weighted sum formula is used: quality score value = hit rate × 0.5 + consistency score × 0.2 + relevance weight × 0.3, that is, 0.85 × 0.5 + 0.84 × 0.2 + 0.3 × 0.3 = 0.425 + 0.168 + 0.09 = 0.683. This calculation process considers the retrieval accuracy, result consistency, and semantic relevance, and compared with the single indicator evaluation in the prior art, the credibility of the quality score is improved.

[0091] By acquiring the semantic query hit rate, the consistency score labeling information, and the semantic relevance weight in multiple dimensions, and integrating the evaluation indicators based on a scientific calculation model, comprehensive quantitative analysis of the quality of the candidate library generation is realized, the problem of single evaluation dimension and inaccurate results in the prior art is solved, and a reliable quality reference basis is provided for optimizing the incremental pre-training stage matching and improving the retrieval effect.

[0092] Further, the method provided in the embodiments of the present application includes the following step A100:

[0093] A110: Obtain a preset period.

[0094] A120: Extract valid historical search statement behaviors in the preset period from the historical search statement behavior library, analyze the first search statement based on the valid historical search statement behaviors, and obtain a search behavior feature vector.

[0095] In the embodiments of the present application, the preset period is a pre-set time range, which is used to limit the time span of extracting historical search behavior data, and the period can be adjusted according to the user search habit change characteristics. The valid historical search statement behavior refers to the historical search record that completes the search process and produces effective feedback (such as search result clicks, collections, etc.) within the preset period, and does not include invalid search behaviors caused by network failure, operation anomaly, etc.

[0096] In one embodiment, the preset period is first obtained, and the specific data is adjusted by a person skilled in the art according to the user search habit change characteristics, and it is assumed that the setting is about 30 days, and the period is set based on the short-term continuity characteristics of user behavior. Taking the user input "application of machine learning in recommendation system" as an example, the valid historical search behavior from May 25, 2025 to June 24, 2025 is extracted from the historical search statement behavior library according to the preset period. The judgment standard of the valid behavior is: the record that completes the search process and produces feedback (such as clicks, collections), excluding invalid requests caused by network failure, etc.

[0097] After the above extraction, for example, the user has 12 same topic search records in the past 30 days, the average candidate library generation time is 3.8 seconds, and the relevant result feedback is obtained 10 times. Based on these valid data, the system analyzes the first search statement, and converts the search frequency, generation time and feedback quantity into feature vectors with dimensions [12, 3.8, 10] respectively, and the vector can accurately represent the current search behavior pattern of the user after standardization processing.

[0098] By setting the preset period and extracting the valid historical search behavior in the period, the current search behavior characteristics of the user are accurately captured, and time-efficient and reliable data support is provided for subsequent normalized co-occurrence frequency calculation and incremental pre-training level matching.

[0099] In summary, the knowledge search candidate library generation method based on incremental pre-training optimization provided in the embodiments of the present application has the following technical effects:

[0100] This application collects the first search statement of the current user, analyzes it using a historical search statement behavior database to obtain search behavior feature vectors, obtains normalized co-occurrence frequencies through weight calculation and mapping, constructs a mapping function relationship between normalized co-occurrence frequency samples and incremental pre-training level samples, obtains matching incremental pre-training levels, and uses a pre-trained statement incremental optimization model to perform multi-level expansion of the first search statement to generate expanded search statements, thereby accurately generating a knowledge retrieval candidate library. This makes the generation results of the knowledge retrieval candidate library more accurate and efficient, achieving the technical effects of expanding semantic coverage, optimizing computing resource allocation, and accurately generating the candidate library.

[0101] Example 2, as Figure 2 As shown, based on the same inventive concept as in Embodiment 1 above, this application provides a knowledge retrieval candidate library generation system based on incremental pre-training optimization, the system comprising:

[0102] The retrieval behavior feature vector acquisition module 1 is used to collect the first retrieval statement of the current user, call the historical retrieval statement behavior library to analyze the first retrieval statement, and obtain the retrieval behavior feature vector.

[0103] The normalized co-occurrence frequency acquisition module 2 is used to calculate the normalized co-occurrence frequency of the retrieval behavior feature vector.

[0104] The mapping function relationship construction module 3 is used to construct the mapping function relationship between normalized co-occurrence frequency samples and incremental pre-training level samples, and to obtain the matching incremental pre-training level corresponding to the normalized co-occurrence frequency using the mapping function relationship.

[0105] The knowledge retrieval candidate library construction module 4 is used to pre-train a statement incremental optimization model. The statement incremental optimization model uses the matching incremental pre-training level to expand the first retrieval statement in multiple levels, outputs the expanded retrieval statement, and generates a knowledge retrieval candidate library based on the expanded retrieval statement.

[0106] Furthermore, the normalized co-occurrence frequency acquisition module 2 is used to perform the following steps:

[0107] The retrieval behavior feature vector is weighted to obtain a feature fusion index, which is then mapped to a target interval to obtain a normalized co-occurrence frequency. The retrieval behavior feature vector includes a retrieval frequency feature vector representing the frequency of historical retrievals, a generation time feature vector representing the time required to generate the retrieval candidate library, and a retrieval feedback index vector representing the number of feedback responses from the generated retrieval candidate library.

[0108] Further, the mapping function relationship construction module 3 is configured to perform the following steps:

[0109] Calculate the score value of each co-occurrence frequency sample in the normalized co-occurrence frequency sample and each incremental pre-training level in the incremental pre-training level sample, and output a score value sample; wherein the score value is the average information entropy of the candidate library generation quality score value and the required computing resource score value of the candidate library; according to the score value sample, obtain the optimal solution of the incremental pre-training level of each co-occurrence frequency sample in the normalized co-occurrence frequency sample, fit the function relationship between each co-occurrence frequency sample in the normalized co-occurrence frequency sample and the corresponding optimal solution of the incremental pre-training level, and output the mapping function relationship.

[0110] Further, the mapping function relationship construction module 3 is configured to perform the following steps:

[0111] Fit the function relationship between each co-occurrence frequency sample in the normalized co-occurrence frequency sample and the corresponding optimal solution of the incremental pre-training level by fitting a model, and the fitting model is an exponential decreasing function model.

[0112] Further, the knowledge retrieval candidate library construction module 4 is configured to perform the following steps:

[0113] Input the first retrieval statement into the statement incremental optimization model, extract a retrieval semantic vector; perform L rounds of expansion iteration on the retrieval semantic vector according to the matching incremental pre-training level, obtain L rounds of related retrieval statements, wherein L is the number of matching incremental pre-training levels; extract the semantic vectors of the L rounds of related retrieval statements for semantic similarity calculation, merge related retrieval statements greater than a preset semantic similarity, and output the processed related retrieval statements as expansion retrieval statements.

[0114] Further, the knowledge retrieval candidate library construction module 4 is configured to perform the following steps:

[0115] Calculate a round of semantic neighborhood of the retrieval semantic vector; select a round of semantic related entities from the round of semantic neighborhood of the retrieval semantic vector, generate a round of related retrieval statements, and in this way, output L rounds of related retrieval statements of L rounds of expansion iteration of the retrieval semantic vector.

[0116] Further, the knowledge retrieval candidate library construction module 4 is configured to perform the following steps:

[0117] Obtain the synonymous semantic neighborhood of the retrieval semantic vector; select synonymous semantic related entities from the synonymous semantic neighborhood of the retrieval semantic vector, generate synonymous related retrieval statements of the retrieval semantic vector; and update the synonymous related retrieval statements to the L rounds of related retrieval statements.

[0118] Further, the mapping function relationship construction module 3 is configured to perform the following steps:

[0119] The semantic query hit rate, the consistency score label information and the semantic correlation weight of the generated candidate library are acquired, and a candidate library generation quality score value is output according to the calculation results of the semantic query hit rate, the consistency score label information and the semantic correlation weight.

[0120] Further, the retrieval behavior feature vector acquisition module 1 is configured to perform the following steps:

[0121] A preset period is acquired, valid historical retrieval statement behaviors in the preset period in the historical retrieval statement behavior library are extracted, the first retrieval statement is analyzed based on the valid historical retrieval statement behaviors, and a retrieval behavior feature vector is acquired.

[0122] The knowledge retrieval candidate library generation system based on the incremental pre-training optimization provided in the embodiments of the present application can execute the knowledge retrieval candidate library generation method based on the incremental pre-training optimization provided in any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0123] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or server, and each unit and module included is only divided according to the functional logic, but is not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy mutual differentiation, and do not limit the protection scope of the present application.

[0124] The above specific embodiments do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps described in the present application can be executed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

Claims

1. A knowledge retrieval candidate library generation method based on incremental pre-training optimization, characterized in that, The method includes: Collect the first search statement of the current user, call the historical search statement behavior library to analyze the first search statement, and obtain the search behavior feature vector; The normalized co-occurrence frequency is obtained by calculating the feature vector of the retrieval behavior, including: first, extracting a three-dimensional feature vector containing the retrieval frequency feature vector, the generation duration feature vector, and the retrieval feedback index vector; second, using a preset weight system to perform linear weighting calculation to obtain the fusion index; and finally, mapping the fusion index to the target interval [0,1] through a linear mapping function to obtain the normalized co-occurrence frequency. Construct a mapping function relationship between normalized co-occurrence frequency samples and incremental pre-training level samples, and use the mapping function relationship to obtain the matching incremental pre-training level corresponding to the normalized co-occurrence frequency; A pre-trained statement incremental optimization model is used to expand the first retrieval statement in multiple levels using the matching incremental pre-training levels, and outputs expanded retrieval statements. A knowledge retrieval candidate library is generated based on the expanded retrieval statements. Weights are calculated on the feature vector of the retrieval behavior to obtain a feature fusion index. The feature fusion index is then mapped to the target interval to obtain the normalized co-occurrence frequency. The retrieval behavior feature vector includes a retrieval frequency feature vector representing the frequency of historical retrievals, a generation time feature vector representing the time required to generate the retrieval candidate database, and a retrieval feedback index vector representing the number of feedback responses from the generated retrieval candidate database. The statement incremental optimization model uses the matching incremental pre-training levels to expand the first retrieval statement at multiple levels, and outputs an expanded retrieval statement. The method includes: Input the first search statement into the statement incremental optimization model to extract the search semantic vector; The retrieval semantic vector is expanded and iterated L times according to the number of matching incremental pre-training levels to obtain L rounds of related retrieval statements, where L is the number of matching incremental pre-training levels; Extract the semantic vectors of the relevant search statements from the L rounds, calculate their semantic similarity, merge the relevant search statements with a semantic similarity greater than the preset threshold, and output the processed relevant search statements as extended search statements.

2. The method as described in claim 1, characterized in that, The mapping relationship between normalized co-occurrence frequency samples and incremental pre-training series samples is constructed using the following methods: Calculate the score values ​​of each co-occurrence frequency sample in the normalized co-occurrence frequency samples and each incremental pre-training level in the incremental pre-training level samples, and output the score value samples. The scoring value is the average information entropy of the quality score value generated by the candidate library and the score value of the computational resources required for the candidate library. Based on the score value samples, obtain the optimal solution of the incremental pre-training series for each co-occurrence frequency sample in the normalized co-occurrence frequency samples, fit the functional relationship between each co-occurrence frequency sample in the normalized co-occurrence frequency samples and the corresponding optimal solution of the incremental pre-training series, and output the mapping function relationship.

3. The method as described in claim 2, characterized in that, The fitting model is an exponentially decreasing function model that fits the functional relationship between each co-occurrence frequency sample in the normalized co-occurrence frequency samples and the corresponding incremental pre-training level optimal solution.

4. The method as described in claim 1, characterized in that, The method involves performing L rounds of expansion iterations on the retrieval semantic vector according to the aforementioned matching increment pre-training levels, including: Calculate the semantic neighborhood of the retrieved semantic vector in one round; Select a semantically related entity from the semantic neighborhood of the retrieval semantic vector, generate a related retrieval statement, and so on, outputting L rounds of related retrieval statements by expanding and iterating the retrieval semantic vector in L rounds.

5. The method as described in claim 1, characterized in that, The method further includes performing L rounds of expansion iterations on the retrieval semantic vector according to the aforementioned matching increment pre-training levels: Obtain the synonymous semantic neighborhood of the retrieved semantic vector; Select synonymous semantic related entities from the synonymous semantic neighborhood of the retrieved semantic vector, and generate synonymous related search statements for the retrieved semantic vector; Update the synonym-related search statement to the Lth round of related search statements.

6. The method as described in claim 2, characterized in that, Methods for obtaining quality scores from candidate libraries include: Obtain the semantic query hit rate, consistency score annotation information, and semantic relevance weight of the generated candidate library; Based on the calculation results of the semantic query hit rate, consistency score annotation information, and semantic relevance weight, the quality score value of the candidate library is output.

7. The method as described in claim 1, characterized in that, The method of analyzing the first search statement by calling the historical search statement behavior library includes: Get the preset period; Extract valid historical search statement behaviors from the historical search statement behavior library that are within the preset period, analyze the first search statement based on the valid historical search statement behaviors, and obtain the search behavior feature vector.

8. A knowledge retrieval candidate library generation system based on incremental pre-training optimization, characterized in that, The system is used to implement the knowledge retrieval candidate library generation method based on incremental pre-training optimization as described in any one of claims 1-7, the system comprising: The retrieval behavior feature vector acquisition module is used to collect the first retrieval statement of the current user, call the historical retrieval statement behavior library to analyze the first retrieval statement, and obtain the retrieval behavior feature vector. The normalized co-occurrence frequency acquisition module is used to calculate the normalized co-occurrence frequency from the feature vector of the retrieval behavior. The mapping function relationship construction module is used to construct the mapping function relationship between normalized co-occurrence frequency samples and incremental pre-training level samples, and to obtain the matching incremental pre-training level corresponding to the normalized co-occurrence frequency using the mapping function relationship; The knowledge retrieval candidate library construction module is used to pre-train a statement incremental optimization model. The statement incremental optimization model uses the matching incremental pre-training levels to expand the first retrieval statement in multiple levels, outputs expanded retrieval statements, and generates a knowledge retrieval candidate library based on the expanded retrieval statements.

Images