Text retrieval method, apparatus, device, and computer program product

Abstract

Embodiments of the present application relate to the technical field of retrieval, and specifically provide a text retrieval method, device, equipment and computer program product. The text retrieval method comprises: obtaining a text retrieval request for a to-be-retrieved text; determining a sparse vector representation and a dense vector representation of the to-be-retrieved text, wherein the dense vector representation is used to identify a text feature of the to-be-retrieved text, the sparse vector representation is used to identify a text word in the to-be-retrieved text, and the dimension of the sparse vector representation is higher than the dimension of the dense vector representation; fusing the sparse vector representation and the dense vector representation to obtain a fused vector representation; and performing a text retrieval operation based on the text retrieval request and the fused vector representation to obtain a text retrieval result. In the embodiment, the dense vector representation and the sparse vector representation of two different dimensions are fused to obtain the fused vector representation, and then the text retrieval operation is performed based on the fused vector representation, thereby effectively improving the accuracy of the text retrieval operation.

Description

Technical Field

[0001] This invention relates to the field of retrieval technology, and in particular to a text retrieval method, apparatus, device, and computer program product. Background Technology

[0002] With the rapid development of information technology, text retrieval has become one of the basic and key capabilities in information search scenarios. Text retrieval refers to the operations of searching, classifying, and filtering text collections based on text content (such as keywords, semantics, etc.).

[0003] Currently, text retrieval operations can be implemented using pre-trained dense vector models. Dense vector models are pre-trained network models used to convert text data into numerical vectors. The data in dense vector models have the characteristics of dense vectors. Dense vectors refer to vectors in which the values ​​of almost all dimensions in the vector space are not zero.

[0004] However, since dense vector models are often trained on data from a specific domain, the accuracy of search results may be low when using dense vector models to retrieve text from unknown domains. Summary of the Invention

[0005] This invention provides a text retrieval method, apparatus, device, and computer program product that can improve the accuracy of text retrieval operations.

[0006] In a first aspect, embodiments of the present invention provide a text retrieval method, including:

[0007] Obtain the text search request for the text to be searched;

[0008] The sparse vector representation and dense vector representation of the text to be retrieved are determined, wherein the dense vector representation is used to identify the text features of the text to be detected, and the sparse vector representation is used to identify the text words in the text to be detected. The dimension of the sparse vector representation is higher than that of the dense vector representation.

[0009] The sparse vector representation and the dense vector representation are fused to obtain a fused vector representation;

[0010] Based on the text retrieval request and the fused vector representation, a text retrieval operation is performed to obtain the text retrieval results.

[0011] Secondly, embodiments of the present invention provide a text retrieval device, comprising:

[0012] The first acquisition module is used to acquire text search requests for the text to be searched;

[0013] The first determining module is used to determine the sparse vector representation and the dense vector representation of the text to be retrieved, wherein the dense vector representation is used to identify the text features of the text to be detected, and the sparse vector representation is used to identify the text words in the text to be detected, and the dimension of the sparse vector representation is higher than the dimension of the dense vector representation.

[0014] The first processing module is used to fuse the sparse vector representation and the dense vector representation to obtain a fused vector representation;

[0015] The first retrieval module is used to perform text retrieval operations based on the text retrieval request and the fused vector representation to obtain text retrieval results.

[0016] Thirdly, embodiments of the present invention provide an electronic device, including: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the text retrieval method in the first aspect described above.

[0017] Fourthly, embodiments of the present invention provide a computer storage medium for storing a computer program, which, when executed by a computer, implements the text retrieval method described in the first aspect above.

[0018] Fifthly, embodiments of the present invention provide a computer program product, comprising: a computer-readable storage medium storing computer instructions, wherein when the computer instructions are executed by one or more processors, the one or more processors cause the one or more processors to perform the steps in the text retrieval method described in the first aspect above.

[0019] The text retrieval method, apparatus, device, and computer program product provided in this invention obtain a text retrieval request for the text to be retrieved; determine the sparse vector representation and dense vector representation of the text to be retrieved, and fuse the sparse vector representation and the dense vector representation to obtain a fused vector representation; then perform a text retrieval operation based on the text retrieval request and the fused vector representation to obtain the text retrieval result. Since the text retrieval operation is implemented based on the fused vector representation after fusing two vector representations of different dimensions, the accuracy of the text retrieval operation is effectively improved. In addition, the implementation of this technical solution can achieve stable text retrieval operation without making significant modifications to the retrieval architecture, which effectively improves the ease of use of the method and further ensures the practicality of the method. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic diagram illustrating a scenario of a text retrieval method provided in an embodiment of the present invention;

[0022] Figure 2 A flowchart illustrating a text retrieval method provided in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the process of fusing the sparse vector representation and the dense vector representation to obtain the fused vector representation, provided for an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram of the process of performing dimensionality reduction processing on the sparse vector representation based on the dimensionality parameter to obtain the dimensionality-reduced vector representation, as provided in an embodiment of the present invention.

[0025] Figure 5 A schematic diagram illustrating the principle of a text retrieval method based on a hybrid sparse vector model and a dense vector model, provided for an application embodiment of the present invention;

[0026] Figure 6 This is a schematic diagram of the structure of a text retrieval device provided in an embodiment of the present invention;

[0027] Figure 7 To and Figure 6 A schematic diagram of the electronic device corresponding to the text retrieval device provided in the embodiment is shown. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise; “multiple” generally includes at least two, but does not exclude the inclusion of at least one.

[0030] It should be understood that the term "and / or" used in this document is merely a description of the relationship between associated devices, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this document generally indicates that the preceding and following associated devices are in an "or" relationship.

[0031] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0032] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes that element.

[0033] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0034] Terminology Explanation:

[0035] Sparse embedding: In a vector space, most dimensions have values ​​of 0, and only a few dimensions have values.

[0036] Dense embedding: In a vector space, the values ​​of almost all dimensions are not 0.

[0037] LLM (Large Language Model): A language model with a large number of parameters, possessing powerful language understanding and logical reasoning capabilities.

[0038] Dimensionality reduction: reducing a high-dimensional vector to a relatively low-dimensional vector using a certain algorithm.

[0039] To facilitate understanding of the specific implementation process and effects of the text retrieval method, apparatus, device, and computer program product in this embodiment, the relevant technologies are briefly described below:

[0040] With the rapid development of information technology, text retrieval has become one of the basic and key capabilities in information search scenarios. Text retrieval refers to the operations of searching, classifying, and filtering text collections based on text content (such as keywords, semantics, etc.).

[0041] Currently, text retrieval operations can be implemented using pre-trained dense vector models or sparse vector models. Dense vector models are pre-trained network models used to convert text data into numerical vectors. The data in dense vector models exhibits the characteristics of dense vectors, meaning that almost all dimensions in the vector space have non-zero values. Sparse vector models are pre-trained network models used to identify text words in text data. The data in sparse vector models exhibits the characteristics of sparse vectors, meaning that almost all dimensions in the vector space have non-zero values.

[0042] For example, taking the product model term "hwy14-186bku1" as an example in a real-world scenario, we can use both sparse vector models and dense vector models to perform text queries on the keyword "hwy14-186bku1", and then obtain the text search results, as shown in the table below:

[0043]

[0044] Specifically, when performing text retrieval using the Dense model, it ignores the similarity between the query and model numbers in the search text. Instead, it identifies the similarity score based on the similarity of sentence structure. Therefore, when the Dense model searches for "hwy14-186bku1", the search text "hwy14-186bku2" has the highest similarity, while the search text "Haier dishwasher hwy14-186bku1" has the lowest similarity. In contrast, when using the Sparse model, it considers the similarity between the query and model numbers in the search text to obtain more accurate search results. This ensures that perfectly matching texts are ranked first. Thus, when searching for "hwy14-186bku1", the search text "hwy15-186bku1" has the lowest similarity, while the search text "Haier dishwasher hwy14-186bku1" has the highest similarity.

[0045] As can be seen from the above, because the data reasoning principles of dense vector models and sparse vector models are different, the applicable application scenarios of sparse vector models are completely different from those of dense vector models. Furthermore, with the continuous and rapid development of dense vector models, their application scope is becoming wider and wider, but they still have the problem of low accuracy in retrieving text outside their domain. However, the above-mentioned problems are precisely the parts that sparse vector models are good at handling. Therefore, how to use two vector models, sparse and dense, for text retrieval operations has become an urgent problem to be solved.

[0046] To address the aforementioned technical issues, related technologies provide a solution that integrates sparse vector models and dense vector models for text retrieval. Specifically, sparse vector models and dense vector models can perform text retrieval operations using different index structures, resulting in two different text retrieval results. Then, the inverse ranking fusion (RRF) algorithm can be used to fuse the two text retrieval results to obtain the final retrieval result.

[0047] However, the two text retrieval results mentioned above need to be fused using vector indexing and RRF algorithms, which not only consumes additional storage and computing resources but also introduces additional computational overhead.

[0048] To address the aforementioned technical problems, embodiments of the present invention provide a text retrieval method, apparatus, device, and computer program product, wherein, with reference to the appendix... Figure 1As shown, the text retrieval method is executed by a text retrieval device 200, which is communicatively connected to a client 100. When the text retrieval device 200 is implemented as a cloud server, the text retrieval method can be executed in the cloud. Several computing nodes (cloud servers) can be deployed in the cloud, each with computing, storage, and other processing resources. In the cloud, multiple computing nodes can be organized to provide a certain service; conversely, a single computing node can provide one or more services. The cloud can provide this service through an external service interface, which users can call to use the corresponding service. Service interfaces include Software Development Kits (SDKs) and Application Programming Interfaces (APIs).

[0049] The text retrieval device 200 refers to a device capable of providing text retrieval operations in a network virtual environment, typically a device that utilizes a network for information planning and text retrieval operations. In physical implementation, the text retrieval device 200 can be any device capable of providing computing services, responding to text retrieval requests, and performing text retrieval operations; for example, it can be a cluster server, a regular server, a cloud server, a cloud host, a virtual data center, etc. The text retrieval device 200 mainly consists of a processor, hard disk, memory, system bus, etc., similar to a general computer architecture.

[0050] Client 100 can be implemented as a mobile phone, personal computer (PC), tablet computer, application program, etc. Furthermore, the basic structure of client 100 may include at least one processor. The number of processors depends on the configuration and type of client 100. Client 100 may also include memory, which can be volatile, such as Random Access Memory (RAM), or non-volatile, such as Read-Only Memory (ROM), flash memory, etc., or both types may be included. The memory typically stores the operating system (OS), one or more applications, and may also store program data. In addition to the processing unit and memory, client 100 also includes some basic configurations, such as a network interface card (NIC) chip, I / O bus, display components, and some peripheral devices. Optionally, some peripheral devices may include, for example, a keyboard, mouse, stylus, printer, etc. Other peripheral devices are well known in the art and will not be described in detail here.

[0051] In this embodiment described above, a network connection is established between the client 100 and the text retrieval device 200. This network connection can be a wireless or wired network connection. If the connection between the client 100 and the text retrieval device 200 is a communication connection, the mobile network standard can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, 5G, 6G, etc.

[0052] Client 100 is used by users to trigger text search operations when users have text search needs. Specifically, when a user has a text search need for a text to be searched, client 100 can obtain a text search request for that text. In some instances, client 100 can display a human-computer interaction interface, through which an execution operation for the text to be searched can be obtained, and based on the execution operation, a text search request for the text to be searched can be obtained. Alternatively, client 100 can be configured with a voice conversion module, which can then be used to obtain a voice interaction operation, and based on the voice interaction operation, a text search request can be obtained. After obtaining the text search request, it can be sent to text search device 200, so that text search device 200 can perform the corresponding text search operation based on the text search request.

[0053] The text retrieval device 200 is communicatively connected to the client 100 and is used to obtain text retrieval requests sent by the client 100. After obtaining the text retrieval request for the text to be retrieved, the device can analyze and process the text to be retrieved based on the text retrieval request to determine the sparse vector representation and dense vector representation of the text to be retrieved. The dense vector representation is used to identify the text features of the text to be detected, and the sparse vector representation is used to identify the text words in the text to be detected. The dimension of the sparse vector representation is higher than that of the dense vector representation.

[0054] Since sparse vector representation and dense vector representation are two different ways of representing the text to be retrieved, in order to improve the accuracy of text retrieval operations, after obtaining the sparse vector representation and dense vector representation, a fusion operation can be performed on the sparse vector representation and dense vector representation to obtain a fused vector representation. Then, a text retrieval operation can be performed based on the text retrieval request and the fused vector representation, so as to accurately obtain the text retrieval results.

[0055] In this embodiment, the sparse vector representation and dense vector representation of the text to be retrieved are determined, and then a fusion operation is performed on the sparse vector representation and dense vector representation to obtain a fused vector representation. Then, the text retrieval operation is performed based on the fused vector representation. Since the text retrieval operation is implemented based on the fusion operation of two vector representations of different dimensions, the accuracy of the text retrieval operation is effectively improved. In addition, this method can achieve stable text retrieval operation without making significant modifications to the retrieval architecture, thus ensuring the ease of use of the method and further ensuring its practicality.

[0056] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features can be combined with each other. Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0057] Figure 2 This is a flowchart illustrating a text retrieval method provided in an embodiment of the present invention; see attached diagram. Figure 2 As shown, this embodiment provides a text retrieval method. The execution subject of this method is a text retrieval device, which can be implemented as software or a combination of software and hardware. When the text retrieval device is implemented as hardware, it can be various electronic devices with text retrieval capabilities, including but not limited to personal computers, servers, etc. When the text retrieval device is implemented as software, it can be installed in the electronic devices listed above. Based on the above-described text retrieval device, the method in this embodiment can include:

[0058] Step S201: Obtain a text search request for the text to be searched.

[0059] Step S202: Determine the sparse vector representation and dense vector representation of the text to be retrieved. The dense vector representation is used to identify the text features of the text to be detected, and the sparse vector representation is used to identify the text words in the text to be detected. The dimensionality of the sparse vector representation is higher than that of the dense vector representation.

[0060] Step S203: Fuse the sparse vector representation and the dense vector representation to obtain the fused vector representation.

[0061] Step S204: Perform text retrieval operations based on the text retrieval request and the fused vector representation to obtain text retrieval results.

[0062] The specific implementation process and effects of each of the above steps are explained in detail below:

[0063] Step S201: Obtain a text search request for the text to be searched.

[0064] When a user has a text search requirement for the text to be searched, the text search device can obtain a text search request for the text to be searched. The obtained text search request can be implemented as text search keywords or text search statements, etc.

[0065] Furthermore, this embodiment does not limit the specific method of obtaining the text retrieval request. In some instances, the text retrieval request can be obtained through human-computer interaction. In this case, obtaining the text retrieval request for the text to be retrieved may include: displaying a human-computer interaction interface through a text retrieval device, wherein the human-computer interaction interface displays the text to be retrieved; obtaining the interactive operation input by the user in the human-computer interaction interface for the text to be retrieved; and obtaining the text retrieval request for the text to be retrieved based on the interactive operation, wherein one text retrieval request may correspond to one or more texts to be retrieved.

[0066] In other instances, text search requests can be obtained not only through human-computer interaction but also through voice interaction. In this case, obtaining a text search request for the text to be searched can include: obtaining the user's voice command through the voice processing module in the text search device; and determining the text search request for the text to be searched based on the voice command. This also allows for the stable acquisition of text search requests for the text to be searched.

[0067] In some other instances, text retrieval requests can be obtained not only through human-computer interaction or voice interaction, but also through a pre-set device (e.g., a client). In this case, obtaining a text retrieval request for the text to be retrieved may include: identifying a client that is communicatively connected to the text retrieval device, wherein the client stores the text retrieval request for the text to be retrieved; by actively or passively accessing the client, the text retrieval request for the text to be retrieved can be reliably obtained.

[0068] Step S202: Determine the sparse vector representation and dense vector representation of the text to be retrieved. The dense vector representation is used to identify the text features of the text to be detected, and the sparse vector representation is used to identify the text words in the text to be detected. The dimensionality of the sparse vector representation is higher than that of the dense vector representation.

[0069] To improve the accuracy of retrieval of the target text, after obtaining the target text, it can be analyzed and processed to determine the sparse vector representation and dense vector representation of the target text. The dense vector representation is used to identify the text features of the target text, while the sparse vector representation is used to identify the text words in the target text. Furthermore, the dimensionality of the sparse vector representation is higher than that of the dense vector representation. For example, the sparse vector representation can have 10,000 dimensions, while the dense vector representation can have 512 dimensions; or the sparse vector representation can have 20,000 dimensions, while the dense vector representation can have 256 dimensions, and so on.

[0070] Specifically, this embodiment does not limit the specific method for determining sparse vector representations and dense vector representations. In some instances, sparse vector representations and dense vector representations can be determined by analyzing and processing the text to be retrieved using a pre-trained machine learning model. In this case, determining the sparse vector representations and dense vector representations of the text to be retrieved may include: obtaining a pre-trained machine learning model; using the machine learning model to analyze and process the text to be retrieved, thereby obtaining the sparse vector representations and dense vector representations of the text to be retrieved output by the machine learning model. This effectively ensures the stability and reliability of determining the sparse vector representations and dense vector representations.

[0071] In other instances, sparse vector representations and dense vector representations can be determined not only by a single machine learning model but also by different network models. Specifically, sparse vector representations can be obtained by analyzing and processing the text to be retrieved using a pre-trained sparse vector model. In this case, determining the sparse vector representation of the text to be retrieved may include: obtaining a sparse vector model, which is trained based on preset sample data; and using the sparse vector model to analyze and process the text to be retrieved to obtain the sparse vector representation of the text to be retrieved. The sparse vector representation includes multiple text words corresponding to the text to be retrieved and the weight information corresponding to each of the multiple text words.

[0072] Specifically, a sparse vector model is pre-trained based on sample data. This sparse vector model is used to determine the sparse vector representation of the text. Therefore, after obtaining the text to be retrieved, the text to be retrieved can be input into the sparse vector model to obtain the sparse vector representation output by the sparse vector model. This effectively ensures the accuracy and reliability of determining the sparse vector representation.

[0073] For sparse vector representation, it can specifically include multiple text words corresponding to the text to be retrieved. For example, when the text to be retrieved is "a text retrieval method based on sparse vector representation and dense vector representation", the sparse vector representation of the text to be retrieved can be obtained by analyzing and processing the text through the sparse vector model. The sparse vector representation can include the following multiple text words: "a", "based on", "sparse", "vector", "representation", "and", "dense", "realize", "of", "text", "retrieval", "method", etc.

[0074] It's important to note that for sparse vector representations, the dimensionality of the sparse vector representation is the same as the dimensionality of the words included in the predefined vocabulary. The predefined vocabulary is set within the sparse vector model, which then analyzes and processes the retrieved text based on the predefined vocabulary to obtain a sparse vector representation with the same dimensionality as the predefined vocabulary. For example, if the predefined vocabulary has 10,000 dimensions, the obtained sparse vector representation will also have 10,000 dimensions; if the predefined vocabulary has 20,000 dimensions, the obtained sparse vector representation will also have 20,000 dimensions. For text words that exist in the predefined vocabulary but not in the retrieved text, the sparse vector representation can be represented by null values ​​(e.g., "0"). This ensures a stable sparse vector representation that includes most values ​​of 0.

[0075] Furthermore, sparse vector representation can include not only multiple text words corresponding to the text to be retrieved, but also the weight information corresponding to each of the multiple text words. The weight information corresponding to each of the multiple text words is used to identify the importance of each text word. Generally, the weight information is positively correlated with the importance of each text word. That is, the larger the weight information, the more important the text word corresponding to the weight information is; the smaller the weight information, the less important the text word corresponding to the weight information is.

[0076] Similarly, dense vector representations can be obtained by analyzing and processing the text to be retrieved using a pre-trained dense vector model. In this case, determining the dense vector representation of the text to be retrieved can include: obtaining a dense vector model, which is obtained by training based on preset sample data; and using the dense vector model to analyze and process the text to be retrieved to obtain the dense vector representation of the text to be retrieved, wherein the dimension of the dense vector representation satisfies the preset dimension parameters.

[0077] Specifically, a dense vector model is pre-trained based on sample data. This dense vector model is used to determine the dense vector representation of the text. Therefore, after obtaining the text to be retrieved, the text can be input into the dense vector model for analysis and processing to obtain the dense vector representation output by the dense vector model. This effectively ensures the accuracy and reliability of determining the dense vector representation.

[0078] In dense vector representation, the dimensions of the dense vector representation satisfy preset dimension parameters. Specifically, preset dimension parameters for feature extraction from text are pre-configured. Different application scenarios can configure different preset dimension parameters. For example, the preset dimension parameters can include at least one of the following: 512 dimensions, 768 dimensions, 1024 dimensions, etc. After determining the preset dimension parameters, the dense vector model can analyze and process the text to be retrieved based on the preset dimension parameters, thereby obtaining a dense vector representation that satisfies the preset dimension parameters. For example, when the preset dimension parameter is 512 dimensions, a 512-dimensional dense vector representation can be obtained; when the preset dimension parameter is 768 dimensions, a 768-dimensional dense vector representation can be obtained; and when the preset dimension parameter is 1024 dimensions, a 1024-dimensional dense vector representation can be obtained. This effectively ensures the accuracy and reliability of determining the dense vector representation.

[0079] Step S203: Fuse the sparse vector representation and the dense vector representation to obtain the fused vector representation.

[0080] In order to perform text retrieval operations by fusing sparse vector representations and dense vector representations, after obtaining the sparse vector representations and dense vector representations, a fusion operation can be performed on the sparse vector representations and dense vector representations to obtain a fused vector representation corresponding to the text to be retrieved.

[0081] Specifically, this embodiment does not limit the specific implementation of the fusion operation of sparse vector representation and dense vector representation. In some instances, the fusion operation can be implemented by a machine learning model. In this case, fusing sparse vector representation and dense vector representation to obtain fused vector representation may include: obtaining a pre-trained machine learning model; and using the machine learning model to perform fusion processing on sparse vector representation and dense vector representation to obtain fused vector representation.

[0082] In other instances, the fusion operation can be implemented not only through a pre-trained machine learning model, but also through a preset algorithm. In this case, fusing sparse vector representations and dense vector representations to obtain a fused vector representation may include: obtaining a preset algorithm for implementing the vector fusion operation, which may include at least one of the following: vector concatenation algorithm, vector weighted fusion algorithm, vector product fusion algorithm, etc. Then, the preset algorithm can be used to perform the fusion operation on the sparse vector representations and dense vector representations to obtain the fused vector representation.

[0083] For example: When the preset algorithm is a vector concatenation algorithm, and the sparse vector representation is L1-dimensional and the dense vector representation is L2-dimensional, the vector concatenation algorithm can be used to fuse the sparse and dense vector representations to obtain a fused vector representation of L1+L2 dimensions. When the preset algorithm is a vector weighted fusion algorithm, the vector weighted fusion algorithm can be used to perform a weighted fusion operation on the sparse and dense vector representations to obtain a fused vector representation of L1 or L2 dimensions. When the preset algorithm is a vector outer product fusion algorithm, the vector product fusion algorithm can be used to perform a product fusion operation on the sparse and dense vector representations to obtain a fused vector representation of L1*L2 dimensions. As can be seen, different fusion methods can obtain fused vector representations with different forms, meaning that the fused vector representation can be determined in different ways, thus effectively ensuring the flexibility and reliability of determining the fused vector representation.

[0084] Step S204: Perform text retrieval operations based on the text retrieval request and the fused vector representation to obtain text retrieval results.

[0085] After obtaining the fused vector representation, text retrieval operations can be performed based on the text retrieval request and the fused vector representation to obtain text retrieval results corresponding to the text retrieval request. In some instances, text retrieval operations can be implemented using a retrieval model. In this case, performing text retrieval operations based on the text retrieval request and the fused vector representation to obtain text retrieval results can include: obtaining a pre-trained retrieval model; inputting the text retrieval request and the fused vector representation into the retrieval model to perform text retrieval operations; and obtaining the text retrieval results output by the retrieval model, thereby effectively achieving efficient and stable text retrieval operations.

[0086] For example, when the text to be retrieved includes Text 1, Text 2, Text 3, and Text 4, and the text retrieval request is "preset product model", the fused vector representation of the text to be retrieved can be obtained by processing the text. This fused vector representation can include sparse and dense vector representations corresponding to each of the multiple texts. Then, the retrieval model can be used to analyze and process the "preset product model" and the "fused vector representation" to obtain the text retrieval results output by the retrieval model. These results can be used to identify whether the text to be retrieved contains "preset product model", or to identify the text similarity between the text to be retrieved and "preset product model".

[0087] In other instances, to improve the accuracy and reliability of text retrieval operations, these operations can be implemented not only through a single retrieval model but also based on both a retrieval model and a large language model. In this case, performing text retrieval based on a text retrieval request and a fused vector representation to obtain text retrieval results may include: determining the retrieval model and the large language model used to implement the text retrieval operation based on the text retrieval request; inputting the fused vector representation into the retrieval model for preliminary retrieval to obtain related text fragments corresponding to the text retrieval request; and inputting the text retrieval request and related text fragments into the large language model for text retrieval to obtain the text retrieval results.

[0088] Specifically, a retrieval model is pre-configured for initial text retrieval. When a text retrieval operation is required, the retrieval model and the large language model used to perform the text retrieval operation can be determined based on the text retrieval request. Then, the fused vector representation can be input into the retrieval model for initial retrieval to obtain the associated text fragments corresponding to the text retrieval request. Subsequently, the text retrieval request and associated text fragments can be input into the large language model for text retrieval, thereby stably obtaining text retrieval results. Compared with the text retrieval operation implemented by only one retrieval model, the text retrieval operation implemented by fusing the large language model can effectively improve the accuracy of the text retrieval operation.

[0089] The text retrieval method provided in this embodiment obtains a text retrieval request for the text to be retrieved; determines the sparse vector representation and dense vector representation of the text to be retrieved, and fuses the sparse vector representation and dense vector representation to obtain a fused vector representation; then, a text retrieval operation is performed based on the text retrieval request and the fused vector representation to obtain the text retrieval result. Since the text retrieval operation is implemented based on the fused vector representation after fusing two vector representations of different dimensions, the accuracy of the text retrieval operation is effectively improved. In addition, the implementation of this technical solution can achieve stable text retrieval operation without significant modification to the retrieval architecture. This not only ensures the utilization of computing resources by eliminating the need for additional storage and computing resources, but also effectively improves the ease of use of the method, further ensuring its practicality.

[0090] Figure 3 This is a flowchart illustrating the process of fusing sparse and dense vector representations to obtain a fused vector representation, provided for embodiments of the present invention. Based on the above embodiments, refer to the appendix... Figure 3 As shown, for sparse vector representations and dense vector representations, since the dimension of sparse vector representations is larger than that of dense vector representations, in order to ensure the accuracy and reliability of the fusion operation between sparse and dense vector representations, a dimensionality reduction operation can be performed on the sparse vector representations first. Then, the fusion operation can be performed based on the dimensionality-reduced sparse vector representations. In this case, fusing the sparse and dense vector representations to obtain the fused vector representation can include:

[0091] Step S301: Obtain the dimension parameters of the dense vector representation.

[0092] Since dense vector representations and sparse vector representations are vector representations of different dimensions, where the dimension of the sparse vector representation is greater than the dimension of the dense vector representation, or the dimension of the sparse vector representation is much greater than the dimension of the dense vector representation, in order to accurately perform the fusion operation on sparse and dense vector representations, after obtaining the dense vector representation, the dense vector representation can be analyzed and processed to obtain the dimension parameters of the dense vector representation.

[0093] In some instances, the dimension parameters of a dense vector representation can be the column dimension parameters of that representation. In this case, the dimension parameters can be obtained by scanning the dense vector representation. Alternatively, since the dense vector representation is obtained by analyzing and processing the retrieved text based on a dense vector model, its dimensions satisfy preset dimension parameters. Specifically, when it is necessary to determine the dense vector representation based on a dense vector model, preset dimension parameters can be configured first. After configuring the preset dimension parameters, they can be used as the dimension parameters of the dense vector representation, thus effectively ensuring the accuracy and reliability of obtaining the dimension parameters of the dense vector representation.

[0094] Step S302: Perform dimensionality reduction processing on the sparse vector representation based on the dimension parameter to obtain the dimensionality-reduced vector representation.

[0095] Since the dimensionality of sparse vector representations is higher than that of dense vector representations, in order to accurately achieve the fusion operation of sparse and dense vector representations, after obtaining the dimensionality parameters of the dense vector representation and the sparse vector representation, the sparse vector representation can be dimensionality reduced based on the dimensionality parameters of the dense vector representation to obtain the dimensionality-reduced vector representation. The dimensionality of the dimensionality-reduced vector representation can satisfy the aforementioned dimensionality parameters, or the dimensionality of the dimensionality-reduced vector representation can be close to the aforementioned dimensionality parameters. Here, "close" can mean that the dimensionality difference between the dimensionality of the dimensionality-reduced vector representation and the dimensionality parameters is less than or equal to a preset threshold. The preset threshold is the maximum allowable dimensionality difference that can guarantee the fusion effect of sparse and dense vector representations.

[0096] Furthermore, this embodiment does not limit the specific implementation method of dimensionality reduction operation on sparse vector representation. In some instances, dimensionality reduction operation can be implemented by a pre-trained machine learning model. In this case, dimensionality reduction of sparse vector representation based on dimension parameters to obtain dimensionality-reduced vector representation may include: obtaining a machine learning model for implementing dimensionality reduction operation; inputting dimension parameters and sparse vector representation into the machine learning model for dimensionality reduction, thereby obtaining the dimensionality-reduced vector representation output by the machine learning model.

[0097] In other instances, dimensionality reduction can be achieved not only through pre-trained machine learning models but also through random dimensionality reduction. In this case, dimensionality reduction of sparse vector representations based on dimension parameters can include: obtaining the element dimension of each vector element in the sparse vector representation; performing a modulo operation on the dimension parameter to obtain the dimensionality-reduced element dimension of each vector element; and determining the dimensionality-reduced vector representation based on the dimensionality-reduced element dimension of each vector element and the vector elements themselves.

[0098] The sparse vector representation includes multiple vector elements, each of which can correspond to a specific dimension parameter. For example, when the sparse vector representation is [a1 a2 a3 a4 a5 a6], the vector element a6 has a corresponding dimension parameter (6), and the vector element a3 has a corresponding dimension parameter (3). Specifically, since sparse vector representation and dense vector representation often need to be fused along the column vector dimension, after obtaining the sparse vector representation, the element dimension corresponding to each vector element can be directly obtained through the sparse vector representation. This element dimension can be the column dimension parameter of the vector element.

[0099] It is important to note that the element dimension of each vector element in a sparse vector representation can be obtained not only directly by analyzing and identifying the sparse vector representation, but also by sorting multiple vector elements in the sparse vector representation. In this case, obtaining the element dimension of each vector element in the sparse vector representation can include: obtaining multiple text words included in the sparse vector representation and their respective weight information; sorting the multiple text words based on their respective weight information to obtain word sorting information; and determining the element dimension of each vector element based on their respective word sorting information.

[0100] Specifically, since sparse vector representations include not only multiple text words and their respective weights, but also the weights that indicate the importance of each text word, in order to ensure that the dimensionality-reduced vector representation includes important text words, after obtaining the sparse vector representation, we can obtain the multiple text words and their respective weights. Then, we can sort the text words based on their respective weights, specifically from largest to smallest, to obtain word ranking information. For example, when the sparse vector is represented as [a1_v1 a2_v2 a3_v3 a4_v4 a5_v5 a6_v6], multiple text words can be sorted using their respective weight information (v1, v2, v3, v4, v5, v6) to obtain the following word sorting information [a5_v5 a1_v1 a3_v3 a6_v6 a2_v2 a4_v4].

[0101] After obtaining the word ranking information for each of the multiple text words, the element dimensions of each vector element can be directly determined based on this information. In some instances, the word ranking information directly determines the element dimensions of each vector element, effectively ensuring the accuracy and reliability of obtaining these dimensions. Dimensionality reduction can then be performed based on these dimensions. This avoids important text words appearing in the same positions in the reduced vector representation, thus ensuring the accuracy and reliability of the resulting vector representation.

[0102] Specifically, after obtaining the element dimensions of each vector element in the sparse vector representation, the modulo operation of the element dimensions of each vector element with respect to the dimension parameter can be performed to obtain the dimension-reduced element dimensions of each vector element. For example, the dimension-reduced element dimensions of each vector element can be id, and the dimension parameter can be d. The dimension-reduced element dimension = id % d is obtained by performing the modulo operation of the dimension-reduced element dimensions of each vector element with respect to the dimension parameter. After obtaining the dimension-reduced element dimensions, the dimension-reduced vector representation can be determined based on the dimension-reduced element dimensions of each vector element and the individual vector elements. This effectively ensures the accuracy and reliability of obtaining the dimension-reduced vector representation.

[0103] Step S303: Fuse the dimensionality-reduced vector representation and the dense vector representation to obtain the fused vector representation.

[0104] Since the dimensionality-reduced vector representation is a sparse vector representation obtained after the dimensionality reduction operation, in order to accurately perform text retrieval operations, after obtaining the dimensionality-reduced vector representation, a fusion operation can be performed on the dimensionality-reduced vector representation and the dense vector representation to obtain a fused vector representation. This effectively ensures the accuracy and reliability of obtaining the fused vector. The specific implementation method and principle of the fusion operation in this embodiment are similar to those in the above embodiments, and can be referred to the above descriptions for details, which will not be repeated here.

[0105] In this embodiment, the dimensionality parameter of the dense vector representation is obtained, and then the sparse vector representation is reduced based on the dimensionality parameter to obtain the dimensionality-reduced vector representation. The dimensionality-reduced vector representation and the dense vector representation are then fused to obtain a stable fused vector representation. Text retrieval operations can then be performed based on the obtained fused vector representation, further improving the practicality of the method.

[0106] Figure 4 This invention provides a flowchart illustrating the process of dimensionality reduction of sparse vector representations based on dimensionality parameters to obtain the dimensionality-reduced vector representations. Based on the above embodiments, refer to the appendix... Figure 4 As shown, dimensionality reduction can be achieved not only through pre-trained machine learning models or random dimensionality reduction methods, but also through clustering algorithms. In this embodiment, the dimensionality reduction of sparse vector representations based on dimensionality parameters to obtain the dimensionality-reduced vector representations can include:

[0107] Step S401: Obtain multiple text words included in the sparse vector representation.

[0108] Since a sparse vector representation includes multiple text words corresponding to the text to be retrieved, these text words may include the same or similar words. To represent the sparse vector representation, in order to represent the same or similar text words through the same position in the vector representation, and to ensure that different positions in the dimensionality-reduced vector representation express text words with different meanings, an aggregation operation can be performed on the multiple text words included in the sparse vector representation. Dimensionality reduction is then performed based on the result of this aggregation operation. Specifically, to achieve dimensionality reduction based on the aggregation operation, the multiple text words included in the sparse vector representation can be obtained first. After obtaining the sparse vector representation, multiple text words can be obtained through it, and these multiple text words correspond to the text to be retrieved.

[0109] Step S402: Cluster multiple text words based on dimension parameters to obtain word clustering results.

[0110] After obtaining multiple text words, clustering operations can be performed on the multiple text words based on the dimension parameters to obtain word clustering results. The word clustering results can be implemented as a set of text words, and each set of text words can include multiple text words with the same or similar meanings.

[0111] In some instances, clustering operations can be implemented using pre-trained machine learning models, or they can be obtained by clustering the word representation information corresponding to each of multiple text words. In this case, clustering multiple text words based on dimensional parameters to obtain word clustering results can include: obtaining the word representation information corresponding to each of multiple text words; and clustering the word representation information corresponding to each of multiple text words based on dimensional parameters to obtain word clustering results.

[0112] Specifically, after obtaining multiple text words, a preset algorithm or model can be used to extract features from them, thereby obtaining the word representation information corresponding to each text word. Then, clustering can be performed on the word representation information based on dimensional parameters. In some examples, clustering the word representation information based on dimensional parameters to obtain the word clustering results can include: determining the dimensional parameter as the K value used to implement the clustering operation; using the K-means clustering algorithm based on the K value to cluster the word representation information corresponding to each text word, thereby accurately obtaining the word clustering results. This effectively realizes the clustering operation based on dimensional parameters for multiple text words, thus effectively ensuring the accuracy and reliability of the word clustering results.

[0113] Step S403: Perform dimensionality reduction on the sparse vector representation based on the word clustering results to obtain the dimensionality-reduced vector representation.

[0114] To perform dimensionality reduction on sparse vector representations, after obtaining the word clustering results, the dimensionality reduction operation can be performed based on the word clustering results to obtain the dimensionality-reduced vector representation. In some instances, the dimensionality reduction operation can be implemented using a pre-trained machine learning model. In this case, performing dimensionality reduction on the sparse vector representation based on the word clustering results to obtain the dimensionality-reduced vector representation may include: obtaining a pre-trained machine learning model; inputting the word clustering results and the sparse vector representation into the machine learning model for dimensionality reduction, thereby obtaining the dimensionality-reduced vector representation. In this case, the dimensionality-reduced vector representation may include the word clustering results after the clustering operation.

[0115] In other instances, dimensionality reduction can be achieved not only through pre-trained machine learning models but also through statistical operations on text word sets. In this case, performing dimensionality reduction on sparse vector representations based on word clustering results to obtain dimensionality-reduced vector representations may include: determining at least one text word set in the sparse vector representation based on word clustering results, where different text word sets correspond to different word clustering results; determining the set dimension corresponding to each of the at least one text word set; and determining the dimensionality-reduced vector representation based on the set dimension corresponding to each of the at least one text word set and the at least one text word set.

[0116] Since word clustering results can aggregate identical or similar text words, in order to represent text words belonging to the same word clustering result through a position in the dimensionality-reduced vector representation, after obtaining the word clustering results, at least one set of text words in the sparse vector representation can be directly determined based on the word clustering results. Then, at least one set of text words can be analyzed and processed to determine the set dimension corresponding to each of the at least one set of text words.

[0117] In some instances, the set dimension corresponding to each of at least one text word set can be determined by the number of text word sets. In this case, determining the set dimension corresponding to each of at least one text word set may include: obtaining the number of text word sets; determining the set dimension corresponding to each of at least one text word set based on the number of sets. In this case, the set sorting information corresponding to each text word set can be directly determined as the set dimension corresponding to each text word set within the range limited by the number of sets.

[0118] In other instances, determining the set dimension corresponding to at least one set of text words may include: obtaining the number of text word sets; obtaining the importance of each text word set, which can be determined by analyzing the weight information of multiple text words in each set. For example, importance can be obtained by averaging the weight information of multiple text words, or by summing the weight information of multiple text words, or by using the median weight information of multiple text words. Then, at least one set of text words can be sorted based on its importance and the number of sets, obtaining sorting information, and the sorting information corresponding to each text word set can be used as the set dimension corresponding to each text word set. This effectively ensures the accuracy and reliability of determining the set dimension.

[0119] After obtaining the set dimension corresponding to at least one set of text words and at least one set of text words, the set dimension corresponding to at least one set of text words and at least one set of text words are analyzed and processed, so that the dimension-reduced vector representation can be stably determined. This effectively ensures the accuracy and reliability of determining the dimension-reduced vector representation.

[0120] In this embodiment, multiple text words included in the sparse vector representation are obtained, and then the multiple text words are clustered based on the dimension parameter to obtain the word clustering result. The sparse vector representation is then reduced in dimensionality based on the word clustering result, so that the dimensionality-reduced vector representation can be stably obtained. Then, text retrieval operations can be performed based on the dimensionality-reduced vector representation. This effectively ensures the accuracy and reliability of the text retrieval operation and further improves the accuracy and reliability of the method.

[0121] For specific applications, please refer to the appendix. Figure 5 As shown, this application embodiment provides a text retrieval method that combines sparse and dense vector models. This method fuses sparse and dense vector features based on their respective feature dimensions, and then performs text retrieval based on the fused vector representation. This solves the problem that sparse vector features, due to their extremely high dimensions (e.g., tens of thousands or hundreds of thousands of dimensions), cannot be calculated together with dense vector features, thereby improving the quality and efficiency of text retrieval. Specifically, the method may include the following steps:

[0122] Step 1: Obtain multiple texts to be searched and search questions to perform the text search operation.

[0123] The multiple texts to be searched may include Text1, Text2, Text3...Textn. These multiple texts are used to identify the text content that needs to be searched. The search question can be implemented as at least one of the following: search statement, search keywords, etc.

[0124] Step 2: Process multiple texts to be retrieved using a dense vector model to obtain dense vector representations.

[0125] The dense vector model can be trained on pre-defined sample data. Specifically, it can be a model based on architectures such as Natural Language Processing (BERT) or Large Language Model (LLM), as long as it can perform text feature extraction in the retrieval scenario. After obtaining multiple texts to be retrieved, these texts can be input into the dense vector model to obtain a fixed-dimensional dense vector representation output by the model. It's important to note that this fixed dimension can be a pre-configured scene dimension, such as 512, 768, or 1024 dimensions.

[0126] Step 3: Process multiple texts to be retrieved using a sparse vector model to obtain sparse vector representations. The sparse vector representations include a series of text words corresponding to the texts to be retrieved and the word weights of each text word.

[0127] Among them, the sparse vector model can be obtained by training on a preset sample data. It can be a model implemented based on information retrieval technology (e.g., bm25 algorithm), which can make the sparse vector model have high accuracy and strong domain generalization. Alternatively, the sparse vector model can be replaced by a neural network model, as long as it can output sparse vector representations corresponding to multiple texts to be retrieved.

[0128] In addition, the dimensional space of the multiple text words included in the sparse vector representation may depend on the dimension of a preset vocabulary, and the dimension of the above preset vocabulary may be implemented at the order of ten thousand or higher; moreover, different sparse vector models may correspond to different preset vocabularies, and the text words included in the preset vocabulary may be continuously updated, which can improve the accuracy of text retrieval; and the word weight corresponding to each of the above text words is used to identify the importance degree of the text word.

[0129] It should be noted that the execution order between step 3 and step 2 in this embodiment is not limited to the execution order defined in the above embodiments, and those skilled in the art can flexibly adjust the execution order between step 2 and step 2 according to specific application scenarios or application requirements, for example: step 2 and step 3 may be executed simultaneously, or step 3 may be executed before step 2.

[0130] Furthermore, in the process of processing multiple to-be-retrieved texts by using the sparse vector model and the dense vector model respectively, in order to avoid the situation that the efficiency of the text retrieval operation is reduced due to the large data volume of the to-be-retrieved texts, before processing the multiple to-be-retrieved texts by using the sparse vector model and the dense vector model respectively, the data volume corresponding to the multiple to-be-retrieved texts may be acquired. When the data volume is greater than or equal to a preset threshold, it indicates that the data volume of the multiple to-be-retrieved texts is large, and then the multiple to-be-retrieved texts may be split to obtain a plurality of text segments, and then the sparse vector model and the dense vector model may be used to analyze and process the plurality of text segments, so as to obtain the sparse vector representation and the dense vector representation.

[0131] Step 4: performing dimensionality reduction processing on the sparse vector representation to obtain a dimensionality-reduced sparse vector.

[0132] Since the dimension of the sparse vector representation is often at the order of ten thousand (D dimension), and the dimension of the dense vector representation is often a fixed dimension (d dimension), where d<<D, in order to facilitate the subsequent hybrid processing operation on the sparse vector representation and the dense vector representation, the D-dimensional sparse vector representation may be reduced in dimension to the same or similar d dimension as the dense vector representation, so that the dimensionality-reduced sparse vector can be obtained, wherein the dimension of the dimensionality-reduced sparse vector may be the same as the dimension of the dense vector representation, or the dimension of the dimensionality-reduced sparse vector may be between 1 time and 5 times the dimension of the dense vector representation.

[0133] To ensure the quality and effectiveness of text retrieval, the dimensionality reduction factor parameters used to implement the dimensionality reduction operation can be determined first. Specifically, all dimensionality reduction factor parameters that allow dimensionality reduction operations on sparse vector representations can be obtained. These dimensionality reduction factor parameters are used to identify whether dimensionality reduction operations can be performed on sparse vector representations to obtain dimensionality-reduced sparse vectors. These dimensionality-reduced sparse vectors are between 1 and 5 times the dimension of the dense vector representation. For example, all dimensionality reduction factor parameters can include: parameter a, which makes the dimensionality-reduced sparse vector 1 times the dimension of the dense vector representation; parameter b, which makes the dimensionality-reduced sparse vector 2 times the dimension of the dense vector representation; parameter c, which makes the dimensionality-reduced sparse vector 3 times the dimension of the dense vector representation; parameter d, which makes the dimensionality-reduced sparse vector 4 times the dimension of the dense vector representation; and parameter e, which makes the dimensionality-reduced sparse vector 5 times the dimension of the dense vector representation.

[0134] After obtaining all the dimensionality reduction factor parameters, the accuracy of text retrieval operations can be estimated based on each dimensionality reduction factor parameter. The dimensionality reduction factor parameter with the highest retrieval accuracy can then be used as the final dimensionality reduction factor parameter for dimensionality reduction operations on sparse vector representations.

[0135] Then, the actual reference dimension can be determined based on the final dimensionality reduction factor parameter and the fixed dimension of the dense vector representation. For example, when the final dimensionality reduction factor parameter is parameter b, which is used to make the sparse vector of the dimensionality reduction twice the dimension of the dense vector representation, and the fixed dimension of the dense vector representation is d, then the actual reference dimension can be determined to be 2*d. When the final dimensionality reduction factor parameter is parameter d, which is used to make the sparse vector of the dimensionality reduction four times the dimension of the dense vector representation, and the fixed dimension of the dense vector representation is d, then the actual reference dimension can be determined to be 4*d. This effectively ensures the accuracy and reliability of the dimensionality reduction operation, and also ensures the accuracy of text retrieval, while reducing the time cost of text retrieval operations.

[0136] In addition, dimensionality reduction can be implemented in the following ways:

[0137] Implementation method (1): Random dimensionality reduction

[0138] After obtaining the sparse vector representation, the word dimension (id) corresponding to each text word in the sparse vector representation can be obtained. The word dimension can be the column dimension of the D-dimensional space corresponding to each text word in the sparse vector representation. For the dense vector representation, the fixed dimension of the dense vector representation can be determined as d-dimensional. Then, the word dimension (id) can be moduloed by the fixed dimension (d-dimensional) to obtain the dimensionality-reduced word dimension of each text word in the d-dimensional space. That is, the dimensionality-reduced word dimension of each text word in the d-dimensional space can be id%d. Then, based on each text word and the dimensionality-reduced word dimension, a dimensionality-reduced sparse vector can be obtained.

[0139] Implementation method (2): Word importance dimensionality reduction method

[0140] After obtaining the sparse vector representation, we can obtain each text word and its weight in the sparse vector representation. Then, we can sort the text words according to their importance based on their weights. After that, we can obtain the word dimension (id) of each text word after sorting. The word dimension of each text word can be the column dimension of each text word in the sorted D-dimensional space.

[0141] For dense vector representations, the fixed dimension of the dense vector representation can be determined as 'd'. Then, the word dimension (id) can be moduloed by the fixed dimension (d) to obtain the dimensionality-reduced word dimension of each text word in the d-dimensional space. That is, the dimensionality-reduced word dimension of each text word in the d-dimensional space can be id%d. Then, based on each text word and the dimensionality-reduced word dimension, the dimensionality-reduced sparse vector can be obtained. Compared with the random dimensionality reduction method mentioned above, this implementation method can avoid the problem of the first d important text words appearing in the same position, causing text conflicts and affecting text retrieval, thus ensuring the accuracy and reliability of text retrieval operations.

[0142] Implementation method (3): Word clustering dimensionality reduction

[0143] After obtaining the sparse vector representation, we can obtain the individual text words and their weights within that representation. We then process each text word using a dense vector model to obtain its word representation vector. The fixed dimension (d-dimensionality) of the dense vector representation can then be used as the number of clusters (K-value). The k-means clustering algorithm is applied to cluster the word representation vectors to obtain the clustering results. Finally, we perform a dimensionality reduction operation based on the clustering results to obtain a dimensionality-reduced sparse vector. Specifically, the dimensionality-reduced sparse vector can include text words that are synonyms or identical words placed in the same position based on the clustering results. This not only reduces text word conflict but also introduces the additional expansion capability of synonyms.

[0144] Step 5: Fuse the dimensionality-reduced sparse vector and dense vector representations to obtain the fused vector.

[0145] In order to improve the quality and effectiveness of text retrieval operations, after obtaining the dense vector representation with fixed dimensions and the sparse vector after dimensionality reduction, the sparse vector and the dense vector representation can be concatenated to obtain the fused vector.

[0146] Step 6: Perform text retrieval based on the fused vectors to obtain the text retrieval results.

[0147] Specifically, after obtaining the fused vector, the fused vector can be input into the retrieval model for preliminary retrieval to obtain related text fragments corresponding to the retrieval question; then the retrieval question and related text fragments are input into the large language model for text retrieval to obtain the text retrieval results.

[0148] The technical solution provided in this application embodiment obtains the sparse and dense vector representations corresponding to multiple texts to be retrieved, and then performs a fusion operation on the sparse and dense vector representations at the feature vector dimension. This solves the problem that sparse vectors with very large dimensions cannot be calculated together with dense vectors, thus making the technical solution easy to use and friendly to the retrieval engine architecture. Then, text retrieval is performed based on the fused vectors, which not only improves the accuracy of text retrieval operations, but also reduces the additional consumption of computing and storage resources because stable text retrieval operations can be achieved without significant modifications to the retrieval architecture. This further improves the ease of use and reliability of the method, which is conducive to market promotion and application.

[0149] Figure 6 This is a schematic diagram of the structure of a text retrieval device provided in an embodiment of the present invention; see attached diagram. Figure 6 As shown, this embodiment provides a text retrieval device for performing the above-described... Figure 2The text retrieval method shown is specific; the device may include:

[0150] The first acquisition module 11 is used to acquire a text retrieval request for the text to be retrieved;

[0151] The first determining module 12 is used to determine the sparse vector representation and dense vector representation of the text to be retrieved. The dense vector representation is used to identify the text features of the text to be detected, and the sparse vector representation is used to identify the text words in the text to be detected. The dimension of the sparse vector representation is higher than that of the dense vector representation.

[0152] The first processing module 13 is used to fuse sparse vector representations and dense vector representations to obtain fused vector representations;

[0153] The first retrieval module 14 is used to perform text retrieval operations based on text retrieval requests and fused vector representations to obtain text retrieval results.

[0154] In some instances, when the first determining module 12 determines the sparse vector representation of the text to be retrieved, the first determining module 12 is used to perform: obtaining a sparse vector model, which is obtained by training based on preset sample data; and using the sparse vector model to analyze and process the text to be retrieved to obtain the sparse vector representation of the text to be retrieved, wherein the sparse vector representation includes multiple text words corresponding to the text to be retrieved and the weight information corresponding to each of the multiple text words.

[0155] In some instances, the dimensionality of the sparse vector representation is the same as the dimensionality of the words included in the pre-defined vocabulary, which is set in the sparse vector model.

[0156] In some instances, when the first determining module 12 determines the dense vector representation of the text to be retrieved, the first determining module 12 is used to perform: obtaining a dense vector model, which is obtained by training based on preset sample data; and using the dense vector model to analyze and process the text to be retrieved to obtain a dense vector representation of the text to be retrieved, wherein the dimension of the dense vector representation satisfies the preset dimension parameters.

[0157] In some instances, when the first processing module 13 fuses the sparse vector representation and the dense vector representation to obtain the fused vector representation, the first processing module 13 is used to perform: obtaining the dimension parameter of the dense vector representation; performing dimensionality reduction processing on the sparse vector representation based on the dimension parameter to obtain the dimensionality-reduced vector representation; and fusing the dimensionality-reduced vector representation and the dense vector representation to obtain the fused vector representation.

[0158] In some instances, when the first processing module 13 performs dimensionality reduction on the sparse vector representation based on the dimension parameter, the first processing module 13 is used to perform: obtaining the element dimension corresponding to each vector element in the sparse vector representation; performing a modulo operation on the dimension parameter on the element dimension corresponding to each vector element to obtain the dimension-reduced element dimension corresponding to each vector element; and determining the dimension-reduced vector representation based on the dimension-reduced element dimension corresponding to each vector element and each vector element.

[0159] In some instances, when the first processing module 13 obtains the element dimensions corresponding to each vector element in the sparse vector representation, the first processing module 13 is used to perform: obtaining multiple text words included in the sparse vector representation and the weight information corresponding to each of the multiple text words; sorting the multiple text words based on the weight information corresponding to each of the multiple text words to obtain word sorting information; and determining the element dimensions corresponding to each of the multiple vector elements based on the word sorting information corresponding to each of the multiple text words.

[0160] In some instances, when the first processing module 13 performs dimensionality reduction processing on the sparse vector representation based on the dimension parameter to obtain the dimensionality-reduced vector representation, the first processing module 13 is used to perform: obtaining multiple text words included in the sparse vector representation; performing clustering processing on the multiple text words based on the dimension parameter to obtain the word clustering result; and performing dimensionality reduction operation on the sparse vector representation based on the word clustering result to obtain the dimensionality-reduced vector representation.

[0161] In some instances, when the first processing module 13 performs clustering processing on multiple text words based on dimension parameters to obtain word clustering results, the first processing module 13 is used to perform: obtaining word representation information corresponding to each of the multiple text words; performing clustering processing on the word representation information corresponding to each of the multiple text words based on dimension parameters to obtain word clustering results.

[0162] In some instances, when the first processing module 13 performs clustering processing on the word representation information corresponding to each of multiple text words based on the dimension parameter to obtain word clustering results, the first processing module 13 is used to perform: determining the dimension parameter as the K value used to implement the clustering operation; and performing clustering processing on the word representation information corresponding to each of multiple text words based on the K value and using the K-means clustering algorithm to obtain word clustering results.

[0163] In some instances, when the first processing module 13 performs dimensionality reduction on the sparse vector representation based on the word clustering results to obtain the dimensionality-reduced vector representation, the first processing module 13 is used to perform the following: based on the word clustering results, determine at least one set of text words in the sparse vector representation, wherein different sets of text words correspond to different word clustering results; determine the set dimension corresponding to each of the at least one set of text words; and determine the dimensionality-reduced vector representation based on the set dimension corresponding to each of the at least one set of text words and the at least one set of text words.

[0164] In some instances, when the first retrieval module 14 performs a text retrieval operation based on the text retrieval request and the fused vector representation to obtain the text retrieval results, the first retrieval module 14 is used to perform the following: based on the text retrieval request, determine the retrieval model and the large language model used to implement the text retrieval operation; input the fused vector representation into the retrieval model for preliminary retrieval to obtain the associated text fragments corresponding to the text retrieval request; input the text retrieval request and the associated text fragments into the large language model for text retrieval to obtain the text retrieval results.

[0165] Figure 6 The device shown can perform Figures 1-5 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figures 1-5 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figures 1-5 The descriptions in the illustrated embodiments will not be repeated here.

[0166] In one possible design, Figure 6 The structure of the text retrieval device shown can be implemented as an electronic device, which can be various devices such as a controller, personal computer, or server. Figure 7 As shown, the electronic device may include a first processor 21 and a first memory 22. The first memory 22 is used to store data executed by the corresponding electronic device. Figures 1-5 In the text retrieval method program provided in the illustrated embodiment, the first processor 21 is configured to execute the program stored in the first memory 22.

[0167] The program includes one or more computer instructions, wherein when executed by the first processor 21, the one or more computer instructions can perform the following steps: obtaining a text retrieval request for the text to be retrieved; determining the sparse vector representation and dense vector representation of the text to be retrieved, wherein the dense vector representation is used to identify the text features of the text to be detected, and the sparse vector representation is used to identify the text words in the text to be detected, and the dimension of the sparse vector representation is higher than that of the dense vector representation; fusing the sparse vector representation and the dense vector representation to obtain a fused vector representation; and performing a text retrieval operation based on the text retrieval request and the fused vector representation to obtain the text retrieval results.

[0168] Furthermore, the first processor 21 is also used to perform the aforementioned Figures 1-5 All or part of the steps in the illustrated embodiments.

[0169] The structure of the electronic device may also include a first communication interface 23 for communication between the electronic device and other devices or communication networks.

[0170] In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used by an electronic device, which includes instructions for executing the above-described... Figures 1-5 The procedure involved in the text retrieval method in the illustrated embodiment.

[0171] Furthermore, embodiments of the present invention provide a computer program product, comprising: a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause one or more processors to perform the aforementioned... Figures 1-5 The steps in the text retrieval method in the illustrated method embodiment.

[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0173] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0176] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0177] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text retrieval method, characterized in that, include: Obtain the text search request for the text to be searched; Determine the sparse vector representation and dense vector representation of the text to be retrieved, wherein the dense vector representation is used to identify the text features of the text to be retrieved, and the sparse vector representation is used to identify the text words in the text to be retrieved, and the dimension of the sparse vector representation is higher than the dimension of the dense vector representation; Obtain the dimension parameters represented by the dense vector; The sparse vector representation is reduced in dimensionality based on the dimensionality parameter to obtain the dimensionality-reduced vector representation. The dimensionality-reduced vector representation and the dense vector representation are fused to obtain a fused vector representation; A text retrieval operation is performed based on the text retrieval request and the fused vector representation to obtain text retrieval results; The process of reducing the dimensionality of the sparse vector representation based on the dimensionality parameter to obtain the dimensionality-reduced vector representation includes: Obtain the multiple text words included in the sparse vector representation and the weight information corresponding to each of the multiple text words; The multiple text words are sorted based on their respective weight information to obtain word sorting information; Based on the word sorting information corresponding to each of the multiple text words, the element dimension corresponding to each of the multiple vector elements is determined. The element dimension corresponding to each vector element is moduloed by the dimension parameter to obtain the dimension-reduced element dimension corresponding to each vector element. The dimension-reduced vector representation is determined based on the dimension of each vector element and the vector elements themselves.

2. The method according to claim 1, characterized in that, Determining the sparse vector representation of the text to be retrieved includes: Obtain a sparse vector model, which is obtained by training based on preset sample data; The sparse vector model is used to analyze and process the text to be retrieved to obtain a sparse vector representation of the text to be retrieved. The sparse vector representation includes multiple text words corresponding to the text to be retrieved and the weight information of each text word.

3. The method according to claim 2, characterized in that, The dimensionality of the sparse vector representation is the same as the dimensionality of the words included in the preset vocabulary, which is set in the sparse vector model.

4. The method according to claim 1, characterized in that, Determining the dense vector representation of the text to be retrieved includes: Obtain a dense vector model, which is obtained by training based on preset sample data; The dense vector model is used to analyze and process the text to be retrieved to obtain a dense vector representation of the text to be retrieved, wherein the dimension of the dense vector representation satisfies a preset dimension parameter.

5. The method according to claim 1, characterized in that, The sparse vector representation is reduced in dimensionality based on the dimensionality parameter to obtain the dimensionality-reduced vector representation, including: Obtain multiple text words included in the sparse vector representation; Based on the dimensional parameters, the multiple text words are clustered to obtain word clustering results; The sparse vector representation is reduced in dimensionality based on the word clustering results to obtain the dimensionality-reduced vector representation.

6. The method according to claim 5, characterized in that, Based on the aforementioned dimensional parameters, clustering is performed on the multiple text words to obtain word clustering results, including: Obtain the word representation information corresponding to each of the multiple text words; Based on the dimensional parameters, the word representation information corresponding to each of the multiple text words is clustered to obtain the word clustering results.

7. The method according to claim 6, characterized in that, Based on the dimensional parameters, the word representation information corresponding to each of the multiple text words is clustered to obtain word clustering results, including: The dimension parameter is determined as the K value used to implement the clustering operation; Based on the K value, and using the K-means clustering algorithm, the word representation information corresponding to each of the multiple text words is clustered to obtain the word clustering results.

8. The method according to claim 5, characterized in that, Based on the word clustering results, the sparse vector representation is reduced in dimensionality to obtain the dimensionality-reduced vector representation, including: Based on the word clustering results, at least one set of text words in the sparse vector representation is determined, wherein different sets of text words correspond to different word clustering results; Determine the set dimension corresponding to at least one set of text words; The dimensionality-reduced vector representation is determined based on the set dimension corresponding to each of the at least one set of text words and the at least one set of text words.

9. The method according to any one of claims 1-8, characterized in that, Based on the text retrieval request and the fused vector representation, a text retrieval operation is performed to obtain text retrieval results, including: Based on the text retrieval request, determine the retrieval model and large language model used to implement the text retrieval operation; The fusion vector representation is input into the retrieval model for preliminary retrieval to obtain the associated text fragments corresponding to the text retrieval request; The text retrieval request and the associated text fragment are input into the large language model to perform text retrieval and obtain the text retrieval results.

10. A text retrieval device, characterized in that, include: The first acquisition module is used to acquire text search requests for the text to be searched; The first determining module is used to determine the sparse vector representation and the dense vector representation of the text to be retrieved, wherein the dense vector representation is used to identify the text features of the text to be retrieved, and the sparse vector representation is used to identify the text words in the text to be retrieved, and the dimension of the sparse vector representation is higher than the dimension of the dense vector representation. The first processing module is used to obtain the dimension parameters of the dense vector representation; perform dimensionality reduction processing on the sparse vector representation based on the dimension parameters to obtain the dimensionality-reduced vector representation; and fuse the dimensionality-reduced vector representation and the dense vector representation to obtain the fused vector representation. The first retrieval module is used to perform text retrieval operations based on the text retrieval request and the fused vector representation to obtain text retrieval results; When the first processing module performs dimensionality reduction processing on the sparse vector representation based on the dimension parameter to obtain the dimensionality-reduced vector representation, the first processing module is configured to: obtain multiple text words included in the sparse vector representation and the weight information corresponding to each of the multiple text words; sort the multiple text words based on the weight information corresponding to each of the multiple text words to obtain word sorting information; determine the element dimension corresponding to each of the multiple vector elements based on the word sorting information corresponding to each of the multiple text words; perform a modulo operation on the dimension parameter by the element dimension corresponding to each vector element to obtain the dimensionality-reduced element dimension corresponding to each vector element; and determine the dimensionality-reduced vector representation based on the dimensionality-reduced element dimension corresponding to each vector element and each vector element.

11. An electronic device, characterized in that, include: A memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method of any one of claims 1-9.

12. A computer storage medium, characterized in that, Used to store a computer program that, when executed by a computer, implements the method of any one of claims 1-9.

13. A computer program product, characterized in that, include: A computer program, when executed by a processor of an electronic device, causes the processor to perform the steps of the method of any one of claims 1-9.

Images