Article retrieval method, article retrieval device and electronic equipment

Abstract

The embodiment of the invention discloses an article retrieval method, an article retrieval device and electronic equipment. A specific embodiment of the method comprises the steps of generating a request vector in response to a received article retrieval request, and generating retrieval attribute data according to a retrieval condition in the article retrieval request; according to the request vector, retrieving an article vector set under a target leaf node matched with the request vector from a pre-constructed article vector tree; and according to the description attribute data of the articles, retrieving a recalled article set matched with the retrieval attribute data from the articles indicated by the article vector set so as to be screened by the business layer. The implementation mode is related to an information retrieval technology, and some filtering rules of a business layer can be integrated into a vector recall stage of an engine layer in advance. In this way, invalid commodity advertisements which do not meet user requirements can be filtered out in advance, so that the data volume of invalid data needing to be processed by a subsequent service layer is reduced, and the overall retrieval efficiency is improved.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of information retrieval technology, specifically to item retrieval methods, item retrieval devices, and electronic devices. Background Technology

[0002] In advertising system architecture, the ad retrieval platform often plays a crucial role, typically located upstream of the advertising system, completing the initial matching of users and ad materials. This involves real-time filtering from a massive ad material database to select a set of hundreds of ads that match user interests, allowing downstream processes to further filter and sort them. The key capabilities of such ad retrieval platforms are mainly concentrated in the following two core modules:

[0003] 1. Engine Layer Recall: This layer is responsible for using various recall strategies to select tens of thousands of ads that are relevant to user interests from a massive amount of advertising materials.

[0004] 2. Business layer filtering and coarse sorting: Further filtering of the recalled ad set according to business rules to ensure that the selected ads comply with the rule constraints under the specific business scenario.

[0005] However, the inventors discovered that because the engine-level recall process failed to fully consider the business rule constraints under various scenarios, the recalled ad set was often heavily filtered when the business layer executed business rule filtering, resulting in a reduced number of final returned results. This also affected the retrieval efficiency of the ad retrieval platform.

[0006] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0008] Some embodiments of this disclosure provide gait feature determination methods, gait feature determination devices, electronic devices, computer-readable media, and computer program products to solve one or more of the technical problems mentioned in the background section above.

[0009] In a first aspect, some embodiments of this disclosure provide an item retrieval method, including: in response to receiving an item retrieval request, generating a request vector; and generating retrieval attribute data based on retrieval conditions in the item retrieval request, wherein the request vector and the retrieval attribute data represent different data information; retrieving, based on the request vector, an item vector set under a target leaf node matching the request vector from a pre-constructed item vector tree, wherein the item vector tree is obtained by clustering the item vectors of items; and retrieving a recall item set matching the retrieval attribute data from the items indicated by the item vector set, based on the item description attribute data, for filtering processing by the business layer.

[0010] In some embodiments, generating search attribute data based on search conditions in an item search request includes: generating a Boolean expression based on the search conditions in the item search request; calculating the hash value of each search condition in the Boolean expression according to the hash algorithm of the Bloom filter to obtain the bit array of the search condition; and using the bit array of each search condition in the Boolean expression as the search attribute data of the item search request.

[0011] In some embodiments, retrieving a recall item set that matches the retrieval attribute data from items indicated by an item vector set based on the item's descriptive attribute data includes: for each item indicated by the item vector set, determining a bit array of descriptive attribute data for each item based on a Bloom filter; matching the bit array of descriptive attribute data for each item with the bit array of retrieval attribute data; and identifying the items indicated by the bit array of descriptive attribute data that matches the bit array of each retrieval condition in the retrieval attribute data as recall items, thereby obtaining a recall item set.

[0012] In some embodiments, the method further includes: constructing a Bloom filter object corresponding to the item to store and retrieve the item's descriptive attribute data; calculating the hash value of the item's descriptive attribute data, storing the calculated hash value in the corresponding Bloom filter, and obtaining a bit array of the item's descriptive attribute data.

[0013] In some embodiments, the method further includes: standardizing the attribute information of the item according to a preset standard attribute; and generating descriptive attribute data of the item in a preset format according to the standardized attribute and the corresponding attribute value.

[0014] In some embodiments, the method further includes: in response to detecting a change in item data, obtaining data in the latest Bloom filter containing the descriptive attribute data of the item, and obtaining the latest item vector tree for storage in memory; for the item indicated by the item vector under each leaf node in the latest item vector tree, determining the information of the Bloom filter object corresponding to the item, and storing the information under the leaf node to which the item belongs.

[0015] In some embodiments, in response to detecting a change in item data, obtaining the latest data in the Bloom filter storing the item's descriptive attribute data, and obtaining the latest item vector tree, includes: in response to detecting a change in the version information of the index file, obtaining the latest version of the index file, wherein the index file is obtained by serialization processing using a preset data description language after the item vector tree and Bloom filter have been constructed; and deserializing the obtained index file using the preset data description language to obtain the Bloom filter data and item vector tree in the file.

[0016] In some embodiments, the item vector tree is constructed by: obtaining relevant information about items on the platform and generating item vectors, wherein the relevant information includes at least one of the following: text title information in the item display page, identification information of the manufacturer, and item image; performing clustering analysis on the item vectors using a clustering algorithm until the clustering termination condition is met, thereby obtaining the item vector tree; wherein, for a node in the item vector tree, the average value of the item vectors of all items belonging to that node is used as the node vector of that node, and the item identifier of the item belonging to that leaf node is stored under each leaf node.

[0017] In some embodiments, the method further includes: filtering recalled items in the recalled item set according to the item retrieval request and preset business rules, and using the processing result as the retrieval result of the item retrieval request.

[0018] Secondly, some embodiments of this disclosure provide an item retrieval device, including: a request data generation unit configured to generate a request vector in response to receiving an item retrieval request, and to generate retrieval attribute data according to retrieval conditions in the item retrieval request, wherein the request vector and the retrieval attribute data represent different data information; a vector tree retrieval unit configured to retrieve, according to the request vector, a set of item vectors under a target leaf node that matches the request vector from a pre-constructed item vector tree, wherein the item vector tree is obtained by clustering the item vectors of items; and an attribute retrieval unit configured to retrieve a set of recalled items that match the retrieval attribute data from the items indicated by the item vector set, according to the item's descriptive attribute data, for filtering processing by a business layer.

[0019] In some embodiments, the request data generation unit is further configured to generate a Boolean expression based on the search conditions in the item search request; for each search condition in the Boolean expression, calculate the hash value of the search condition according to the hash algorithm of the Bloom filter to obtain the bit array of the search condition; and use the bit array of each search condition in the Boolean expression as the search attribute data of the item search request.

[0020] In some embodiments, the attribute retrieval unit is further configured to, for each item indicated by the item vector set, determine a bit array of descriptive attribute data for each item according to a Bloom filter; match the bit array of descriptive attribute data for each item with the bit array of retrieval attribute data; and determine the item indicated by the bit array of descriptive attribute data that matches the bit array of each retrieval condition in the retrieval attribute data as the recall item, thereby obtaining a recall item set.

[0021] In some embodiments, the item retrieval device further includes an item data storage unit configured to construct a Bloom filter object corresponding to the item for storing and retrieving the item's descriptive attribute data; calculate the hash value of the item's descriptive attribute data; store the calculated hash value in the corresponding Bloom filter to obtain a bit array of the item's descriptive attribute data.

[0022] In some embodiments, the item retrieval device further includes a data standardization unit configured to standardize the attribute information of the item according to a preset standard attribute; and to generate descriptive attribute data of the item in a preset format according to the standardized attributes and corresponding attribute values.

[0023] In some embodiments, the item retrieval device further includes an item data acquisition unit, configured to, in response to detecting an item data change, acquire the latest data in the Bloom filter containing the item's descriptive attribute data, and acquire the latest item vector tree for storage in memory; for the item indicated by the item vector under each leaf node in the latest item vector tree, determine the information of the Bloom filter object corresponding to the item, and store the information under the leaf node to which the item belongs.

[0024] In some embodiments, the item data acquisition unit is further configured to acquire the latest version of the index file in response to detecting a change in the version information of the index file, wherein the index file is obtained by serialization processing using a preset data description language after the item vector tree and Bloom filter have been constructed; and the acquired index file is deserialized using the preset data description language to obtain the Bloom filter data and item vector tree in the file.

[0025] In some embodiments, the item data generation unit is further configured to acquire relevant information about items on the platform and generate item vectors, wherein the relevant information includes at least one of the following: text title information in the item display page, identification information of the manufacturer, and item image; perform clustering analysis on the item vectors using a clustering algorithm until the clustering termination condition is met to obtain an item vector tree; wherein, for a node in the item vector tree, the average value of the item vectors of each item belonging to that node is used as the node vector of that node, and the item identifier of the item belonging to that leaf node is stored under each leaf node.

[0026] In some embodiments, the item retrieval device further includes a business filtering unit, configured to filter recalled items in the recalled item set according to the item retrieval request and preset business rules, and to use the processing result as the retrieval result of the item retrieval request.

[0027] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the item retrieval method described in any of the implementations of the first aspect above.

[0028] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the item retrieval method described in any of the implementations of the first aspect above.

[0029] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the item retrieval method described in any of the implementations of the first aspect above.

[0030] The above-described embodiments of this disclosure have the following beneficial effects: The item retrieval methods of some embodiments of this disclosure can filter out invalid items that do not meet user needs in advance during the recall process, reducing the amount of invalid data to be processed in subsequent business filtering stages, thereby improving overall retrieval efficiency. Specifically, in related retrieval technologies, the engine-layer recall stage fails to fully consider business rule constraints in various scenarios, such as user-specific filtering options and geographical location restrictions. Therefore, the recalled ad set is usually heavily filtered when the business layer executes business rule filtering, which not only reduces the number of final returned results but also affects the retrieval efficiency of the ad retrieval platform. In extreme cases, all recall results may be filtered.

[0031] To alleviate these problems, theoretically, increasing the number of ads recalled at the engine layer could increase the number of results filtered by the business layer. However, this method often introduces new issues: as the number of recalled ads increases, retrieval time also rises significantly, affecting the system's throughput, especially when handling complex query conditions or high-concurrency requests. Furthermore, in scenarios with extremely high filtering ratios, where most recalled ads are filtered out by the business layer, simply increasing the number of recalled ads cannot effectively alleviate the problem.

[0032] Based on this, the item retrieval method disclosed herein addresses the inefficiency of the aforementioned advertising retrieval process by proposing a scenario-based retrieval solution that integrates multi-scenario business rule constraint filtering into the engine layer vector recall stage in advance. By pre-constructing an item vector tree, items can be quickly retrieved based on the request vector of the item retrieval request. Subsequently, based on the retrieval attribute data of the item retrieval request and the item's descriptive attribute data, further retrieval and filtering of the retrieved items can be performed. This allows invalid items that do not meet user needs to be filtered out in advance during the recall process. This reduces the amount of invalid data that the subsequent business layer needs to process, reduces resource consumption, and improves overall retrieval efficiency. Furthermore, by performing business rule constraint filtering in advance during the recall process, the amount of valid items among the recalled items can be guaranteed, thereby reducing the number of items filtered out by the subsequent business layer and ensuring that the final returned advertising queue contains a sufficient number of valid results. Attached Figure Description

[0033] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0034] Figure 1 This is a flowchart of some embodiments of the item retrieval method disclosed herein;

[0035] Figure 2 This is a schematic diagram of the overall architecture of some application scenarios of the item retrieval method disclosed herein;

[0036] Figure 3A yes Figure 2 A schematic diagram illustrating the construction process of the item vector tree and Bloom filter;

[0037] Figure 3B yes Figure 2 A schematic diagram illustrating the process of generating Boolean expressions for search criteria;

[0038] Figure 3C yes Figure 2A flowchart illustrating the process of updating data for online services;

[0039] Figure 4 These are schematic diagrams illustrating the structure of some embodiments of the item retrieval device disclosed herein;

[0040] Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0041] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0042] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0043] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0044] Furthermore, the terms “a” and “a plurality” used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as “one or more”.

[0045] Figure 1 A flow 100 is shown, illustrating some embodiments of an item retrieval method according to this disclosure. The method may include the following steps:

[0046] Step 101: In response to receiving an item retrieval request, generate a request vector and generate retrieval attribute data based on the retrieval conditions in the item retrieval request.

[0047] In some embodiments, the entity executing the item retrieval method of this disclosure (such as...) Figure 2 The online search server (in the context of the system) can communicate with other electronic devices via wired or wireless connections, such as... Figure 2 This can be achieved through offline index building servers or Bloom filters. Users can interact with the executing entity using their terminal devices. For example, various applications can be installed on the terminal device, such as shopping apps, where users can search for items.

[0048] In this scenario, the terminal device can generate an item retrieval request and send it to the executing entity. Upon receiving the item retrieval request, the executing entity can generate a request vector. Furthermore, based on the retrieval criteria in the item retrieval request, it can generate retrieval attribute data. The request vector and the retrieval attribute data represent different data information.

[0049] As an example, a request vector can represent the initial request data entered by the user. Search attribute data, on the other hand, can represent the search criteria added by the user. For instance, a user enters "mobile phone" in the search box. Additionally, the user sets a price range and a screen size range as optional filters. In this case, the executing entity can convert the word "mobile phone" into a vector representation as the request vector. Furthermore, the set price range and screen size range can be used as the search attribute data for this request.

[0050] In some embodiments, the executing entity can generate a request vector based on the item retrieval request and the user's historical behavior data. The historical behavior data may include, but is not limited to, at least one of the following: historically viewed item data, historically saved item data, historical order data, etc. As an example, after receiving an item retrieval request, the executing entity can determine the user identifier (such as an account) that sent the request. Then, based on the user identifier, it can obtain the user's historical behavior data on the application. Next, the item retrieval request and historical behavior data can be input into a pre-built recall model to generate a request vector. This recall model can employ various machine learning model structures, which will not be elaborated upon here.

[0051] In addition, the executing entity can utilize a pre-defined constraint database to determine the search conditions in the item search request that belong to the constraints in the constraint database, or the search conditions that match them. Then, it can generate search attribute data based on these search conditions.

[0052] Step 102: Based on the request vector, retrieve the set of item vectors under the target leaf node that matches the request vector from the pre-built item vector tree.

[0053] In some embodiments, based on the request vector obtained in step 201, the executing entity can retrieve the set of item vectors under the target leaf node that matches the request vector from a pre-constructed item vector tree. The item vector tree is generally obtained by clustering the item vectors of the items.

[0054] Here, the classic BeamSearch algorithm can be used to search layer by layer in the item vector tree. Starting from the root node, the distance between the request vector and the node vectors of the current layer is calculated, and the K closest nodes are selected after sorting. Then, the child nodes of these K nodes are used as a new candidate set, and the K closest nodes to the request vector are calculated from this set. This process is repeated at each layer until the K closest target leaf nodes to the request vector are found. The number K and the distance algorithm can be set according to actual needs. For example, the distance can be a dot product or cosine distance.

[0055] Understandably, when the number of items is insufficient, the executing entity can use all item vectors under these target leaf nodes as the set of retrieved item vectors. When the number of items is sufficient, for all item vectors under these target leaf nodes, the executing entity can recalculate the distance between them and the request vector, and then use the K item vectors closest to the request vector as the set of retrieved item vectors.

[0056] In some embodiments, such as Figure 2 and 3A As shown, the item vector tree can be constructed using the following methods. First, relevant information about items on the platform can be obtained to generate item vectors (i.e., item embeddings). This relevant information can include at least one of the following: text title information from the item display page, identification information of the manufacturer (such as brand), item images, etc. Similarly, the relevant information about the items can be input into a pre-built recall model to generate item vectors. Next, a clustering algorithm can be used to cluster the item vectors until the clustering termination condition is met, thus obtaining the item vector tree (i.e., vector tree index). The clustering termination condition is not limited, such as the number of tree levels reaching a set value, or the number of items in the cluster reaching a set value, etc. Here, for a node (i.e., an intermediate node) in the item vector tree, the average of the item vectors of all items belonging to that node can be used as the node vector of that node. Furthermore, under each leaf node (i.e., an item node), the item identifier of the items belonging to that leaf node can be stored, such as item_id (IdentityDocument, identity number).

[0057] It's important to note that the literal meaning of "item" generally includes terms like "product," "item," or "strip width." In a retrieval system, "item" typically refers to a single object or record that can be retrieved, and its physical meaning depends on the specific retrieval scenario. For example, in an e-commerce advertising retrieval system, "item" generally represents the goods within the advertising materials. Furthermore, the item vector tree can be built offline by the advertising pipeline system or pre-built by the executing entity.

[0058] Step 103: Based on the item description attribute data, retrieve the recall item set that matches the retrieval attribute data from the items indicated by the item vector set, so that the business layer can perform filtering processing.

[0059] In some embodiments, the executing entity can obtain the descriptive attribute data of the items indicated by the item vector set obtained in step 202. Then, based on the descriptive attribute data of each item, it can retrieve items that match the retrieved attribute data from the items indicated by the item vector set, and use these as recalled items to obtain a recalled item set. These recalled items are those retrieved by the engine layer and require further filtering processing by the business layer.

[0060] Here, descriptive attribute data typically refers to data describing the attributes of an item, and can be obtained in various ways. It's important to note that the attribute (constraint) descriptions may differ between different items, and the number of attributes may also vary. To reduce or avoid retrieval errors caused by attribute descriptions, the item's attribute information can first be standardized. For example, a standard attribute table (i.e., an algorithm vocabulary) can be used to unify the item's attribute descriptions. Alternatively, the intersection of the standard attribute table and the item's attributes can be found, and the attributes that exist in both can be used as the standardized attributes of the item. Based on the standardized attributes and their corresponding attribute values, the item's descriptive attribute data can be obtained.

[0061] In some embodiments, the executing entity may directly store the raw fields of the item's descriptive attribute data in memory, thereby performing matching analysis with the retrieved attribute data.

[0062] As described above, the item retrieval method disclosed herein addresses the inefficiency of the aforementioned advertising retrieval process by proposing a scenario-based retrieval solution that integrates multi-scenario business rule constraints and filtering into the engine-layer vector recall stage in advance. By pre-constructing an item vector tree, items can be quickly retrieved based on the request vector of the item retrieval request. Subsequently, based on the retrieval attribute data of the item retrieval request and the item's descriptive attribute data, further retrieval and filtering of the retrieved items can be performed. This allows invalid items that do not meet user needs to be filtered out in advance during the recall process, retaining the set of valid items. This reduces the amount of invalid data that the subsequent business layer needs to process, reduces resource consumption, and improves overall retrieval efficiency. Furthermore, by performing business rule constraint filtering in advance during the recall process, the amount of valid items in the recalled items can be guaranteed, thereby reducing the number of items filtered out by the subsequent business layer and ensuring that the final returned advertising queue contains a sufficient number of valid results.

[0063] It should be noted that directly storing raw fields generally has the following drawbacks: 1) It consumes a lot of memory, and the storage space increases linearly with the number of condition fields, which increases the system's memory burden. 2) Using raw fields for condition matching is less efficient, which will affect the overall performance of the engine layer recall. 3) Introducing raw fields means that the engine layer recall is coupled with complex business filtering logic at the code level.

[0064] Because directly storing raw fields has the aforementioned drawbacks, in some application scenarios, the item's attribute information can first be standardized based on preset standard attributes (such as the standard attribute table mentioned above). Then, based on the standardized attributes and their corresponding values, descriptive attribute data in a preset format can be generated for the item. Finally, this descriptive attribute data can be stored for item retrieval.

[0065] As an example, such as Figure 2 and 3A As shown, the attributes (i.e., constraints) of an item can be processed. The attribute (i.e., constraint) type can be represented by `attr_type`, and the specific value of the attribute, i.e., the attribute value, can be recorded as `attr_value`. Thus, they can be uniformly concatenated into a string according to the format "attr_type^attr_value". This processing method is applicable to condition values ​​of any type, such as integers, floating-point numbers, strings, etc.

[0066] Furthermore, a Bloom filter object corresponding to each item can be constructed. This establishes a mapping between items (item_id) and Bloom filters. Next, the hash value of the item's descriptive attribute data is calculated, and this hash value is stored in the corresponding Bloom filter, resulting in a bit array of the item's descriptive attribute data. In other words, a Bloom filter object can be created for each item, and all the processed attribute values ​​are inserted into the Bloom filter. These attribute values ​​are ultimately mapped to bits in the bit array.

[0067] In other words, when constructing a Bloom filter for an item, the different attributes of the item are standardized and converted into hash values. Each attribute has its "attribute type - attr_type" and "attribute value - attr_value". During implementation, the hash value is calculated uniformly in the form of attr_type^attr_value, and then the hash value is written into the bits of the Bloom filter. Moreover, the size of the Bloom filter for each item is usually fixed.

[0068] It should be noted that Bloom filters are typically a probabilistic data structure with extremely high space and query efficiency. They use a bit array to store data at the underlying level and can quickly determine whether an element is in a set. Their core principle is as follows:

[0069] 1) The length of a bit array is usually represented by the parameter m. During initialization, all m bits in the array are set to 0.

[0070] 2) When adding an element, hash the element using k hash functions to generate k hash values, which are the indices of bit arrays. Then set all bits at these positions to 1.

[0071] 3) When querying an element, the input element needs to be hashed using the same k hash functions to obtain k indices, and then the k positions in the bit array are checked. If any position is 0, the element is definitely not in the set; if all positions are 1, the element may exist in the set.

[0072] The embodiments of this disclosure employ a Bloom filter to store the scalar attribute fields of items, i.e., business rule constraint information, constructing a forward index data of "item_id->Bloom filter". The Bloom filter maps multiple elements to a finite bit array using k hash functions. With fixed parameters, the storage space occupied by the bit array is fixed, typically much smaller than the actual space occupied by the element set. Furthermore, it does not increase linearly with the addition of elements, making it suitable for storing a large number of scalar condition fields and exhibiting excellent scalability. In addition, regarding matching efficiency, the time complexity of determining whether a condition field matches is O(k), independent of the number of elements stored in the Bloom filter.

[0073] Alternatively, a standard bitmap can be used for data storage. Each bit represents a specific constraint value. This method is simpler in data structure but is typically only suitable for storing elements with a limited value space. However, in advertising retrieval systems, the constraint types and values ​​for different products in advertising materials are numerous and lack a fixed value space. This makes it difficult to map these constraint values ​​to a bitmap according to a unified rule. If a hash function is introduced to map the constraint values ​​to integers, i.e., their positions in the bitmap, the bitmap effectively becomes equivalent to a Bloom filter.

[0074] Understandably, when using a Bloom filter to store and process the descriptive attribute data of items, in order to further improve recall efficiency, the retrieval attribute data of the item retrieval request can usually be processed accordingly. For example... Figure 3BAs shown, firstly, the executing entity can generate Boolean expressions based on the search conditions in the item search request. That is, it can parse the constraint information carried in the advertisement request and construct Boolean expressions according to the AND / OR relationships between different types of conditions. If each conditional expression contains multiple conditions, the conditions can be related by AND (&&). Each condition may contain multiple attribute values, which can be related by OR (||). Next, for each search condition in the Boolean expression, the hash value of the search condition can be calculated using the hash algorithm of a Bloom filter, obtaining the bit array of the search condition. The bit arrays of each search condition in the Boolean expression are then used as the search attribute data for the item search request.

[0075] like Figure 3B As shown in the example, K hash algorithms can be used to calculate K hash values ​​corresponding to each attribute value (attr_value1-5) in each search condition (cond1, 2, 3). These hash values ​​are then used to construct a corresponding bit array. For example, if K=2, and the calculated hash values ​​are 2 and 8 respectively, then the 2nd and 8th bits in the corresponding bit array are set to 1, and the remaining bits are set to 0.

[0076] Furthermore, when retrieving the recall item set, the first step is to obtain the bit array of descriptive attribute data for each item indicated by the item vector set. This bit array is then matched against the bit array of the search attribute data (i.e., the search conditions in the item search request). Next, items indicated by the bit array of descriptive attribute data that matches the bit array of each search condition in the search attribute data are identified as recall items, thus obtaining the recall item set. In other words, during the engine-layer vector recall process, the item vector tree index is first queried based on the request vector to find the items under the leaf nodes. Then, business constraint filtering is performed based on the scalar conditions of the items (i.e., the Bloom filter forward index). This involves matching the Bloom filter information corresponding to the item against the bit array in the Boolean expression of the search conditions. If any condition does not match, the current item is filtered out, ultimately retaining the valid item set.

[0077] It is understandable that the item vector tree and item description attribute data constructed above may change as items are added, removed, or adjusted on the platform. Therefore, when the executing entity detects changes in item data, such as when the offline indexing system sends a change notification, it can retrieve the data from the Bloom filter containing the latest item description attribute data, as well as the latest item vector tree, and store it in memory. Alternatively, the executing entity can also update and load the index data through periodic full updates. For the item vector indicated by the item vector under each leaf node in the latest item vector tree, the information of the corresponding Bloom filter object can be determined, and this information can be stored under the leaf node to which the item belongs.

[0078] In other words, such as Figure 3C As shown, the execution entity first loads the Bloom filter forward index, which can be stored in memory using a hash map. Then, it loads the item vector tree index file, constructing a tree index storage structure in memory to store the index information. Specifically, when loading item information for leaf nodes, the execution entity can retrieve the pointer to the Bloom filter object corresponding to the current item from the Bloom filter hash map based on the item identifier (e.g., item_id), and store it in memory along with other item information. That is, during item data updates, the execution entity can pre-establish a method for mapping item identifiers to Bloom filter information. This way, during subsequent online retrieval, when an item under a leaf node is found, the corresponding Bloom filter data can be directly retrieved, avoiding performance hotspots caused by excessive hash map queries. This reduces the number of hash map queries and improves overall query and retrieval efficiency.

[0079] In some embodiments, to facilitate data management and updates, after the item vector tree index and Bloom filter positive index are constructed, a predefined data description language (protobuf) can be used to serialize them into a binary index file and write version information. This version information can be determined based on the update time or version number. Here, protobuf, or Protocol Buffers, is typically a data description language, similar to XML, capable of serializing structured data and used for data storage, communication protocols, etc. Figure 2 As shown, the protobuf protocol can be used to enable data interaction between the "offline index building system" and the "online retrieval service".

[0080] In this scenario, the offline index building system can send the latest version information to the executing entity. Alternatively, the executing entity can query the file's version information upon receiving an item retrieval request. If a change in the index file's version information is detected—that is, a difference from the version information stored in memory—then the latest version of the index file can be obtained. Next, the obtained index file can be deserialized using the aforementioned preset data description language to extract the Bloom filter data and item vector tree from the file.

[0081] In some embodiments, such as Figure 2 As shown, for items recalled at the engine layer, the executing entity can also perform business layer filtering. Specifically, it can filter the recalled items in the recalled item set based on the item retrieval request and preset business rules. That is, based on the retrieval conditions in the item retrieval request, the specific business filtering rules at the business layer can be determined for the request. These are the parameters of the business rule filter. Furthermore, the processing results can be used as the retrieval results for the item retrieval request.

[0082] It should be noted that since Bloom filters are typically a probabilistic data storage structure, they can often accurately identify items that do not match the search criteria in the recall stage. However, there is a certain false positive rate for successfully matched items. That is, there is a certain probability that some of the recalled items that pass the filter do not actually meet the search criteria. Therefore, the method disclosed in this paper can adjust the Bloom filter parameters, such as setting the number of attributes (i.e., constraints) to within 50, thereby controlling the false positive rate to within 1% and ensuring that most invalid items are filtered out in the recall stage. At the same time, it can be combined with the rule filters of the back-end business layer to perform supplementary and accurate screening for items that are misidentified by the Bloom filter. Thus, while ensuring that a certain number of items are retrieved, the accuracy of the search results can be further improved.

[0083] Further reference Figure 4 As a response to the above Figures 2 to 3C The present disclosure provides some embodiments of an item retrieval device to implement the method shown. These device embodiments are similar to... Figures 2 to 3C The methods and embodiments shown correspond to those described. This device can be specifically applied to various electronic devices.

[0084] like Figure 4As shown, the item retrieval device 400 in some embodiments may include: a request data generation unit 401, configured to generate a request vector in response to receiving an item retrieval request, and to generate retrieval attribute data according to the retrieval conditions in the item retrieval request, wherein the request vector and the retrieval attribute data represent different data information; a vector tree retrieval unit 402, configured to retrieve, according to the request vector, a set of item vectors under the target leaf node that matches the request vector from a pre-constructed item vector tree, wherein the item vector tree is obtained by clustering the item vectors of the items; and an attribute retrieval unit 403, configured to retrieve a set of recalled items that match the retrieval attribute data from the items indicated by the item vector set according to the item's descriptive attribute data, for filtering processing by the business layer.

[0085] In some embodiments, the request data generation unit 401 may be further configured to generate a Boolean expression based on the search conditions in the item search request; for each search condition in the Boolean expression, calculate the hash value of the search condition according to the hash algorithm of the Bloom filter to obtain the bit array of the search condition; and use the bit array of each search condition in the Boolean expression as the search attribute data of the item search request.

[0086] In some embodiments, the attribute retrieval unit 403 is further configured to, for each item indicated by the item vector set, determine the bit array of the descriptive attribute data of each item according to the Bloom filter; match the bit array of the descriptive attribute data of each item with the bit array of the retrieval attribute data; and determine the item indicated by the bit array of the descriptive attribute data that matches the bit array of each retrieval condition in the retrieval attribute data as the recall item, thereby obtaining the recall item set.

[0087] In some embodiments, the item retrieval device 400 may further include an item data storage unit (not shown in the figure), configured to construct a Bloom filter object corresponding to the item, for storing and retrieving the item's descriptive attribute data; calculate the hash value of the item's descriptive attribute data, and store the calculated hash value in the corresponding Bloom filter to obtain a bit array of the item's descriptive attribute data.

[0088] In some embodiments, the item retrieval device 400 may further include a data standardization unit (not shown in the figure), configured to standardize the attribute information of the item according to a preset standard attribute; and generate descriptive attribute data of the item in a preset format according to the standardized attribute and the corresponding attribute value.

[0089] In some embodiments, the item retrieval device 400 may further include an item data acquisition unit (not shown in the figure), configured to, in response to detecting an item data change, acquire the latest data in the Bloom filter containing the item's descriptive attribute data, and acquire the latest item vector tree for storage in memory; for the item indicated by the item vector under each leaf node in the latest item vector tree, determine the information of the Bloom filter object corresponding to the item, and store the information under the leaf node to which the item belongs.

[0090] In some embodiments, the item data acquisition unit may be further configured to acquire the latest version of the index file in response to detecting a change in the version information of the index file, wherein the index file is obtained by serialization processing using a preset data description language after the item vector tree and Bloom filter have been constructed; the acquired index file is deserialized using the preset data description language to obtain the Bloom filter data and item vector tree in the file.

[0091] In some embodiments, the item data generation unit may also be configured to obtain relevant information about items on the platform and generate item vectors, wherein the relevant information includes at least one of the following: text title information in the item display page, identification information of the manufacturer, and item image; clustering analysis of the item vectors is performed using a clustering algorithm until the clustering termination condition is met to obtain an item vector tree; wherein, for a node in the item vector tree, the average value of the item vectors of each item belonging to that node is used as the node vector of that node, and the item identifier of the item belonging to that leaf node is stored under each leaf node.

[0092] In some embodiments, the item retrieval device 400 may further include a business filtering unit (not shown in the figure), configured to filter recalled items in the recalled item set according to the item retrieval request and preset business rules, and to use the processing result as the retrieval result of the item retrieval request.

[0093] It is understandable that the units described in the item retrieval device 400 are related to the reference. Figures 2 to 3C The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the item retrieval device 400 and the units contained therein, and will not be repeated here.

[0094] The following is for reference. Figure 5 It shows a schematic diagram of the structure of an electronic device 500 suitable for implementing some embodiments of the present disclosure. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0095] like Figure 5 As shown, the electronic device 500 may include a processing unit 501 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0096] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, speakers, vibrators, etc.; storage devices 508 including, for example, memory cards, disks, hard drives, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0097] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 509, or installed from storage device 508, or installed from ROM 502. When the computer program is executed by processing device 501, it performs the functions defined in the methods of some embodiments of this disclosure.

[0098] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0099] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0100] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: generate a request vector in response to receiving an item retrieval request, and generate retrieval attribute data based on the retrieval conditions in the item retrieval request, wherein the request vector and the retrieval attribute data represent different data information; retrieve, based on the request vector, a set of item vectors under the target leaf node matching the request vector from a pre-constructed item vector tree, wherein the item vector tree is obtained by clustering the item vectors of the items; and retrieve a set of recalled items matching the retrieval attribute data from the items indicated by the item vector set, for filtering processing by the business layer, based on the item's descriptive attribute data.

[0101] Furthermore, computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0103] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a request data generation unit, a vector tree retrieval unit, and an attribute retrieval unit. The names of these units do not necessarily limit the specific unit; for example, the request data generation unit may also be described as "a unit that generates retrieval attribute data based on the retrieval conditions in an item retrieval request."

[0104] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0105] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the above-described item retrieval methods.

[0106] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. An item retrieval method, comprising: In response to receiving an item retrieval request, a request vector is generated, and retrieval attribute data is generated based on the retrieval conditions in the item retrieval request, wherein the request vector and the retrieval attribute data represent different data information; Based on the request vector, retrieve the set of item vectors under the target leaf node that matches the request vector from the pre-constructed item vector tree, wherein the item vector tree is obtained by clustering the item vectors of the items; Based on the item description attribute data, a set of recall items matching the retrieval attribute data is retrieved from the items indicated by the item vector set, for filtering processing by the business layer.

2. The item retrieval method according to claim 1, wherein, The step of generating search attribute data based on the search criteria in the item search request includes: Generate a Boolean expression based on the search criteria in the item search request; For each search condition in the Boolean expression, the hash value of the search condition is calculated according to the hash algorithm of the Bloom filter, and the bit array of the search condition is obtained. The bit arrays of each search condition in the Boolean expression are used as the search attribute data for the item search request.

3. The item retrieval method according to claim 2, wherein, The step of retrieving a set of recall items that match the retrieval attribute data from the items indicated by the item vector set based on the item's descriptive attribute data includes: For each item indicated by the item vector set, a bit array of descriptive attribute data for each item is determined according to the Bloom filter; Match the bit array of the description attribute data of each item with the bit array of the search attribute data; The items indicated by the bit array of the description attribute data that matches the bit array of each search condition in the search attribute data are identified as the recalled items, thus obtaining the recalled item set.

4. The item retrieval method according to claim 1, wherein, The method further includes: Construct a Bloom filter object corresponding to the item to store and retrieve the item's descriptive attribute data; Calculate the hash value of the item's description attribute data, store the calculated hash value in the corresponding Bloom filter, and obtain the bit array of the item's description attribute data.

5. The item retrieval method according to claim 1, wherein, The method further includes: The attribute information of the items is standardized according to the preset standard attributes. Based on the standardized attributes and their corresponding attribute values, generate description attribute data for items in a preset format.

6. The item retrieval method according to claim 4, wherein, The method further includes: In response to the detection of changes in item data, the latest data in the Bloom filter containing the item's descriptive attribute data is retrieved, and the latest item vector tree is retrieved and stored in memory; For the item vector in the latest item vector tree, determine the information of the Bloom filter object corresponding to the item under each leaf node, and store the information under the leaf node to which the item belongs.

7. The item retrieval method according to claim 6, wherein, The step of responding to the detection of a change in item data by obtaining the latest data from the Bloom filter containing the item's descriptive attribute data, and obtaining the latest item vector tree, includes: In response to the detection of a version information change in the index file, the latest version of the index file is obtained, wherein the index file is obtained by serialization processing using a preset data description language after the item vector tree and Bloom filter have been constructed; The obtained index file is deserialized using the preset data description language to obtain Bloom filter data and item vector trees from the file.

8. The item retrieval method according to claim 1, wherein, The item vector tree is constructed using the following method: Obtain relevant information about items on the platform and generate item vectors, wherein the relevant information includes at least one of the following: text title information on the item display page, identification information of the manufacturer, and item image; Clustering algorithms are used to perform clustering analysis on the item vectors until the clustering termination condition is met, resulting in an item vector tree. Specifically, for each node in the item vector tree, the average value of the item vectors of all items belonging to that node is used as the node vector of that node, and the item identifier of the item belonging to that leaf node is stored under each leaf node.

9. The item retrieval method according to any one of claims 1-8, wherein, The method further includes: Based on the item retrieval request and preset business rules, the recalled items in the recalled item set are filtered, and the processing results are used as the retrieval results of the item retrieval request.

10. An item retrieval device, comprising: The request data generation unit is configured to generate a request vector in response to receiving an item retrieval request, and to generate retrieval attribute data based on the retrieval conditions in the item retrieval request, wherein the request vector and the retrieval attribute data represent different data information; The vector tree retrieval unit is configured to retrieve, based on the request vector, a set of item vectors under the target leaf node that matches the request vector from a pre-constructed item vector tree, wherein the item vector tree is obtained by clustering the item vectors of the items; The attribute retrieval unit is configured to retrieve a set of recall items that match the retrieval attribute data from the items indicated by the item vector set, based on the item's descriptive attribute data, for filtering by the business layer.

11. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the item retrieval method as described in any one of claims 1-9.

12. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the item retrieval method as described in any one of claims 1-9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the item retrieval method as described in any one of claims 1-9.

Images