Commodity search method and its device, equipment and medium

Abstract

The present application relates to a commodity search method and its device, equipment, and medium in the field of computer technology, the method comprising: responding to a user search request, obtaining the multi-dimensional commodity information of the commodity corresponding to the search text pointed to by the request and the historical access behavior of the user, and constructing a historical commodity data sequence; using the user branch in the double tower model to obtain the user feature vector generated by the interaction of the features corresponding to the search text and the historical commodity data sequence; using the commodity branch in the double tower model to obtain the commodity feature vector generated by the interaction of the features corresponding to the multi-dimensional commodity information of the commodity in the commodity database; according to the similarity between the user feature vector and the commodity feature vector of each commodity in the commodity database, recall the commodities with higher similarity and construct them into a commodity set. The present application can accurately represent the correlation between the commodity and the search keyword input by the user, so that more relevant commodities can be searched.

Description

Technical Field

[0001] The present application relates to the field of e-commerce technology, and in particular to a commodity search method and its corresponding device, computer equipment, and computer-readable storage medium. Background Art

[0002] At present, users of e-commerce service platforms can search for the desired products by inputting search text. Usually, the background of the platform uses a relevance control module to filter the products retrieved according to the search text, and only retain those products that can completely match the search text. In the subsequent sorting stage, the sorted completely matched products are displayed to users, while a large number of products with low relevance are filtered out. However, in reality, the relevance of these products is not low, but the relevance of these products cannot be reflected under the condition of completely matching the search text. Therefore, the filtered products not only consume a lot of computing resources, but also cause the products that could have entered the subsequent sorting to be unable to be sorted, reducing the exposure rate.

[0003] Therefore, how to accurately represent the correlation between products and search keywords entered by users so that more relevant products can be displayed to users is an urgent problem to be solved. Summary of the invention

[0004] The primary purpose of the present application is to solve at least one of the above problems and to provide a product search method and its corresponding device, computer equipment, and computer-readable storage medium.

[0005] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0006] A product search method provided for one of the purposes of this application comprises the following steps:

[0007] In response to a user search request, obtain multi-dimensional product information of a product corresponding to the search text pointed to by the request and the user's historical access behavior, and construct a historical product data sequence, wherein the product information of the product includes a product identification code, a product title, and a product image;

[0008] A user feature vector is generated by interactively obtaining the features corresponding to the search text and the historical product data sequence using the user branch in the double tower model;

[0009] The commodity branch in the double tower model is used to obtain the corresponding features of the multi-dimensional commodity information of the commodities in the commodity database to interactively generate a commodity feature vector;

[0010] According to the similarity between the user feature vector and the product feature vector of each product in the product database, products with higher similarity are recalled to form a product set.

[0011] On the other hand, a product search device provided to meet one of the purposes of the present application includes a request response module, a user branch module, a product branch module and a product recall module, wherein: the request response module is used to respond to user search requests, obtain the multi-dimensional product information of the product corresponding to the search text pointed to by the request and the historical access behavior of the user, and construct a historical product data sequence, wherein the product information of the product includes a product identification code, a product title and a product picture; the user branch module is used to use the user branch in the double tower model to obtain the user feature vector generated by interactively obtaining the features corresponding to the search text and the historical product data sequence; the product branch module is used to use the product branch in the double tower model to obtain the product feature vector generated by interactively obtaining the features corresponding to the multi-dimensional product information of the products in the product database; the product recall module is used to recall the products with higher similarity according to the similarity between the user feature vector and the product feature vector of each product in the product database, and construct them into a product set.

[0012] On the other hand, a computer device provided to meet one of the purposes of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the product search method described in the present application.

[0013] On the other hand, a computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the product search method in the form of computer-readable instructions, and when the computer program is called and executed by a computer, it executes the steps included in the method.

[0014] The technical solution of this application has many advantages, including but not limited to the following aspects:

[0015] This application uses the user branch in the twin tower model to interact with the search text currently input by the user and the features corresponding to the historical product sequence constructed by the multi-dimensional product information of the product corresponding to the historical access behavior of the user, and obtains the corresponding user feature vector. The product branch in the twin tower model is used to interact with the features corresponding to the multi-dimensional product information of the product, and obtains the corresponding product feature vector, and then recalls the products with high similarity according to the similarity between the user feature vector and the product feature vector of each product in the product database, and constructs a product set. It can be seen that in the user branch in the twin tower model, it can ensure that the historical access behavior related to the search text is captured, and the semantic / intention representation of the user under the current search is enriched, so that the obtained user feature vector can fully semantically represent the search text, which helps to match more products related to the user search. In addition, in the product branch in the twin tower model, it can ensure that the product information related to the product is captured, so that the obtained product feature vector can fully and accurately semantically represent the product, which helps to ensure the accuracy of the similarity between the user feature vector and the product feature vector, and accurately recall products with high similarity. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0017] Figure 1 A flowchart of a typical embodiment of the commodity search method of the present application;

[0018] Figure 2 A schematic diagram of a flow chart of outputting a user feature vector for a user branch in a dual-tower model in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of a process of obtaining a user feature vector by performing multiple feature interactions based on personalized behavior features and comprehensive feature information in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of a flow chart of outputting a commodity feature vector for a commodity branch in a double tower model in an embodiment of the present application;

[0021] Figure 5 A schematic diagram of a process for obtaining a logo feature vector, a text feature vector, and a picture feature vector in an embodiment of the present application;

[0022] Figure 6 A schematic diagram of a process of obtaining an associated feature vector based on feature interaction between a text feature vector and an image feature vector in an embodiment of the present application;

[0023] Figure 7A schematic diagram of a process for constructing a product set in an embodiment of the present application;

[0024] Figure 8 This is a functional block diagram of the product search device of this application;

[0025] Fig. 9 A schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0026] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0027] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0028] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0029] It will be understood by those skilled in the art that the "client", "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication system), which can combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant, personal digital assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System, global positioning system) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable for and / or configured to run locally, and / or in a distributed form, at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playing terminal, for example, a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.

[0030] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0031] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0032] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.

[0033] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0034] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0035] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0036] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0037] A product search method of the present application can be programmed as a computer program product and deployed in a client or server for execution. For example, in the exemplary application scenario of the present application, it can be deployed and implemented in a server of an e-commerce platform, thereby accessing an interface opened after the computer program product is running and performing human-computer interaction with the process of the computer program product through a graphical user interface to execute the method.

[0038] See also Figure 1 The commodity search method of the present application, in its typical embodiment, comprises the following steps:

[0039] Step S1100: respond to a user search request, obtain multi-dimensional product information of a product corresponding to the search text pointed to by the request and the user's historical access behavior, and construct a historical product data sequence, wherein the product information of the product includes a product identification code, a product title, and a product picture;

[0040] Generally speaking, in an e-commerce platform, a user's access behaviors such as clicking, purchasing, and collecting products indicate to a certain extent that the product corresponding to the access behavior is related to the user, for example, the product is a daily consumable for the user, the product is a favorite of the user, the product is a gift from the user, etc. Therefore, the e-commerce platform can maintain a historical behavior database to store the historical access behaviors of each user on the platform and the product identification code of the corresponding product. The user's historical access behaviors include any one or more behaviors of clicking, purchasing, and collecting. The product identification code is to set a unique identification code for each product to facilitate the distinction between the products.

[0041] Generally speaking, an e-commerce platform can maintain a product database to store product information of products listed by online stores on the platform. The product information includes product titles, product images, and product identification codes, so that the platform can list and display corresponding products according to the product information. The product title is the text entered as the title by the merchant user of the online store of the e-commerce platform when listing the product. The product image is the image uploaded by the merchant user of the online store of the e-commerce platform for display when listing the product, including the product header image, product detail image, etc. In subsequent calls, one or more product images can be called according to the specific situation.

[0042] Users of the e-commerce platform can operate on the product search page displayed on their client and enter search text. The search text can be entered as keywords or complete text related to the product, such as product attributes, product names, product functions, etc. Then, they touch the relevant submission control on the page to trigger the generation of a user search request on the client. Then, the client submits the user search request containing the search text entered by the user to the server of the e-commerce service platform to obtain products related to the search text.

[0043] The server receives the user search request, responds to it, and obtains the search text input by the user pointed to by the request. In addition, with the timestamp corresponding to the currently received user search request, multiple time dimensions can be divided into different time lengths from the timestamp, thereby the server obtains multi-dimensional commodity information of commodities corresponding to the historical access behaviors of the user corresponding to various time dimensions, and constructs historical commodity data sequences corresponding to various time dimensions, wherein the multiple time dimensions can be short-term and long-term, wherein the short-term can be ten days from the timestamp, and the long-term can be one month from the timestamp.

[0044] Specifically, when the time dimension is short-term, the user's historical access behavior in the short term is obtained from the historical behavior database, which is to click on the corresponding product identification code of one or more products. According to the product identification code, the product title and product image of the corresponding product are obtained from the product database. The product image can only take one image that best represents the product, such as the product header image. For each product clicked by the user, the encoding layer is used to encode its product title, product image, and product identification code respectively, and the encoded feature information corresponding to the encoded product title, product image, and product identification code is spliced ​​to obtain the encoded representation of the product. Further, the encoded representation of each product is used as a sequence element to construct a short-term historical product data sequence.

[0045] When the time dimension is long-term, the product identification code of one or more products corresponding to the user's historical access behavior in the long term is obtained from the historical behavior database, and the product title and product image of the corresponding product are obtained from the product database according to the product identification code. The product image can only take one image that best represents the product, such as the product header image. For each product clicked by the user, the encoding layer is used to encode its product title, product image, and product identification code respectively, and the encoded feature information corresponding to the encoded product title, product image, and product identification code is spliced ​​to obtain the encoded representation of the product. Further, the encoded representation of each product is used as a sequence element to construct a long-term historical product data sequence in which the user's historical access behavior is clicks. Similarly, referring to the above, a long-term historical product data sequence corresponding to the user's historical access behavior of purchase and collection is constructed.

[0046] Step S1200: using the user branch in the double tower model to obtain a user feature vector generated by interacting between the search text and the features corresponding to the historical product data sequence;

[0047] The search text is segmented using a variety of segmentation granularities, including single-word granularity, multi-word granularity (number of words greater than 1 and less than or equal to 3), and segmentation granularity. The segmentation implementation corresponding to various segmentation granularities can adopt any one of the N-Gram segmentation algorithm, the Jieba segmentation algorithm, etc., so as to obtain a single-word granularity sequence, a multi-word granularity sequence, and a segmentation granularity sequence accordingly. The coding layer is used to perform corresponding encoding on each segmentation sequence to obtain the corresponding encoding feature information of each segmentation sequence. Further, the encoding feature information corresponding to each segmentation sequence is respectively average-pooled to extract the sequence feature vector of each segmentation sequence. The encoding feature information is to map each segmentation sequence to the encoding word table of the encoding layer to obtain the corresponding mapping information. The mapping information is specifically expressed as a dense matrix. The encoding word table is obtained by learning the feature representation of the encoding layer. The specific implementation can be realized by those skilled in the art as needed according to the disclosure herein.

[0048] The encoder module of the Transformer model applies a self-attention layer to the encoded feature information of the word segmentation granularity sequence for feature interaction, and then performs mean pooling on the hidden layer vectors in the last layer of the model, thereby extracting the deep semantic information of the word segmentation granularity sequence and obtaining the deep semantic vector output by the model.

[0049] From the historical search records of the user who sent the user search request, obtain the historical search text corresponding to the historical search behavior of the user, segment the historical search text using the segmentation granularity, obtain the corresponding segmentation sequence, further, use the encoding layer to encode the segmentation sequence, and extract the corresponding encoding feature information. The historical search record can be stored by the e-commerce platform for the search text entered by each user on the platform for each search behavior, so as to be called up.

[0050] The sequence feature vector of the word segmentation granularity sequence is used as the query vector, and the encoded feature information is used as the key vector and the value vector. The key feature information in the encoded feature information is queried through the attention layer. The key feature information represents the feature information that is most relevant to the search text input by the user's historical search and the search text input by the current user search, and is specifically expressed as a vector.

[0051] The sequence feature vectors, deep semantic vectors, and key feature information corresponding to the single-word granularity sequence, multi-word granularity sequence, and word segmentation granularity sequence are added element by element to obtain a mixed feature vector. Further, each sequence feature vector, deep semantic vector, key feature information, and mixed feature vector are vertically spliced ​​to construct comprehensive feature information.

[0052] A multi-head self-attention layer is used to extract the deep semantic information corresponding to the short-term historical commodity data sequence, and a vector of all zeros is added to the deep semantic information to obtain the deep semantic information with zeros added as a short-term personalized behavior feature. In addition, the long-term historical commodity data sequences corresponding to the user's historical access behaviors of click, purchase and collection are respectively averaged and pooled to obtain the deep semantic information corresponding to each long-term historical commodity data sequence for vertical splicing, and a vector of all zeros is added to the spliced ​​deep semantic information to obtain the deep semantic information with zeros added as a long-term personalized behavior feature. Further, an attention layer is applied to interact the short-term personalized behavior features and the long-term personalized behavior features with the comprehensive feature information respectively, to obtain the feature information that is most relevant to the short-term personalized behavior features and the long-term personalized behavior features and the comprehensive feature information respectively, and the feature information corresponding to the short-term personalized behavior features and the long-term personalized behavior features are vertically spliced ​​with the comprehensive feature information and input into the multi-head self-attention layer for feature interaction, and the output of the multi-head self-attention layer is obtained as a user feature vector.

[0053] Step S1300: using the commodity branch in the double tower model to obtain the commodity feature vector interactively generated by the features corresponding to the multi-dimensional commodity information of the commodity in the commodity database;

[0054] Obtain the product identification code, product title, and product image of the stored product from the product database, extract the features corresponding to the product identification code, product title, and product image, and obtain the identification feature vector, text feature vector, and image feature vector accordingly. Further, use the text feature vector as the query vector, the image feature vector as the key vector and the value vector, apply the attention layer to perform feature interaction, obtain the output of the attention layer and add it to the identification feature vector as the product feature vector, and the output of the attention layer represents the most relevant feature information of the image feature vector and the text feature vector.

[0055] Step S1400: According to the similarity between the user feature vector and the product feature vector of each product in the product database, products with high similarity are recalled to form a product set.

[0056] In one embodiment, a mapping relationship is established between the user feature vector and the product feature vector of each product in the product database to construct a vector set and store it in the faiss library. Further, after the vector set is preprocessed and packaged into an index file (index file) for storage, the user feature vector can be used as the input of faiss. According to the similarity between the user feature vector and the product feature vector of each product, the products with higher similarity in the product database are recalled, thereby constructing a product set with the products with higher similarity. The faiss (Facebook AlSimilarity Search) can provide an efficient and reliable similarity clustering and retrieval method for massive data (dense vectors) in high-dimensional space, and can support the search of billions of vectors. It is an approximate neighbor search library.

[0057] In another embodiment, any data distance algorithm may be used, including but not limited to any one of a cosine similarity algorithm, an Euclidean distance algorithm, a Pearson correlation coefficient algorithm, a Jaccard coefficient algorithm, etc., to calculate the similarity between the user feature vector and the product feature vector of each product in the product database, and then sort the corresponding products in order of high to low similarity, and recall the products with higher similarity in the top-ranked set to construct a product set.

[0058] Furthermore, in response to the user search request, the product set is pushed to the client that sends the user search request and displayed on its related product page.

[0059] It can be known from the typical embodiments of the present application that the technical solution of the present application has many advantages, including but not limited to the following aspects:

[0060] In this application, the user branch in the two-tower model is used to interact with the features corresponding to the historical product sequence constructed from the multi-dimensional product information of the products corresponding to the user's current input search text and the user's historical access behavior, so as to obtain the corresponding user feature vector. The product branch in the two-tower model is used to interact with the features corresponding to the multi-dimensional product information of the products to obtain the corresponding product feature vector. Then, according to the similarity between the user feature vector and the product feature vectors of each product in the product database, products with higher similarity are recalled and constructed into a product set. It can be seen that in the user branch of the two-tower model, it can ensure capturing the historical access behavior related to the search text, enriching the semantic / intention representation of the user under the current search, so that the obtained user feature vector can fully perform semantic representation on the search text, which helps to match more products related to the user's search. In addition, in the product branch of the two-tower model, it can ensure capturing the product information related to the products, so that the obtained product feature vector can fully and accurately perform semantic representation on the products, which helps to ensure the accuracy of the similarity between the user feature vector and the product feature vector, and accurately recall products with higher similarity.

[0061] Please refer to Figure 2 , in a further embodiment, in the step S1200 of using the user branch in the two-tower model to obtain the user feature vector generated by interacting with the features corresponding to the search text and the historical product data sequence, the following steps are included:

[0062] Step S1210: Segment the search text using multiple word segmentation granularities, respectively obtain the segmented sequences corresponding to different granularities, and extract the sequence feature vectors of each segmented sequence;

[0063] The search text is segmented using multiple word segmentation granularities, and the multiple word segmentation granularities include single-character granularity, multi-character granularity (the number of characters is greater than 1 and less than or equal to 3), and word segmentation granularity, so as to correspondingly obtain the single-character granularity sequence, multi-character granularity sequence, and word segmentation granularity sequence.

[0064] Using the N-Gram word segmentation algorithm, set the sliding window N for taking words as 1 and the step size as 1, segment the search text at the single-character granularity, and gradually slide and intercept each single character from the search text to construct the single-character granularity sequence.

[0065] Using the N-Gram word segmentation algorithm, since the length of the usually segmented result is less than 3, the sliding window N (1 < N <= 3) can be set, for example, N is 2 and the step size is 1, segment the search text at the multi-character granularity, and gradually slide and intercept each text composed of two single characters from the search text to construct the multi-character granularity sequence.

[0066] The search engine mode using the jieba word segmentation algorithm performs word segmentation on the search text to obtain corresponding word segments. Those skilled in the art can understand that the precise mode of jieba word segmentation is to implement the most accurate segmentation of text information, segmenting the corresponding word segments, and can achieve the absence of redundant data in the word segments. However, based on the precise mode, the search engine mode segments the long words in the word segments again, so that the granularity of the word segmentation text is finer.

[0067] The coding layer is used to encode each word segmentation sequence accordingly to obtain the coding feature information corresponding to each word segmentation sequence. Further, the coding feature information corresponding to each word segmentation sequence is respectively average pooled to extract the sequence feature vector of each word segmentation sequence. The coding feature information is obtained by mapping each word segmentation sequence to the coding word table of the coding layer, and the mapping information is specifically expressed as a dense matrix. The coding word table is obtained by learning the feature representation of the coding layer. The specific implementation can be realized by those skilled in the art as needed according to the disclosure herein.

[0068] Step S1220: extracting deep semantic information of the word segmentation sequence obtained by performing word segmentation at the word segmentation granularity based on the encoded feature information, and obtaining a corresponding deep semantic vector;

[0069] The encoder module of the Transformer model is used to use the encoded feature information of the word segmentation sequence obtained by word segmentation at the word segmentation granularity as the query vector, key vector, and value vector respectively, and the self-attention layer is applied to perform feature interaction to determine the correlation between each word segmentation and other word segmentations in the encoded feature information, so as to deeply dig out the corresponding key semantics in combination with the context semantics, and then mean pooling is performed on the hidden vectors in the last layer of the model, thereby extracting the deep semantic information of the word segmentation granularity sequence and obtaining the deep semantic vector output by the model.

[0070] Step S1230: Segment the historical search text corresponding to the historical search behavior of the user to obtain a corresponding segmentation sequence, and extract the encoding feature information corresponding to the segmentation sequence;

[0071] From the historical search records of the user who sent the user search request, obtain the historical search text corresponding to the historical search behavior of the user, segment the historical search text using the segmentation granularity, obtain the corresponding segmentation sequence, further, use the encoding layer to encode the segmentation sequence, and extract the corresponding encoding feature information. The historical search record can be stored by the e-commerce platform for the search text entered by each user on the platform for each search behavior, so as to be called up.

[0072] Step S1240: using the sequence feature vector of the word segmentation sequence obtained by segmenting the search text at the word segmentation granularity as the query vector, using the encoded feature information of the word segmentation sequence obtained by segmenting the historical search text as the key vector and the value vector, and querying the key feature information in the encoded feature information through the attention layer;

[0073] Specifically, the query vector Q and the key vector K can be matched with their respective learnable weights W Q , W K After that, the dot product operation is performed to realize feature interaction, and the corresponding learnable weight W is matched from the key vector K with the semantics of the current search text as a reference. K The key vector in the historical search text is determined from the obtained results. The key vector contains key feature information that characterizes the correlation between the historical search text and the current search text. The key vector can be normalized using the Softmax function, and the feature values ​​therein are mapped to the confidence interval of [0,1] to obtain a weight vector that is used to characterize the weight of the correlation degree enjoyed by each corresponding feature in the encoded feature information. In order to extract the key feature information from the encoded feature information, the weight vector can be further compared with the value vector V (as mentioned above, the corresponding learnable weight W can be matched in advance as needed). V ), that is, the coded feature information is multiplied to realize the weighted summation of the feature values ​​in the coded feature information, thereby obtaining the final key feature information, wherein each feature value is adjusted under the action of the weight vector, thereby realizing the mining and representation of the most relevant key feature information between the historical search text and the current search text in the coded feature information.

[0074] Step S1250, integrating the sequence feature vector, deep semantic vector and key feature information to construct comprehensive feature information;

[0075] The sequence feature vectors, deep semantic vectors, and key feature information corresponding to the single-word granularity sequence, multi-word granularity sequence, and word segmentation granularity sequence are added element by element to obtain a mixed feature vector. Further, each sequence feature vector, deep semantic vector, key feature information, and mixed feature vector are vertically spliced ​​to construct comprehensive feature information.

[0076] Step S1260: extract the personalized behavior features corresponding to the historical product data sequence, apply at least one attention layer to interact the personalized behavior features with the comprehensive feature information, and obtain the user feature vector.

[0077] A multi-head self-attention layer is used to extract the deep semantic information corresponding to the short-term historical commodity data sequence, and a vector of all zeros is added to the deep semantic information to obtain the deep semantic information with zeros added as a short-term personalized behavior feature. In addition, the long-term historical commodity data sequences corresponding to the user's historical access behaviors of click, purchase and collection are average pooled to obtain the deep semantic information corresponding to each long-term historical commodity data sequence, and a vector of all zeros is added to the added deep semantic information to obtain the deep semantic information with zeros added as a long-term personalized behavior feature. Further, an attention layer is applied to interact the short-term personalized behavior features and the long-term personalized behavior features with the comprehensive feature information, respectively, to obtain the feature information that is most relevant to the short-term personalized behavior features and the long-term personalized behavior features and the comprehensive feature information, and the feature information corresponding to the short-term personalized behavior features and the long-term personalized behavior features are spliced ​​with the comprehensive feature information and input into the multi-head self-attention layer for feature interaction, and the output of the multi-head self-attention layer is obtained as a user feature vector.

[0078] In this embodiment, the search text is organized literally in various ways, including single-word granularity, multi-word granularity, and word segmentation granularity. The search text is represented in various ways, including mean pooling, transformer, attention mechanism, vertical splicing, etc., so that the obtained user feature vector can fully semantically represent the search text, which is helpful for subsequent matching of more products related to the user's search.

[0079] See also Figure 3 In a further embodiment, step S1260, extracting personalized behavior features corresponding to the historical product data sequence, applying at least one attention layer to interact the personalized behavior features with the comprehensive feature information, and obtaining the user feature vector, comprises the following steps:

[0080] Step S1261: extracting personalized behavior features corresponding to the historical commodity data sequence, applying an attention layer to interact the personalized behavior features with the comprehensive feature information, and obtaining associated behavior features;

[0081] A multi-head self-attention layer is used to extract the deep semantic information corresponding to the short-term historical commodity data sequence, and a vector of all zeros is added to the deep semantic information to obtain the deep semantic information with zeros added as a short-term personalized behavior feature. In addition, the long-term historical commodity data sequences corresponding to the user's historical access behaviors of clicks, purchases, and favorites are mean-pooled, and the deep semantic information corresponding to each long-term historical commodity data sequence is obtained for vertical splicing, and a vector of all zeros is added to the spliced ​​deep semantic information to obtain the deep semantic information with zeros added as a long-term personalized behavior feature.

[0082] Further, the comprehensive feature information is used as a query vector, and the short-term individual behavior features are used as key vectors and value vectors, and the associated behavior features in the short-term individual behavior features are queried through the attention layer, which characterize the short-term user's historical access behavior related to the search text input by the current user. In addition, the comprehensive feature information is used as a query vector, and the long-term individual behavior features are used as key vectors and value vectors, and the associated behavior features in the long-term individual behavior features are queried through the attention layer, which characterize the long-term user's historical access behavior related to the search text input by the current user.

[0083] Step S1262: splice the associated behavior features and the comprehensive feature information, apply a multi-head self-attention layer to perform feature interaction on the spliced ​​features, and obtain the user feature vector.

[0084] The [CLS] identifier is added to the first position, and the short-term association behavior features, long-term association behavior features and comprehensive feature information are concatenated to form input. The concatenated features are used as query vectors, key vectors, and value vectors. The multi-head sub-attention layer is applied for feature interaction, and the output of the multi-head self-attention layer is obtained as the user feature vector. The [CLS] identifier imitates the structure in BERT, a learnable vector, and condenses information.

[0085] In this embodiment, the features corresponding to the user's historical access behavior are interacted with the features corresponding to the search text currently input by the user through multiple attention layers, so as to ensure that the historical access behavior related to the search text is captured and the semantic / intent representation of the user under the current search is enriched. In addition, for the deep semantic information corresponding to the short-term and long-term historical commodity data sequences, a vector of all zeros is added to eliminate potential noise and solve the situation where the user's historical behavior and the current search may be completely unrelated.

[0086] See also Figure 4 In a further embodiment, step S1300, the step of interactively generating a product feature vector by using the product branch in the double tower model to obtain the features corresponding to the multi-dimensional product information of the product in the product database, includes the following steps:

[0087] Step S1310: extract features corresponding to the product identification code, product title, and product image in the product information of the product using the product branch in the double tower model, and obtain an identification feature vector, a text feature vector, and a picture feature vector accordingly;

[0088] The commodity identification code is encoded by applying a coding layer to obtain corresponding coding information as a title feature vector.

[0089] The product image is divided into multiple sub-images of equal size to form a sub-image sequence, and an image encoder is applied to extract deep semantic features from each sub-image in the sub-image sequence to obtain an image feature vector composed of feature vectors corresponding to each sub-image.

[0090] The encoding layer is applied to encode the product title, the encoding information obtained by the encoding is pooled, and the pooling result is input into the multi-layer perception layer to extract the corresponding deep semantic information to obtain a text feature vector.

[0091] Step S1320: Apply an attention layer to interact with the text feature vector and the picture feature vector, and query the associated feature vector corresponding to the text feature vector from the picture feature vector;

[0092] The text feature vector is used as the query vector, and the image feature vector is used as the key vector and the value vector. The attention layer is applied to interact. The text features representing the product corresponding to the text feature vector are referred to, and the associated feature vector corresponding to the text feature vector is queried from the image feature vector to ensure that the corresponding features in the product image related to the features corresponding to the product title are captured.

[0093] Step S1330: Add the associated feature vector and the identification feature vector to obtain a product feature vector.

[0094] The association feature vector and the identification feature vector are added element by element to obtain a product feature vector.

[0095] In this embodiment, the multi-dimensional product information of the product is respectively vectorized, and then the attention layer is used to perform feature interaction on the vectorized representations corresponding to the product image and the product title respectively, and the associated feature vector obtained by the feature interaction is combined with the vectorized representation of the product identification code to construct a product feature vector. It can be seen that the product feature vector is obtained by integrating the multi-dimensional product information, which can fully semantically represent the product, and help to ensure the accuracy of the similarity between the subsequent user feature vector and the product feature vector.

[0096] See also Figure 5 In a further embodiment, step S1310, using the product branch in the double tower model to extract features corresponding to the product identification code, product title, and product image in the product information of the product, and correspondingly obtaining the identification feature vector, text feature vector, and image feature vector, includes the following steps:

[0097] Step S1311, applying a coding layer to encode the product identification code, and obtaining corresponding coding information as a title feature vector;

[0098] Obtain the product identification code, product title, and product image of the stored product from the product database. It can be understood that the product identification code is unique, and the corresponding product identification code is different for different products, that is, the product identification code can be used as a representation to distinguish each product from other products. Based on this, the product identification code can provide certain support for the feature representation of the corresponding product, and the coding layer is used to encode the product identification code of the product, and the coding information of the vectorized representation of the product identification code is obtained as the title feature vector. The feature representation of the corresponding product can be constructed based on the title feature vector.

[0099] Step S1312: divide the product image into multiple sub-images of equal size to form a sub-image sequence;

[0100] In one embodiment, Vision Transformer is used as an image encoder to encode the product image of the product. According to the principle of Vision Transformer, the product image is divided into multiple sub-images of equal specifications according to a preset size, thereby obtaining a sub-image sequence.

[0101] Step S1313: applying a picture encoder to extract deep semantic features from each sub-picture in the sub-picture sequence, and obtaining a picture feature vector composed of feature vectors corresponding to each sub-picture;

[0102] The sub-image sequence is input into the image encoder for feature interaction. Vision Transformer is an encoding component based on the self-attention layer. It extracts the feature vectors corresponding to each sub-image by performing deep feature interaction on each sub-image, and concatenates these feature vectors to construct an image feature vector, thereby realizing the encoding of the product image.

[0103] Step S1314: applying the encoding layer to encode the product title, pooling the encoded information obtained by the encoding, and inputting the pooling result into the multi-layer perception layer to extract the corresponding deep semantic information to obtain a text feature vector;

[0104] The decoding layer can carry a corresponding encoding word list, which can be used to encode text information. Therefore, on the basis of obtaining each word unit in the product title by word segmentation, the word list can be used to query the word vector of each word unit in the product title, and these word vectors are concatenated to construct the encoding information of the product title, so as to realize the preliminary feature representation of the product title. It can be seen that the product title is usually composed of keywords stacked and lacks grammatical structure. The semantic information of each keyword in the literal sense is prominent enough, and the context signal is weak, so no complex model is needed to capture the semantics. Based on this, the encoding information can be further averaged and pooled, and the obtained pooling result is input into the multi-layer perception layer (also called ANN, also known as artificial neural network) to extract the corresponding deep semantic information and obtain the text feature vector output by the multi-layer perception layer.

[0105] In this embodiment, the corresponding feature extraction is performed on the multi-dimensional product information of the product, namely the product identification code, product title, and product image, to obtain the corresponding feature vector. Subsequently, the feature representation of the corresponding product, namely the product feature vector, can be constructed through the feature vector corresponding to the multi-dimensional product information, so that the semantic features contained in the enriched product feature vector can accurately represent the product, which helps to ensure the accuracy of the similarity between the subsequent user feature vector and the product feature vector. In addition, the image encoder extracts features from the product image based on the local information corresponding to each sub-image, and can obtain the image semantic features of the product image at a finer granularity to ensure the accuracy of feature extraction.

[0106] See also Figure 6 In a further embodiment, step S1320, the step of applying the attention layer to interact with the text feature vector and the picture feature vector, and querying the associated feature vector corresponding to the text feature vector from the picture feature vector, includes the following steps:

[0107] Step S1321: Use the text feature vector as a query vector, use the image feature vector as a key vector and a value vector, and interact the query vector with the key vector to determine a key vector of the product image;

[0108] Since the key vector is an image feature vector, which is a feature representation of the product image, and the query vector is a text feature vector, which contains the deep semantic information of the corresponding product title, in the embodiment of using the multi-head attention mechanism in the attention layer, the query vector Q and the key vector K can be matched with their respective learnable weights W Q , W K After that, the dot product operation is performed to realize feature interaction, and the corresponding learnable weight W is matched from the key vector K with the text semantics as a reference. K The key vectors in the product images are determined from the obtained results.

[0109] For embodiments where the attention layer uses a single-head self-attention mechanism, the query vector Q does not need to match the learnable weights.

[0110] Step S1322, normalizing the key vector to obtain a weight vector;

[0111] After obtaining the key vector, the Softmax function can be used to normalize it, and the feature values ​​therein can be mapped to a confidence interval of [0,1] to obtain a weight vector for representing the weight of the criticality of each corresponding feature in the image feature vector.

[0112] Step S1323: Superimpose the weight vector on the value vector to obtain an associated feature vector.

[0113] In order to extract key features from the image feature vector, the weight vector and the value vector V (as mentioned above, the corresponding learnable weight W can be matched in advance as needed) can be further V ), that is, multiplying the image feature vectors, to achieve weighted summation of the eigenvalues ​​in the image feature vectors, thereby obtaining the final key feature sequence, in which each feature value is adjusted under the action of the weight vector, thereby realizing the mining and representation of the key features in the product image.

[0114] In some embodiments, whether based on a single-head attention mechanism or a multi-head attention mechanism, the value vector V can directly reuse the key vector K to reduce the weight parameters that need to be learned, so as to accelerate the convergence speed of the model during the training process. It is not difficult to understand that the various learnable weights are back-propagated and modified during the training process of the image encoder, and are solidified after the image encoder is trained to convergence.

[0115] In this embodiment, the attention layer performs self-attention operation on the image feature vector by referring to the text feature vector, so as to achieve deep mining of key features in the product image and obtain key vectors, so that the key vectors can effectively represent the feature information in the product image that is strongly related to the product title, thereby more accurately representing the corresponding product.

[0116] See also Figure 7 In a further embodiment, step S1400, based on the similarity between the user feature vector and the product feature vector of each product in the product database, recalling products with high similarity and constructing a product set includes the following steps:

[0117] Step S1410: Calculate the dot product between the user feature vector and the product feature vector of each product in the product database as the similarity;

[0118] Those skilled in the art should know that the similarity between the user feature vector and each product feature vector can be represented by calculating the dot product between the user feature vector and the product feature vector of each product in the product database.

[0119] Step S1420: Filter out the commodities whose similarity exceeds a preset threshold and construct them into a commodity set.

[0120] A preset threshold may be set to quickly filter out commodities whose similarity exceeds the preset threshold, so as to construct a commodity set with these commodities. The preset threshold may be an empirical threshold or an experimental threshold, and the specific value may be set by technicians in this field as needed.

[0121] In this embodiment, the dot product between two vectors is calculated as the similarity to scientifically quantify the similarity, so that the products whose similarity exceeds a preset threshold can be quickly and accurately screened out later.

[0122] See also Figure 8 , a commodity search device provided to meet one of the purposes of the present application is a functional embodiment of the commodity search method of the present application, the device includes a request response module 1100, a user branch module 1200, a commodity branch module 1300 and a commodity recall module 1400, wherein: the request response module 1100 is used to respond to user search requests, obtain the multi-dimensional commodity information of the commodity corresponding to the search text pointed to by the request and the historical access behavior of the user, and construct a historical commodity data sequence, the commodity information of the commodity includes a commodity identification code, a commodity title and a commodity picture; the user branch module 1200 is used to use the user branch in the double tower model to obtain the user feature vector generated by the interaction between the features corresponding to the search text and the historical commodity data sequence; the commodity branch module 1300 is used to use the commodity branch in the double tower model to obtain the commodity feature vector corresponding to the features of the multi-dimensional commodity information of the commodities in the commodity database to interact and generate; the commodity recall module 1400 is used to recall the commodities with higher similarity according to the similarity between the user feature vector and the commodity feature vector of each commodity in the commodity database, and construct them into a commodity set.

[0123] In a further embodiment, the user branch module 1200 includes: a text segmentation submodule, which is used to segment the search text using multiple segmentation granularities, obtain segmentation sequences corresponding to different granularities, and extract sequence feature vectors of each segmentation sequence respectively; a semantic extraction submodule, which is used to extract deep semantic information based on the encoded feature information of the segmentation sequence obtained by segmenting at the segmentation granularity, and obtain the corresponding deep semantic vector; an encoded feature submodule, which is used to segment the historical search text corresponding to the user's historical search behavior to obtain the corresponding segmentation sequence, and extract the encoded feature information corresponding to the segmentation sequence; a first attention submodule, which is used to The sequence feature vector of the word sequence obtained by segmenting the search text at the word segmentation granularity is used as the query vector, the encoded feature information of the word sequence obtained by segmenting the historical search text is used as the key vector and the value vector, and the key feature information in the encoded feature information is queried through the attention layer; the information construction submodule is used to integrate the sequence feature vector, the deep semantic vector and the key feature information to construct the comprehensive feature information; the second attention submodule is used to extract the personalized behavior features corresponding to the historical commodity data sequence, and apply at least one attention layer to interact the personalized behavior features with the comprehensive feature information to obtain the user feature vector.

[0124] In a further embodiment, the second attention sub-module includes: a first feature interaction unit, used to extract personalized behavior features corresponding to the historical product data sequence, and apply an attention layer to interact the personalized behavior features with the comprehensive feature information to obtain associated behavior features; a second feature interaction unit, used to splice the associated behavior features with the comprehensive feature information, and apply a multi-head self-attention layer to perform feature interaction on the spliced ​​features to obtain the user feature vector.

[0125] In a further embodiment, the product branch module 1300 includes: a vectorization sub-module, which is used to use the product branch in the double-tower model to extract features corresponding to the product identification code, product title, and product image in the product information of the product, and obtain an identification feature vector, a text feature vector, and a picture feature vector accordingly; a third attention sub-module, which is used to apply an attention layer to interact with the text feature vector and the picture feature vector, and query the associated feature vector corresponding to the text feature vector from the picture feature vector; a vector addition sub-module, which is used to add the associated feature vector and the identification feature vector as a product feature vector.

[0126] In a further embodiment, the vectorization submodule includes: a first vector representation unit, used to apply the coding layer to encode the product identification code, and obtain the corresponding coding information as the title feature vector; a picture segmentation unit, used to segment the product picture into a plurality of sub-pictures of equal specifications to form a sub-picture sequence; a second vector representation unit, used to apply the picture encoder to extract deep semantic features from each sub-picture in the sub-picture sequence, and obtain a picture feature vector composed of feature vectors corresponding to each sub-picture; a third vector representation unit, used to apply the coding layer to encode the product title, pool the coding information obtained by the encoding, and input the pooling result into the multi-layer perception layer to extract the corresponding deep semantic information to obtain a text feature vector;

[0127] In a further embodiment, the third attention submodule includes: a third feature interaction unit, used to use the text feature vector as a query vector, the image feature vector as a key vector and a value vector, and interact the query vector with the key vector to determine the key vector of the product image; a normalization processing unit, used to normalize the key vector to obtain a weight vector; and a feature extraction unit, used to superimpose the weight vector on the value vector to obtain an associated feature vector.

[0128] In a further embodiment, the product recall module 1400 includes: a similarity calculation submodule, used to calculate the dot product between the user feature vector and the product feature vector of each product in the product database as the similarity; and a product screening submodule, used to screen out products whose similarity exceeds a preset threshold to form a product set.

[0129] In order to solve the above technical problems, the present application also provides a computer device. Fig. 9 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a commodity search method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the commodity search method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Fig. 9The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0130] In this embodiment, the processor is used to execute Figure 8 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the commodity search device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0131] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the commodity search method of any embodiment of the present application.

[0132] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0133] In summary, the correlation between the product and the search keyword input by the user can be accurately represented, so that more relevant products can be searched and the exposure rate of the product can be reasonably increased.

[0134] Those skilled in the art will appreciate that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, altered, combined, or deleted. Further, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and schemes in the prior art that are similar to those disclosed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted.

[0135] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A commodity search method, characterized in that: The steps include: In response to a user search request, obtain multi-dimensional product information of a product corresponding to the search text pointed to by the request and the user's historical access behavior, and construct a historical product data sequence, wherein the product information of the product includes a product identification code, a product title, and a product image; A user feature vector is generated by interactively obtaining the features corresponding to the search text and the historical product data sequence using the user branch in the double tower model; The commodity branch in the double tower model is used to obtain the corresponding features of the multi-dimensional commodity information of the commodities in the commodity database to interactively generate a commodity feature vector; According to the similarity between the user feature vector and the product feature vector of each product in the product database, recall products with higher similarity to form a product set; The step of using the user branch in the double tower model to obtain the user feature vector generated by interacting between the search text and the features corresponding to the historical product data sequence includes: Segmenting the search text with multiple segmentation granularities, obtaining segmentation sequences corresponding to different granularities, and extracting sequence feature vectors of each segmentation sequence respectively; Extracting deep semantic information from the encoded feature information of the word segmentation sequence obtained by performing word segmentation based on the word segmentation granularity to obtain a corresponding deep semantic vector; Segmenting the historical search text corresponding to the historical search behavior of the user to obtain a corresponding segmentation sequence, and extracting the encoding feature information corresponding to the segmentation sequence; Using the sequence feature vector of the word sequence obtained by segmenting the search text at the word segmentation granularity as the query vector, using the encoded feature information of the word sequence obtained by segmenting the historical search text as the key vector and the value vector, and querying the key feature information in the encoded feature information through the attention layer; Combining the sequence feature vector, the deep semantic vector and the key feature information to construct comprehensive feature information; The personalized behavior feature corresponding to the historical product data sequence is extracted, and at least one attention layer is applied to interact the personalized behavior feature with the comprehensive feature information to obtain the user feature vector.

2. The commodity search method according to claim 1, characterized in that: Extracting personalized behavior features corresponding to the historical product data sequence, applying at least one attention layer to interact the personalized behavior features with the comprehensive feature information, and obtaining the user feature vector, including the following steps: Extracting personalized behavior features corresponding to the historical commodity data sequence, applying an attention layer to interact the personalized behavior features with the comprehensive feature information to obtain associated behavior features; After splicing the associated behavior features and the comprehensive feature information, a multi-head self-attention layer is applied to perform feature interaction on the spliced ​​feature results to obtain the user feature vector.

3. The commodity search method according to claim 1, characterized in that: The step of interactively generating a product feature vector by using the product branch in the double tower model to obtain the features corresponding to the multi-dimensional product information of the products in the product database includes the following steps: The product branch in the double tower model is used to extract features corresponding to the product identification code, product title, and product image in the product information of the product, and obtain an identification feature vector, a text feature vector, and a picture feature vector accordingly; Applying an attention layer to interact with the text feature vector and the picture feature vector, and querying the associated feature vector corresponding to the text feature vector from the picture feature vector; The association feature vector and the identification feature vector are added together to obtain a product feature vector.

4. The commodity search method according to claim 3, characterized in that: The steps of extracting features corresponding to the product identification code, product title, and product image in the product information of the product by using the product branch in the double tower model, and obtaining the identification feature vector, text feature vector, and image feature vector accordingly include the following steps: Applying a coding layer to encode the product identification code to obtain corresponding coding information as a title feature vector; Divide the product image into multiple sub-images of equal size to form a sub-image sequence; Applying a picture encoder to extract deep semantic features from each sub-picture in the sub-picture sequence to obtain a picture feature vector composed of feature vectors corresponding to each sub-picture; The encoding layer is applied to encode the product title, the encoding information obtained by the encoding is pooled, and the pooling result is input into the multi-layer perception layer to extract the corresponding deep semantic information to obtain a text feature vector.

5. The commodity search method according to claim 3, characterized in that: The step of applying an attention layer to interact with the text feature vector and the picture feature vector, and querying the associated feature vector corresponding to the text feature vector from the picture feature vector, includes the following steps: Using the text feature vector as a query vector, using the image feature vector as a key vector and a value vector, and interacting the query vector with the key vector to determine a key vector of the product image; Normalizing the key vector to obtain a weight vector; The weight vector is superimposed on the value vector to obtain an associated feature vector.

6. The commodity search method according to claim 1, characterized in that: According to the similarity between the user feature vector and the product feature vector of each product in the product database, recalling products with high similarity and constructing a product set includes the following steps: Calculating the dot product between the user feature vector and the product feature vector of each product in the product database as the similarity; The commodities whose similarity exceeds a preset threshold are selected and constructed into a commodity set.

7. A commodity search device, characterized in that: include: A request response module is used to respond to a user search request, obtain the multi-dimensional product information of the product corresponding to the search text pointed to by the request and the historical access behavior of the user, and construct a historical product data sequence. The product information of the product includes a product identification code, a product title, and a product picture; A user branch module, used for obtaining a user feature vector generated by interaction between the search text and the features corresponding to the historical commodity data sequence by using the user branch in the twin tower model; A commodity branch module, which is used to obtain the commodity feature vectors interactively generated by the features corresponding to the multi-dimensional commodity information of the commodities in the commodity database by using the commodity branches in the double tower model; A product recall module, used for recalling products with high similarity according to the similarity between the user feature vector and the product feature vector of each product in the product database, and constructing them into a product set; The step of using the user branch in the double tower model to obtain the user feature vector generated by interacting between the search text and the features corresponding to the historical product data sequence includes: Segmenting the search text with multiple segmentation granularities, obtaining segmentation sequences corresponding to different granularities, and extracting sequence feature vectors of each segmentation sequence respectively; Extracting deep semantic information from the encoded feature information of the word segmentation sequence obtained by performing word segmentation based on the word segmentation granularity to obtain a corresponding deep semantic vector; Segmenting the historical search text corresponding to the historical search behavior of the user to obtain a corresponding segmentation sequence, and extracting the encoding feature information corresponding to the segmentation sequence; Using the sequence feature vector of the word sequence obtained by segmenting the search text at the word segmentation granularity as the query vector, using the encoded feature information of the word sequence obtained by segmenting the historical search text as the key vector and the value vector, and querying the key feature information in the encoded feature information through the attention layer; Combining the sequence feature vector, the deep semantic vector and the key feature information to construct comprehensive feature information; The personalized behavior feature corresponding to the historical product data sequence is extracted, and at least one attention layer is applied to interact the personalized behavior feature with the comprehensive feature information to obtain the user feature vector.

8. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 6 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images