Object analysis method, apparatus and electronic device

Abstract

Embodiments of the present application provide an object analysis method, device and electronic equipment, the method comprising: matching, by a similar object acquisition model, a to-be-analyzed object with a target similar object group corresponding to the to-be-analyzed object. Specifically, the similar object acquisition model comprises an object node network graph, and the object node network graph is a node relationship network graph for representing the corresponding relationship between an analysis object and a similar object, which is constructed according to the corresponding relationship between a sample analysis object and a sample target similar object. According to the to-be-analyzed object and the above-mentioned corresponding relationship stored in the object node network graph, the similarity between each target similar object in the target similar object group and the to-be-analyzed object is greater than a preset similarity threshold, and the click rate of each target similar object is greater than a preset click rate threshold. Therefore, the target similar object group matched for the to-be-analyzed object by the above-mentioned method improves the matching rate of the to-be-analyzed object and the target similar object.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an object analysis method, an object analysis device, an electronic device, and a computer storage medium, a similar image recommendation method, device, electronic device, and computer storage medium, and a training method, device, electronic device, and computer storage medium for a similar object acquisition model. Background Technology

[0002] In online shopping scenarios, users typically input known object information into an online shopping platform. The server then matches multiple similar objects based on the user's input, allowing the user to select the target object from among these similar objects.

[0003] However, in the above method, when the server analyzes and matches the object information input by the user, it does so by analyzing based on single factors. For example, it matches similar objects based on the user's historical click behavior, or it matches object information corresponding to similar text based on the text semantic information of the object to be retrieved, or if the object to be retrieved is an image, it matches similar images based on the image semantic information.

[0004] The method for obtaining similar objects described above is based on a single target. This approach results in a low matching rate between the retrieved similar objects and the user's input object. Therefore, improving the matching rate between similar objects and the object to be retrieved is a problem that needs to be solved. Summary of the Invention

[0005] This application provides an object analysis method to improve the matching rate between similar objects and the object to be retrieved. This application also provides an object analysis device, electronic device, and computer storage medium; a similar image recommendation method, device, electronic device, and computer storage medium; and a training method, device, electronic device, and computer storage medium for a similar object acquisition model.

[0006] This application provides an object analysis method, comprising: acquiring the object to be analyzed, the object to be analyzed including an image to be analyzed; inputting the object to be analyzed into a similar object acquisition model to obtain a target similar object group output by the similar object acquisition model that matches the object to be analyzed, wherein the similarity between each target similar object in the target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold; wherein the similar object acquisition model is used to acquire the target similar object group that matches the object to be analyzed based on the multimodal feature information of the object to be analyzed, combined with an object node network graph, the object node network graph being a node relationship network graph constructed based on the correspondence between sample analysis objects and sample target similar objects, used to characterize the correspondence between analysis objects and similar objects.

[0007] Optionally, the step of inputting the object to be analyzed into a similar object acquisition model to obtain a target similar object group that matches the object to be analyzed, output by the similar object acquisition model, includes: inputting the object to be analyzed into a similar object acquisition model to obtain multiple candidate similar objects corresponding to the object to be analyzed; and determining a target similar object group that matches the object to be analyzed from the multiple candidate similar objects based on a first Euclidean distance between the object to be analyzed and each candidate similar object, wherein the first Euclidean distance is used to characterize the matching degree between the object to be analyzed and the candidate similar objects.

[0008] Optionally, the step of inputting the object to be analyzed into a similar object acquisition model to obtain multiple candidate similar objects corresponding to the object to be analyzed includes: inputting the object to be analyzed into a similar object acquisition model to obtain multimodal feature information of the object to be analyzed; determining multimodal feature information of multiple candidate similar objects corresponding to the multimodal feature information of the object to be analyzed based on the multimodal feature information of the object to be analyzed; and determining multiple candidate similar objects corresponding to the object to be analyzed based on the multimodal feature information of the multiple candidate similar objects.

[0009] Optionally, determining a target similar object group matching the object to be analyzed from the plurality of candidate similar objects based on the first Euclidean distance between the object to be analyzed and each candidate similar object includes: obtaining the first Euclidean distance between the object to be analyzed and each candidate similar object; if the first Euclidean distance is less than a first preset Euclidean distance threshold, determining the candidate similar object as a target similar object matching the object to be analyzed; and constructing a target similar object group matching the object to be analyzed based on the obtained plurality of target similar objects matching the object to be analyzed.

[0010] Optionally, obtaining the first Euclidean distance between the object to be analyzed and each candidate similar object includes: obtaining the multimodal feature vector of the object to be analyzed, obtaining the multimodal feature vector of each candidate similar object; and obtaining the first Euclidean distance between the object to be analyzed and each candidate similar object based on the multimodal feature vector of the object to be analyzed and the multimodal feature vector of each candidate similar object.

[0011] Optionally, obtaining the multimodal feature vector of the object to be analyzed includes: obtaining text feature information and non-text feature information of the object to be analyzed; fusing the text feature information and non-text feature information of the object to be analyzed to obtain the multimodal feature information of the object to be analyzed, wherein the multimodal feature information of the object to be analyzed includes individual text feature information, individual non-text feature information, and fused feature information of text features and non-text features; and obtaining the multimodal feature vector of the object to be analyzed through an image representation model based on the multimodal feature information of the object to be analyzed.

[0012] Optionally, obtaining the multimodal feature vector of each candidate similar object includes: obtaining text feature information and non-text feature information of each candidate similar object; fusing the text feature information and non-text feature information of each candidate similar object to obtain the multimodal feature information of each candidate similar object, wherein the multimodal feature information of each candidate similar object includes individual text feature information, individual non-text feature information, and fused feature information of text features and non-text features; and obtaining the multimodal feature vector of each candidate similar object through an image representation model based on the multimodal feature information of each candidate similar object.

[0013] Optionally, determining the multimodal feature information of multiple candidate similar objects corresponding to the multimodal feature information of the object to be analyzed, based on the multimodal feature information of the object to be analyzed, includes: obtaining the multimodal feature information of multiple first candidate similar objects corresponding to the multimodal feature information of the object to be analyzed, based on the multimodal feature information of the object to be analyzed and the pre-stored correspondence between the analysis object and similar objects in the object node network graph; obtaining the multimodal feature information of multiple second candidate similar objects from the multimodal feature information of the multiple first candidate similar objects, wherein the similarity to the multimodal feature information of the object to be analyzed is greater than a preset similarity threshold; and selecting the multimodal feature information of the second candidate similar objects from the multimodal feature information of the multiple second candidate similar objects, wherein the click-through rate is greater than a preset click-through rate threshold, as the multimodal feature information of the multiple candidate similar objects corresponding to the multimodal feature information of the object to be analyzed.

[0014] This application embodiment also provides a similar image recommendation method applied to a user terminal. The method includes: acquiring an image to be analyzed; obtaining a first request message for requesting to acquire a group of target similar images matching the image to be analyzed; acquiring at least one target similar image matching the image to be analyzed according to the first request message; constructing a group of target similar images matching the image to be analyzed based on the at least one target similar image matching the image to be analyzed, wherein the similarity between each target similar image in the group and the image to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar image is greater than a preset click-through rate threshold; displaying the target similar images in the group; wherein each target similar image in the group is acquired by the server in the following manner: acquiring a group of target similar objects matching the image to be analyzed by a similar object acquisition model based on the multimodal feature information of the image to be analyzed and combined with an object node network graph, wherein the object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to characterize the correspondence between the analysis object and the similar object.

[0015] Optionally, acquiring the image to be analyzed includes: acquiring a second request message for requesting the acquisition of the image to be analyzed, the second request message including descriptive information for the image to be analyzed; acquiring a first candidate image group for the image to be analyzed based on the second request message; and in response to detecting a selection operation for at least one candidate image in the first candidate image group, selecting the selected candidate image as the image to be analyzed.

[0016] Optionally, it also includes: displaying a filled area on the first search page for obtaining descriptive information for the image to be analyzed; obtaining a second request message for requesting the image to be analyzed includes: in response to detecting a trigger operation for the filled area, obtaining a second request message for requesting the image to be analyzed, the second request message including descriptive information for the image to be analyzed.

[0017] This application embodiment also provides a training method for a similar object acquisition model, including: acquiring a sample analysis object and a target similar object sample group matching the sample analysis object, wherein the similarity between each sample target similar object in the target similar object sample group and the sample analysis object is greater than a preset similarity threshold, and the click-through rate of each sample target similar object is greater than a preset click-through rate threshold, the sample analysis object includes a sample analysis image, and the sample target similar object includes a sample target similar image matching the sample analysis image; training an initial similar object acquisition model based on the sample analysis object and the target similar object sample group to obtain a trained similar object acquisition model; wherein the trained similar object acquisition model is used to acquire a target similar object group matching the object to be analyzed based on the multimodal feature information of the object to be analyzed, combined with an object node network graph, wherein the object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to characterize the correspondence between the analysis object and similar objects.

[0018] Optionally, the step of training the initial similar object acquisition model based on the sample analysis object and the target similar object sample group to obtain the trained similar object acquisition model includes: inputting the information of the sample analysis object into the initial similar object acquisition model to obtain a first similar object group output by the initial similar object acquisition model for the sample analysis object; obtaining a second Euclidean distance between each first candidate similar object in the first similar object group and the sample analysis object; obtaining a first preset Euclidean distance threshold between the target similar object in the target similar object sample group and the sample analysis object; obtaining an error value between the second Euclidean distance and the first preset Euclidean distance threshold; and adjusting the parameters of the initial similar object acquisition model based on the error value to obtain the trained similar object acquisition model.

[0019] Optionally, the step of inputting the information of the sample analysis object into an initial similar object acquisition model to obtain a first similar object group output by the initial similar object acquisition model for the sample analysis object includes: inputting the sample analysis object into the initial similar object acquisition model to obtain the multimodal feature information of the sample analysis object; obtaining the multimodal feature information of a first candidate similar object corresponding to the multimodal feature information of the sample analysis object based on the multimodal feature information of the sample analysis object and the correspondence between the analysis object and similar objects stored in the initial object node network graph; and constructing a first similar object sample group for the sample analysis object based on the obtained multimodal feature information of multiple first candidate similar objects.

[0020] Optionally, obtaining the second Euclidean distance between each first candidate similar object in the first similar object group and the sample analysis object includes: obtaining the multimodal feature vector of the first candidate similar object, obtaining the multimodal feature vector of the sample analysis object, and calculating the second Euclidean distance between the first candidate similar object and the sample analysis object based on the multimodal feature vector of the first candidate similar object and the multimodal feature vector of the sample analysis object.

[0021] Optionally, adjusting the parameters of the initial similar object acquisition model based on the error value to obtain a trained similar object acquisition model includes: if the error value is greater than a preset error value threshold, continuing to execute the step of inputting the information of the sample analysis object into the initial similar object acquisition model to obtain the first similar object group output by the initial similar object acquisition model for the sample analysis object; if the error value is less than the preset error value threshold, stopping the parameter adjustment of the similar object acquisition model to obtain a trained similar object acquisition model.

[0022] Optionally, it also includes: updating the correspondence between the analyzed objects and similar objects stored in the initial object node network graph based on the error value.

[0023] Optionally, the sample analysis object and the target similar object sample group matching the sample analysis object are positive sample data pairs used to train the similar object acquisition model, and the method further includes: acquiring the sample analysis object and other similar object sample groups that do not match the sample analysis object; using the sample analysis object and other similar object sample groups that do not match the sample analysis object as negative sample data pairs used to train the similar object acquisition model; and using the negative sample data pairs to train the similar object acquisition model.

[0024] Optionally, obtaining the multimodal feature vector of the first candidate similar object includes: obtaining text feature information and non-text feature information of the first candidate similar object; fusing the text feature information and non-text feature information of the first candidate similar object to obtain the multimodal feature information of the first candidate similar object, wherein the multimodal feature information of the first candidate similar object includes individual text feature information, individual non-text feature information, and fused feature information of text features and non-text features; and obtaining the multimodal feature vector of the first candidate similar object through an image representation model based on the multimodal feature information of the first candidate similar object.

[0025] Optionally, obtaining the multimodal feature vector of the sample analysis object includes: obtaining text feature information and non-text feature information of the sample analysis object; fusing the text feature information and non-text feature information of the sample analysis object to obtain the multimodal feature information of the sample analysis object, wherein the multimodal feature information of the sample analysis object includes individual text feature information, individual non-text feature information, and fused feature information of text features and non-text features; and obtaining the multimodal feature vector of the sample analysis object through an image representation model based on the multimodal feature information of the sample analysis object.

[0026] This application embodiment also provides an object analysis device, including: a first acquisition unit, configured to acquire the object to be analyzed, the object to be analyzed including an image to be analyzed; a first obtaining unit, configured to input the object to be analyzed into a similar object acquisition model, and obtain a target similar object group output by the similar object acquisition model that matches the object to be analyzed, wherein the similarity between each target similar object in the target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click rate of each target similar object is greater than a preset click rate threshold; wherein, the similar object acquisition model is configured to obtain a target similar object group that matches the object to be analyzed based on the multimodal feature information of the object to be analyzed, combined with an object node network graph, the object node network graph being a node relationship network graph constructed based on the correspondence between sample analysis objects and sample target similar objects, used to characterize the correspondence between analysis objects and similar objects.

[0027] This application also provides a similar image recommendation device applied to a user terminal. The device includes: a second acquisition unit for acquiring an image to be analyzed; a first request message acquisition unit for acquiring a first request message for requesting to acquire a group of target similar images matching the image to be analyzed; and a target similar image group construction unit for acquiring at least one target similar image matching the image to be analyzed based on the first request message, and constructing a group of target similar images matching the image to be analyzed based on the at least one target similar image matching the image to be analyzed, wherein each target similar image in the group of target similar images has a similarity to the image to be analyzed. The similarity score is greater than a preset similarity threshold, and the click-through rate of each target similar image is greater than a preset click-through rate threshold; a display unit is used to display the target similar images in the target similar image group; wherein, each target similar image in the target similar image group is obtained by the server in the following way: by obtaining a target similar object group matching the image to be analyzed through a similar object acquisition model based on the multimodal feature information of the image to be analyzed and combined with the object node network graph, wherein the object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to characterize the correspondence between the analysis object and the similar object.

[0028] This application embodiment also provides a training device for a similar object acquisition model, including: a third acquisition unit, used to acquire a sample analysis object and a target similar object sample group matching the sample analysis object, wherein the similarity between each sample target similar object in the target similar object sample group and the sample analysis object is greater than a preset similarity threshold, and the click rate of each sample target similar object is greater than a preset click rate threshold, the sample analysis object includes a sample analysis image, and the sample target similar object includes a sample target similar image matching the sample analysis image; a similar object acquisition model acquisition unit, used to train an initial similar object acquisition model based on the sample analysis object and the target similar object sample group to obtain a trained similar object acquisition model; wherein the trained similar object acquisition model is used to acquire a target similar object group matching the object to be analyzed based on the multimodal feature information of the object to be analyzed, combined with an object node network graph, wherein the object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to characterize the correspondence between the analysis object and similar objects.

[0029] This application also provides an electronic device, which includes a processor and a memory; the memory stores a computer program, and the processor executes the above-described method after running the computer program.

[0030] This application also provides a computer storage medium storing a computer program, which, when run by a processor, executes the above-described method.

[0031] Compared with the prior art, the embodiments of this application have the following advantages:

[0032] This application provides an object analysis method, comprising: acquiring an object to be analyzed, the object to be analyzed including an image to be analyzed; inputting the object to be analyzed into a similar object acquisition model to obtain a target similar object group output by the similar object acquisition model that matches the object to be analyzed, wherein the similarity between each target similar object in the target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold; wherein, the similar object acquisition model is used to acquire a target similar object group that matches the object to be analyzed based on the multimodal feature information of the object to be analyzed, combined with an object node network graph, the object node network graph being a node relationship network graph constructed based on the correspondence between sample analysis objects and sample target similar objects, used to characterize the correspondence between analysis objects and similar objects.

[0033] In the above method, a similarity object acquisition model matches the object to be analyzed with a group of target similar objects corresponding to the object to be analyzed. Specifically, the similarity object acquisition model includes an object node network graph, which is a node relationship network graph constructed based on the correspondence between the sample object to be analyzed and the sample target similar objects, used to represent the correspondence between the object to be analyzed and similar objects. Based on the object to be analyzed and the correspondence stored in the object node network graph, the similarity between each target similar object in the acquired target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold. Therefore, by matching the target similar object group to the object to be analyzed using the above method, the matching rate between the object to be analyzed and the target similar objects is improved.

[0034] This application provides a training method for a similar object acquisition model, comprising: acquiring a sample analysis object and a target similar object sample group matching the sample analysis object, wherein the similarity between each sample target similar object in the target similar object sample group and the sample analysis object is greater than a preset similarity threshold, and the click-through rate of each sample target similar object is greater than a preset click-through rate threshold; the sample analysis object includes a sample analysis image, and the sample target similar objects include sample target similar images matching the sample analysis image; training an initial similar object acquisition model based on the sample analysis object and the target similar object sample group to obtain a trained similar object acquisition model; wherein the trained similar object acquisition model is used to acquire a target similar object group matching the object to be analyzed based on the multimodal feature information of the object to be analyzed, combined with an object node network graph, wherein the object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar objects, used to characterize the correspondence between the analysis object and similar objects.

[0035] In the above method, a similarity object acquisition model is trained using a sample analysis object and a target similarity object sample group that matches the sample analysis object. The trained similarity object acquisition model can acquire a target similarity object group that matches the object to be analyzed based on the multimodal feature information of the object to be analyzed and the object node network graph. Specifically, the similarity rate between each similar object in the target similarity object group and the object to be analyzed is greater than a preset similarity rate threshold, and the click-through rate of each similar object is greater than a preset click-through rate threshold. The similarity object acquisition model trained using the above method improves the matching rate between the object to be analyzed and the target similarity object group during the matching process.

[0036] This application provides a similar image recommendation method applied to a user terminal. The method includes: acquiring an image to be analyzed; obtaining a first request message for requesting a group of target similar images matching the image to be analyzed; acquiring at least one target similar image matching the image to be analyzed based on the first request message; constructing a group of target similar images matching the image to be analyzed based on the at least one target similar image matching the image to be analyzed, wherein the similarity between each target similar image in the group and the image to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar image is greater than a preset click-through rate threshold; and displaying the target similar images in the group. Each target similar image in the group is acquired by the server in the following manner: a group of target similar objects matching the image to be analyzed is acquired by a similar object acquisition model based on the multimodal feature information of the image to be analyzed and combined with an object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to characterize the correspondence between the analysis object and the similar object.

[0037] The above method involves a user terminal receiving a first request message from a user requesting the acquisition of target similar images corresponding to the image to be analyzed. Based on this first request message, a group of target similar images corresponding to the image to be analyzed is obtained. This group of target similar images is acquired by the server. Specifically, a similarity object acquisition model acquires the target similar object group based on the multimodal feature information of the object to be analyzed and the object node network graph. The similarity between the target similar images acquired by the similarity object acquisition model and the image to be analyzed is greater than a preset similarity threshold, and the click-through rate of the target similar images is greater than a preset click-through rate threshold, thus improving the matching rate between the object to be analyzed and the target similar objects. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of a first application scenario of the object analysis method provided in the embodiments of this application.

[0039] Figure 2 This is a schematic diagram of a second application scenario for the object analysis method provided in the embodiments of this application.

[0040] Figure 3 The object node network diagram provided for the embodiments of this application.

[0041] Figure 4 for Figure 3 A partial schematic diagram of the object node network graph.

[0042] Figure 5This is a flowchart of an object analysis method provided in the first embodiment of this application.

[0043] Figure 6 This is a schematic diagram of an object analysis device provided in the second embodiment of this application.

[0044] Figure 7 This is a flowchart of a training method for a similar object acquisition model provided in the third embodiment of this application.

[0045] Figure 8 This is a schematic diagram of a training device for a similar object acquisition model provided in the fourth embodiment of this application.

[0046] Figure 9 A flowchart of a similar object image recommendation method provided in the fifth embodiment of this application.

[0047] Figure 10 This is a schematic diagram of a similar object image recommendation device provided in the sixth embodiment of this application.

[0048] Figure 11 This is a schematic diagram of an electronic device provided in the seventh embodiment of this application. Detailed Implementation

[0049] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0050] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The descriptive terms used in this application and the appended claims, such as "a," "first," and "second," are not intended to limit quantity or sequence, but rather to distinguish information of the same type from one another.

[0051] To facilitate understanding of the systems and methods provided in the embodiments of this application, the background of the embodiments of this application will be introduced before introducing the embodiments of this application.

[0052] In online shopping scenarios, users typically input information about the object to be analyzed on an online shopping platform. The server then matches multiple similar objects with the same information based on the user's input, allowing the user to select the target similar object from among them.

[0053] However, in the above methods, when the server analyzes and matches the information of the object to be analyzed input by the user, it usually does so based on a single factor. For example, it matches similar objects based on the user's historical click behavior of the object to be analyzed, or it matches object information corresponding to similar text based on the text semantic information of the object to be retrieved, or if the object to be retrieved is an image, it matches similar images based on the image semantic information.

[0054] The aforementioned method for obtaining similar objects is based on a single target. This approach results in a low matching rate between the retrieved similar objects and the user-inputted object. Therefore, improving the matching rate between similar objects and the object to be retrieved is a problem that needs to be addressed.

[0055] Having provided the background information above, those skilled in the art can understand the problems existing in the prior art. The application scenarios of the object analysis method of this application will now be described in detail. The object analysis method provided in this application can be applied to various scenarios of obtaining similar objects to the object to be analyzed. For example, on an online shopping platform, a user inputs the required product information or provides image information to be searched. The server, based on the product information input by the user, obtains similar product information matching the product information and sends the obtained similar product information to the client, which then displays the similar product information. Furthermore, it can also be applied to other scenarios of searching for similar information, without limitation.

[0056] The following section first illustrates the application scenarios of the object analysis method in the embodiments of this application.

[0057] Figure 1 This is a schematic diagram illustrating an application scenario of the object analysis method provided in the embodiments of this application.

[0058] like Figure 1 As shown, this application scenario includes client 101 and server 102. Figure 1 To illustrate this in detail, client 101 refers to the user's terminal. On the application platform corresponding to the user's terminal, the client obtains the information of the object to be analyzed provided by the user. This information can be text input by the user, or image information provided by the user. Taking a user querying product information on an online shopping platform as an example, the information provided by the user could be the product information of the product being queried. Please refer to [link / reference]. Figure 2 This is a second application scenario diagram of the object analysis method provided in the embodiments of this application. Figure 2 Include Figure 2 Pages 2a to 2a in Figure 2 Pages 2, 2, and 3 describe the information displayed on the client side during the process of obtaining target similar images of the image to be analyzed. Figure 2 Page 2a displays the first query page, where the user provides product information to be queried. This product information includes user-provided image information, text search terms entered in the search box, or attribute information of the product. Client 101 sends the obtained object information to server 102.

[0059] Server 102 retrieves candidate image information matching the product information provided by the user and sends it to client 101, which then displays the first search results page. Figure 2 Page 2b of the search results displays multiple candidate images that match the description information of the object to be analyzed. It was detected that the user was targeting the first candidate image (e.g., ...) on the first search results page. Figure 2 The long-press trigger action in image 1) on page 2b is sent to the server. The server receives this action, which means... Figure 3 The Trigger (long press to trigger product) operation shown in the figure uses the first candidate image as the image to be analyzed. The long press to trigger operation here refers to obtaining similar images of the image to be analyzed (that is, the first candidate image).

[0060] After the server obtains similar images to the image to be analyzed (i.e., the first candidate image), such as Figure 3 The system retrieves images of similar products from the "Item" (Similar Products) list and sends these images to the client. The client then displays the similar images corresponding to the image being analyzed on the second results page, such as... Figure 2 Page 2b displays images of various similar products to image 1.

[0061] Among them, the similarity between the similar images displayed on the second results page and the image to be analyzed is greater than the preset similarity threshold, and the click-through rate of the similar images is greater than the preset click-through rate threshold.

[0062] The aforementioned server, based on the user-provided information about the object to be analyzed, determines the image to be analyzed corresponding to that object. Then, it acquires similar images corresponding to the object to be analyzed using a pre-trained similarity object acquisition model. Specifically, the object to be analyzed information is input into the similarity object acquisition model. The model determines the image to be analyzed based on this information. Then, based on the multimodal feature information of the image to be analyzed and the pre-stored correspondence between the analyzed image and its corresponding similar images in the object node network graph, it acquires multiple candidate similar images that match the image to be analyzed. Finally, it determines the target similar image group that matches the image to be analyzed from these candidate similar images.

[0063] Specifically, the target similar image in the target similar image group is determined from multiple candidate similar images by the Euclidean distance (L2 distance) between the image feature vector of the image to be analyzed and the image feature vector of the candidate similar images. The Euclidean distance (L2 distance) is calculated using the following formula 1:

[0064]

[0065] Where L2 represents the L2 distance between two vectors; x1 represents the x-axis component of the image vector of the image to be analyzed; x2 represents the x-axis component of the image vector of other candidate similar images; y1 represents the y-axis component of the image vector of the image to be analyzed; and y2 represents the y-axis component of the image vector of other candidate similar images.

[0066] The matching degree between the image to be analyzed and the candidate similar images is determined by calculating the L2 distance between the image feature vectors of the image to be analyzed and the image feature vectors of the candidate similar images. A smaller L2 distance indicates that the similarity between the candidate similar image and the image to be analyzed is greater than a preset similarity threshold, and the click-through rate of the candidate similar image is greater than a preset click-through rate threshold. Therefore, the L2 distance between the image feature vectors of the image to be analyzed and the image feature vectors of the candidate similar images is compared with the preset L2 distance threshold. Candidate similar images with an L2 distance less than the preset L2 distance threshold are selected as target similar images that match the image to be analyzed. A target similar image group is constructed based on the obtained target similar images, and the target similar images in the target similar image group are displayed on the second results page.

[0067] To improve the accuracy of calculating the L2 distance between the image feature vector of the image to be analyzed and the image feature vectors of candidate similar images, this application uses multimodal feature information of the image to determine the multimodal feature vector when obtaining the image feature vector. Specifically, the multimodal feature vector of the image is determined in the following way:

[0068] Taking the acquisition of multimodal feature vectors of an image to be analyzed as an example, the process begins with obtaining the multimodal feature information of the image. Specifically, this involves acquiring the textual feature information and the non-textual feature information of the image. The textual feature information refers to textual information related to the image, such as the image's name and accompanying annotations. The non-textual feature information includes visual feature information, which refers to the visual information inherent in the image itself, such as color, texture, and shape—all common visual features of images.

[0069] The textual and non-textual features of the image to be analyzed are multiplied by an outer product to obtain individual textual, non-textual, and image-text fusion features. These individual textual, non-textual, and image-text fusion features are then used as the multimodal feature information of the image to be analyzed. Based on this multimodal feature information, a multimodal feature vector is obtained. This multimodal feature information is then input into an image representation model to obtain the multimodal feature vector output by the model.

[0070] The above describes the method for obtaining the image feature vector of the image to be analyzed. The image feature vector of the candidate image is also obtained in the same way.

[0071] Furthermore, the similarity object acquisition model is obtained by combining multimodal feature technology with the graph learning model and training the graph learning model.

[0072] The following describes in detail the training method for the similarity object acquisition model:

[0073] The first step is to prepare sample data for training the initial similarity object acquisition model (i.e., the graph learning model). This sample data is obtained from user historical behavior log data, which consists of historical query data from the online service platform and historical query results.

[0074] The similarity object acquisition model trained here ensures that the target similar images obtained for the image to be analyzed possess both click-through rate (CTR) and similarity features. Specifically, the similarity between the target similar images in the target similar image group and the image to be analyzed is greater than a preset similarity threshold, and the CTR of the target similar images is greater than a preset CTR threshold. Therefore, when using user historical behavior log data to obtain sample data, not only are click sample data and similar sample data for the historical image to be analyzed obtained, but the sample data also enhances the training effect of the graph learning model on both CTR and similarity features.

[0075] This application embodiment specifically determines the connection relationships between nodes based on the user's historical behavior log data during the process of historically querying similar product images. Based on these connections, an object node network graph is determined. Furthermore, based on these connections, click sample data and similar sample data are determined for training the graph learning model.

[0076] Please refer to Figure 3 This is the object node network diagram provided in the embodiments of this application. Figure 3In the object node network graph, the connecting edges between nodes represent the connections between them. These edges include direct click connections and similarity connections. The connections are determined based on the user's historical query records on the online shopping platform and the historical query results obtained from those records. Figure 3 In this model, connecting edges will be established between two nodes with the following types of relationships to enrich the relationships between object nodes in the object node network graph. Specifically, these include: connections between Query (user query term) and Trigger (long press to trigger product), Trigger (long press to trigger product) and Item (similar product), Item (similar product) and Item (similar product) and Category (product category), which will be described in detail below.

[0077] Query (user query terms) refers to the query terms provided by the user during the historical query process on the online service platform. Among them, Q1, Q2, Q3, Q4, and Q5 are different historical query terms provided by the user. For example, Q1 is straight-leg pants, Q2 is sweatpants, Q3 is slim-fit pants, Q4 is dress pants, and Q5 is shorts.

[0078] A Trigger (long press to trigger product) indicates a historical product image selected by the user that matches the user's historical query term, and initiates the process of retrieving similar historical product images that match the historical product image. A Query-Trigger connection edge indicates that the user provides a search term and long presses on a product image in the first search results page.

[0079] The server retrieves product images corresponding to the product information represented by the user's historical query terms based on the user's provided query, and displays multiple candidate product images for the user's historical query terms on the first search results page. Then, on the first search results page, the client detects the user's triggered actions on the candidate product images displayed on the first search results page.

[0080] For example, in Figure 3 In the search results, after the user provided the historical query terms Q1 (straight-leg pants) or Q2 (sports pants), among the multiple candidate product images that appeared on the first search results page, the user performed a long press to trigger the action for T1 (straight-leg sports pants). That is, T1 (straight-leg sports pants) was selected as the historical product image to be analyzed corresponding to the historical query terms Q1 or Q2.

[0081] For example, when the user provides the historical query term Q3 (slim-fit pants), both T2 (long slim-fit pants) and T3 (short slim-fit pants) are historical product images to be analyzed that match the historical query term Q3. When the user provides the historical query term Q5 (short pants), T3 (short slim-fit pants) is also a historical product image to be analyzed that matches the historical query term Q5.

[0082] The above describes the meaning of the Query-Trigger connection edge, which is to display the historical product images to be analyzed to the user based on the historical query terms provided by the user, and then initiate the operation steps of querying historical target similar images that match the historical product images to be analyzed.

[0083] In this application, the sample data for training the similar object acquisition model consists of the two node data corresponding to the Trigger-Item connection edge and the two node data corresponding to the Item-Item connection edge, the obtained click sample data, and the similar sample data.

[0084] Combined Figure 3 After a user clicks the Trigger (long press to trigger product) on T1 (straight-leg sweatpants), the server retrieves historical similar product images corresponding to the historical target product image T1 (straight-leg sweatpants) and displays these historical similar product images on the second results page, such as... Figure 3 Images of each product in the Item (similar products) category.

[0085] Item (Similar Products) represents historical target similar products displayed on the second results page. These are historical target similar product images that match the historical product image being analyzed, obtained after a Trigger (long-press triggered product) action. The connecting edge of Trigger-Item indicates that after a user long-presses a historical product being analyzed, historical target similar product images matching that product image are displayed on the second results page. Therefore, the two image data corresponding to the connecting edge of Trigger-Item can form click positive sample data and similar positive sample data. Correspondingly, in Figure 3 Two image data sets that do not have a Trigger-Item connection edge can form click negative sample data and similar negative sample data.

[0086] like Figure 3In the image, T1 (straight-leg athletic pants), I1 (straight-leg loose-fitting athletic pants), and I2 (straight-leg slim-fit athletic pants) are the historical product images to be analyzed. T1 (straight-leg athletic pants) and both I1 and I2 belong to the straight-leg athletic pants category, making them similar target product images to the historical product image T1. Therefore, there is a connection edge between T1 and I1, and another connection edge between T1 and I2. The connection edge between T1 and I1 is a direct click connection edge, representing positive click sample data, and the connection edge between T1 and I2 is also a positive click sample data.

[0087] For example, T2 is long slim-fit trousers, I2 is straight-leg slim-fit athletic trousers, and I3 is slim-fit denim shorts. Therefore, the historical product image to be analyzed, T2, is long slim-fit trousers, characterized by being both long and slim-fit; I2's trousers are specifically long, straight-leg, and athletic; I3's trousers are short. Therefore, I2 is a target similar product image matching T2, while I3 is not. T2 and I2 have a direct click relationship, with a connecting edge; that is, T2 and I2 belong to positive click sample data and similar positive sample data. T2 and I3 do not have a direct click relationship, with no connecting edge; that is, T2 and I3 belong to negative click sample data and similar negative sample data.

[0088] For example, if T3 represents short slim-fit trousers, I3 represents slim-fit denim shorts, and I4 represents slim-fit suit shorts, then T3-I3 represents positive click samples and similar positive samples, and T3-I4 represents positive click samples and similar positive samples. If T4 represents suit trousers, I4 represents slim-fit suit shorts, and I5 represents suit trousers, then T4-I4 represents positive click samples and similar positive samples, and T4-I5 represents positive click samples and similar positive samples. Correspondingly, there is no direct click relationship between T3 and I5; therefore, T3-I5 can be considered negative click samples and similar negative samples.

[0089] Additionally, if a user clicks the Trigger (long press to trigger a product) action and multiple similar product images are displayed on the second results page, and the user clicks on two similar product images simultaneously, it indicates that the similarity between these two images is relatively high, and both belong to historical similar target product images matched with historical product images to be analyzed. For example... Figure 3 In the diagram, the connecting edge between I1 (straight-leg loose-fitting athletic pants) and I2 (straight-leg slim-fit athletic pants) indicates that the user first clicked on the details page of product I1, and then clicked on the details page of product I2. The similarity between products I1 and I2 is greater than the preset similarity threshold. I1-I2 belongs to similar positive sample data. I3 is slim-fit denim shorts, which is a different type of pants from I2 (straight-leg slim-fit athletic pants). I2-I3 cannot form a connecting edge; therefore, I2-I3 belongs to similar negative sample data.

[0090] For example, I3 is slim-fit denim shorts, and I4 is slim-fit suit shorts. Both items are target similar items matching T3, and I3 and I4 share the characteristics of being slim-fit and shorts, thus classifying them as similar items. Therefore, I3-I4 represents similar positive sample data. However, the difference between I4 (slim-fit suit shorts) and I5 (suit trousers) is that I4 is short trousers, while I5 is long trousers. Since there is a distinction between short and long trousers, there is no connecting edge between I4 and I5, and I4-I5 represents similar negative sample data.

[0091] Additionally, Category (product category) refers to the leaf category attribute of a product. For example, the leaf category attribute of I1 (straight-leg loose-fitting athletic pants) is casual pants, the leaf category attribute of I2 (straight-leg slim-fit athletic pants) and I3 (slim-fit denim shorts) is slim-fit pants, and the leaf category attribute of I4 (slim-fit suit shorts) and I5 (suit pants) is suit pants.

[0092] exist Figure 3 and Figure 4 middle, Figure 4 for Figure 3 A partial schematic diagram of the object node network graph. Figure 4 The image shows click negative sample data and similar negative sample data. In the connection relationships between the various nodes of the user historical behavior log data above, the two images corresponding to the connection edges between TI are click positive sample data and similar positive sample data, as shown below. Figure 4 In the T1-I1 model, the two product images correspond to each other; the two images without a connecting edge between T1 and I1 represent click negative sample data and similar negative sample data, such as... Figure 4 The two product images corresponding to T2-I3 in the diagram. The two images corresponding to the connecting edge between II are similar positive sample data, such as... Figure 4 In the diagram, the two product images corresponding to I1 and I2, and the two images without a connecting edge between I1 and I2, are considered similar negative sample data. Figure 4 The two product images corresponding to I2-I3 in the diagram are shown. Therefore, the graph learning model is trained based on sample data between nodes in the user's historical behavior log data, enabling the graph learning model to construct an object node network graph between various object nodes.

[0093] In the process of acquiring sample data, it is also necessary to acquire difficult negative sample data to enhance the training effect of the graph learning model. For example, if Ia represents a short-sleeved shirt and Ib represents pants, then Ia and Ib are simple similar negative sample pairs. When training the graph learning model with simple negative sample pairs, the model can quickly identify the difference between short-sleeved shirts and pants. However, when Ia represents a crew neck short-sleeved shirt and Ic represents a V-neck short-sleeved shirt, the distinguishing features between Ia and Ic are relatively small, making it difficult for the graph learning model to distinguish between crew neck and V-neck short-sleeved shirts. Therefore, this embodiment also employs various difficult negative samples to train the graph learning model, improving the accuracy of the trained similar object acquisition model in identifying subtle differences between similar images of the image to be analyzed.

[0094] The above is a brief description of the sample data for preparing to train a graph learning model and obtaining similar objects to acquire the model.

[0095] In addition, during the process of obtaining sample data for training the graph learning model based on user historical behavior log data, it is necessary to take corresponding click sample data and similar sample data according to the preset sampling ratio, so that the target similar images of the image to be analyzed obtained by the finally trained similar object acquisition model have both click rate and similarity features.

[0096] The preset sampling ratio can be obtained according to the following formula 2:

[0097] sample_weight(t p i q )=α×similar_score(t p i q )+β×click_score(t p i q )

[0098] Wherein, sample_weight(t) p i q ) represents the sampling weight of the sample data, α represents the sampling coefficient of the similar sample scores of similar sample data, and similar_score(t) represents the sampling weight of the sample data. p i q ) represents the similarity score corresponding to similar sample data; β represents the sampling coefficient of the click score of the click sample data, click_score(t) p i q The ) represents the click score corresponding to the clicked sample data.

[0099] According to Formula 2 above, the click score of the click sample data and the similarity score of the similar sample data are determined according to the above proportions. The click score and similarity score control the importance of the two types of sample data. Then, the graph learning model is trained based on these sample data, so that the final similar object acquisition model identifies similar images that have both click-through rate and similarity effects on the target similar images of the image to be analyzed.

[0100] The above describes the first step of obtaining sample data and the second step of constructing an object node network graph based on the sample data, so that the object node network graph stores the correspondence between the analysis object and similar objects that match the analysis object.

[0101] Based on the above description, after obtaining sample data and constructing an object node network graph based on the sample data, the third step is to train the initial similar object acquisition model (i.e., the graph learning model).

[0102] Step 1: Input the sample analysis object information into the graph learning model to obtain the sample analysis image corresponding to the sample analysis object information.

[0103] In this step, the sample analysis object information includes user query terms provided by the user on the client side, or image information provided by the user. The system matches the user query terms with corresponding sample analysis images, or scans the image information provided by the user to provide the user with sample analysis images corresponding to the image information.

[0104] Step 2: Based on the sample analysis image and the initial object node network graph, obtain the first similar image group corresponding to the sample analysis image.

[0105] The initial object node network graph is built based on the connection relationships between the sample analysis image and the sample target similar images. It primarily references sample similar images in the object node network graph that have a direct click relationship with the sample analysis image, as well as the connection relationships between multiple sample similar images targeting the same sample analysis image. A sample similar image that has a direct click relationship with the sample analysis image refers to an image identified by the server when a long-press trigger operation is detected, indicating the need to retrieve a target similar image matching the sample analysis image. In other words, the similarity between the sample similar image and the sample analysis image is greater than a preset similarity threshold, and the click rate of the sample similar image is greater than a preset click rate threshold.

[0106] For example, Figure 3In the analysis, the sample image is T1 (straight-leg sports pants). Based on the long-press trigger operation for T1, the server determines that the sample similar images that match T1 (straight-leg sports pants) include I1 (straight-leg loose sports pants) and I2 (straight-leg slim-fit sports pants).

[0107] Step 3: Calculate the second Euclidean distance L2 between the sample analysis image and each first candidate similar image in the first similar image group.

[0108] The second Euclidean distance L2 is calculated using Formula 1 above, based on the image feature vectors of the sample analysis image and the image feature vectors of the first candidate similar image.

[0109] To improve the accuracy of the first Euclidean distance L2, the image feature vector of the sample analysis image is obtained by acquiring the multimodal feature vector of the sample analysis image and the multimodal feature vector of the first candidate similar image.

[0110] Obtaining the multimodal feature vector of an image involves performing multimodal feature fusion processing on the image information to obtain the multimodal feature information of the image. This multimodal feature information is then input into an image representation model to obtain the multimodal feature vector of the image output by the image representation model.

[0111] Step 4: Calculate the Euclidean distance L2 between the sample analysis image and the image similar to the sample target, where the Euclidean distance L2 between the sample analysis image and the image similar to the sample target is a preset Euclidean distance threshold.

[0112] This section calculates a preset Euclidean distance threshold between the sample analysis image and similar images of the target. The sample data includes positive and negative samples. Positive samples indicate that the similarity between the sample analysis image and the similar image of the target is greater than a preset similarity threshold, and the click-through rate of the similar image of the target is greater than a preset click-through rate threshold. Therefore, the L2 distance corresponding to the positive samples is less than the first preset Euclidean distance threshold.

[0113] Negative sample data indicates that the similarity between the sample analysis image and the sample target similar image is less than a preset similarity threshold, and the click-through rate of the sample target similar image is less than a preset click-through rate threshold. Therefore, the L2 distance corresponding to negative sample data is greater than the second preset Euclidean distance threshold, and the second preset Euclidean distance threshold is greater than the first preset Euclidean distance threshold.

[0114] The initial graph learning model is trained using both positive and negative sample data to continuously improve the matching rate between the image to be analyzed and candidate similar images. The matching rate is defined as follows: the similarity between the image to be analyzed and the candidate similar images is greater than a preset similarity threshold, and the click-through rate of the candidate similar images is greater than a preset click-through rate threshold.

[0115] Step 5: Calculate the error value based on the first Euclidean distance L2 and the preset Euclidean distance threshold. Train the graph learning model based on the error value until the error value tends to a stable value. Stop training the graph learning model to obtain the trained similar object acquisition model.

[0116] Based on the error between the first Euclidean distance L2 and the preset Euclidean distance threshold, the graph learning model is continuously trained so that the final trained similar object acquisition model obtains a similarity between the target similar image and the image to be analyzed that is greater than the preset similarity threshold, and the click rate of the target similar image is greater than the preset click rate threshold.

[0117] The above describes the process of training the initial similarity object acquisition model (i.e., the graph learning model) to obtain the similarity object acquisition model.

[0118] This application provides an object analysis method, comprising: acquiring an object to be analyzed, the object to be analyzed including an image to be analyzed; inputting the object to be analyzed into a similar object acquisition model to obtain a target similar object group output by the similar object acquisition model that matches the object to be analyzed, wherein the similarity between each target similar object in the target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold; wherein, the similar object acquisition model is used to acquire a target similar object group that matches the object to be analyzed based on the multimodal feature information of the object to be analyzed, combined with an object node network graph, the object node network graph being a node relationship network graph constructed based on the correspondence between sample analysis objects and sample target similar objects, used to characterize the correspondence between analysis objects and similar objects.

[0119] In the above method, a similarity object acquisition model matches the object to be analyzed with a group of target similar objects corresponding to the object to be analyzed. Specifically, the similarity object acquisition model includes an object node network graph, which is a node relationship network graph constructed based on the correspondence between the sample object to be analyzed and the sample target similar objects, used to represent the correspondence between the object to be analyzed and similar objects. Based on the object to be analyzed and the correspondence stored in the object node network graph, the similarity between each target similar object in the acquired target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold. Therefore, by matching the target similar object group to the object to be analyzed using the above method, the matching rate between the object to be analyzed and the target similar objects is improved.

[0120] First Embodiment

[0121] The first embodiment of this application provides an object analysis method. Figure 5 A flowchart of an object analysis method provided in the first embodiment of this application is shown below. Figure 5 The method provided in this embodiment will be described in detail. The embodiments described below are for illustrative purposes only and are not intended to limit actual use.

[0122] like Figure 5 As shown, the object analysis method provided in this embodiment includes the following steps:

[0123] like Figure 5 As shown, in step S501, the object to be analyzed is obtained, and the object to be analyzed includes the image to be analyzed.

[0124] This step is used to obtain the object to be analyzed. The object to be analyzed can be a text search term, such as... Figure 3 The query in the text can also be an image provided by the user for searching. For example, when a user searches for product information on an online shopping platform, they first provide an image. The online shopping platform then identifies the image information and provides the user with the corresponding image to be analyzed.

[0125] This step obtains the object to be analyzed, which is used as the input object for the similarity object acquisition model to obtain the target similar object group that matches the object to be analyzed.

[0126] like Figure 5As shown, in step S502, the object to be analyzed is input into the similar object acquisition model to obtain a target similar object group that matches the object to be analyzed, output by the similar object acquisition model. The similarity between each target similar object in the target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold. The similar object acquisition model is used to obtain a target similar object group that matches the object to be analyzed based on the multimodal feature information of the object to be analyzed and the object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to characterize the correspondence between the analysis object and the similar object.

[0127] This step is used to obtain multiple target similar objects matching the object to be analyzed obtained in the previous step through the similar object acquisition model. The target similar object group constructed from the multiple target similar objects is used as the similar object retrieval result, and multiple target similar objects in the target similar object group are displayed.

[0128] The step of inputting the object to be analyzed into the similar object acquisition model and obtaining the target similar object group that matches the object to be analyzed, output by the similar object acquisition model, can be obtained in the following way:

[0129] The object to be analyzed is input into a similar object acquisition model to obtain multiple candidate similar objects corresponding to the object to be analyzed; based on the first Euclidean distance between the object to be analyzed and each candidate similar object, a target similar object group matching the object to be analyzed is determined from the multiple candidate similar objects, where the first Euclidean distance is used to characterize the matching degree between the object to be analyzed and the candidate similar objects.

[0130] After acquiring the information of the object to be analyzed, the similarity object acquisition model obtains multiple candidate similar objects corresponding to that information. Whether these candidate similar objects match the object to be analyzed is determined by calculating the first Euclidean distance between each candidate similar object and the object to be analyzed using Formula 1. A smaller first Euclidean distance indicates a higher degree of matching between the candidate similar object and the object to be analyzed. This matching degree has two characteristics: first, the similarity between the candidate similar object and the object to be analyzed; and second, the click-through rate of the candidate similar object. Therefore, a higher degree of matching between the candidate similar object and the object to be analyzed indicates a greater similarity and a higher click-through rate.

[0131] The step of inputting the object to be analyzed into the similar object acquisition model to obtain multiple candidate similar objects corresponding to the object to be analyzed can be achieved in the following way:

[0132] The object to be analyzed is input into a similar object acquisition model to obtain the multimodal feature information of the object to be analyzed; based on the multimodal feature information of the object to be analyzed, the multimodal feature information of multiple candidate similar objects corresponding to the multimodal feature information of the object to be analyzed is determined; based on the multimodal feature information of the multiple candidate similar objects, multiple candidate similar objects corresponding to the object to be analyzed are determined.

[0133] To improve the matching degree between the acquired target similar objects and the object to be analyzed, this application obtains the multimodal feature information of the object to be analyzed after the similar object acquisition model obtains the object to be analyzed. The multimodal feature information of the object to be analyzed is obtained by fusing the textual feature information and non-textual feature information of the object to be analyzed. Specifically, the textual feature information and non-textual feature information of the object to be analyzed are subjected to an outer product to obtain the individual textual feature information, individual non-textual feature information, and fused feature information of the object after fusing the textual and non-textual features.

[0134] By obtaining the multimodal feature information of the object to be analyzed and comparing it with the multimodal feature information of the candidate similar objects, the matching degree between the obtained candidate similar objects and the object to be analyzed can be improved.

[0135] Specifically, determining the multimodal feature information of multiple candidate similar objects corresponding to the multimodal feature information of the object to be analyzed, based on the multimodal feature information of the object to be analyzed, can be achieved through the following method:

[0136] Based on the multimodal feature information of the object to be analyzed and the pre-stored correspondence between the object to be analyzed and similar objects in the object node network graph, the multimodal feature information of multiple first candidate similar objects corresponding to the multimodal feature information of the object to be analyzed is obtained; among the multimodal feature information of the multiple first candidate similar objects, the multimodal feature information of multiple second candidate similar objects whose similarity to the multimodal feature information of the object to be analyzed is greater than a preset similarity threshold is obtained; among the multimodal feature information of the multiple second candidate similar objects, the multimodal feature information of the second candidate similar objects whose click-through rate is greater than a preset click-through rate threshold is taken as the multimodal feature information of the multiple candidate similar objects corresponding to the multimodal feature information of the object to be analyzed.

[0137] The similar object retrieval model includes an object node network graph, which is a node graph constructed based on the sample data used to train the similar object retrieval model. The connection edges between each node include: Query (user query term) - Trigger (long press to trigger product), Trigger (long press to trigger product) - Item (similar product), Item (similar product) - Item (similar product), and Item (similar product) - Cate (product category).

[0138] When using similar objects to obtain models in practice, the main methods are to determine candidate similar objects that have a direct click relationship with the object to be analyzed, or other candidate similar objects that have a similar relationship with the candidate similar objects of the object to be analyzed, based on the two types of connection edges in the object node network graph: Trigger (long press to trigger product) - Item (similar product) and Item (similar product) - Item (similar product).

[0139] Therefore, based on the multimodal feature information of the object to be analyzed and the connection relationship of each node in the object node network graph, the analysis object node that matches the multimodal feature information of the object to be analyzed is determined. Then, based on the connection relationship of each node in the object node network graph, the multimodal feature information of multiple first candidate similar objects corresponding to the multimodal feature information of the object to be analyzed is determined.

[0140] Among the multimodal feature information of multiple first candidate similar objects, candidate similar objects with similarity greater than the preset similarity threshold and click-through rate greater than the preset click-through rate threshold are selected based on the similarity threshold for the object to be analyzed and the click-through rate threshold for the candidate similar objects, and are used as candidate similar objects corresponding to the object to be analyzed.

[0141] The above is a detailed introduction to obtaining candidate similar objects corresponding to the object to be analyzed by acquiring a model through similar objects.

[0142] In addition, after obtaining candidate similar objects to the object to be analyzed, the first Euclidean distance is used to determine whether the candidate similar object is a target similar object that matches the object to be analyzed.

[0143] The step of determining a target similarity object group matching the object to be analyzed from the plurality of candidate similar objects based on the first Euclidean distance between the object to be analyzed and each candidate similar object includes:

[0144] Obtain the first Euclidean distance between the object to be analyzed and each candidate similar object; if the first Euclidean distance is less than a first preset Euclidean distance threshold, then determine the candidate similar object as a target similar object matching the object to be analyzed; construct a target similar object group matching the object to be analyzed based on the multiple target similar objects matching the object to be analyzed.

[0145] The first preset Euclidean distance threshold is determined based on the Euclidean distance between the sample analysis object and the sample target similar object, where the sample target similar object is the positive sample data that matches the sample analysis object. The Euclidean distance between the sample analysis object and the sample target similar object in a positive sample data pair is used as the first preset Euclidean distance threshold, and the Euclidean distance between the sample analysis object and the negative sample similar object in a negative sample data pair is used as the second preset Euclidean distance threshold. The first preset Euclidean distance threshold is less than the second preset Euclidean distance threshold.

[0146] The first Euclidean distance between the object to be analyzed and the candidate similar objects is compared with the first preset Euclidean distance threshold to determine whether the candidate similar objects are target similar objects that match the object to be analyzed.

[0147] The step of obtaining the first Euclidean distance between the object to be analyzed and each candidate similar object includes:

[0148] Obtain the multimodal feature vector of the object to be analyzed, and obtain the multimodal feature vector of each candidate similar object; based on the multimodal feature vector of the object to be analyzed and the multimodal feature vector of each candidate similar object, obtain the first Euclidean distance between the object to be analyzed and each candidate similar object.

[0149] Here, the object to be analyzed is the image to be analyzed, and the candidate similar objects are the candidate similar images. According to Formula 1 above, calculating the Euclidean distance between two images requires obtaining the image vectors of both images. To improve the matching degree between the object to be analyzed and the candidate similar objects, this application obtains the multimodal feature vectors of the object to be analyzed and the candidate similar objects. That is, it compares the modal features of each mode of the image to be analyzed and the candidate similar images to determine the matching degree between them.

[0150] The process of obtaining the multimodal feature vector of the object to be analyzed includes:

[0151] The textual and non-textual feature information of the object to be analyzed is obtained; the textual and non-textual feature information of the object to be analyzed is fused to obtain the multimodal feature information of the object to be analyzed, which includes individual textual feature information, individual non-textual feature information, and fused feature information of textual and non-textual features; based on the multimodal feature information of the object to be analyzed, the multimodal feature vector of the object to be analyzed is obtained through an image representation model.

[0152] The step of obtaining the multimodal feature vector of each candidate similar object includes:

[0153] The textual and non-textual feature information of each candidate similar object is obtained; the textual and non-textual feature information of each candidate similar object is fused to obtain the multimodal feature information of each candidate similar object, which includes individual textual feature information, individual non-textual feature information, and fused feature information of textual and non-textual features; based on the multimodal feature information of each candidate similar object, the multimodal feature vector of each candidate similar object is obtained through an image representation model.

[0154] Here, the image representation model obtains the multimodal feature vector of the image based on its multimodal feature information. Using the multimodal feature vectors of two images, the Euclidean distance between them is calculated to determine the matching degree between the two images. Specifically, it checks whether the similarity between the candidate similar image and the image to be analyzed is greater than a preset similarity threshold, and whether the click-through rate of the candidate similar image is greater than a preset click-through rate threshold.

[0155] This application provides an object analysis method, comprising: acquiring an object to be analyzed, the object to be analyzed including an image to be analyzed; inputting the object to be analyzed into a similar object acquisition model to obtain a target similar object group output by the similar object acquisition model that matches the object to be analyzed, wherein the similarity between each target similar object in the target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold; wherein, the similar object acquisition model is used to acquire a target similar object group that matches the object to be analyzed based on the multimodal feature information of the object to be analyzed, combined with an object node network graph, the object node network graph being a node relationship network graph constructed based on the correspondence between sample analysis objects and sample target similar objects, used to characterize the correspondence between analysis objects and similar objects.

[0156] In the above method, a similarity object acquisition model matches the object to be analyzed with a group of target similar objects corresponding to the object to be analyzed. Specifically, the similarity object acquisition model includes an object node network graph, which is a node relationship network graph constructed based on the correspondence between the sample object to be analyzed and the sample target similar objects, used to represent the correspondence between the object to be analyzed and similar objects. Based on the object to be analyzed and the correspondence stored in the object node network graph, the similarity between each target similar object in the acquired target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold. Therefore, by matching the target similar object group to the object to be analyzed using the above method, the matching rate between the object to be analyzed and the target similar objects is improved.

[0157] Second Embodiment

[0158] The first embodiment described above provides an object analysis method. Correspondingly, the second embodiment of this application also provides an object analysis apparatus. Since the apparatus embodiment is basically similar to the method embodiment, it is described simply. For details of the relevant technical features, please refer to the corresponding description of the method embodiment provided above. The following description of the apparatus embodiment is merely illustrative. Figure 6 The diagram shown is a schematic representation of an object analysis device provided in the second embodiment of this application, comprising:

[0159] The first acquisition unit 601 is used to acquire the object to be analyzed, the object to be analyzed including the image to be analyzed;

[0160] The first obtaining unit 602 is used to input the object to be analyzed into a similar object acquisition model to obtain a target similar object group that matches the object to be analyzed, output by the similar object acquisition model. The similarity between each target similar object in the target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold. The similar object acquisition model is used to obtain a target similar object group that matches the object to be analyzed based on the multimodal feature information of the object to be analyzed and combined with an object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar objects, used to characterize the correspondence between the analysis object and the similar objects.

[0161] Third Embodiment

[0162] The third embodiment of this application provides a training method for a similar object acquisition model, the specific process of which is as follows: Figure 7 As shown, it is a flowchart of a training method for a similar object acquisition model provided in the third embodiment of this application. Figure 7The training method for the similar object acquisition model shown includes steps S701 to S702.

[0163] like Figure 7 As shown, in step S701, a sample analysis object and a target similar object sample group matching the sample analysis object are obtained. The similarity between each sample target similar object in the target similar object sample group and the sample analysis object is greater than a preset similarity threshold, and the click rate of each sample target similar object is greater than a preset click rate threshold. The sample analysis object includes a sample analysis image, and the sample target similar object includes a sample target similar image matching the sample analysis image.

[0164] This step obtains sample data, which is used to train the similarity object acquisition model.

[0165] Here, the sample analysis object and the target similar object sample group that matches the sample analysis object are positive sample data pairs used to train the similar object acquisition model.

[0166] To improve the training accuracy of the similarity object model, this application trains the model using both positive and negative sample data. The positive sample data consists of the sample analysis object and a group of target similar objects matching the sample analysis object. The similarity between the target similar objects in the target similar object group and the sample analysis object is greater than a preset similarity threshold, and the click-through rate of each target similar object is greater than a preset click-through rate threshold. The Euclidean distance between the target similar object sample and the sample analysis object is a first Euclidean distance threshold, indicating that the matching degree between the target similar object sample and the sample analysis object meets a preset matching requirement.

[0167] Positive sample data pairs are used; for example, the sample analysis object is a crew neck short-sleeved shirt, and the target sample similar object is a blue crew neck short-sleeved shirt. The similarity object acquisition model is trained using positive sample data, enabling the model to identify target similar objects that match the object being analyzed.

[0168] Negative sample data refers to negative samples of similar objects whose similarity to the sample analysis object is no greater than a preset similarity threshold, or negative samples of similar objects whose click-through rate is lower than a preset click-through rate threshold. For example, if the sample analysis object is a crew neck short-sleeved shirt, the negative sample similar object is a V-neck short-sleeved shirt.

[0169] The method further includes: acquiring a sample analysis object and other similar object sample groups that do not match the sample analysis object; using the sample analysis object and other similar object sample groups that do not match the sample analysis object as negative sample data pairs for training a similar object acquisition model; and using the negative sample data pairs to train the similar object acquisition model.

[0170] By training the similarity object acquisition model with negative sample data, the model can increase the difficulty in distinguishing between the object to be analyzed and the candidate similar objects, thereby improving the matching degree of the similarity object acquisition model in identifying the target similar objects of the object to be analyzed.

[0171] Furthermore, in this embodiment of the application, to improve the matching degree between the target similar objects and the object to be analyzed in the similar object acquisition model analysis, the matching degree here includes both similarity and click-through rate factors. Therefore, when taking sample data, not only click sample data but also similar sample data are taken. The specific sampling ratio can be taken according to Formula 2 described in the scenario embodiment, which will not be repeated here.

[0172] like Figure 7 As shown, in step S702, the initial similar object acquisition model is trained based on the sample analysis object and the target similar object sample group to obtain the trained similar object acquisition model; wherein, the trained similar object acquisition model is used to obtain the target similar object group matching the object to be analyzed based on the multimodal feature information of the object to be analyzed and combined with the object node network graph, the object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to characterize the correspondence between the analysis object and the similar object.

[0173] This step is used to train the initial similar object acquisition model, so that the trained similar object acquisition model can analyze the similarity between the target similar object corresponding to the object to be analyzed and the object to be analyzed, which is greater than the preset similarity threshold, and the click rate of each target similar object is greater than the preset click rate threshold.

[0174] The trained similar object acquisition model includes an object node network graph, which is constructed based on the connection relationship between the sample analysis object and the sample target similar object. The specific construction process can be referred to the description in the scenario implementation example, and will not be repeated here.

[0175] The step of training the initial similarity object acquisition model based on the sample analysis object and the target similarity object sample group to obtain the trained similarity object acquisition model includes:

[0176] The information of the sample analysis object is input into an initial similar object acquisition model to obtain a first similar object group output by the initial similar object acquisition model for the sample analysis object; a second Euclidean distance is obtained between each first candidate similar object in the first similar object group and the sample analysis object; a first preset Euclidean distance threshold is obtained between the target similar object in the target similar object sample group and the sample analysis object; an error value is obtained between the second Euclidean distance and the first preset Euclidean distance threshold; based on the error value, the parameters of the initial similar object acquisition model are adjusted to obtain a trained similar object acquisition model.

[0177] During the training of the similarity object acquisition model, the first group of similar objects is obtained from the sample analysis based on the initial similarity object acquisition model. The difference between the first group of similar objects and the target group of similar objects is analyzed and represented by Euclidean distance.

[0178] Specifically, the error value between the second Euclidean distance and the first preset Euclidean distance threshold is compared, and the model parameters of the similar object acquisition model are adjusted based on the error value.

[0179] The step of inputting the information of the sample analysis object into the initial similar object acquisition model to obtain the first similar object group output by the initial similar object acquisition model for the sample analysis object can be determined in the following way:

[0180] The sample analysis object is input into an initial similarity object acquisition model to obtain the multimodal feature information of the sample analysis object; based on the multimodal feature information of the sample analysis object and the correspondence between the analysis object and similar objects stored in the initial object node network graph, the multimodal feature information of the first candidate similar object corresponding to the multimodal feature information of the sample analysis object is obtained; based on the obtained multimodal feature information of multiple first candidate similar objects, a first similar object sample group for the sample analysis object is constructed.

[0181] In this application, to train the initial similarity object acquisition model and improve the accuracy of the trained model in identifying target similar objects of the object to be analyzed, multimodal feature technology is incorporated to acquire multimodal feature information of the sample analysis object and multimodal feature information of the first candidate similar object for the sample analysis object. Based on the multimodal feature information of the sample analysis object, the first candidate similar object is acquired, thereby increasing the matching degree between the first candidate similar object and the sample analysis object.

[0182] In addition, obtaining the second Euclidean distance between each first candidate similar object in the first similar object group and the sample analysis object includes:

[0183] Obtain the multimodal feature vector of the first candidate similar object and the multimodal feature vector of the sample analysis object; calculate the second Euclidean distance between the first candidate similar object and the sample analysis object based on the multimodal feature vector of the first candidate similar object and the multimodal feature vector of the sample analysis object.

[0184] The second Euclidean distance between the first candidate similar object and the sample analysis object is calculated. The specific description can be calculated according to Formula 1 in the scenario embodiment. The relevant technical methods are the same as the calculation process of the first Euclidean distance in the first embodiment, and will not be repeated here.

[0185] Here, obtaining the multimodal feature vector of the first candidate similar object includes:

[0186] The textual and non-textual feature information of the first candidate similar object are obtained; the textual and non-textual feature information of the first candidate similar object are fused to obtain the multimodal feature information of the first candidate similar object, which includes individual textual feature information, individual non-textual feature information, and fused feature information of textual and non-textual features; based on the multimodal feature information of the first candidate similar object, the multimodal feature vector of the first candidate similar object is obtained through an image representation model.

[0187] The process of obtaining the multimodal feature vector of the sample analysis object includes:

[0188] The textual and non-textual feature information of the sample analysis object is obtained; the textual and non-textual feature information of the sample analysis object is fused to obtain the multimodal feature information of the sample analysis object, which includes individual textual feature information, individual non-textual feature information, and fused feature information of textual and non-textual features; based on the multimodal feature information of the sample analysis object, the multimodal feature vector of the sample analysis object is obtained through an image representation model.

[0189] The above describes how the information of the sample analysis object is input into the initial similar object acquisition model to obtain the description of the first similar object group output by the initial similar object acquisition model for the sample analysis object.

[0190] Furthermore, after obtaining the first group of similar objects for the sample analysis object, a second Euclidean distance is calculated between the sample analysis object and the first candidate similar object in the first group of similar objects. Based on the error value between the second Euclidean distance and the first preset Euclidean distance threshold, the parameters of the similar object acquisition model are adjusted.

[0191] The step of adjusting the parameters of the initial similarity object acquisition model based on the error value to obtain the trained similarity object acquisition model includes:

[0192] If the error value is greater than a preset error value threshold, the step of inputting the information of the sample analysis object into the initial similar object acquisition model to obtain the first similar object group output by the initial similar object acquisition model for the sample analysis object continues; if the error value is less than the preset error value threshold, the parameter adjustment of the similar object acquisition model is stopped to obtain the trained similar object acquisition model.

[0193] In this embodiment, the error value is used to determine whether to continue training the similar object acquisition model. It also includes updating the correspondence between the analyzed objects and similar objects stored in the initial object node network graph based on the error value.

[0194] If the error value here is greater than the preset error value threshold, it means that the similarity or click-through rate between the first similar object sample group obtained by the similar object acquisition model and the target similar object sample group is too low, and the model parameters of the similar object acquisition model need to be further trained.

[0195] The training continues until the error value is less than the preset error value threshold. This indicates that the model parameters of the similar object acquisition model can meet the requirements, so that the matching degree between the first similar object in the first similar object group acquired by the trained similar object acquisition model and the sample analysis object meets the preset matching degree condition. That is, the similarity between the first candidate similar object and the sample analysis object is greater than the preset similarity threshold, and the click rate of the first candidate similar object is greater than the preset click rate threshold.

[0196] This application provides a training method for a similar object acquisition model, comprising: acquiring a sample analysis object and a target similar object sample group matching the sample analysis object, wherein the similarity between each sample target similar object in the target similar object sample group and the sample analysis object is greater than a preset similarity threshold, and the click-through rate of each sample target similar object is greater than a preset click-through rate threshold; the sample analysis object includes a sample analysis image, and the sample target similar objects include sample target similar images matching the sample analysis image; training an initial similar object acquisition model based on the sample analysis object and the target similar object sample group to obtain a trained similar object acquisition model; wherein the trained similar object acquisition model is used to acquire a target similar object group matching the object to be analyzed based on the multimodal feature information of the object to be analyzed, combined with an object node network graph, wherein the object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar objects, used to characterize the correspondence between the analysis object and similar objects.

[0197] In the above method, a similarity object acquisition model is trained using a sample analysis object and a target similarity object sample group that matches the sample analysis object. The trained similarity object acquisition model can acquire a target similarity object group that matches the object to be analyzed based on the multimodal feature information of the object to be analyzed and the object node network graph. Specifically, the similarity rate between each similar object in the target similarity object group and the object to be analyzed is greater than a preset similarity rate threshold, and the click-through rate of each similar object is greater than a preset click-through rate threshold. The similarity object acquisition model trained using the above method improves the matching rate between the object to be analyzed and the target similarity object group during the matching process.

[0198] Fourth embodiment

[0199] The third embodiment described above provides a training method for a similar object acquisition model. Correspondingly, the fourth embodiment of this application also provides a training device for a similar object acquisition model. Since the device embodiment is basically similar to the method embodiment, it is described simply. For details of the relevant technical features, please refer to the corresponding description of the method embodiment provided above. The following description of the device embodiment is merely illustrative. Figure 8 The diagram shown is a schematic of a training apparatus for a similar object acquisition model provided in the fourth embodiment of this application, comprising:

[0200] The third acquisition unit 801 is used to acquire a sample analysis object and a target similar object sample group that matches the sample analysis object. The similarity between each sample target similar object in the target similar object sample group and the sample analysis object is greater than a preset similarity threshold, and the click rate of each sample target similar object is greater than a preset click rate threshold. The sample analysis object includes a sample analysis image, and the sample target similar object includes a sample target similar image that matches the sample analysis image.

[0201] The similar object acquisition model obtaining unit 802 is used to train an initial similar object acquisition model based on the sample analysis object and the target similar object sample group to obtain a trained similar object acquisition model; wherein, the trained similar object acquisition model is used to obtain a target similar object group matching the object to be analyzed based on the multimodal feature information of the object to be analyzed and combined with the object node network graph, wherein the object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to characterize the correspondence between the analysis object and similar objects.

[0202] Fifth embodiment

[0203] The fifth embodiment of this application provides a similar image recommendation method, the specific process of which is as follows: Figure 9 As shown, it is a flowchart of a training method for a similar object acquisition model provided in the third embodiment of this application. Figure 9 The similar image recommendation method shown includes steps S901 to S904.

[0204] like Figure 9 As shown, in step S901, the image to be analyzed is acquired.

[0205] The acquisition of the image to be analyzed includes:

[0206] A second request message is obtained for requesting to acquire an image to be analyzed, the second request message including descriptive information for the image to be analyzed; a first candidate image group for the image to be analyzed is obtained according to the second request message; in response to detecting a selection operation for at least one candidate image in the first candidate image group, the selected candidate image is taken as the image to be analyzed.

[0207] This embodiment applies to a user terminal. The online service platform page of the user terminal displays image information that the user wants to query. Obtaining the image to be analyzed can be achieved by the user entering product information into the search box on the user terminal's display page, and the system matching the corresponding image information based on the product information. Alternatively, the user can provide image information to be queried, and the user terminal, after recognizing the image information, matches the corresponding image information for the user.

[0208] The process of obtaining a second request message for requesting the image to be analyzed may include the following: displaying a filled area on a first search page for obtaining descriptive information about the image to be analyzed; obtaining the second request message for requesting the image to be analyzed includes: in response to detecting a trigger operation on the filled area, obtaining a second request message for requesting the image to be analyzed, wherein the second request message includes descriptive information about the image to be analyzed.

[0209] The first search page is as follows Figure 2 On page 2a, users can enter text information to be searched or provide image information to be searched on the first search page of the user terminal.

[0210] The user terminal receives the second request message provided by the user, matches at least one candidate image for the user based on the description information of the image to be analyzed, and displays the obtained at least one candidate image on the first search results page, such as... Figure 2 Page 2b. Based on the user's selection of at least one candidate image in the first search results page, the selected candidate image is determined as the image to be analyzed.

[0211] like Figure 9 As shown, in step S902, a first request message is obtained for requesting to obtain a group of target similar images that match the image to be analyzed.

[0212] like Figure 9 As shown, in step S903, at least one target similar image matching the image to be analyzed is obtained according to the first request message. Based on the at least one target similar image matching the image to be analyzed, a target similar image group matching the image to be analyzed is constructed. The similarity between each target similar image in the target similar image group and the image to be analyzed is greater than a preset similarity threshold, and the click rate of each target similar image is greater than a preset click rate threshold.

[0213] This step is used to obtain target similar images that match the image to be analyzed. After the client receives the user's first request message, it sends the first request message to the server. The server then obtains the target similar object group that matches the object to be analyzed based on the similar object acquisition model.

[0214] Each target similar image in the target similar image group is obtained by the server in the following way: by obtaining a target similar object group matching the image to be analyzed through a similar object acquisition model based on the multimodal feature information of the image to be analyzed and combined with the object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to represent the correspondence between the analysis object and the similar object.

[0215] like Figure 9 As shown, in step S904, target similar images in the target similar image group are displayed.

[0216] This step displays each target-similar image in the obtained target-similar image group on the user's terminal page, such as... Figure 2 Page 2c. On this page, the similarity between the target similar image and the image to be analyzed is greater than a preset similarity threshold, and the click-through rate of the target similar object is greater than a preset click-through rate threshold.

[0217] Sixth Embodiment

[0218] The fifth embodiment described above provides a method for recommending similar images. Correspondingly, the sixth embodiment of this application also provides a device for recommending similar images. Since the device embodiment is basically similar to the method embodiment, it is described simply. For details of the relevant technical features, please refer to the corresponding description of the method embodiment provided above. The following description of the device embodiment is merely illustrative. Figure 10 The diagram shown is a schematic representation of a similar image recommendation device provided in the sixth embodiment of this application, comprising:

[0219] The second acquisition unit 1001 is used to acquire the image to be analyzed;

[0220] The first request message obtaining unit 1002 is used to obtain a first request message for requesting to obtain a group of target similar images that match the image to be analyzed;

[0221] The target similar image group construction unit 1003 is used to obtain at least one target similar image that matches the image to be analyzed according to the first request message, and construct a target similar image group that matches the image to be analyzed according to the at least one target similar image that matches the image to be analyzed, wherein the similarity between each target similar image in the target similar image group and the image to be analyzed is greater than a preset similarity threshold, and the click rate of each target similar image is greater than a preset click rate threshold;

[0222] Display unit 1004 is used to display target similar images in the target similar image group;

[0223] Each target similar image in the target similar image group is obtained by the server in the following way: by obtaining a target similar object group matching the image to be analyzed through a similar object acquisition model based on the multimodal feature information of the image to be analyzed and combined with the object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to represent the correspondence between the analysis object and the similar object.

[0224] Seventh Embodiment

[0225] Corresponding to the methods of the first, third, and fifth embodiments of this application, the seventh embodiment of this application also provides an electronic device. For example... Figure 11 As shown, Figure 11 This is a schematic diagram of an electronic device provided in the seventh embodiment of this application. The electronic device includes: at least one processor 1101, at least one communication interface 1102, at least one memory 1103, and at least one communication bus 1104. Optionally, the communication interface 1102 can be an interface for a communication module, such as the interface for a GSM module. The processor 1101 may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The memory 1103 may include high-speed RAM and may also include non-volatile memory, such as at least one disk storage device. The memory 1103 stores a program, and the processor 1101 calls the program stored in the memory 1103 to execute the methods of the first, third, and fifth embodiments of the present invention.

[0226] Eighth embodiment

[0227] Corresponding to the methods of the first, third, and fifth embodiments of this application, the eighth embodiment of this application also provides a computer storage medium. The computer storage medium stores a computer program, which is executed by a processor to perform the methods of the first, third, and fifth embodiments.

[0228] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0229] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, a network interface, and memory. Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0230] 1. Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0231] 2. Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. An object analysis method, characterized in that, include: Obtain the object to be analyzed, which includes the image to be analyzed; The object to be analyzed is input into the similar object acquisition model to obtain the target similar object group that matches the object to be analyzed. The similarity between each target similar object in the target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click rate of each target similar object is greater than a preset click rate threshold. The similar object acquisition model is used to obtain a group of target similar objects that match the object to be analyzed based on the multimodal feature information of the object to be analyzed and in combination with the object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar objects, used to represent the correspondence between the analysis object and the similar objects. The connection edges between each object node in the object node network graph include direct click connection edges and similar connection edges. The similar object acquisition model is trained based on click sample data and similar sample data obtained from user historical behavior log data. The target similar object group obtained by the similar object acquisition model in the application process has both click rate and similarity features.

2. The method according to claim 1, characterized in that, The step of inputting the object to be analyzed into the similar object acquisition model and obtaining the target similar object group that matches the object to be analyzed, output by the similar object acquisition model, includes: The object to be analyzed is input into the similar object acquisition model to obtain multiple candidate similar objects corresponding to the object to be analyzed. Based on the first Euclidean distance between the object to be analyzed and each candidate similar object, a target similar object group matching the object to be analyzed is determined from the plurality of candidate similar objects. The first Euclidean distance is used to characterize the matching degree between the object to be analyzed and the candidate similar objects.

3. The method according to claim 2, characterized in that, The step of inputting the object to be analyzed into the similarity object acquisition model to obtain multiple candidate similar objects corresponding to the object to be analyzed includes: The object to be analyzed is input into the similar object acquisition model to obtain the multimodal feature information of the object to be analyzed; Based on the multimodal feature information of the object to be analyzed, determine the multimodal feature information of multiple candidate similar objects corresponding to the multimodal feature information of the object to be analyzed; Based on the multimodal feature information of the multiple candidate similar objects, multiple candidate similar objects corresponding to the object to be analyzed are determined.

4. The method according to claim 2, characterized in that, The step of determining a target similarity object group matching the object to be analyzed from the plurality of candidate similar objects based on the first Euclidean distance between the object to be analyzed and each candidate similar object includes: Obtain the first Euclidean distance between the object to be analyzed and each candidate similar object; If the first Euclidean distance is less than the first preset Euclidean distance threshold, then the candidate similar object is determined to be a target similar object that matches the object to be analyzed; Based on the multiple target similar objects that match the object to be analyzed, a group of target similar objects that match the object to be analyzed is constructed.

5. The method according to claim 4, characterized in that, Obtaining the first Euclidean distance between the object to be analyzed and each candidate similar object includes: Obtain the multimodal feature vector of the object to be analyzed, and obtain the multimodal feature vector of each candidate similar object; Based on the multimodal feature vector of the object to be analyzed and the multimodal feature vector of each candidate similar object, the first Euclidean distance between the object to be analyzed and each candidate similar object is obtained.

6. The method according to claim 5, characterized in that, The process of obtaining the multimodal feature vector of the object to be analyzed includes: Obtain the textual and non-textual feature information of the object to be analyzed; The text feature information and non-text feature information of the object to be analyzed are fused to obtain the multimodal feature information of the object to be analyzed. The multimodal feature information of the object to be analyzed includes the individual text feature information, the individual non-text feature information, and the fused feature information of the text feature and non-text feature. Based on the multimodal feature information of the object to be analyzed, the multimodal feature vector of the object to be analyzed is obtained through an image representation model.

7. The method according to claim 5, characterized in that, The step of obtaining the multimodal feature vector of each candidate similar object includes: Obtain the textual and non-textual feature information of each candidate similar object; The text feature information and non-text feature information of each candidate similar object are fused to obtain the multimodal feature information of each candidate similar object. The multimodal feature information of each candidate similar object includes the individual text feature information, the individual non-text feature information, and the fused feature information of the text feature and non-text feature. Based on the multimodal feature information of each candidate similar object, the multimodal feature vector of each candidate similar object is obtained through an image representation model.

8. The method according to claim 3, characterized in that, The step of determining the multimodal feature information of multiple candidate similar objects corresponding to the multimodal feature information of the object to be analyzed, based on the multimodal feature information of the object to be analyzed, includes: Based on the multimodal feature information of the object to be analyzed and the correspondence between the object to be analyzed and similar objects pre-stored in the object node network graph, the multimodal feature information of multiple first candidate similar objects corresponding to the multimodal feature information of the object to be analyzed is obtained; Among the multimodal feature information of the plurality of first candidate similar objects, the multimodal feature information of a plurality of second candidate similar objects with a similarity greater than a preset similarity threshold to the multimodal feature information of the object to be analyzed is obtained; Among the multimodal feature information of the plurality of second candidate similar objects, the multimodal feature information of the second candidate similar objects whose click rate is greater than a preset click rate threshold is taken as the multimodal feature information of the plurality of candidate similar objects corresponding to the multimodal feature information of the object to be analyzed.

9. A method for recommending similar images, characterized in that, Applied to a user terminal, the method includes: Acquire the image to be analyzed; Obtain a first request message for requesting to obtain a group of target similar images that match the image to be analyzed; According to the first request message, at least one target similar image matching the image to be analyzed is obtained. Based on the at least one target similar image matching the image to be analyzed, a target similar image group matching the image to be analyzed is constructed. The similarity between each target similar image in the target similar image group and the image to be analyzed is greater than a preset similarity threshold, and the click rate of each target similar image is greater than a preset click rate threshold. Display the target-similar images in the target-similar image group; Each target similar image in the target similar image group is obtained by the server in the following way: the target similar object group matching the image to be analyzed is obtained by the similar object acquisition model based on the multimodal feature information of the image to be analyzed and combined with the object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to represent the correspondence between the analysis object and the similar object. The connection edges between each object node in the object node network graph include direct click connection edges and similar connection edges. The similar object acquisition model is trained based on click sample data and similar sample data obtained from the user's historical behavior log data. The target similar object group obtained by the similar object acquisition model in the application process has both click rate and similarity features.

10. The method according to claim 9, characterized in that, The acquisition of the image to be analyzed includes: A second request message is obtained for requesting the acquisition of an image to be analyzed, the second request message including descriptive information for the image to be analyzed; Based on the second request message, a first candidate image group is obtained for the image to be analyzed; In response to detecting a selection operation for at least one candidate image in the first candidate image group, the selected candidate image is taken as the image to be analyzed.

11. The method according to claim 10, characterized in that, Also includes: The first search page displays a filled area for obtaining descriptive information for the image to be analyzed; The process of obtaining a second request message for requesting the image to be analyzed includes: In response to detecting a trigger operation for the filled region, a second request message is obtained for requesting to obtain the image to be analyzed, the second request message including descriptive information for the image to be analyzed.

12. A training method for a similar object acquisition model, characterized in that, include: Obtain a sample analysis object and a target similar object sample group that matches the sample analysis object. The similarity between each target similar object in the target similar object sample group and the sample analysis object is greater than a preset similarity threshold, and the click rate of each target similar object is greater than a preset click rate threshold. The sample analysis object includes a sample analysis image, and the target similar object includes a target similar image that matches the sample analysis image. Based on the sample analysis object and the target similar object sample group, the initial similar object acquisition model is trained to obtain the trained similar object acquisition model; The trained similar object acquisition model is used to obtain a group of target similar objects that match the object to be analyzed, based on the multimodal feature information of the object to be analyzed and combined with the object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar objects, used to represent the correspondence between the analysis object and similar objects. The connection edges between each object node in the object node network graph include direct click connection edges and similarity connection edges. The similar object acquisition model is trained based on click sample data and similar sample data obtained from user historical behavior log data. The target similar object group obtained by the similar object acquisition model in the application process has both click rate and similarity features.

13. The method according to claim 12, characterized in that, The step of training the initial similarity object acquisition model based on the sample analysis object and the target similar object sample group to obtain the trained similarity object acquisition model includes: The information of the sample analysis object is input into the initial similar object acquisition model to obtain the first similar object group output by the initial similar object acquisition model for the sample analysis object; Obtain the second Euclidean distance between each first candidate similar object in the first similar object group and the sample analysis object; Obtain a first preset Euclidean distance threshold between the target similar object in the target similar object sample group and the sample analysis object; Obtain the error value between the second Euclidean distance and the first preset Euclidean distance threshold; Based on the error value, the parameters of the initial similar object acquisition model are adjusted to obtain the trained similar object acquisition model.

14. The method according to claim 13, characterized in that, The step of inputting the information of the sample analysis object into an initial similarity object acquisition model to obtain the first similarity object group output by the initial similarity object acquisition model for the sample analysis object includes: The sample analysis object is input into the initial similarity object acquisition model to obtain the multimodal feature information of the sample analysis object; Based on the multimodal feature information of the sample analysis object and the correspondence between the analysis object and similar objects stored in the initial object node network graph, the multimodal feature information of the first candidate similar object corresponding to the multimodal feature information of the sample analysis object is obtained; Based on the multimodal feature information of multiple first candidate similar objects obtained, a first similar object sample group is constructed for the sample analysis object.

15. The method according to claim 13, characterized in that, Obtaining the second Euclidean distance between each first candidate similar object in the first similar object group and the sample analysis object includes: Obtain the multimodal feature vector of the first candidate similar object, and obtain the multimodal feature vector of the sample analysis object; The second Euclidean distance between the first candidate similar object and the sample analysis object is calculated based on the multimodal feature vector of the first candidate similar object and the multimodal feature vector of the sample analysis object.

16. The method according to claim 13, characterized in that, The step of adjusting the parameters of the initial similarity object acquisition model based on the error value to obtain the trained similarity object acquisition model includes: If the error value is greater than the preset error value threshold, then continue to execute the step of inputting the information of the sample analysis object into the initial similar object acquisition model to obtain the first similar object group output by the initial similar object acquisition model for the sample analysis object; If the error value is less than the preset error value threshold, then stop adjusting the parameters of the similar object acquisition model and obtain the trained similar object acquisition model.

17. The method according to claim 14, characterized in that, Also includes: Based on the error value, the correspondence between the analyzed objects and similar objects stored in the initial object node network graph is updated.

18. The method according to claim 12, characterized in that, The sample analysis object and the target similar object sample group matching the sample analysis object are positive sample data pairs used to train the similar object acquisition model. The method further includes: Obtain the sample analysis object and other similar object sample groups that do not match the sample analysis object; The sample analysis object and other similar object sample groups that do not match the sample analysis object are used as negative sample data pairs for training the similar object acquisition model; The similarity object acquisition model is trained using the negative sample data pairs.

19. The method according to claim 15, characterized in that, The step of obtaining the multimodal feature vector of the first candidate similar object includes: Obtain the textual and non-textual feature information of the first candidate similar object; The text feature information and non-text feature information of the first candidate similar object are fused to obtain the multimodal feature information of the first candidate similar object. The multimodal feature information of the first candidate similar object includes the individual text feature information, the individual non-text feature information, and the fused feature information of the text feature and non-text feature. Based on the multimodal feature information of the first candidate similar object, the multimodal feature vector of the first candidate similar object is obtained through an image representation model.

20. The method according to claim 15, characterized in that, The process of obtaining the multimodal feature vector of the sample analysis object includes: Obtain the textual and non-textual feature information of the sample analysis object; The textual and non-textual feature information of the sample analysis object are fused to obtain the multimodal feature information of the sample analysis object. The multimodal feature information of the sample analysis object includes the individual textual feature information, the individual non-textual feature information, and the fused feature information of the textual and non-textual features. Based on the multimodal feature information of the sample analysis object, the multimodal feature vector of the sample analysis object is obtained through an image representation model.

21. An object analysis device, characterized in that, include: The first acquisition unit is used to acquire the object to be analyzed, the object to be analyzed including the image to be analyzed; The first obtaining unit is used to input the object to be analyzed into the similar object acquisition model to obtain a target similar object group that matches the object to be analyzed, output by the similar object acquisition model. The similarity between each target similar object in the target similar object group and the object to be analyzed is greater than a preset similarity threshold, and the click rate of each target similar object is greater than a preset click rate threshold. The similar object acquisition model is used to obtain a group of target similar objects that match the object to be analyzed based on the multimodal feature information of the object to be analyzed and in combination with the object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar objects, used to represent the correspondence between the analysis object and the similar objects. The connection edges between each object node in the object node network graph include direct click connection edges and similar connection edges. The similar object acquisition model is trained based on click sample data and similar sample data obtained from user historical behavior log data. The target similar object group obtained by the similar object acquisition model in the application process has both click rate and similarity features.

22. A similar image recommendation device, characterized in that, The device, applied to a user terminal, includes: The second acquisition unit is used to acquire the image to be analyzed; The first request message obtaining unit is used to obtain a first request message for requesting to obtain a group of target similar images that match the image to be analyzed; The target similar image group construction unit is configured to obtain at least one target similar image that matches the image to be analyzed according to the first request message, and construct a target similar image group that matches the image to be analyzed according to the at least one target similar image that matches the image to be analyzed, wherein the similarity between each target similar image in the target similar image group and the image to be analyzed is greater than a preset similarity threshold, and the click rate of each target similar image is greater than a preset click rate threshold. The display unit is used to display target-similar images in the target-similar image group; Each target similar image in the target similar image group is obtained by the server in the following way: the target similar object group matching the image to be analyzed is obtained by the similar object acquisition model based on the multimodal feature information of the image to be analyzed and combined with the object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar object, used to represent the correspondence between the analysis object and the similar object. The connection edges between each object node in the object node network graph include direct click connection edges and similar connection edges. The similar object acquisition model is trained based on click sample data and similar sample data obtained from the user's historical behavior log data. The target similar object group obtained by the similar object acquisition model in the application process has both click rate and similarity features.

23. A training device for a similar object acquisition model, characterized in that, include: The third acquisition unit is used to acquire a sample analysis object and a target similar object sample group that matches the sample analysis object. The similarity between each target similar object in the target similar object sample group and the sample analysis object is greater than a preset similarity threshold, and the click rate of each target similar object is greater than a preset click rate threshold. The sample analysis object includes a sample analysis image, and the target similar object includes a target similar image that matches the sample analysis image. The similar object acquisition model obtaining unit is used to train the initial similar object acquisition model based on the sample analysis object and the target similar object sample group to obtain the trained similar object acquisition model. The trained similar object acquisition model is used to obtain a group of target similar objects that match the object to be analyzed, based on the multimodal feature information of the object to be analyzed and combined with the object node network graph. The object node network graph is a node relationship network graph constructed based on the correspondence between the sample analysis object and the sample target similar objects, used to represent the correspondence between the analysis object and similar objects. The connection edges between each object node in the object node network graph include direct click connection edges and similarity connection edges. The similar object acquisition model is trained based on click sample data and similar sample data obtained from user historical behavior log data. The target similar object group obtained by the similar object acquisition model in the application process has both click rate and similarity features.

24. An electronic device, characterized in that, The electronic device includes a processor and a memory; The memory stores a computer program, and after the processor runs the computer program, it executes the method described in any one of claims 1-20.

25. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, performs the method described in any one of claims 1-20.

Images