Method and apparatus for generating object combination, and electronic device and readable medium

Abstract

Disclosed in the present application are a method and apparatus for generating an object combination, and an electronic device and a readable medium. The method comprises: acquiring reference combination object information, which comprises object attribute information respectively corresponding to a plurality of reference objects; for each reference object, on the basis of the object attribute information corresponding to the reference object, determining from a candidate object set a target candidate object corresponding to the reference object, wherein the degree of matching between object attribute information of the target candidate object and the object attribute information corresponding to the reference object meets a preset condition; and using target candidate objects respectively corresponding to the plurality of reference objects as an object combination.

Description

Method, device, electronic device and readable medium for generating object combination

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 27, 2023, with application number 202311265365.2 and invention name “A method, device, electronic device and readable medium for generating object combinations”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a method, device, electronic device, and readable medium for generating an object combination. Background Art

[0004] With the development of Internet technology, people can operate online objects through online behaviors, and staff can also better display objects to people through editing, management, etc. For example, online store staff can display the products and / or services in the online store through editing, management, etc., and users can browse, collect, or purchase the products and / or services in the online store.

[0005] Currently, staff can manually combine multiple objects on the line into object combinations on an online management platform (hereinafter referred to as the management platform), and display the object combinations to people, who can browse, view, and even purchase the object combinations online.

[0006] However, the manual matching of object combinations by staff is not convenient enough for the staff, and the matched object combinations are also of varying quality. The purpose of optimizing online marketing cannot be achieved by matching object combinations.

[0007] Summary of the Invention

[0008] In order to solve the above technical problems, the present application provides a method, device, electronic device and readable medium for generating object combinations, which can automatically and intelligently generate object combinations, which not only facilitates the staff, but also ensures the quality of the automatically generated object combinations, thereby making it possible to optimize online marketing situations.

[0009] In order to achieve the above objectives, the technical solutions provided by this application are as follows:

[0010] In a first aspect, the present application provides a method for generating an object combination, comprising:

[0011] Acquire reference combination object information, where the reference combination object information includes object attribute information corresponding to each of a plurality of reference objects;

[0012] For each reference object, determining a target candidate object corresponding to the reference object from a set of candidate objects based on the object attribute information corresponding to the reference object; wherein the degree of matching between the object attribute information of the target candidate object and the object attribute information corresponding to the reference object satisfies a preset condition;

[0013] The target candidate objects corresponding to each of the plurality of reference objects are respectively regarded as an object combination.

[0014] In a possible implementation, obtaining the reference combination object information includes:

[0015] identifying, from a collocation main image, a plurality of reference objects included in the collocation main image;

[0016] According to the sub-picture corresponding to each reference object in the collocation main picture, object attribute information of the corresponding reference object is obtained.

[0017] As an example, obtaining object attribute information of each reference object among the multiple reference objects according to a sub-image corresponding to the collocation main image includes:

[0018] The sub-image corresponding to each reference object in the collocation main image is input into the feature detection model to obtain object attribute information of the corresponding reference object.

[0019] In a possible implementation, determining the target candidate object corresponding to the reference object from a candidate object set according to the object attribute information corresponding to the reference object includes:

[0020] From a preset retrieval library, determine the object attribute information whose matching degree with the object attribute information corresponding to the reference object satisfies the preset conditions, and record the candidate object corresponding to the determined object attribute information as the target candidate object, the retrieval library includes a mapping relationship between each candidate object in a plurality of candidate objects and the corresponding object attribute information, and the candidate object set includes the plurality of candidate objects involved in the retrieval library.

[0021] In a possible implementation, the process of constructing the preset search library includes:

[0022] Obtaining a target image for each candidate object, wherein the target image includes only one candidate object;

[0023] Based on the feature detection model, obtaining object attribute information corresponding to each of the target images;

[0024] The mapping relationship between each candidate object and the corresponding object attribute information is saved accordingly to obtain the search library.

[0025] In one possible implementation, the feature detection model training process includes:

[0026] Based on the images in the training sample, the initial model is pre-trained using a self-supervised method to obtain the first model;

[0027] Based on the images in the training samples and the classification labels corresponding to the images, the first model is trained in a clustering manner to obtain a second model;

[0028] Based on the two pictures in the training sample and the sample attributes of the two pictures, the second model is trained in two stages at least once using a binary classification method, and the second model obtained from the last two-stage training is recorded as the feature detection model.

[0029] In a possible implementation, the object combination includes a first combination of reference objects included in the collocation main image, and the method further includes:

[0030] Corresponding anchor points are added to preset positions of the areas where the reference objects in the matching main image are located to obtain the combined image of the first combination, and the anchor points are used to associate preset attribute information of the corresponding reference objects.

[0031] In a second aspect, the present application further provides a device for generating an object combination, comprising:

[0032] An acquiring unit, configured to acquire reference combination object information; wherein the reference combination object information includes object attribute information corresponding to each of a plurality of reference objects;

[0033] a matching unit configured to determine, for each reference object, a target candidate object corresponding to the reference object from a set of candidate objects based on object attribute information corresponding to the reference object; wherein a degree of matching between the object attribute information of the target candidate object and the object attribute information corresponding to the reference object satisfies a preset condition;

[0034] The generating unit is configured to combine the target candidate objects corresponding to the plurality of reference objects into an object combination.

[0035] In a possible implementation, the acquiring unit includes:

[0036] an identification subunit, configured to identify, from a collocation main image, a plurality of reference objects included in the collocation main image;

[0037] The first obtaining subunit is configured to obtain object attribute information of each reference object among the multiple reference objects according to a sub-picture corresponding to the reference object in the matching main picture.

[0038] As an example, the first obtaining subunit is specifically configured to:

[0039] The sub-image corresponding to each reference object in the collocation main image is input into the feature detection model to obtain object attribute information of the corresponding reference object.

[0040] In a possible implementation, the matching unit is specifically configured to:

[0041] From a preset retrieval library, determine the object attribute information whose matching degree with the object attribute information corresponding to the reference object satisfies the preset conditions, and record the candidate object corresponding to the determined object attribute information as the target candidate object, the retrieval library includes a mapping relationship between each candidate object in a plurality of candidate objects and the corresponding object attribute information, and the candidate object set includes the plurality of candidate objects involved in the retrieval library.

[0042] In a possible implementation, the device further includes: a construction unit;

[0043] The construction unit is used to construct a preset retrieval library.

[0044] As an example, the building block includes:

[0045] A second obtaining subunit is configured to obtain a target image for each candidate object, wherein the target image includes only one candidate object;

[0046] A third obtaining subunit is configured to obtain object attribute information corresponding to each of the target images based on the feature detection model;

[0047] The saving subunit is used to save the mapping relationship between each candidate object and the corresponding object attribute information to obtain the search library.

[0048] In a possible implementation, the apparatus further includes: a training unit;

[0049] The training unit is used to train the feature detection model.

[0050] As an example, the training unit includes:

[0051] A first training subunit is configured to pre-train the initial model in a self-supervised manner based on images in the training sample to obtain a first model;

[0052] A second training subunit is configured to perform a one-stage training on the first model in a clustering manner based on the images in the training samples and the classification labels corresponding to the images to obtain a second model;

[0053] The third training subunit is used to perform at least one two-stage training on the second model using a binary classification method based on the two pictures in the training sample and the sample attributes of the two pictures, and record the second model obtained from the last two-stage training as the feature detection model.

[0054] In a possible implementation, the object combination includes a first combination of reference objects included in the collocation main image, and the apparatus further includes: an anchor unit;

[0055] The anchor point unit is used to add corresponding anchor points at preset positions of the areas where each reference object in the matching main image is located to obtain the combined image of the first combination, and the anchor points are used to associate preset attribute information of the corresponding reference objects.

[0056] It should be noted that the specific implementation method of the device and the technical effects achieved can be found in the relevant description of the method provided in the first aspect or any implementation method of the first aspect.

[0057] In a third aspect, the present application further provides an electronic device, the electronic device comprising: a processor and a memory;

[0058] The memory is used to store instructions or programs;

[0059] The processor is used to execute the instructions or programs in the memory so that the electronic device executes the method provided by the first aspect or any one of the implementations of the first aspect.

[0060] In a fourth aspect, the present application also provides a readable medium, which stores instructions or programs. When the instructions or programs are run on a processor, the processor executes the method provided in the first aspect or any one of the implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0062] FIG1 is a flow chart of a method for generating an object combination provided in an embodiment of the present application;

[0063] FIG2 is a schematic diagram of an object detection method according to an embodiment of the present application;

[0064] FIG3 is a schematic diagram of a feature detection model provided in an embodiment of the present application;

[0065] FIG4 is a schematic diagram of a training process of a feature detection model provided in an embodiment of the present application;

[0066] FIG5 is a schematic diagram of a process for generating an object combination based on a graph recognition algorithm according to an embodiment of the present application;

[0067] FIG6 is a schematic diagram of another process for generating object combinations based on a graph recognition algorithm according to an embodiment of the present application;

[0068] FIG7 is a schematic diagram of determining an object combination anchor point in a matching main image according to an embodiment of the present application;

[0069] FIG8 is a schematic structural diagram of an apparatus 800 for generating an object combination provided in an embodiment of the present application;

[0070] FIG9 is a schematic structural diagram of an electronic device 900 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In some cases, users want to view and purchase multiple objects with specific relationships through online entities. To facilitate this, online entities often display multiple objects adjacent to each other or as a group. Displaying multiple objects as a group can be understood as combining multiple objects. For example, an online store might combine nursing pads, postpartum caps, and postpartum shoes into a maternity bag. Another example is combining shirts and trousers into a commuter suit.

[0072] Currently, online store staff typically manually enter pre-matched object combinations on the online store's operating platform based on their own experience or sales data for each object. For example, an online store's operating platform has a "product matching" function. Staff members enter the editing page through "Create New Matching" and create matching combinations on the editing page by uploading matching main images, adding products, and other operations. Once the matching combinations pass review, they can be published and put on the shelves for users to view and / or purchase. Although this process can facilitate users to view and / or purchase online, it is not very user-friendly for online store staff. On the one hand, staff members need to select the objects they want to match and perform complex editing operations. On the other hand, the quality of the object combinations generated by the staff's manual object matching cannot be guaranteed. In many cases, the object combinations provided cannot meet the needs of users. Therefore, the current object matching method is time-consuming and labor-intensive and has little effect.

[0073] Based on this, an embodiment of the present application provides a method for generating object combinations. Taking the management platform of an online entity as an example, an automatic matching function is integrated in the background of the management platform, so that the management platform can provide the automatic matching function for the staff of the online entity. Taking the automatic matching needs of the staff of the online entity as an example, the device for generating object combinations can obtain reference combination object information, and the reference combination object information includes object attribute information corresponding to each of multiple reference objects; for each reference object, according to the object attribute information corresponding to the reference object, the target candidate object corresponding to the reference object is determined from the candidate object set, wherein the degree of matching between the object attribute information of the target candidate object and the object attribute information corresponding to the reference object meets the preset conditions; then, the target candidate objects corresponding to each of the multiple reference objects are taken as an object combination.

[0074] In this way, there is no need for staff to manually match objects. Due to the automatic matching function integrated in the background of the management platform, the staff only needs to use the automatic matching function, and the management platform can use the multiple reference objects included in the known matching as the basis for matching, and match at least one target candidate object corresponding to each reference object based on the object attribute information of the reference object and the object attribute information of the candidate object in the candidate object set, so as to generate the combination of target candidate objects corresponding to different reference objects as an object combination, thereby achieving the purpose of automatically generating object combinations. This not only facilitates the staff, but also because the known matching of reference objects is used as a benchmark, the quality of the automatically generated object combination can also be guaranteed, thereby making it possible to optimize online marketing.

[0075] It should be noted that the management platform of an online entity can be: a management platform provided by the e-commerce enterprise hosting the online entity, which can be operated by the online entity after logging in. The management platform can be in the form of a client or a web page.

[0076] Compared with the prior art, the embodiments of the present application have at least the following advantages:

[0077] In the technical solution provided by this application, an automatic matching function is provided online. After the staff executes this function, they can obtain reference combination object information, which includes object attribute information corresponding to each of the multiple reference objects; for each reference object, based on the object attribute information corresponding to the reference object, determine the target candidate object corresponding to the reference object from the candidate object set, wherein the degree of match between the object attribute information of the target candidate object and the object attribute information corresponding to the reference object meets the preset conditions; then, the target candidate objects corresponding to the multiple reference objects are regarded as an object combination. In this way, there is no need for staff to manually match objects. Due to the automatic matching function integrated in the background of the management platform, the staff only needs to use this automatic matching function, and the management platform can use the multiple reference objects included in the known matching as the matching basis. Based on the object attribute information of the reference object and the object attribute information of the candidate objects in the candidate object set, at least one target candidate object corresponding to each reference object is matched, thereby generating the combination of target candidate objects corresponding to different reference objects into an object combination, achieving the purpose of automatically generating object combinations, which not only facilitates the staff, but also ensures the quality of the automatically generated object combinations based on the known matching of the reference objects, thereby making it possible to optimize online marketing.

[0078] It should be noted that the subject implementing the method of generating an object combination may be the device for generating an object combination provided in the embodiment of the present application. The device for generating an object combination may be carried in an electronic device or a functional module of an electronic device. Taking the scenario of the management platform of an online entity as an example, the device for generating an object combination may include the management platform of an online entity, or the device for generating an object combination may be connected to the management platform of an online entity; the electronic device may be installed with a client corresponding to the management platform of the online entity, or may access the webpage corresponding to the management platform of the online entity.

[0079] FIG1 is a flow chart of a method for generating an object combination provided in an embodiment of the present application. The method can be applied to an apparatus for generating an object combination, such as apparatus 800 for generating an object combination shown in FIG8 below. In the scenario of an online entity management platform, the apparatus for generating an object combination can include the online entity management platform or be connected to the online entity management platform. Therefore, the method can also be considered to be applied to the online entity management platform.

[0080] As shown in FIG1 , the method may include, for example, the following S101 to S103:

[0081] S101: Acquire reference combination object information, where the reference combination object information includes object attribute information corresponding to each of a plurality of reference objects.

[0082] In an embodiment of the present application, the online entity's management platform includes an operation item corresponding to the automatic matching function. By performing an action on the operation item, a staff member is deemed to have performed the target operation on the management platform. The execution of the target operation triggers the management platform's backend to automatically match objects for the staff member's corresponding target entity. For example, the interface corresponding to the online entity's management platform may include a "Smart Matching Recommendation" button. The "Smart Matching Recommendation" button corresponds to the operation item corresponding to the automatic matching function on the online entity's management platform. When a user clicks the "Smart Matching Recommendation" button, it is equivalent to performing the target operation on the management platform, thereby triggering the management platform to execute the method.

[0083] In the process of automatically matching objects for the target entity in the background of the management platform, it is necessary to first obtain the matching information representing the known matching in the target entity, and use the objects included in the matching information as reference objects to implement the method provided in the embodiment of the present application, that is, the matching information can be understood as the data basis for generating object combinations in the embodiment of the present application. Matching information can refer to any information that can indicate a known matching, which can be text description information or a picture. For example, the matching information can be a plurality of pictures corresponding to each object in the target entity, which includes, in addition to the object, at least one picture of other objects in the target entity, namely, the matching main picture referred to below. The multiple pictures corresponding to the object may, for example, include the header picture, detail picture or carousel picture of the object. In order to display more information, the detail picture or carousel picture of the object may include other objects in addition to the object itself. Therefore, the detail picture or carousel picture including multiple objects can be used as the matching main picture.

[0084] As an example, S101 may include: S101a, identifying multiple reference objects included in the matching main image from the matching main image; S101b, obtaining object attribute information of the corresponding reference object based on the sub-image corresponding to each of the multiple reference objects in the matching main image. S101b may include, for example, inputting the sub-image corresponding to each of the multiple reference objects in the matching main image into a feature detection model to obtain object attribute information of the corresponding reference object. The object attribute information may, for example, represent any information representing the attributes of the object, such as the style, color, category, and other attributes of the object. The object attribute information may be represented by a feature vector of the object.

[0085] The implementation process of S101a may include, for example: S101a1, performing object detection (also known as subject detection) on the matching main image to determine that the matching main image includes a first object and at least one other object; S101a2, cropping the objects detected in the matching main image to crop a first image including the first object and a second image including each other object from the matching main image. Both the first image and the second image are partial images in the matching main image, and the first image only includes the first object and a portion of the background of the first object in the matching main image, and the background portion does not include the content of other objects as much as possible. Similarly, the second image only includes one other object and a portion of the background of the other object in the matching main image, and the background portion does not include the content of any object other than the other object as much as possible. The first object and the at least one other object are both reference objects in S101.

[0086] For example, the main matching image obtained from the image of the first object of the target entity can be seen in Picture 1 in Figure 2. The first object is top A, and the main matching image also includes skirt B (i.e., other objects). Through S101a, the first and second images can be obtained, where the first image corresponds to Picture 2 in Figure 2, and the second image can correspond to Picture 3 in Figure 2. Picture 2 is the part of Picture 1 corresponding to top A, and Picture 3 is the part of Picture 1 corresponding to skirt B.

[0087] The implementation process of S101b may include, for example: inputting the first image into a feature detection model, the feature detection model outputting a first feature vector; similarly, inputting each of the at least one second image into the feature detection model, the feature detection model outputting a second feature vector corresponding to the input second image. The feature detection model may also be referred to as an image encoder based on its function. The input of the feature detection model is an image, and the output is a feature vector of the image. The output feature vector can represent the attributes of the object included in the input image (also referred to as a subject feature).

[0088] For example, a feature detection model could be the Swin Transformer. Figure 3 shows the overall architecture of the Swin Transformer. The Swin Transformer employs a hierarchical design consisting of four stages. Each stage reduces the resolution of the input image, gradually expanding the receptive field, similar to convolutional neural networks (CNNs). As shown on the left side of Figure 3, the Swin Transformer input begins with a patch embedding, which slices the image into individual tiles and embeds them into the embedding. Each stage consists of a patch merging module and at least one block. The patch merging module downsamples the image data entering each stage to reduce the image resolution. The specific structure of the block is shown on the right side of Figure 3, primarily comprising layer normalization (LN), a multi-layer perceptron (MLP), window attention, and shifted window attention. Swin Transformer, a model with a hierarchical design that includes a sliding window operation, is used as a feature detection model in the embodiments of the present application. Its sliding window operation includes non-overlapping local windows and overlapping cross-windows, which limits the attention calculation to one window. On the one hand, it can introduce the locality of the CNN convolution operation, and on the other hand, it can save computational complexity.

[0089] In order to better obtain the feature vector of the picture, it is necessary to train the feature detection model to a model with better feature detection performance. The embodiment of the present application provides a method for multi-stage training of the feature detection model. For example, the training of the feature detection model may include: S11, based on the pictures in the training sample, the initial model is pre-trained in a self-supervised manner to obtain a first model; S12, based on the pictures in the training sample and the classification labels corresponding to the pictures, the first model is trained in a clustering manner in one stage to obtain a second model; S13, based on the two pictures in the training sample and the sample attributes of the two pictures, the second model is trained in a two-stage manner at least once, and the second model obtained from the last two-stage training is recorded as the feature detection model.

[0090] As shown in Figure 4, for the pre-training in S11, it can be implemented through multimodal pre-training models such as Contrastive Language-Image Pre-training (CLIP), Bootstraping Language-Image Pre-training (BLIP), BEiT2 and BEiT3. For the one-stage training of S12, it can be implemented by spectral clustering of subcenter, arcface, partial-fc, id classification, infonce and triplet. For the two-stage training in S13, for example, it can include: constructing correlation classification data + rule-based difficult sample mining, which can be implemented through classification, main or whole image fusion, or smoothL1; for example, it can include: manually labeling difficult samples, which can be implemented through binary classification, Triplet or InfoNce, or continuous mining of difficult samples.

[0091] Taking Picture 1 shown in FIG2 as an example, the reference combination object information obtained in S101 may include: feature vector 2 corresponding to Picture 2 and feature vector 3 corresponding to Picture 3. Top A in Picture 2 and skirt B in Picture 3 are the two reference objects corresponding to the reference combination object information.

[0092] S102, for each reference object, determining a target candidate object corresponding to the reference object from a candidate object set based on the object attribute information corresponding to the reference object, wherein the degree of matching between the object attribute information of the target candidate object and the object attribute information corresponding to the reference object meets a preset condition.

[0093] It should be noted that for each of the multiple reference objects corresponding to the reference combination object information obtained in S101, at least one target candidate object corresponding to the reference object can be obtained based on S102. For example, the two reference objects corresponding to the reference combination object information obtained in S101 are top A in Picture 2 and skirt B in Picture 3 in Figure 2. Then, for top A, S102 is executed to obtain at least one target candidate object 1 corresponding to top A; for skirt B, S102 is executed to obtain at least one target candidate object 2 corresponding to skirt B.

[0094] As an example, for each reference object, S102 may include: determining, from a preset search library, object attribute information whose matching degree with the object attribute information corresponding to the reference object satisfies a preset condition, and recording a candidate object corresponding to the determined object attribute information as the target candidate object, wherein the search library includes a mapping relationship between each candidate object in a plurality of candidate objects and the corresponding object attribute information, and the candidate object set includes the plurality of candidate objects involved in the search library. Hereinafter, object attribute information is described using a feature vector of the object.

[0095] The retrieval library can be a database that is pre-built and maintained before executing S102, and includes mapping relationships between objects and their feature vectors. The set of objects included in the retrieval library is the candidate object set in S102. The retrieval library is used to provide a data basis for retrieving target candidate objects that match the reference objects, so as to determine target candidate objects that have the same and / or similar attributes in the target entity as the multiple reference objects included in the matching main graph.

[0096] In some possible implementations, before S102, the method may further include a process of constructing a search library. The process of constructing the search library may, for example, include: S100a, obtaining a target image for each candidate object, wherein the target image includes only one candidate object; S100b, obtaining object attribute information corresponding to each target image based on the feature detection model; S100c, storing the mapping relationship between each candidate object and the corresponding object attribute information, thereby obtaining the search library. The target image for each candidate object is an image that only includes the candidate object and does not include other objects. For example, it may be a Stock Keeping Unit (SKU) image. A SKU image refers to an image that displays attributes that can distinguish the object. For example, for each product, its brand, trademark, color, model, and other attributes are different from other products. Therefore, the SKU image is an image that displays the aforementioned attributes of the product. In this embodiment, the SKU image is merely exemplary. In S100a, any image that includes the object itself and does not include other objects can also be used to replace the SKU image of the object. The feature detection model in S100b can be the same as the feature detection model for implementing S101b, both of which are trained models for extracting feature vectors of images. After S100c, the detection library may include, for example: candidate object 1-feature vector 1, candidate object 2-feature vector 2, ..., candidate object n-feature vector n, where n is an integer greater than or equal to 1, and the candidate object set may include: candidate object 1, candidate object 2, ..., candidate object n. It should be noted that the content in the retrieval library can be continuously updated based on the increase, decrease or change of the objects included in the target entity. For example, when the objects included in the target entity change, the embodiment of the present application can execute S100a to S100c on the candidate objects involved in the change, thereby obtaining an updated retrieval library, so that the staff of the target entity can obtain the most accurate automatic matching results at any time.

[0097] Among them, the degree of matching between the object attribute information of the target candidate object and the object attribute information corresponding to the reference object meets the preset conditions, which can also be understood as the object attribute information of the target candidate object matches the object attribute information of the reference object. Specifically, it can mean that the object attribute information of the target candidate object is the same or similar to the object attribute information of the reference object. The similarity between the object attribute information of the target candidate object and the object attribute information of the reference object can mean that the difference between the object attribute information of the target candidate object and the object attribute information of the reference object is within a preset range.

[0098] S103: Taking the target candidate objects corresponding to each of the plurality of reference objects as an object combination.

[0099] As an example, S103 may include: arbitrarily combining each reference object among multiple reference objects with the target candidate objects corresponding to other reference objects based on the target candidate object obtained in S102 to obtain multiple object combinations, each object combination including a target candidate object corresponding to each reference object in at least two reference objects among the multiple reference objects.

[0100] Still taking the matching main picture shown in Figure 2 as an example, the reference objects include top A and skirt B. If for top A, executing S102 obtains target candidate objects 1 corresponding to top A including {top A1, top A2 and top A3}; if for skirt B, executing S102 obtains at least one target candidate object 2 corresponding to skirt B including {skirt B1 and skirt B2}; then, the object combinations obtained based on S103 may include: object combination 1 {top A1, skirt B1}, object combination 2 {top A1, skirt B2}, object combination 3 {top A2, skirt B1}, object combination 4 {top A2, skirt B2}, object combination 5 {top A3, skirt B1} and object combination 6 {top A3, skirt B2}. Alternatively, the six object combinations may include the combination {top A, skirt B}, i.e., one top among tops A1, A2, and A3 is top A, and one skirt among skirts B1 and B2 is skirt B. Alternatively, if the six object combinations do not include the combination {top A, skirt B}, then the object combinations obtained in S103 may also include {top A, skirt B}.

[0101] In other implementations, if the reference objects corresponding to the reference combination object information are more than two, then an object combination can be generated based on the target candidate objects corresponding to at least two reference objects. Assuming that the reference objects corresponding to the reference combination object information include A, B, and C, if for A, executing S102 obtains target candidate object 1 corresponding to A including {A1, A2, and A3}; if for B, executing S102 obtains at least one target candidate object 2 corresponding to B including {B1 and B2}; if for C, executing S102 obtains at least one target candidate object 3 corresponding to C including {C1}, then the object combinations obtained based on S103 may include: object combination 1 {A1, B1}, object combination 2 {A1, B2}, object combination 3 {A2, B1}, object combination 4{A2, B2}, object combination 5{A3, B1}, object combination 6{A3, B2}, object combination 7{A1, C1}, object combination 8{A2, C1}, object combination 9{A3, C1}, object combination 10{B1, C1}, object combination 11{B2, C1}, object combination 12{A1, B1, C1}, object combination 13{A1, B2, C1}, object combination 14{A2, B1, C1}, object combination 15{A2, B2, C1}, object combination 16{A3, B1, C1} and object combination 17{A3, B2, C1}.

[0102] The method provided in the embodiment of the present application can be shown in FIG5 , for example, including steps 100 a to 100 c of constructing a search library and steps S101 a 1 , S101 a 2 , S101 b , and S102 to S103 of automatic matching. Taking Picture 1 shown in Figure 2 as an example, the automatic matching process can be seen in Figure 6, including: S21, performing object detection and cropping from Picture 1 to obtain Picture 2 and Picture 3; S22, inputting Picture 2 and Picture 3 into the feature detection model, and the feature detection model outputs feature vector 2 corresponding to Picture 2 and feature vector 3 corresponding to Picture 3; S23, searching feature vector 2 in the retrieval library to obtain target candidate object 1, and target candidate object 1 includes at least top A, searching feature vector 3 in the retrieval library to obtain target candidate object 2, and target candidate object 2 includes at least skirt B, and the retrieval library includes a correspondence 1 between the feature vector 2' of each target candidate object 1 and the corresponding target candidate object 1, and a correspondence 2 between the feature vector 3' of each target candidate object 2 and the corresponding target candidate object 2; S24, determining at least one object combination in the target entity, and the at least one object combination includes a first combination: top A and skirt B. Among them, the degree of matching between eigenvector 2 and eigenvector 2' meets the preset conditions, for example, it can mean that eigenvector 2 is identical or similar to eigenvector 2', and the similarity between eigenvector 2 and eigenvector 2' can mean that the difference between eigenvector 2 and eigenvector 2' is within a preset range; similarly, eigenvector 3 and eigenvector 3' match, for example, it can mean that eigenvector 3 is identical or similar to eigenvector 3', and the similarity between eigenvector 3 and eigenvector 3' can mean that the difference between eigenvector 3 and eigenvector 3' is within a preset range.

[0103] In some possible implementations, the object combination obtained in S103 may include a first combination of reference objects included in the main image, such as {top A, skirt B} in the above example. Then, after S103, the method may further include: adding corresponding anchor points at preset positions in the region where each reference object is located in the main image to obtain a combined image of the first combination, wherein the anchor points are used to associate preset attribute information of the corresponding reference objects. The preset positions may be, for example, positions [0.6, 0.7] in the region where the objects are located. An anchor point refers to a mark that can be linked to the object details page of an object. For example, when there is no operation on a certain anchor point, the anchor point can be displayed as a dot; when the user moves the cursor to the preset area where the anchor point is located, the preset attribute information associated with the anchor point can be, for example, certain key information of the corresponding reference object (such as product name, price, etc.), and the certain key information of the reference object corresponding to the anchor point is displayed near the anchor point; when the user clicks on the anchor point, the preset attribute information associated with the anchor point can be, for example, the Uniform Resource Locator (URL) of the object details page of the corresponding reference object, and the page jumps to the object details page corresponding to the URL. Optionally, other relevant information of the reference object can be added in the direction corresponding to each reference object in the first combination, for example, at least one of the name of the object, the category to which it belongs, the header image, or the SKU image. For the matching main image shown in Figure 2, the object combination obtained based on this method includes the first combination of {top A, skirt B}. Then, the combination image of the first object obtained in this embodiment can be shown in Figure 7. In this way, the object images in the object database of the target entity in the online platform are first captured, and the matching main image including multiple objects is screened. For the images including objects on the matching main image, they are captured, matched with the objects published by the target entity, and the locations of the determined objects are marked on the matching main image, so as to complete the construction of the object combination intelligently and quickly.

[0104] It is understandable that after generating at least one object combination of the target entity, the at least one object combination of the target entity can be saved in the form of a file for viewing, editing and other operations by staff of the target entity.

[0105] In some possible implementations, in order to subsequently optimize the matching algorithm and to provide a better experience for the online entity, the method may further include: calculating the correlation of the object combination based on the number of views, sales, buyer reviews, staff reviews, and sales correlation of each object in the object combination within the target entity, thereby determining the matching score of the object combination based on the correlation. Optionally, the matching score can be displayed on the management platform to guide the display and / or release of the object combination. For example, the staff can choose to put the object combination with a higher matching score on the shelf. For another example, the staff can choose to "adjust" the display order of each object combination on the display page of the online entity according to the size of the matching score. The staff can even set a matching score threshold for the display. Object combinations corresponding to matching scores below the matching score threshold will not be displayed on the display page. For another example, the staff can also influence the matching score of the object combination by filling in a review.

[0106] It should be noted that, for the at least one object combination generated by S103, it can be an object combination that has been deduplicated by the same combination and / or filtered by the same category in the matching; or, at least one object combination generated by S103 can be an object combination that has not been deduplicated by the same combination and / or filtered by the same category in the matching. Then, after S103, the method can also perform deduplication by the same combination and / or filtering by the same category in the matching, so that the object combination finally provided to the staff of the target entity is of better quality, so as to improve the user experience of using the automatic matching function.

[0107] It can be seen that in this method, there is no need for staff to manually match objects. Due to the automatic matching function integrated in the background of the management platform, the staff only needs to use the automatic matching function, and the management platform can use the multiple reference objects included in the known matching as the basis for matching, and match at least one target candidate object corresponding to each reference object based on the object attribute information of the reference object and the object attribute information of the candidate object in the candidate object set, so as to generate the combination of target candidate objects corresponding to different reference objects as an object combination, thereby achieving the purpose of automatically generating object combinations. It is not only convenient for the staff, but also because the known matching of the reference objects is used as a benchmark, the quality of the automatically generated object combination can also be guaranteed, thereby making it possible to optimize online marketing.

[0108] Accordingly, an embodiment of the present application further provides an apparatus 800 for generating an object combination. As shown in FIG8 , the apparatus 800 may include:

[0109] The acquisition unit 801 is configured to acquire reference combination object information; wherein the reference combination object information includes object attribute information corresponding to each of a plurality of reference objects;

[0110] A matching unit 802 is configured to determine, for each reference object, a target candidate object corresponding to the reference object from a set of candidate objects based on object attribute information corresponding to the reference object; wherein a degree of matching between the object attribute information of the target candidate object and the object attribute information corresponding to the reference object satisfies a preset condition;

[0111] The generating unit 803 is configured to combine the target candidate objects corresponding to the multiple reference objects into an object combination.

[0112] In a possible implementation, the acquiring unit 801 includes:

[0113] an identification subunit, configured to identify, from a collocation main image, a plurality of reference objects included in the collocation main image;

[0114] The first obtaining subunit is configured to obtain object attribute information of each reference object among the multiple reference objects according to a sub-picture corresponding to the reference object in the matching main picture.

[0115] As an example, the first obtaining subunit is specifically configured to:

[0116] The sub-image corresponding to each reference object in the collocation main image is input into the feature detection model to obtain object attribute information of the corresponding reference object.

[0117] In a possible implementation, the matching unit 802 is specifically configured to:

[0118] From a preset retrieval library, determine the object attribute information whose matching degree with the object attribute information corresponding to the reference object satisfies the preset conditions, and record the candidate object corresponding to the determined object attribute information as the target candidate object, the retrieval library includes a mapping relationship between each candidate object in a plurality of candidate objects and the corresponding object attribute information, and the candidate object set includes the plurality of candidate objects involved in the retrieval library.

[0119] In a possible implementation, the apparatus 800 further includes: a construction unit;

[0120] The construction unit is used to construct a preset retrieval library.

[0121] As an example, the building block includes:

[0122] A second obtaining subunit is configured to obtain a target image for each candidate object, wherein the target image includes only one candidate object;

[0123] A third obtaining subunit is configured to obtain object attribute information corresponding to each of the target images based on the feature detection model;

[0124] The saving subunit is used to save the mapping relationship between each candidate object and the corresponding object attribute information to obtain the search library.

[0125] In a possible implementation, the apparatus 800 further includes: a training unit;

[0126] The training unit is used to train the feature detection model.

[0127] As an example, the training unit includes:

[0128] A first training subunit is configured to pre-train the initial model in a self-supervised manner based on images in the training sample to obtain a first model;

[0129] A second training subunit is configured to perform a one-stage training on the first model in a clustering manner based on the images in the training samples and the classification labels corresponding to the images to obtain a second model;

[0130] The third training subunit is used to perform at least one two-stage training on the second model using a binary classification method based on the two pictures in the training sample and the sample attributes of the two pictures, and record the second model obtained from the last two-stage training as the feature detection model.

[0131] In a possible implementation, the object combination includes a first combination of reference objects included in the collocation main image, and the apparatus 800 further includes: an anchor unit;

[0132] The anchor point unit is used to add corresponding anchor points at preset positions of the areas where each reference object in the matching main image is located to obtain the combined image of the first combination, and the anchor points are used to associate preset attribute information of the corresponding reference objects.

[0133] It should be noted that the specific implementation of the device 800 and the technical effects achieved can be found in the relevant description of the method shown in FIG. 1 .

[0134] In addition, an embodiment of the present application also provides an electronic device, which includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any implementation of the method provided in the embodiment of the present application.

[0135] Referring to FIG9 , a schematic diagram of the structure of an electronic device 900 suitable for implementing embodiments of the present disclosure is shown. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG9 is merely an example and should not limit the functionality or scope of use of embodiments of the present disclosure.

[0136] As shown in Figure 9, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0137] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although FIG9 shows the electronic device 900 with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0138] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0139] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0140] An embodiment of the present application further provides a computer-readable medium, in which instructions or computer programs are stored. When the instructions or computer programs are executed on a device, the device executes any implementation of the method provided in the embodiment of the present application.

[0141] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0142] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0143] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0144] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can perform the method.

[0145] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0146] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0147] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit / module does not, in some cases, limit the unit itself.

[0148] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0149] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0150] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0151] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0152] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0153] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0154] It should be noted that in the embodiments of the present application, user sensitive information is not involved, and user-related information is obtained, used and determined after user authorization. In one example, before obtaining user-related information, the corresponding interface displays prompt information related to obtaining data use authorization. This prompt information is based on relevant laws and regulations and informs the user of the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure in an appropriate manner, so that the user can determine whether to agree to the authorization based on the prompt information. It can be understood that the above-mentioned notification and the process of obtaining user authorization are only illustrative and do not constitute a limitation on the implementation method of this disclosure. Other methods that meet relevant laws and regulations can also be applied to the implementation method of this disclosure.

[0155] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating an object combination, comprising: Acquire reference combination object information, wherein the reference combination object information includes object attribute information corresponding to each of a plurality of reference objects; For each reference object, determining a target candidate object corresponding to the reference object from a candidate object set according to the object attribute information corresponding to the reference object; wherein the degree of matching between the object attribute information of the target candidate object and the object attribute information corresponding to the reference object satisfies a preset condition; The target candidate objects respectively corresponding to the multiple reference objects are taken as an object combination.

2. The method according to claim 1, wherein obtaining reference combination object information comprises: From the collocation main image, identifying a plurality of reference objects included in the collocation main image; According to the sub-picture corresponding to each reference object among the multiple reference objects in the matching main picture, object attribute information of the corresponding reference object is obtained.

3. The method according to claim 2, wherein obtaining object attribute information of a corresponding reference object according to a sub-image corresponding to each reference object in the matching main image comprises: The sub-image corresponding to each reference object among the multiple reference objects in the matching main image is input into the feature detection model to obtain object attribute information of the corresponding reference object.

4. The method according to claim 3, wherein determining the target candidate object corresponding to the reference object from the candidate object set according to the object attribute information corresponding to the reference object comprises: From a preset retrieval library, object attribute information whose matching degree with the object attribute information corresponding to the reference object satisfies the preset conditions is determined, and the candidate object corresponding to the determined object attribute information is recorded as the target candidate object, the retrieval library includes a mapping relationship between each candidate object in a plurality of candidate objects and the corresponding object attribute information, and the candidate object set includes the plurality of candidate objects involved in the retrieval library.

5. The method according to claim 4, wherein the process of constructing the preset search library comprises: Obtaining a target image for each candidate object, wherein the target image includes only one candidate object; Based on the feature detection model, obtaining object attribute information corresponding to each of the target images; The mapping relationship between each candidate object and the corresponding object attribute information is saved accordingly to obtain the search library.

6. The method according to any one of claims 3 to 5, wherein the training process of the feature detection model comprises: Based on the pictures in the training samples, the initial model is pre-trained in a self-supervised manner to obtain a first model; Based on the pictures in the training samples and the classification labels corresponding to the pictures, the first model is trained in one stage by clustering to obtain a second model; Based on the two images in the training sample and the sample attributes of the two images, the second model is trained in two stages at least once using a binary classification method, and the second model obtained from the last two-stage training is recorded as the feature detection model.

7. The method according to any one of claims 2 to 6, wherein the object combination includes a first combination of reference objects included in the matching main image, and the method further includes: Corresponding anchor points are added at preset positions of the regions where the reference objects in the matching main image are located, respectively, to obtain the combined image of the first combination, wherein the anchor points are used to associate with preset attribute information of the corresponding reference objects.

8. A device for generating a combination of objects, comprising: An acquisition unit, configured to acquire reference combination object information; wherein the reference combination object information includes object attribute information corresponding to each of a plurality of reference objects; A matching unit, configured to determine, for each reference object, a target candidate object corresponding to the reference object from a candidate object set according to the object attribute information corresponding to the reference object; wherein the degree of matching between the object attribute information of the target candidate object and the object attribute information corresponding to the reference object satisfies a preset condition; The generating unit is used to combine the target candidate objects corresponding to each of the plurality of reference objects into an object combination.

9. An electronic device, comprising: Processor and memory; The memory is used to store instructions or programs; The processor is used to execute the instructions or programs in the memory so that the electronic device executes the method according to any one of claims 1 to 7.

10. A readable medium storing instructions or programs, which, when executed on a processor, enable the processor to execute the method according to any one of claims 1 to 7.

Images