Image processing method, device, equipment, readable storage medium and product

Abstract

The embodiment of the present application discloses an image processing method, device, equipment, readable storage medium and product, wherein the method includes: obtaining an image to be processed, and the title information and category information of the image to be processed; performing image detection on the image to be processed, and determining the main frame corresponding to each commodity in the image to be processed; the main frame corresponding to each commodity is used to select the corresponding commodity in the image to be processed; performing classification prediction on the main frame corresponding to each commodity in the image to be processed, and determining the predicted category information of the main frame corresponding to each commodity in the image to be processed; based on the predicted category information, the title information and the category information, determining the target commodity feature of the image to be processed, and pushing commodities based on the target commodity feature. The embodiment of the present application can improve the accuracy of commodity push.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, equipment, readable storage medium and product. Background Art

[0002] Image processing tasks are widely used in various scenarios. In many scenarios, there may be multiple products in one image, for example, an image may contain clothes, pants, shoes, accessories, etc. The current image processing tasks generally combine the positions or areas occupied by the multiple products contained in the image to determine the main product features of the image. However, due to factors such as the camera angle and image layout, small objects in the image are generally ignored. Therefore, it is difficult to accurately determine the main product features in the image, resulting in low accuracy in product push. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, apparatus, device, readable storage medium and product, which can improve the accuracy of product push.

[0004] In a first aspect, the present application provides an image processing method, comprising:

[0005] Obtaining an image to be processed, as well as title information and category information of the image to be processed;

[0006] Performing image detection on the image to be processed to determine the main frame corresponding to each commodity in the image to be processed; the main frame corresponding to each commodity is used to select the corresponding commodity in the image to be processed;

[0007] Classify and predict the main frame corresponding to each commodity in the image to be processed, and determine the predicted category information of the main frame corresponding to each commodity in the image to be processed;

[0008] Based on the predicted category information, the title information and the category information, target product features of the image to be processed are determined, and product push is performed based on the target product features.

[0009] In a second aspect, the present application provides an image processing device, comprising:

[0010] An information acquisition unit, used to acquire an image to be processed, and title information and category information of the image to be processed;

[0011] An image detection unit is used to perform image detection on the image to be processed, and determine a main frame corresponding to each commodity in the image to be processed; the main frame corresponding to each commodity is used to select the corresponding commodity in the image to be processed;

[0012] An image prediction unit, used to classify and predict the main frame corresponding to each commodity in the image to be processed, and determine the predicted category information of the main frame corresponding to each commodity in the image to be processed;

[0013] The information pushing unit is used to determine the target commodity feature of the image to be processed based on the predicted category information, the title information and the category information, and push the commodity based on the target commodity feature.

[0014] In a third aspect, the present application provides a computer device, including: a processor, a memory, and a network interface;

[0015] The processor is connected to a memory and a network interface, wherein the network interface is used to provide a data communication function, the memory is used to store computer program codes, and the processor is used to call the computer program codes so that a computer device including the processor executes the image processing method.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor performs the above-mentioned image processing method.

[0017] In a fifth aspect, the present application provides a computer program product or a computer program, characterized in that the computer program product or the computer program includes computer instructions, which, when executed by a processor, implement the image processing method provided in the various optional methods in the first aspect of the present application.

[0018] In the embodiment of the present application, by obtaining the title information and category information of the image to be processed, image detection is performed on the image to be processed to determine the main frame corresponding to each commodity in the image to be processed, and the main frame corresponding to each commodity in the image to be processed is classified and predicted to determine the predicted category information of the main frame corresponding to each commodity, and then the predicted category information of the main frame corresponding to each commodity, the title information and category information of the image to be processed are combined to determine the target commodity features in the image to be processed, and then push the commodity. Since the main frame corresponding to each commodity in the image to be processed is classified and predicted, and is judged with the title information and category information of the image to be processed, any commodity in the image will not be ignored because of the position of the commodity in the image to be processed or the small area in the image to be processed. By judging the commodities in each detected main frame, the main commodities in the image to be processed can be determined, and then the target commodity features of the image to be processed can be obtained, which can improve the accuracy of commodity determination, and then when commodity push is performed based on the determined target commodity features, the accuracy of commodity push can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a network architecture diagram of an image processing system provided by an embodiment of the present application;

[0021] Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0022] Figure 3 is a flowchart of another image processing method provided in an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of the composition structure of an image processing device provided in an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of the composition structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The technical solution of the present application can be applied to scenarios where the main features of a product are determined from an image. For example, it can be applied to a product push system, where the main product in the image can be determined by performing image detection on multiple products in the image, and then the product push can be performed based on the main product in the image, which can improve the accuracy of the product push. The technical solution of the embodiment of the present application can also be applied to any scenario where the main product in the image needs to be determined, for example, it can include but is not limited to an image recognition system, an image search system or other systems, which is not limited in the embodiment of the present application. The collection, use and processing of data such as the title information, category information, merchant information, etc. of the image to be processed involved in the embodiment of the present application requires the consent of the corresponding personal information subject, and needs to comply with the provisions of relevant laws and regulations.

[0027] Please refer to Figure 1 , Figure 1 is a network architecture diagram of an image processing system provided in an embodiment of the present application, such as Figure 1 As shown, the computer device can exchange data with the terminal device, and the number of the terminal devices can be one or at least two. For example, when the number of the terminal devices is multiple, the terminal device may include Figure 1 The terminal device 101a, the terminal device 101b and the terminal device 101c in the computer device 102 are as follows. Taking the terminal device 101a as an example, the computer device 102 can obtain the image to be processed, as well as the title information and category information of the image to be processed. Further, the computer device 102 can perform image detection on the image to be processed, and determine the main body frame corresponding to each commodity in the image to be processed. Further, the computer device 102 can classify and predict the main body frame corresponding to each commodity in the image to be processed, and determine the predicted category information of the main body frame corresponding to each commodity in the image to be processed. Further, the computer device 102 can determine the target commodity features of the image to be processed based on the predicted category information, title information and category information. Optionally, the computer device 102 can also push commodities based on the target commodity features, thereby pushing the corresponding commodities to the terminal device 101a.

[0028] The computer device mentioned in the embodiments of the present application may refer to a server or a terminal device, or may also be a system composed of a server and a terminal device, which is not limited in the embodiments of the present application. The terminal device may be an electronic device, including but not limited to a mobile phone, a tablet computer, a laptop computer, a vehicle-mounted device, etc. The server mentioned above may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, vehicle-road collaboration, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. When the server is an independent physical server, the server can perform image processing independently. When the server is multiple physical servers, multiple physical servers can collaborate to perform image processing.

[0029] For further information, see Figure 2 , Figure 2 is a flowchart of an image processing method provided by an embodiment of the present application; Figure 2 As shown, the image processing method can be applied to a computer device, and the image processing method includes but is not limited to the following steps:

[0030] S101, obtaining an image to be processed, as well as title information and category information of the image to be processed.

[0031] In some image processing tasks, such as product retrieval based on image processing tasks or image search for products, it is necessary to build an accurate product feature library. The accuracy of the product feature library will affect the product search and recall experience. The richness of product images is a means for merchants to attract consumers, that is, there may be a large number of product combinations in product images, such as clothes, pants, shoes, socks, bags, etc., or the product images include a variety of combinations such as makeup effects. Therefore, it is difficult to accurately locate the main product from the product image, and then in subsequent product retrieval or image search tasks, it will lead to the accuracy of product retrieval or image search. The current method generally determines the main product in the product image based on the position and area ratio of the product in the product image, but this method is not friendly to small objects in the product image, such as shoes, socks, etc., and the product image is greatly affected by factors such as the shooting angle, which will lead to inaccurate selection of the main product.

[0032] In view of this, the image processing method provided in the embodiment of the present application detects the main body frames corresponding to multiple commodities from the commodity image, and selects the main body frame corresponding to the most important commodity from the multiple main body frames by combining the classification prediction results of the multiple main body frames, the title information and category information of the commodity image, so as to accurately select the main commodity in the commodity image, thereby improving the accuracy of subsequent commodity push.

[0033] In an embodiment of the present application, the image to be processed may refer to a commodity image. For example, the image to be processed may include one or more commodities of the same type or different types. For example, the image to be processed may be any one of an image of clothing, a beauty image, a furniture image, an accessories image, a digital image, or a food image, or may be an image obtained by combining the above multiple categories. The image to be processed may include commodities, commodity specification information, and commodity details information, etc. The title information of the image to be processed may include, for example, the name of the commodity in the image to be processed, the efficacy description of the commodity, the model of the commodity, the net content of the commodity, etc. The category information of the image to be processed may include, for example, the category to which the commodity in the image to be processed belongs. For example, the category to which the commodity belongs may include, but is not limited to, clothing, shoes, bags, accessories, furniture, beauty, digital, food, and the like.

[0034] Optionally, the category information may include multi-level category information, and the first-level category information may include but is not limited to clothing, shoes, bags, accessories, furniture, beauty, digital, food, etc., and the second-level category information may refer to the sub-classification of the first-level category information. For example, when the first-level category information is shoes, the second-level category information may include men's shoes, women's shoes, children's shoes, etc. Optionally, the multi-level category information may also include third-level category information, and the third-level category information may refer to the sub-classification of the second-level category information. For example, when the second-level category information is women's shoes, the third-level category information may include sports shoes, leather shoes, and sandals. Optionally, the multi-level category information may also include fourth-level category information, and the fourth-level category information may refer to the sub-classification of the third-level category information. For example, when the third-level category information is sports shoes, the fourth-level category information may include basketball shoes, football shoes, running shoes, outdoor shoes, etc. Optionally, the multi-level category information may also include more levels, such as five, six or more levels of category information, which is not limited in the embodiments of the present application.

[0035] Among them, the title information and category information of the image to be processed may refer to the title information and category information set when uploading the image to be processed, and the title information and category information of the image to be processed may indicate the title information and category information of the main commodities in the image to be processed. That is to say, the title information and category information of the image to be processed may refer to the title information and category information of the main commodities in the image to be processed selected by the user account that uploaded the image to be processed. Among them, the user account may refer to the account used by the user who logs in to the application interaction platform. The title information and category information of the image to be processed may, for example, refer to a commodity description category, that is, a category used to describe the real commodity in the image to be processed. Subsequently, by obtaining the predicted category information of the commodities in each main frame in the image to be processed and judging the consistency with the commodity description category, it can be determined whether the commodities in each main frame are consistent.

[0036] In one embodiment, the title information and category information of the to-be-processed image uploaded by the merchant can be obtained. Specifically, the image upload instruction of the first terminal can be detected, and the image corresponding to the image upload instruction can be determined as the to-be-processed image; at least one reference title information and at least one reference category information can be sent to the first terminal; the title selection operation of the first terminal for at least one reference title information and the category selection operation for at least one reference category information can be obtained respectively; the reference title information corresponding to the title selection operation can be determined as the title information, and the reference category information corresponding to the category selection operation can be determined as the category information.

[0037] Among them, the image to be processed may refer to an image uploaded by the first terminal, and the first terminal may refer to a terminal held by a merchant. For example, in the application interaction platform, the merchant may refer to a user account that uploads the merchant's merchant information and the merchant's corresponding product information on the application interaction platform. The merchant uploads the corresponding product information on the application interaction platform by using the first terminal. The customer can view the product information on the application interaction platform, thereby being able to operate the product and obtain the corresponding product.

[0038] In a specific implementation, the first terminal can trigger an image upload instruction, and upload the image corresponding to the image upload instruction to the computer device, and the computer device can send at least one reference title information and at least one reference category information to the first terminal. The merchant can select the title information and category information corresponding to the image to be processed from the at least one reference title information and at least one reference category information through the first terminal. For example, at least one reference title information and at least one reference category information can be displayed through the first terminal, and the merchant can select any of the title information and category information. The merchant can trigger a title selection operation for at least one reference title information, and trigger a category selection operation for at least one reference category information through the first terminal, and the computer device can determine the reference title information corresponding to the title selection operation as the title information of the image to be processed, and determine the reference category information corresponding to the category selection operation as the category information. It can be understood that when the first terminal uploads the image to be processed, the image upload instruction may include one image to be processed or a set of images to be processed, that is, one image to be processed or a set of images to be processed may be uploaded. The set of images to be processed may include multiple images, and the embodiment of the present application does not limit this. The title information and category information of each image to be processed in the set of images to be processed may be the same.

[0039] In an embodiment of the present application, when using the first terminal to upload an image to be processed, by selecting the title information and category information of the uploaded image to be processed, it can be used for consistency judgment later, so as to determine the most important commodity in the image to be processed, and then the commodity features of the most important commodity in the image to be processed can be used to construct a commodity feature library, which is convenient for subsequent application in corresponding scenarios, such as commodity retrieval scenarios or image search commodity scenarios, and so on.

[0040] S102, performing image detection on the image to be processed to determine a main body frame corresponding to each commodity in the image to be processed.

[0041] In an embodiment of the present application, since an image to be processed is acquired and the image to be processed may include multiple commodities, image detection can be performed on the image to be processed to determine the main body frame corresponding to each commodity in the image to be processed. The main body frame corresponding to each commodity is used to frame the corresponding commodity in the image to be processed, and one main body frame can frame one commodity. Among them, the main body frame corresponding to each commodity refers to the main body frame in the image to be processed that selects each commodity. If the image to be processed contains one commodity, the main body frame corresponding to each commodity refers to the main body frame corresponding to the one commodity. If the image to be processed contains multiple commodities, the main body frame corresponding to each commodity refers to the main body frame corresponding to each commodity in the multiple commodities. Since the image to be processed may include one commodity or multiple commodities, the main body frame corresponding to each commodity included in the image to be processed can be determined by performing image detection on the image to be processed.

[0042] In one implementation, when determining the main frame in the image to be processed, an image detection model can be used to perform image detection on the image to be processed to determine the main frame corresponding to each commodity in the image to be processed. The image detection model is used to detect objects that may be commodities in the image to be processed, and mark the objects that may be commodities to obtain the main frame. Optionally, the image detection model may include, but is not limited to, a convolutional neural network model (CNN), a recurrent neural network model (RNN), a deep neural network model (DNN), and the like.

[0043] Optionally, before using the image detection model, in order to improve the detection accuracy of the image detection model, the image detection model can be pre-trained using a large number of sample images, so that the trained image detection model has the ability to detect the main frame corresponding to each commodity in the image, and then the trained image detection model is used to perform image detection on the image to be processed, so that the main frame corresponding to each commodity in the image to be processed can be determined. In the process of selecting sample images, sample images of multiple commodity categories can be selected, and then after the image detection model is trained, the trained image detection model can detect images of various commodity categories to improve the accuracy of image detection.

[0044] Optionally, when selecting sample images, some of the large number of sample images selected may include one product, some include multiple products, some include multiple categories of products, some include multiple products of the same category, etc. By selecting sample images containing multiple products when selecting sample images, the selection of sample images can be enriched, and then when the image detection model is subsequently trained, the detection accuracy of the image detection model for images containing one or more products can be improved.

[0045] S103, classifying and predicting the main body frame corresponding to each commodity in the image to be processed, and determining the predicted category information of the main body frame corresponding to each commodity in the image to be processed.

[0046] In the embodiment of the present application, since the image to be processed may include main frames corresponding to multiple commodities, the main frames corresponding to each commodity in the image to be processed are classified and predicted to determine the predicted category information of the main frames corresponding to each commodity in the image to be processed. By determining the predicted category information of the main frames corresponding to each commodity, it can be compared with the title information and category information of the image to be processed, thereby determining which of the multiple commodities in the image to be processed is the most important commodity in the image to be processed, which can improve the accuracy of subsequent commodity push.

[0047] In one embodiment, the predicted category information of the main frame corresponding to each commodity in the image to be processed can be determined in the following manner: classify and predict the main frame corresponding to each commodity in the image to be processed to obtain the probability that the commodity in each main frame belongs to multiple commodity categories; obtain the multi-level category information of the commodity category corresponding to the maximum probability among the probabilities of multiple commodity categories; determine the multi-level category information of the commodity category corresponding to the maximum probability as the predicted category information of the main frame corresponding to each commodity in the image to be processed.

[0048] Among them, the multiple commodity categories may include but are not limited to clothing, shoes, bags, accessories, furniture, beauty, digital, food, etc. By predicting the probability that the main frame corresponding to each commodity in the image to be processed is a plurality of commodity categories, the probability of the plurality of commodity categories can be used to indicate the possibility that the commodity is a plurality of commodity categories. The greater the probability, the greater the possibility that the commodity is of the commodity category, and the smaller the probability, the smaller the possibility that the commodity is of the commodity category. Predicting the probability that the main frame corresponding to each commodity in the image to be processed is each commodity category means: predicting the probability that each commodity in the image to be processed is each commodity category.

[0049] Optionally, when classifying and predicting the probability that the goods in each main frame are multiple categories of goods, it is possible to predict that the goods in each main frame are multiple first-level categories of goods, or it is also possible to predict that the goods in each main frame are multiple second-level categories, third-level categories, or more. For example, for any one product, it is possible to directly predict the probability that the four-level category information corresponding to the goods is basketball shoes, football shoes, running shoes, and outdoor shoes. For example, if the maximum probability is the probability that the goods are outdoor shoes, it can be determined that the multi-level category information corresponding to the maximum probability includes: first-level category information shoes, second-level category information women's shoes, third-level category information sports shoes, and fourth-level category information outdoor shoes. The predicted category information of the main frame corresponding to the goods may include first-level category information shoes, second-level category information women's shoes, third-level category information sports shoes, and fourth-level category information outdoor shoes, and so on. By predicting the probability of each product belonging to multiple product categories, the product category corresponding to the maximum probability is determined, and the multi-level category information to which the product category corresponding to the maximum probability belongs is determined as the predicted category information of the main frame corresponding to each product. The predicted category information can be compared with the category information and title information of the image to be processed to determine the main products in the image to be processed, thereby improving the accuracy of product determination.

[0050] It is understandable that if there is only one product in the image to be processed, the above operation can be performed on the product to obtain the predicted category information of the main frame corresponding to the product. If there are multiple products in the image to be processed, the above operation can be performed on each product to obtain the predicted category information of the main frame corresponding to multiple products in the image to be processed.

[0051] Optionally, an image classification model may be used to determine the predicted category information of the main frame corresponding to each commodity in the image to be processed. For example, an image classification model may be used to classify and predict the main frame corresponding to each commodity in the image to be processed, obtain the probability that the commodity in each main frame is a plurality of commodity categories, obtain the multi-level category information of the commodity category corresponding to the maximum probability among the probabilities of the plurality of commodity categories; and determine the multi-level category information of the commodity category corresponding to the maximum probability as the predicted category information of the main frame corresponding to each commodity in the image to be processed. Among them, the image classification model may include, for example, but is not limited to, a convolutional neural network model (CNN), a recurrent neural network model (RNN), a deep neural network model (DNN), and the like. The image classification model may be used to determine the category information of the main frame corresponding to each commodity in the image to be processed. The image classification model may determine the primary category information or the multi-level category information of the main frame corresponding to each commodity in the image to be processed. Specifically, the image classification model may be trained according to the demand, so that the image classification model has the corresponding classification ability.

[0052] Optionally, before using the image classification model, in order to improve the accuracy of the image classification model, the image classification model can be pre-trained using a large number of sample images, so that the trained image classification model has the ability to determine the predicted category information of the main frame corresponding to each commodity in the image to be processed, and then the trained image classification model is used to perform classification prediction on the image to be processed, so as to determine the predicted category information of the main frame corresponding to each commodity in the image to be processed. In the process of selecting sample images, sample images of multiple commodity categories can be selected, and then after training the image classification model, the trained image classification model can perform classification prediction on images of various commodity categories, thereby improving the accuracy of image classification.

[0053] S104, determining target product features of the image to be processed based on the predicted category information, title information and category information, and pushing products based on the target product features.

[0054] In the embodiment of the present application, the consistency of each commodity in the image to be processed can be determined based on the predicted category information, title information and category information, so that the commodities with consistency in the image to be processed can be determined as the main commodities in the image to be processed, and then the characteristics of the main commodities in the image to be processed can be determined as the target commodity characteristics of the image to be processed. In specific application scenarios, commodity push can also be performed based on the target commodity characteristics. Since the accuracy of commodity determination is improved when the main commodity in the image to be processed is determined, the accuracy of commodity push will also be improved when the commodity push is performed subsequently.

[0055] Optionally, feature extraction can be performed on each main frame in the image to be processed to obtain image features of the products in each main frame. Since the categories of the products in each main frame are different, the image features of the products in each main frame are different. By extracting the product features of the products in each main frame, they can be stored in a product feature library and then used for product push.

[0056] Optionally, when extracting features from each main frame in the image to be processed to obtain image features of the commodities in each main frame, an image feature extraction model may be used to extract features from each main frame in the image to be processed to obtain image features of the commodities in each main frame. The image feature extraction model may include, but is not limited to, a convolutional neural network model (CNN), a recurrent neural network model (RNN), a deep neural network model (DNN), and the like.

[0057] Optionally, the image feature extraction model may also include visual feature models corresponding to multiple commodity categories, for example, including but not limited to clothing feature extraction models, beauty feature extraction models, digital feature extraction models, furniture feature extraction models, and the like. It is understandable that among the visual feature models corresponding to multiple commodity categories, each model can be used to extract features of commodities of the corresponding category. For example, the clothing feature extraction model can extract features of clothing commodities, such as clothes, pants, shoes, etc., and the beauty feature extraction model can extract features of beauty commodities, and so on. For example, before using the visual feature model corresponding to each commodity category, the corresponding model can be trained using sample images of the corresponding category, so that the corresponding model has the ability to extract features of the corresponding category. Since the training samples of each feature extraction model are different, the content of attention is different when extracting the image features of the commodity. Therefore, by using the feature extraction model corresponding to each category to extract features of the commodity, the accuracy of feature extraction can be improved.

[0058] In one embodiment, the method for determining the target commodity features of the image to be processed based on the predicted category information, title information and category information can be as follows: based on the predicted category information, title information and category information of each main frame in the image to be processed, determine whether the commodities in each main frame are consistent; if only one commodity in each main frame is consistent, then use the image features of the commodity in the main frame as the target commodity features of the image to be processed. If at least two commodities in each main frame are consistent, then select a target main frame from the at least two main frames, and use the image features of the commodity in the target main frame as the target commodity features of the image to be processed.

[0059] Among them, the consistency of the commodity may refer to that the commodity matches the title information and category information of the image to be processed. If there is only one commodity in the image to be processed and the commodity is consistent, it can be considered that the commodity is the main commodity in the image to be processed, so that the characteristics of the main commodity can be extracted as the target commodity characteristics of the image to be processed. If there are multiple commodities in the image to be processed and the multiple commodities are consistent, the multiple commodities can be further processed to determine which commodity is the main commodity in the image to be processed, so that the characteristics of the main commodity can be extracted as the target commodity characteristics of the image to be processed. Specifically, based on the predicted category information, title information and category information of each main frame in the image to be processed, determining whether the commodities in each main frame are consistent can refer to: matching the predicted category information of each main frame in the image to be processed with the title information and category information of the image to be processed, and determining whether the commodities in each main frame are consistent. For example, the predicted category information of each main frame in the image to be processed can be matched with the title information of the image to be processed and / or the category information of the image to be processed to obtain text similarity, and determining whether the commodities in each main frame in the image to be processed are consistent based on the text similarity. For example, if the text similarity is greater than the similarity threshold, it is determined that the commodities in each main frame in the image to be processed are consistent. If the text similarity is less than or equal to the similarity threshold, it is determined that the commodities in each main frame in the image to be processed are inconsistent, and so on.

[0060] In the embodiment of the present application, by matching the predicted category information of each main frame in the image to be processed with the title information and category information of the image to be processed, it is possible to determine whether the goods in each main frame are consistent. Since the processing is performed on each product in the image to be processed, the product will not be ignored due to factors such as the area ratio or position in the image, so the accuracy of product determination can be improved.

[0061] In one embodiment, the predicted category information may include at least one level of category information, and the category information may include at least one level of category information. The consistency of the goods may be determined by the following method: matching the at least one level of category information included in the predicted category information with the at least one level of category information or title information included in the category information; if the at least one level of category information included in the predicted category information matches the title information, then the goods in each main frame are determined to be consistent. Alternatively, if the at least one level of category information included in the predicted category information matches the at least one level of category information included in the category information, then the goods in each main frame are determined to be consistent.

[0062] Among them, at least one first-level category information included in the predicted category information matches the title information, which may mean that: at least one first-level category information included in the predicted category information appears in the title information. For example, when the predicted category information includes first-level category information, the predicted category information matches the title information, which may mean that: the first-level category information included in the predicted category information appears in the title information. For example, when the predicted category information includes second-level category information, the predicted category information matches the title information, which may mean that: the first-level category information and / or the second-level category information included in the predicted category information appear in the title information. For example, when the predicted category information includes third-level category information, the predicted category information matches the title information, which may mean that: the first-level category information and / or the second-level category information and / or the third-level category information included in the predicted category information appear in the title information. For example, when the predicted category information includes fourth-level category information, the predicted category information matches the title information may mean that: the first-level category information and / or the second-level category information and / or the third-level category information and / or the fourth-level category information included in the predicted category information appear in the title information, or the third-level category information and / or the fourth-level category information included in the predicted category information appear in the title information, and so on.

[0063] Among them, the at least one first-level category information included in the predicted category information matches the at least one first-level category information included in the category information, which may mean that: the text similarity between the at least one first-level category information included in the predicted category information and the at least one first-level category information included in the category information is greater than the similarity threshold. For example, when the predicted category information includes first-level category information, and the category information includes first-level category information, the at least one first-level category information included in the predicted category information matches the at least one first-level category information included in the category information, which may mean that: the text similarity between the first-level category information included in the predicted category information and the first-level category information included in the category information is greater than the similarity threshold. For example, when the predicted category information includes second-level category information, and the category information includes second-level category information, the at least one first-level category information included in the predicted category information matches the at least one first-level category information included in the category information, which may mean that: the text similarity between the first-level category information and / or second-level category information included in the predicted category information and the first-level category information and / or second-level category information included in the category information is greater than the similarity threshold. For example, when the predicted category information includes three-level category information, and the category information includes three-level category information, the matching of at least one first-level category information included in the predicted category information and at least one first-level category information included in the category information may mean: the text similarity between the first-level category information and / or the second-level category information and / or the third-level category information included in the predicted category information and the first-level category information and / or the second-level category information and / or the third-level category information included in the category information is greater than the similarity threshold. For example, when the predicted category information includes four-level category information, and the category information includes four-level category information, the matching of at least one first-level category information included in the predicted category information and at least one first-level category information included in the category information may mean: the text similarity between the first-level category information and / or the second-level category information and / or the third-level category information and / or the fourth-level category information included in the predicted category information and the first-level category information and / or the second-level category information and / or the third-level category information and / or the fourth-level category information included in the category information is greater than the similarity threshold.

[0064] Optionally, for example, when the predicted category information includes four-level category information, and the category information includes four-level category information, matching of at least one level of category information included in the predicted category information and at least one level of category information included in the category information may also refer to: the text similarity between the four-level category information of the predicted category information and the four-level category information of the category information is greater than a similarity threshold. Alternatively, the four-level category information of the predicted category information is in the four-level category information of the category information. Alternatively, the text similarity between the sum of the three-level category information and the four-level category information of the predicted category information and the sum of the three-level category information and the four-level category information of the category information is greater than a similarity threshold. For example, if the three-level category information of the predicted category information is facial makeup, and the four-level category information of the predicted category information is liquid foundation, then the sum of the three-level category information and the four-level category information of the predicted category information is "facial makeup, liquid foundation". The third-level category information of the category information is facial makeup, and the fourth-level category information of the category information is foundation cream. The sum of the third-level category information and the fourth-level category information of the category information is "facial makeup, foundation cream". The text similarity between "facial makeup, liquid foundation" and "facial makeup, foundation cream" is calculated. For example, they can be converted into feature vectors respectively, and then the similarity distance between the two feature vectors is calculated, and the text similarity is determined based on the similarity distance, and so on. The embodiments of the present application are not limited to this.

[0065] In one embodiment, if there are at least two main body frames in each of which the commodities are consistent, a target main body frame can be selected from the at least two main body frames in the following manner: obtaining the mean of commodity features; respectively calculating the feature distances between the image features of the commodities in at least two main body frames and the mean of the commodity features; and determining the main body frame corresponding to the image feature with the smallest feature distance as the target main body frame.

[0066] Among them, the mean value of the commodity feature can be used to reflect the features of the commodity under the title information and category information. By calculating the feature distance between the image feature of any commodity and the mean value of the commodity feature, it can be determined whether the image feature of any commodity is the commodity corresponding to the title information and category information. For example, the smaller the feature distance between the image feature of any commodity and the mean value of the commodity feature, the greater the possibility that the arbitrary commodity is the main commodity in the image to be processed. The larger the feature distance between the image feature of any commodity and the mean value of the commodity feature, the smaller the possibility that the arbitrary commodity is the main commodity in the image to be processed. The mean value of the commodity feature can be used to reflect the image features of the main commodity in the image to be processed. By calculating the feature distance between the image feature of any commodity and the mean value of the commodity feature (i.e., the reference image feature of the main commodity in the image to be processed), the smaller the feature distance, the greater the possibility that the commodity is the main commodity. By determining the main frame corresponding to the image feature with the smallest feature distance as the target main frame, the commodity in the target main frame can refer to the main commodity in the image to be processed, so that the main commodity can be selected from multiple commodities in the image to be processed, thereby improving the accuracy of commodity determination.

[0067] Optionally, the commodity feature mean value can be obtained by, for example, presetting or by obtaining the commodity feature mean value that matches the title information and category information from the historical records when the title information and category information of the image to be processed are obtained. For example, when there are multiple images to be processed, that is, the merchant uploads a set of images to be processed at the same time, and the set of images to be processed includes multiple images to be processed, the feature mean value can be obtained based on the features of the main commodities in the images to be processed that have been determined. For example, if the set of images to be processed includes 3 images to be processed, for the first image to be processed, if the first image to be processed includes a consistent commodity, the commodity feature of the commodity can be added to the commodity feature library, and when the second image to be processed includes multiple consistent commodities, the commodity feature mean value in the commodity feature library is calculated, and the commodity feature mean value at this time is the commodity feature of the main commodity in the first image to be processed. If there are multiple consistent commodities in the first image to be processed, the main commodity in the image to be processed can be determined based on the commodity feature mean value preset or obtained from the historical records. If the second image to be processed includes a consistent product, the product features of the product in the second image to be processed can be added to the product feature library. At this time, the mean product feature is the average value between the product features of the main product in the first image to be processed and the product features of the main product in the second image to be processed. If the second image to be processed includes multiple consistent products, the mean product feature in the product feature library can be used to determine the main product among the multiple products. Similarly, when determining the main product in each image to be processed in the set of images to be processed, the features of the product can be extracted and added to the product feature library. When calculating the image to be processed when there are multiple products in the set of images to be processed, the mean product feature can be obtained from the product feature library for further judgment to determine the main product among the multiple products.

[0068] Optionally, the method for calculating the feature distance may include, but is not limited to, cosine distance, Euclidean distance, Mahalanobis distance, Manhattan distance, etc. The embodiment of the present application does not limit the method for calculating the feature distance.

[0069] It can be understood that the embodiment of the present application is explained for one image to be processed, and the main commodities in the image to be processed are determined, so that the main commodities in the image to be processed are used as the target image features of the image to be processed. For multiple images to be processed, the processing method of the image to be processed can be referred to, so as to determine the main commodities in each image to be processed and the target image features of each image to be processed, so as to achieve accurate product push.

[0070] In one embodiment, product push can be performed based on target product features, thereby improving the accuracy of product push. Specifically, the target product features can be added to a product feature library; when a product selection operation of the second terminal is obtained, candidate product features are determined based on the product selection operation; based on the candidate product features, recommended product features and recommended merchant information associated with the recommended product features are matched from the product feature library, and the recommended product features and the products corresponding to the recommended merchant information are pushed to the second terminal.

[0071] Among them, the product feature library can store product features, merchant information associated with the product, product information associated with the product features, and the association between the three, etc. Product features can refer to features extracted by an image feature extraction model. Merchant information can include the merchant's logo, the merchant's product type, the merchant's product sales, etc. Product information associated with product features can refer to product information corresponding to the product features, or products matching the product features, such as products similar to the product features, etc. The second terminal can refer to a terminal that needs to push products, for example, the second terminal can refer to a terminal used by a customer, and the customer can perform operations such as product retrieval and image search for products through the second terminal. For example, a customer can trigger a selection operation on the second terminal, and the selection operation can refer to inputting or uploading an image on the second terminal, and then candidate product features can be matched from the product feature library based on the selection operation. For example, the image features of the image uploaded by the customer can be extracted to obtain candidate product features, and the similarity between the candidate product features and each product feature in the product feature library is calculated respectively, and the product features with a similarity greater than a similarity threshold are determined as recommended product features. And the merchant information associated with the recommended commodity feature in the commodity feature library is determined as the recommended merchant information, and the commodity corresponding to the recommended commodity feature and the recommended merchant information is pushed to the second terminal.

[0072] In an embodiment of the present application, by determining the main commodity features from multiple commodities in the image to be processed, and constructing a commodity feature library based on the commodity features of the main commodities, the commodity feature library can be accurately established to improve the accuracy of subsequent commodity push. By judging the consistency between the predicted category information of multiple commodities in the image to be processed and the title information and category information of the image to be processed, the main commodity in the image to be processed can be determined. In addition, by calculating the feature distance between the mean of the commodity features and the features of multiple commodities in the image to be processed, that is, the prior knowledge that the commodity has a high probability of appearing under the logo of the same merchant, the selection of the main frame of the commodity layer by layer from simple to difficult can be realized, which can well solve the problem of inaccurate determination of the main commodity caused by the irregular commodity information in the image, such as the arbitrary position and arbitrary area proportion of the commodity in the image, thereby improving the accuracy of commodity selection. Combining the image detection model, the image classification model, and the prior knowledge such as the title information and the category information, the designed commodity consistency judgment method can provide weakly supervised multimodal information for the accurate selection of the target main frame. In addition, since the merchant uploads a set of images to be processed for the same product, the principle of maximum probability of product appearance is described. By calculating the main frame with the smallest feature distance from the mean of product features by using multiple main frames in a single image to be processed, the main frame is used as the target main frame of the main product. This can effectively avoid false detection of image detection models and wrong product selection caused by non-standard product information, and the use of product feature mean calculation method can greatly reduce the amount of calculation.

[0073] In the embodiment of the present application, by obtaining the title information and category information of the image to be processed, image detection is performed on the image to be processed to determine the main frame corresponding to each commodity in the image to be processed, and the main frame corresponding to each commodity in the image to be processed is classified and predicted to determine the predicted category information of the main frame corresponding to each commodity, and then the predicted category information of the main frame corresponding to each commodity, the title information and category information of the image to be processed are combined to determine the target commodity features in the image to be processed, and then push the commodity. Since the main frame corresponding to each commodity in the image to be processed is classified and predicted, and is judged with the title information and category information of the image to be processed, any commodity in the image will not be ignored because of the position of the commodity in the image to be processed or the small area in the image to be processed. By judging the commodities in each detected main frame, the main commodities in the image to be processed can be determined, and then the target commodity features of the image to be processed can be obtained, which can improve the accuracy of commodity determination, and then when commodity push is performed based on the determined target commodity features, the accuracy of commodity push can be improved.

[0074] For further information, see Figure 3 , Figure 3 is a flow chart of another image processing method provided in an embodiment of the present application; Figure 3 As shown, the image processing method can be applied to a computer device, and the image processing method includes but is not limited to the following steps:

[0075] S201, obtaining an image to be processed, as well as title information and category information of the image to be processed.

[0076] Here, since a merchant may upload multiple images at the same time, for example, the set of images to be processed uploaded by the merchant includes multiple images to be processed, then for each image in the set of images to be processed, the main frame (bbox) in each image to be processed can be detected based on the image detection model, and the main frame is used to select the corresponding product in the image to be processed. Furthermore, the main frame corresponding to each product in the image to be processed can be classified and predicted by the image classification model to determine the predicted category information of the main frame corresponding to each product in the image to be processed, for example, the four-level category information of the product in each main frame from coarse to fine can be determined. Optionally, the image feature extraction model can be used to extract features from each main frame to obtain the visual representation information of the product in each main frame, that is, the image feature of each product.

[0077] S202, performing image detection on the image to be processed, determining a main body frame corresponding to each commodity in the image to be processed, and extracting image features of the commodity in each main body frame.

[0078] In the embodiment of the present application, the specific implementation of steps S201 to S202 can refer to the implementation of the above-mentioned steps S101 to S102, which will not be repeated here.

[0079] S203: If the number of main frames corresponding to the commodity in the image to be processed is one, determine whether the commodities in the main frame are consistent.

[0080] In an embodiment of the present application, the main body frames corresponding to each commodity in the image to be processed can be classified and predicted, and the predicted category information of the main body frames corresponding to each commodity in the image to be processed can be determined. Based on the predicted category information, title information and category information, the number of main body frames in the image to be processed can be determined, and it can be determined whether the commodities in each main body frame are consistent, thereby determining the target commodity characteristics of the main commodities in the image to be processed.

[0081] Specifically, when judging whether the commodities in each main frame are consistent, the predicted category information of the main frame corresponding to each commodity includes four-level category information, and the category information of the image to be processed includes four-level category information. For example, the predicted category information of any main frame includes four-level category information of "beauty, makeup, facial makeup, liquid foundation", when the third-level category information "facial makeup" or the fourth-level category information "liquid foundation" in the predicted category information of the main frame appears in the title information of the image to be processed, or the predicted category of the main frame If the text similarity between the fourth-level category information in the information and the fourth-level category information in the category information of the image to be processed is greater than 70%; or the fourth-level category information in the predicted category information of the main frame is in the fourth-level category information in the category information of the image to be processed; or the text similarity between the third-level category information "facial makeup" plus the fourth-level category information "liquid foundation" in the predicted category information of the main frame and the third-level category information plus the fourth-level category information in the category information of the image to be processed is greater than 65%, then it is determined that the products in the main frame are consistent, otherwise they are not consistent.

[0082] S204: If the product is not consistent, stop processing.

[0083] Here, the inconsistency of the product may mean that the number of products detected in the image to be processed is one, and the inconsistency of the product means that the product is not the main product in the image to be processed, and subsequent processing may not be performed, for example, the image features of the product do not need to be added to the product feature library.

[0084] Optionally, when performing image detection on the image to be processed and determining the main body frame corresponding to each product in the image to be processed, it is possible that the main body frame is not detected, that is, the main body in the image to be processed is not detected, and the image to be processed is not processed subsequently, for example, the image features of the product do not need to be added to the product feature library. For example, there is no product in the image to be processed, or the main body frame is not detected when performing image detection on the image to be processed due to the position of the product in the image, the viewing angle, or the incomplete product information.

[0085] S205: If the product is consistent, the image features of the product in the main frame are used as target product features of the image to be processed, and the target product features are added to a product feature library.

[0086] Here, since when a main body frame is detected, the goods in the main body frame are consistent, that is, the goods in the main body frame are consistent with the title information and category information (i.e., the store description category) of the image to be processed, it means that the main body frame may be a target main body frame that is not misdetected, and the image feature (feature) of the goods in the target main body frame can be added to the product feature library. Optionally, the image feature (feature) of the goods in the target main body frame, the product image (bbox information) in the main body frame, the identification of the image to be processed, or the identification of the merchant of the image to be processed can also be added to the product library, and the product retrieval and product push can be performed based on the product library later. Alternatively, the image feature (feature) of the goods in the target main body frame, the product image (bbox information) in the main body frame, the identification of the image to be processed, or the identification of the merchant of the image to be processed can also be directly added to the product feature library, and the product retrieval and product push can be performed based on the product feature library later, and the embodiment of the present application is not limited to this.

[0087] S206: If there are multiple main body frames corresponding to the commodity in the image to be processed, determine whether the commodities in the multiple main body frames are consistent.

[0088] Here, multiple may refer to two or more than two. When multiple subject frames exist in the image to be processed, it means that when multiple subjects are detected in the image to be processed, consistency judgment can be performed on the products in the multiple subject frames respectively to select the target subject frame.

[0089] S207: If there is a product that is consistent, the image feature of the product in the main frame is used as the target product feature of the image to be processed, and the target product feature is added to the product feature library.

[0090] Here, if it is determined that there is only one product in the main frame that is consistent, the image feature (feature) of the product in the main frame can be added to the product feature library. Optionally, the image feature (feature) of the product in the main frame, the product image (bbox information) in the main frame, the identifier of the image to be processed, or the identifier of the merchant of the image to be processed can also be added to the product library, and the product retrieval and product push can be performed based on the product library later. Alternatively, the image feature (feature) of the product in the main frame, the product image (bbox information) in the main frame, the identifier of the image to be processed, or the identifier of the merchant of the image to be processed can also be directly added to the product feature library, and the product retrieval and product push can be performed based on the product feature library later, and the embodiment of the present application does not limit this.

[0091] S208, if there are at least two products that are consistent, the feature distances between at least two product image features and the product feature mean are calculated respectively, the main body frame corresponding to the image feature with the smallest feature distance is determined as the target main body frame, and the image features of the product in the target main body frame are added to the product feature library.

[0092] Here, since at least two commodities are consistent, the mean of the commodity features corresponding to the commodity can be obtained for judgment. Specifically, for multiple images to be processed corresponding to the same commodity, after the target main frame is determined for each image to be processed, the image features of the commodity in the target main frame can be added to the commodity feature library, so as to calculate the feature mean of all the main frames corresponding to the commodity image to be processed in the commodity feature library as the commodity feature mean corresponding to the commodity image to be processed. For the image to be processed where the accurate target main frame cannot be obtained, the main frame with the closest main frame bbox to the commodity feature mean can be taken as the target main frame, and the commodity in the target main frame is the main commodity in the image to be processed. It can be understood that by utilizing the principle that the commodity described in all images of the same commodity appears most frequently, and by calculating the feature distance between the feature mean and the commodity feature, it is possible to obtain the main frame that is most likely to be the main commodity with a smaller amount of calculation.

[0093] S209, push products based on target product features.

[0094] In the embodiment of the present application, the product push method of step S209 can refer to the product push method of the above-mentioned step S104, which will not be repeated here.

[0095] In a possible implementation, the target commodity features of the image to be processed may also be determined in combination with the area of ​​each main frame in the image to be processed. For example, the target commodity features of the image to be processed may be determined based on the predicted category information of each main frame in the image to be processed, the area of ​​each main frame, the title information, and the category information.

[0096] In another possible implementation, the target commodity features of the image to be processed can also be determined in combination with the historical operation records of the second terminal. For example, the target commodity features of the image to be processed can be determined based on the historical operation records of the second terminal, the predicted category information, title information and category information of each main frame in the image to be processed. Among them, the historical operation records of the second terminal can be used to reflect the preferences of users associated with the user account using the second terminal for various commodities. The historical operation records of the second terminal may include the number of operations, operation frequency, preferences, etc. of the second terminal for each category of commodities. For example, when there are multiple commodities in the image to be processed that are consistent, the main frame corresponding to the commodity that matches the historical operation record of the second terminal among the multiple commodities is determined as the target main frame. By selecting the target main frame from multiple main frames in the image to be processed in combination with the historical operation records of the second terminal, and determining the image features of the commodity in the target main frame as the target commodity features, targeted commodity push can be achieved to improve user experience.

[0097] In the embodiment of the present application, by obtaining the title information and category information of the image to be processed, image detection is performed on the image to be processed to determine the main frame corresponding to each commodity in the image to be processed, and the main frame corresponding to each commodity in the image to be processed is classified and predicted to determine the predicted category information of the main frame corresponding to each commodity, and then the predicted category information of the main frame corresponding to each commodity, the title information and category information of the image to be processed are combined to determine the target commodity features in the image to be processed, and then push the commodity. Since the main frame corresponding to each commodity in the image to be processed is classified and predicted, and is judged with the title information and category information of the image to be processed, any commodity in the image will not be ignored because of the position of the commodity in the image to be processed or the small area in the image to be processed. By judging the commodities in each detected main frame, the main commodities in the image to be processed can be determined, and then the target commodity features of the image to be processed can be obtained, which can improve the accuracy of commodity determination, and then when commodity push is performed based on the determined target commodity features, the accuracy of commodity push can be improved.

[0098] The method of the embodiment of the present application is introduced above, and the device of the embodiment of the present application is introduced below.

[0099] See also Figure 4 , Figure 4 : is a schematic diagram of the composition structure of an image processing device provided in an embodiment of the present application. The above-mentioned image processing device can be deployed on a computer device; the image processing device can be used to execute the corresponding steps in the image processing method provided in an embodiment of the present application. The image processing device 40 includes:

[0100] The information acquisition unit 401 is used to acquire the image to be processed, as well as the title information and category information of the image to be processed;

[0101] The image detection unit 402 is used to perform image detection on the image to be processed, and determine the main frame corresponding to each commodity in the image to be processed; the main frame corresponding to each commodity is used to select the corresponding commodity in the image to be processed;

[0102] The image prediction unit 403 is used to classify and predict the main frame corresponding to each commodity in the image to be processed, and determine the predicted category information of the main frame corresponding to each commodity in the image to be processed;

[0103] The information pushing unit 404 is used to determine the target product feature of the image to be processed based on the predicted category information, the title information and the category information, and push the product based on the target product feature.

[0104] Optionally, the image prediction unit 403 is specifically configured to:

[0105] Classify and predict the main frame corresponding to each commodity in the image to be processed, and obtain the probability that the commodity in each main frame belongs to multiple commodity categories;

[0106] Obtain multi-level category information to which the commodity category corresponding to the maximum probability among the probabilities of the multiple commodity categories belongs;

[0107] The multi-level category information to which the commodity category corresponding to the maximum probability belongs is determined as the predicted category information of the main frame corresponding to each commodity in the image to be processed.

[0108] Optionally, the image processing device 40 may further include a feature extraction unit 405, wherein the feature extraction unit 405 is configured to:

[0109] Extract features of each main frame in the image to be processed to obtain image features of the products in each main frame;

[0110] The information push unit 404 is specifically used for:

[0111] Determining whether the commodities in each main frame are consistent based on the predicted category information of each main frame in the image to be processed, the title information and the category information;

[0112] If only one of the main frames has the same commodity, the image feature of the commodity in the main frame is used as the target commodity feature of the image to be processed;

[0113] If there are at least two main body frames whose commodities are consistent among the main body frames, a target main body frame is selected from the at least two main body frames, and the image features of the commodities in the target main body frame are used as the target commodity features of the image to be processed.

[0114] Optionally, the predicted category information includes at least one level of category information, and the category information includes at least one level of category information; the information pushing unit 404 is specifically used to:

[0115] If at least one level of category information included in the predicted category information matches the title information, it is determined that the commodities in each main frame are consistent; or,

[0116] If at least one level of category information included in the predicted category information matches at least one level of category information included in the category information, it is determined that the commodities in each main frame are consistent.

[0117] Optionally, the information pushing unit 404 is specifically used to:

[0118] Get the mean of product features;

[0119] Calculating feature distances between image features of the products in the at least two main frames and feature mean values ​​of the products respectively;

[0120] The main body frame corresponding to the image feature with the smallest feature distance is determined as the target main body frame.

[0121] Optionally, the information acquisition unit 401 is specifically used to:

[0122] Detecting an image upload instruction of the first terminal, and determining an image corresponding to the image upload instruction as the image to be processed;

[0123] sending at least one reference title information and at least one reference category information to the first terminal;

[0124] Respectively acquiring a title selection operation of the first terminal for the at least one reference title information and a category selection operation for the at least one reference category information;

[0125] The reference title information corresponding to the title selection operation is determined as the title information, and the reference category information corresponding to the category selection operation is determined as the category information.

[0126] Optionally, the information pushing unit 404 is specifically used to:

[0127] Add the target product feature to the product feature library;

[0128] When a product selection operation of the second terminal is obtained, determining the characteristics of the candidate products based on the product selection operation;

[0129] Based on the candidate product feature, a recommended product feature and recommended merchant information associated with the recommended product feature are matched from the product feature library, and the recommended product feature and the product corresponding to the recommended merchant information are pushed to the second terminal.

[0130] It should be noted that Figure 4 For the contents not mentioned in the corresponding embodiments, please refer to the description of the method embodiments, which will not be repeated here.

[0131] In the embodiment of the present application, by obtaining the title information and category information of the image to be processed, image detection is performed on the image to be processed to determine the main frame corresponding to each commodity in the image to be processed, and the main frame corresponding to each commodity in the image to be processed is classified and predicted to determine the predicted category information of the main frame corresponding to each commodity, and then the predicted category information of the main frame corresponding to each commodity, the title information and category information of the image to be processed are combined to determine the target commodity features in the image to be processed, and then push the commodity. Since the main frame corresponding to each commodity in the image to be processed is classified and predicted, and is judged with the title information and category information of the image to be processed, any commodity in the image will not be ignored because of the position of the commodity in the image to be processed or the small area in the image to be processed. By judging the commodities in each detected main frame, the main commodities in the image to be processed can be determined, and then the target commodity features of the image to be processed can be obtained, which can improve the accuracy of commodity determination, and then when commodity push is performed based on the determined target commodity features, the accuracy of commodity push can be improved.

[0132] See also Figure 5 , Figure 5 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 5 As shown, the above-mentioned computer device 50 may include: a processor 501, a network interface 504 and a memory 505. In addition, the above-mentioned computer device 50 may also include: a user interface 503, and at least one communication bus 502. Among them, the communication bus 502 is used to realize the connection and communication between these components. Among them, the user interface 503 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 503 may also include a standard wired interface and a wireless interface. The network interface 504 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 505 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 505 may optionally also be at least one storage device located away from the aforementioned processor 501. As Figure 5As shown, the memory 505 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0133] exist Figure 5 In the computer device 50 shown, the network interface 504 can provide a network communication function; the user interface 503 is mainly used to provide an input interface for the user; and the processor 501 can be used to call the device control application stored in the memory 505 to achieve:

[0134] Obtaining an image to be processed, as well as title information and category information of the image to be processed;

[0135] Performing image detection on the image to be processed to determine the main frame corresponding to each commodity in the image to be processed; the main frame corresponding to each commodity is used to select the corresponding commodity in the image to be processed;

[0136] Classify and predict the main frame corresponding to each commodity in the image to be processed, and determine the predicted category information of the main frame corresponding to each commodity in the image to be processed;

[0137] Based on the predicted category information, the title information and the category information, target product features of the image to be processed are determined, and product push is performed based on the target product features.

[0138] It should be understood that the computer device 50 described in the embodiment of the present application can execute the above Figure 2 and Figure 3 The description of the above-mentioned image processing method in the corresponding embodiment can also be performed as described above. Figure 4 The description of the above-mentioned image processing device in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here either.

[0139] In the embodiment of the present application, by obtaining the title information and category information of the image to be processed, image detection is performed on the image to be processed to determine the main frame corresponding to each commodity in the image to be processed, and the main frame corresponding to each commodity in the image to be processed is classified and predicted to determine the predicted category information of the main frame corresponding to each commodity, and then the predicted category information of the main frame corresponding to each commodity, the title information and category information of the image to be processed are combined to determine the target commodity features in the image to be processed, and then push the commodity. Since the main frame corresponding to each commodity in the image to be processed is classified and predicted, and is judged with the title information and category information of the image to be processed, any commodity in the image will not be ignored because of the position of the commodity in the image to be processed or the small area in the image to be processed. By judging the commodities in each detected main frame, the main commodities in the image to be processed can be determined, and then the target commodity features of the image to be processed can be obtained, which can improve the accuracy of commodity determination, and then when commodity push is performed based on the determined target commodity features, the accuracy of commodity push can be improved.

[0140] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, which includes program instructions, which, when executed by a computer, causes the computer to execute the method of the aforementioned embodiment, and the computer may be part of the computer device mentioned above. For example, it is the above-mentioned processor 501. As an example, the program instructions may be deployed on a computer device for execution, or deployed on multiple computer devices located at one location for execution, or, executed on multiple computer devices distributed at multiple locations and interconnected by a communication network, and multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain network.

[0141] Those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0142] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image to be processed, as well as title information and category information of the image to be processed; Performing image detection on the image to be processed to determine a main frame corresponding to each commodity in the image to be processed; the main frame corresponding to each commodity is used to select the corresponding commodity in the image to be processed; Classify and predict the main frame corresponding to each commodity in the image to be processed, and determine the predicted category information of the main frame corresponding to each commodity in the image to be processed; Based on the predicted category information, the title information and the category information of the main frame corresponding to each product, a target main frame is determined to determine the target product features of the image to be processed, and products are pushed based on the target product features; the image to be processed includes multiple products, the target product features refer to the features of the products in the target main frame in the image to be processed, and the predicted category information of the target main frame matches the title information and / or the category information.

2. The method according to claim 1, characterized in that The classifying and predicting the main frame corresponding to each commodity in the image to be processed to determine the predicted category information of the main frame corresponding to each commodity in the image to be processed includes: Classify and predict the main frame corresponding to each commodity in the image to be processed to obtain the probability that the commodity in each main frame belongs to multiple commodity categories; Obtain multi-level category information to which the commodity category corresponding to the maximum probability among the probabilities of the multiple commodity categories belongs; The multi-level category information to which the commodity category corresponding to the maximum probability belongs is determined as the predicted category information of the main frame corresponding to each commodity in the image to be processed.

3. The method according to claim 1, characterized in that The method further comprises: Extracting features from each main frame in the image to be processed to obtain image features of the commodities in each main frame; The step of determining a target main frame based on the predicted category information, the title information, and the category information of the main frame corresponding to each commodity, so as to determine the target commodity feature of the image to be processed, includes: Determining whether the commodities in each main frame are consistent based on the predicted category information of each main frame in the image to be processed, the title information and the category information; If there is only one commodity in each of the main frames that is consistent, the image features of the commodity in the main frame are used as the target commodity features of the image to be processed; If there are at least two main body frames whose commodities are consistent among the main body frames, a target main body frame is selected from the at least two main body frames, and image features of the commodities in the target main body frame are used as target commodity features of the image to be processed.

4. The method according to claim 3, characterized in that The predicted category information includes at least one level of category information, and the category information includes at least one level of category information; The determining whether the commodities in each main frame are consistent based on the predicted category information of each main frame in the image to be processed, the title information and the category information includes: If at least one level of category information included in the predicted category information matches the title information, it is determined that the commodities in each main frame are consistent; or, If at least one level of category information included in the predicted category information matches at least one level of category information included in the category information, it is determined that the commodities in each main frame are consistent.

5. The method according to claim 3, characterized in that: The step of selecting a target main body frame from the at least two main body frames comprises: Acquire a product feature mean; the product feature mean is used to reflect the features of the products under the title information and the category information, and the product feature mean is obtained by the features of the products matching the title information and the category information in a product feature library, wherein the product feature library includes features of the products in a plurality of images to be processed; Calculating feature distances between image features of the commodities in the at least two main frames and mean values ​​of the commodity features respectively; The main body frame corresponding to the image feature with the smallest feature distance is determined as the target main body frame.

6. The method according to claim 1, characterized in that The acquiring of the image to be processed, and the title information and category information of the image to be processed includes: detecting an image upload instruction of the first terminal, and determining an image corresponding to the image upload instruction as the image to be processed; sending at least one reference title information and at least one reference category information to the first terminal; Respectively acquiring a title selection operation of the first terminal for the at least one reference title information and a category selection operation for the at least one reference category information; The reference title information corresponding to the title selection operation is determined as the title information, and the reference category information corresponding to the category selection operation is determined as the category information.

7. The method according to any one of claims 1 to 6, characterized in that: The pushing of products based on the target product features includes: Adding the target product features to a product feature library; When a product selection operation of the second terminal is obtained, determining the characteristics of the candidate products based on the product selection operation; Based on the candidate product features, recommended product features and recommended merchant information associated with the recommended product features are matched from the product feature library, and products corresponding to the recommended product features and the recommended merchant information are pushed to the second terminal.

8. An image processing device, characterized in that: The device comprises: An information acquisition unit, used to acquire an image to be processed, and title information and category information of the image to be processed; An image detection unit is used to perform image detection on the image to be processed, and determine a main frame corresponding to each commodity in the image to be processed; the main frame corresponding to each commodity is used to select the corresponding commodity in the image to be processed; An image prediction unit, configured to classify and predict the main frame corresponding to each commodity in the image to be processed, and determine the predicted category information of the main frame corresponding to each commodity in the image to be processed; An information push unit is used to determine a target main frame based on the predicted category information, the title information and the category information of the main frame corresponding to each product, so as to determine the target product features of the image to be processed, and push products based on the target product features; the image to be processed includes multiple products, the target product features refer to the features of the products in the target main frame in the image to be processed, and the predicted category information of the target main frame matches the title information and / or the category information.

9. A computer device, characterized in that: include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store program code, and the processor is used to call the program code so that the computer device executes the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images